From patchwork Thu Sep 30 14:30:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2D72C433FE for ; Thu, 30 Sep 2021 14:33:34 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7694F60FC2 for ; Thu, 30 Sep 2021 14:33:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7694F60FC2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Y4Eb6sS93HoOqU+F9JOyR0QPNTgNbAH5vAoRYbtSnRs=; b=WB/rauubF3EEWx 0//7q0DthVPoWSRrWvQw2V0lDVlyzE68RidWdI8eh2QNPYtuObC45Xeh9sv4OU22Iad77GVr8Uben 0WYDuQTuOgxu37r1+mCo+tEewGzev3BvRzaGHgNCi2YjRspN5tZHmOQ2w9/FxIA0D4wlhda1aYUhe nfs4FPBsBQ0HA7WrENL1SLxdbABxINrEJpNU8UG0530GlnguGIfQJX27WTs21CqsVp+O5zp9FxkMk 4eNrQDykjbAnga1wR+ZbwDpUwqugzvK2YHDwTXopqY5AkRYIbtPYBqUg7GvxeBaHCXFoHa/MA+XVP K554noYmESvncfwknubw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx6E-00EhfY-3D; Thu, 30 Sep 2021 14:31:50 +0000 Received: from mail-qt1-x835.google.com ([2607:f8b0:4864:20::835]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5h-00EhN2-GX for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:21 +0000 Received: by mail-qt1-x835.google.com with SMTP id d8so5836087qtd.5 for ; Thu, 30 Sep 2021 07:31:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZgbetCXmU2I4AxLznQ7nGzSUyVlFTduK3mMsOCeyR2E=; b=NvUabrPKnpzcu5TeS9l5XS+8SzQvkmleB8d/hakxdXIDBVUCJ9US6MWCXxhdTzzOig rr/9DOSe6KjgBFl4joofHr//byFKjS0nSWBIndIorTah/fBqaOu/+2XRbb7FbH0UuTwv Qs+9FnH6mov9SV8ogh3PfFtbpic6oXECOAXGs49oQpo/gRXLDCIhgwUHOgU2toON5fDa ZxNRn+d1qmt4sgmUbA+DVxfghB99G1F4g68Ox7xXTW6zqsK4O3qyumJD+UElSbXa4FRJ ayCH2de8a5bJEypbyBdzpWcQ30ht6ipCMWAJuJAaNNHEDZVYI6LaWbsuZo+3w/8KAA/g cSew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZgbetCXmU2I4AxLznQ7nGzSUyVlFTduK3mMsOCeyR2E=; b=ZCtCXoF3QBw5tbEk01bZF5siMsB0SX+eG2hWn3GocMaG39QRBrFCRT8FodO1tf2uTQ UyNjj4fpul2F3liBgO9N/yxarD7HHWuhk/l0/Q8XWA7LMs4BXD6cmlFeC7dL0DJdKBdI J3rl5vOUhulW7s7meW/azAjXf7gkYmM2KCx+YQCLica26NluJClK7qCj4qWgk/jVC0fU P0wUg9cz6hJPeJoVx8NSjjZqyiw6vHH9gsIInjch7ewfvuqF3mtCpA57Atha9ER9GDGP F1qZIrwDboie2pQaGq237VJ50vHqlDA8JGpUTpCKyg9ni73nLkEERL+5WQCMjgBkXk5M uIlA== X-Gm-Message-State: AOAM530P7ACiE14bqaSE+5qIT71Pnvf+KwJUkpTGZOhobeNapFP/st8D DOe/X1vlYjFhxqzuutpYYq9fHQ== X-Google-Smtp-Source: ABdhPJxvgW9/FtyJRraTI54gn/Z60LChe52Uu0d4MLes/qY0mj18LYUViK9I0CDtWXyZAWETFAM8yw== X-Received: by 2002:ac8:4c84:: with SMTP id j4mr6839446qtv.411.1633012276760; Thu, 30 Sep 2021 07:31:16 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:16 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 01/15] arm64: kernel: add helper for booted at EL2 and not VHE Date: Thu, 30 Sep 2021 14:30:59 +0000 Message-Id: <20210930143113.1502553-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073117_582099_ED10BD8E X-CRM114-Status: GOOD ( 15.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_nvhe(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 2 +- arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..a9457e96203c 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_nvhe(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 9a7b1262ef17..81b3d0fe7a63 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 46a0b4d6e251..4c9533f4c0c4 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -49,7 +49,7 @@ extern int in_suspend; /* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) +#define el2_reset_needed() (is_hyp_nvhe()) /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 47f77d1234cb..d20620a1c51a 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -202,7 +202,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_nvhe()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Thu Sep 30 14:31:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEF46C433F5 for ; Thu, 30 Sep 2021 14:33:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9244361423 for ; Thu, 30 Sep 2021 14:33:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9244361423 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WlXXQSPsB8nb2po5MSQd4kZJ4vYBL1uv2Dpi1F5k1K0=; b=jltXuWmgIsZ+uN Ry1NXpzu4WlaT+QvPGUo9xVNs2AaNGyV2sl05Hn2q5c9BeQKupCbDymSMaBTWcaYLEIeMz45l2Gt4 xakCoK7HvPOoQ0fLXk5wEuuNwXT7A332rZRuw6NuZ8cse7Kjqe3ZH+dqBlwhFEll0xCdZXRT68AIM Qj2l/aoku80vUNXWGYh4rwCdqlCJX6twPljZZTbsbYcbFxEt+6wcsdh08/9FBz4XuE1yz6t+gf6x9 +0a3deTbah22XVOMv6SmdYDzthDjglVbj1IWh70FkPCL25ACuNboQZiBHWxugExtOFTQsPww+PbU3 hOONqN0Qj/o53lgJtA0w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx6R-00Ehkz-T6; Thu, 30 Sep 2021 14:32:04 +0000 Received: from mail-qv1-xf34.google.com ([2607:f8b0:4864:20::f34]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5j-00EhNy-Da for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:22 +0000 Received: by mail-qv1-xf34.google.com with SMTP id d20so260114qvm.8 for ; Thu, 30 Sep 2021 07:31:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+4++pnD78fv084ZJJb31cAwxAfjQdVgmdWKFv3ofNWM=; b=PwSkFB5lUIJlNNvHorXkdi5cHBr8TW91BMYYJo/Le3xqGzArqbpyx08n0Ln1FNN8V9 FAFTqIVj/f6d2W2gsY177xVTv/r9AbnqH4GQjLc5PtOUs8+4OQmadzz3A1Xs43xchghI Wh4vRLWrbaVRb9Pb5gtLeFoVmTZXCEB68pVST4gG3jNkJaIE4sHBDcgxA2xRJF9rcc5i CM08jBrUniz8B0qRG8PFfjRS7s22iK1OMmYBeFI8eXLY2MI7XDHeoUOCgT3O2uDFTCHN esQ15Pe1NoByE+7fmDdFl6Uf/mCv2brr7uhwPGQwntr7Nt5pqreknxdkGI2gzzHDVK1Y ipZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+4++pnD78fv084ZJJb31cAwxAfjQdVgmdWKFv3ofNWM=; b=54sx6nkUO9/iTT5sFKX7OcV28oJa641OtcS9whycr1zmQchyQmDYLONkpJl9s+RsOT 97rWYsEznzy1Dj9F4gu4ER/iOoudgVYsqsiaxW04GHO8zJJyfwVTDfGJ1KijRhP74Hrw J4tf69pTiTm0X8XR+w9XwYBC84vJLWv9pf3Q4Jog0k23fydosI+UXwiUUTYGD0RAYTH0 hg8mm4B4jMOlkJjT8pXaxI7mIygO4cYjb2NM1oofcg59NGYKDlJ0XxbvP1jzELgAci+X Xhd4PWyqQE7zQfOtSh0zIxISlDEn2gXTgFBsoVu9AZPWf0mjBjvMSYJAZ5l4I05BV1W3 fURg== X-Gm-Message-State: AOAM533PcZtyNC9wZcF/DvZuJkiGRAWenbkj9VZ9br3OWioAEonyS5vj 7G6HbW1PXA17AxDgp5Cl+V1vHw== X-Google-Smtp-Source: ABdhPJzkISSDNj6DI6kaZcbl1IAI6lqibUXU1DRHLZaDVciLYY9K0DrqkYiXrO51Bv8hV2XWzusKEg== X-Received: by 2002:a0c:f38b:: with SMTP id i11mr1177471qvk.20.1633012277774; Thu, 30 Sep 2021 07:31:17 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:17 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 02/15] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Thu, 30 Sep 2021 14:31:00 +0000 Message-Id: <20210930143113.1502553-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073119_476685_30A37FDF X-CRM114-Status: GOOD ( 23.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/trans_pgd.h | 9 +++-- arch/arm64/include/asm/virt.h | 2 ++ arch/arm64/kernel/hibernate-asm.S | 52 --------------------------- arch/arm64/kernel/hibernate.c | 26 ++++++-------- arch/arm64/mm/Makefile | 1 + arch/arm64/mm/trans_pgd-asm.S | 58 ++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd.c | 27 ++++++++++++-- 7 files changed, 103 insertions(+), 72 deletions(-) create mode 100644 arch/arm64/mm/trans_pgd-asm.S diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..7b04d32b102c 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -1,8 +1,8 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Copyright (c) 2020, Microsoft Corporation. - * Pavel Tatashin + * Copyright (c) 2021, Microsoft Corporation. + * Pasha Tatashin */ #ifndef _ASM_TRANS_TABLE_H @@ -36,4 +36,9 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + +extern char trans_pgd_stub_vectors[]; + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index a9457e96203c..3c8af033a997 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,8 @@ */ extern u32 __boot_cpu_mode[2]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 81c0186a5e32..a30a2c3f905e 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -112,56 +112,4 @@ alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE hvc #0 3: ret SYM_CODE_END(swsusp_arch_suspend_exit) - -/* - * Restore the hyp stub. - * This must be done before the hibernate page is unmapped by _cpu_resume(), - * but happens before any of the hyp-stub's code is cleaned to PoC. - * - * x24: The physical address of __hyp_stub_vectors - */ -SYM_CODE_START_LOCAL(el1_sync) - msr vbar_el2, x24 - eret -SYM_CODE_END(el1_sync) - -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - -/* el2 vectors - switch el2 here while we restore the memory image. */ - .align 11 -SYM_CODE_START(hibernate_el2_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t - - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h - - ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 -SYM_CODE_END(hibernate_el2_vectors) - .popsection diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 4c9533f4c0c4..b96ef9060e4c 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -51,9 +51,6 @@ extern int in_suspend; /* Do we need to reset el2? */ #define el2_reset_needed() (is_hyp_nvhe()) -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - /* hyp-stub vectors, used to restore el2 during resume from hibernate. */ extern char __hyp_stub_vectors[]; @@ -434,6 +431,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -461,6 +459,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (el2_reset_needed()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -473,26 +479,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - dcache_clean_inval_poc((unsigned long)hibernate_exit, - (unsigned long)hibernate_exit + exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (el2_reset_needed()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index f188c9092696..ff1e800ba7a1 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o +obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o KASAN_SANITIZE_physaddr.o += n diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S new file mode 100644 index 000000000000..8c4bffe3089d --- /dev/null +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +/* + * Copyright (c) 2021, Microsoft Corporation. + * Pasha Tatashin + */ + +#include +#include +#include + +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + +.macro el1_sync_vector +SYM_CODE_START_LOCAL(el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS /* Called from hibernate */ + b.ne 1f + msr vbar_el2, x1 + mov x0, xzr + eret +1: /* Unexpected argument, set an error */ + mov_q x0, HVC_STUB_ERR + eret +SYM_CODE_END(el1_sync) +.endm + +SYM_CODE_START(trans_pgd_stub_vectors) + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t + + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h + + el1_sync_vector // Synchronous 64-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__trans_pgd_stub_vectors_end, SYM_L_LOCAL) +SYM_CODE_END(trans_pgd_stub_vectors) + +# Check the trans_pgd_stub_vectors didn't overflow +.org . - (__trans_pgd_stub_vectors_end - trans_pgd_stub_vectors) + SZ_2K diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..26bd8f2d95af 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -5,8 +5,8 @@ * * This file derived from: arch/arm64/kernel/hibernate.c * - * Copyright (c) 2020, Microsoft Corporation. - * Pavel Tatashin + * Copyright (c) 2021, Microsoft Corporation. + * Pasha Tatashin * */ @@ -322,3 +322,26 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &trans_pgd_stub_vectors, ARM64_VECTOR_TABLE_LEN); + caches_clean_inval_pou((unsigned long)hyp_stub, + (unsigned long)hyp_stub + + ARM64_VECTOR_TABLE_LEN); + dcache_clean_inval_poc((unsigned long)hyp_stub, + (unsigned long)hyp_stub + + ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Thu Sep 30 14:31:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEB66C433F5 for ; Thu, 30 Sep 2021 14:34:51 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9DA8C60FC2 for ; Thu, 30 Sep 2021 14:34:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9DA8C60FC2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QF5TuBiBN/+yiLm/EwWKfUv2MkDNSHsx8wzsgTu9sQo=; b=mLyUFCbT+ZeiBK t8Htto9GUqLa4CcGxLErvidJynGCsdCS1AUe9ZSHjSKsoyLFiYMVmFapH7PjjHBicNeiAI8FuuJLn prQKOPgd5IN7Tfw52WZH5fKn/shIrFXmxcMuOACqbcr5afA1/BpVrEG5ialqjg5ryHYhZbsluNg3i h4RddAr0kgSqiRCPXZtOMwL/yDYHA//Y4Io1fJmq1g06GUexpir5TmZ/DEhRchA4KtBLOeXHa9Wsg ymMTkYVohPdAwJBu57Ggz6clegFZhthVYXWvmx5A+8pC8inlaIlrySB7FRV5roONWi+vzDN49DjDI uskJxQ+WIfIsbxbNZvYw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx7Z-00Ei7b-Pb; Thu, 30 Sep 2021 14:33:14 +0000 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5j-00EhNz-Oq for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:23 +0000 Received: by mail-qv1-xf31.google.com with SMTP id a14so3683790qvb.6 for ; Thu, 30 Sep 2021 07:31:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=e39R8ZsIeIwklE8cpr9UY0qKnjifmGWiOSmlNRknozY=; b=Dvw31wgsS8G8oxRKiLY9AhC7vsIrGYUqnkg0rdzHudMErK/hqx/f2xKLuM+d+wFz94 JUmD5np8vuwG2sGXNGsbm1qYTnL5SwmKk3BvG6nraqjwNtTgysYN2X6REwY/fvMqNdAz WLoSnNhuaCUYjqopKhwffkXgoe7lX17Re98nrYZngFujO2+9S5P4HTM+0zeBJdgd5nLG 2YrkHbtNBlzYOOJByqwcPTwT9QKuRm31Mkw6DL6XZxRQc6JsHrc/vF3FUunfcRX4Tuxg zJNVbfpaez0v6k+uebd2NVVfyA93/+wABN7kKOsZMg78c45ftAn4QhH3QWl22IAt60Vj 3CeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e39R8ZsIeIwklE8cpr9UY0qKnjifmGWiOSmlNRknozY=; b=JJMMpvXPEnlmsW4UWp6J/PEOSwx1pj0PEmQIyHLdb7oVdTd4kPFGxPtS8Dlych9ZTE H7xKeWyJlkAS/fHJpQ81Nc0ExNqWGFc2LySsEdA8lQTwW4cLkTbp0hk0svvR8IBO6G8+ 4kQs5gON0Iyde+51bpVcoklg992pJvg/rhzbkcpNJVt7pcWUiFr/HNxIiyrr9gFbs3uH g4KUk3m7oG4G7M1smbwsJiyRbScTl8k4TxZBcA3O4Ndag9yE/SooF+cQiKEzEDNUkqJZ 3qzRgYzo9xx0gcToV48yQ9kKypKyzv1QAwy/tC7EV88gQlxfVYQG3AoBMhUlQfQ2xqGy n7iA== X-Gm-Message-State: AOAM5327TOek9QKATAER8fmdRDL0Hxijgwn02s9v5uf74B5d/nWbWRGw 9r+2xhAYakG3dGGqkYBS60bmFQ== X-Google-Smtp-Source: ABdhPJyKP/5js7mH/GPTqK5GuDttLVlJNJvbT8t4LfCfXnHZzGw54YFenib9JG+PlW6v9j7cikxMqQ== X-Received: by 2002:a05:6214:2d1:: with SMTP id g17mr5468778qvu.63.1633012278962; Thu, 30 Sep 2021 07:31:18 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:18 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 03/15] arm64: hibernate: abstract ttrb0 setup function Date: Thu, 30 Sep 2021 14:31:01 +0000 Message-Id: <20210930143113.1502553-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073119_885536_94B4A4CC X-CRM114-Status: GOOD ( 15.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functionality when relocation code is going to be idmapped. Move the setup sequence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index f4ba93d4ffeb..6770667b34a3 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b96ef9060e4c..2758f75d6809 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -212,26 +212,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Thu Sep 30 14:31:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64D6EC433F5 for ; Thu, 30 Sep 2021 14:34:04 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33E4860FC2 for ; Thu, 30 Sep 2021 14:34:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 33E4860FC2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=igwy2fkadwzASFWCd3+gfrD/W3MsNsdzBqjRbIzwRdY=; b=j5C2rZHc4Z1kqr P8e6hFdSxPQVrYUT9MzYgOKmmPd2lmDHiLTedi2oqdDbl0p13+FBg5c5J+V1QpQbCcJHGHtNFU0AC SjU+oq2yBQ9gFb2JtZ+KjjMQUv6sXaP6AwVNXgMVDM/UAVlGOj7RZLrTjAudJM36A8+HErFMbIe2w R4+9BfagM8V9YSDfdtkPAn4bJGhqf6Xz0xROg26k0unXV8/XI3Ez173zPSXIdexK9jU8SCheymXdu Omj54+nGvv1cjUd2uwQM/vlnV0Ni0CK7+ARJVXZ9+6CgDROoOL8AblnLfBSi5/f2kkLZdyNU/y+zg TNytNTfVzHw9VDOD3OjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx6r-00EhrF-As; Thu, 30 Sep 2021 14:32:30 +0000 Received: from mail-qv1-xf36.google.com ([2607:f8b0:4864:20::f36]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5l-00EhPT-Ex for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:24 +0000 Received: by mail-qv1-xf36.google.com with SMTP id v14so2326494qvl.9 for ; Thu, 30 Sep 2021 07:31:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wvhp7Shwzj2V1AcHSUNo3T+jO//upg214s/8JF7+8X4=; b=WzTIvObi8BmW8NTQjOqb0FOv0H/1NPIY+oIISBMpm8fDw4WooAALyuQ8nS0FbKt/SD oG484TyexjpEF9PLSUf6FoVGHUmJa7wCuNbtCA+2gWYOBP7IMxuKsxo2SB2qce1734pU xsEoWg7TnfyCy6fwEslgvv5G4WPjSmJb/ebM1dBbO3PAOPhAMBgXqHNADEKlZZw80hfB UHXFu0nSxEI896qgpyWWYjO5i4k8yAw4Zdf5kcNiCy9l3kfnM4g1YbizWJfXBfT0X1pO i58tA/zr32BfcRs2J3R6EKHkuX/khIa1jSiNc1733FUETlk2PlBrtbTQuqXUQUOrPrxJ 0vIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wvhp7Shwzj2V1AcHSUNo3T+jO//upg214s/8JF7+8X4=; b=HODBZzBIqKU8skthwQFMSwFVPDlpi+X4jBR3BSPFshB/sDyYOE9vizCbbQKAbQMq/v msUMmq3SEE+KQphRUbgJ2Kt1Eqmeg6i7FdckOvzzkwpZXpZCMtTaXHoR8RzFju4Jfu4j 9FwMK7RcLwOeE+FBsDB9idNAA5jQfOfQuUpcckaoRuuGvAQiJR9JfmIbSCjt+CE7Ur8Y MlawnZtiBDGm6kALareCK8GBRKEKPibWzRKYL7+xJLc4Oxh9Jgi3kt0lMSh/2kDoGwXp j3ps1tpjFl36yeXXKpp6GqKZmwZ3k6K9z9k3PruzAnF41EhPqYDfRYkhdo5zhSXF6N8y ccYA== X-Gm-Message-State: AOAM533wYxrecPdAw01cf8TCel7YIS7qKfxmhNBYcPjdQXOLbGfRJCrv yuxmhwkQ+U8R2Np6P6m1mUoOjw== X-Google-Smtp-Source: ABdhPJyRpe+0jXAcRshmB05WHMM4GvLP/u4h4SOYX1etnyUhK8wl9VzQ7qbqdCBBQWvQ1rXkOCyYsA== X-Received: by 2002:a05:6214:90:: with SMTP id n16mr4150225qvr.1.1633012279941; Thu, 30 Sep 2021 07:31:19 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:19 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 04/15] arm64: kexec: flush image and lists during kexec load time Date: Thu, 30 Sep 2021 14:31:02 +0000 Message-Id: <20210930143113.1502553-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073121_529006_0D6C5204 X-CRM114-Status: GOOD ( 16.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/kernel/machine_kexec.c | 58 ++++++++++++++----------------- 1 file changed, 26 insertions(+), 32 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 213d56c14f60..b6d5a02cba2e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -60,29 +60,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* - * For execution with the MMU off, reloc_code needs to be cleaned to the - * PoC and invalidated from the I-cache. - */ - dcache_clean_inval_poc((unsigned long)reloc_code, - (unsigned long)reloc_code + - arm64_relocate_new_kernel_size); - icache_inval_pou((uintptr_t)reloc_code, - (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -163,6 +140,32 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + dcache_clean_inval_poc((unsigned long)reloc_code, + (unsigned long)reloc_code + + arm64_relocate_new_kernel_size); + icache_inval_pou((uintptr_t)reloc_code, + (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -180,13 +183,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -261,8 +257,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Thu Sep 30 14:31:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CED4C433EF for ; Thu, 30 Sep 2021 14:35:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EF64160FC2 for ; Thu, 30 Sep 2021 14:35:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org EF64160FC2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=E5lxmsYqlzLTZgjsh/+3xLO28wzqu8RRdYjxpssakrg=; b=z4nuMx9AHvexJx jWcvclWVdKJ9qs7a7VO2z0/QKuvmRBx0MHyWENjqc+za3u1fFDVHOMWHRWv6Pd0vYC5nDrbELU0Vr 3buj/FdPJZLJd/0V2wK6735TgIz1QffoRGTxDOqWTXTjnHSNVsrEgZrR8K4Gw3peijzQxCjIdPGTD jsJ6V39kvfq/RxGHmpL7dE3US9OHeTSVr5iSM0aQx6jYDlN8dPudB82nGAVQ7LA8iFWMAw9KiuL2H rr2LRcJPqdNRjyM6nhdJ9QANPP8oFEWBVZXrctn+3B3r2HZ0oEKFq1wbbD0x/R0u1MESOOO6NpXhG PbE50ola2P27kftzoHjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx80-00EiPA-Lf; Thu, 30 Sep 2021 14:33:40 +0000 Received: from mail-qv1-xf33.google.com ([2607:f8b0:4864:20::f33]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5m-00EhPy-LQ for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:25 +0000 Received: by mail-qv1-xf33.google.com with SMTP id dk4so3693789qvb.2 for ; Thu, 30 Sep 2021 07:31:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2byFwZTqL6H5T1HwZ3UMFR9jHqf1akMl4mBN/vDMX94=; b=CnuAcVa/z2KIcemUvmuCQG3R70S7y0YjOx0S55gdxctiytScgPrJDP6M1fLmRGPYXJ YAT3mIXZVlLiki2mw9xhu8NnySDq7DVByfEQjdj7oIecSzThsjjHYPVJQ4VGIOBh5cqz VlYyGJ6x1aJFFhC+csTa7NfnPJGU6R7oBZD8MmNrJ4z6rW5X+/OgtPXmPF9tCdlDw8iv 5gBfKad8IQFE/KHebxTVW7m95rJ16FTAVG6bAoPT9dRSVK92yIipf0bAMwhKETG99lhJ JBbcprcZPUyjLRS4jyYAX+2cwdviTgJKdrRmaXT6Z1jEDDgCDbBiPpKZPoxA4uQvLe6A kMFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2byFwZTqL6H5T1HwZ3UMFR9jHqf1akMl4mBN/vDMX94=; b=Q4wLGIUtHfPGaFuWl9tWoMkBtbhVfjJsdw+fANWeBbAyPUWuGrMDz10O+2f+Leq7Qq I8O0bLaXthiiACbPMHtRlPxKjOszf2zXKA/2LkMe4lOW2sOH//6+MgXLxUQNfnmPusbv 9dvmhEvm+o0mT7fP9rc1gzNmL62sU1stIAD/GnLeEp5fpij0d6ofmzfL4wdxOwg1L+fB kct+7zeyc04A6vSAvuyY6eqjqqCc3E3RPS+0Onid1DuF1OmbW6sHdDReIbzTOueCybb8 Unj0TIaZRf6yVPEgjFELQDTneqFbfWnUcKolrNkA0jz00MAtmrubYSitgtFugtok3q8s L4lw== X-Gm-Message-State: AOAM530Z0fJwX9SCFKMrTHD+NjSY1+INjiKmyWufoHNVFMzOFgC4zkVk qz/NK1X6KgCI/PKtM7+6/ZL0RA== X-Google-Smtp-Source: ABdhPJx2GA26RZN03kRYmBDCi/gcboAxmrhaWBYq1chXopjjFm1J8tvOWaWK45xUGQBMAOa3uCnDFA== X-Received: by 2002:ad4:45f0:: with SMTP id q16mr331461qvu.4.1633012281074; Thu, 30 Sep 2021 07:31:21 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:20 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 05/15] arm64: kexec: skip relocation code for inplace kexec Date: Thu, 30 Sep 2021 14:31:03 +0000 Message-Id: <20210930143113.1502553-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073122_754937_98664128 X-CRM114-Status: GOOD ( 18.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pasha Tatashin Suggested-by: James Morse Acked-by: Catalin Marinas --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b6d5a02cba2e..7f1cb5a2a463 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -144,16 +144,16 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, @@ -162,6 +162,8 @@ int machine_kexec_post_load(struct kimage *kimage) icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -188,19 +190,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Thu Sep 30 14:31:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94A09C433EF for ; Thu, 30 Sep 2021 14:37:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5B79B61269 for ; Thu, 30 Sep 2021 14:37:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5B79B61269 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kGNAKM0pCivALP3Hsm3ry+oQj0dccRkZOYQ06UuTuDQ=; b=yIG5Nl4uZWpIGW DkycSX7afr7cWFVFmK/FkONJl2G6GlkFU7rz1GLY+TOonzEyMT+MyGGC76AE88+gywtDsztTcUNTR 0ks5cWmQDWlNaM81Bkw/9r6zF70HldSoIP9QGBGtz2PrWm2NE9CSw0OF5jcIbK7f1k3r+2/lwrCt/ EHgD/nJDr1xnxZWkLkij3ywM5xLDlY5CQ0PoE+Ea4/dQDipohd7c4BNU54B6y4Dk8fasXAU06q7IJ m8piySu4XufgNRKm16OoglASz+4JkhyxF6djkcloA6pcMyfRzP7u33cNvp0osMDmTx3td0pvysEYR C5FEnEluuw1CDSCyRuBg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx8Y-00Eihw-FC; Thu, 30 Sep 2021 14:34:14 +0000 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5n-00EhQk-Jd for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:27 +0000 Received: by mail-qk1-x731.google.com with SMTP id i132so5999752qke.1 for ; Thu, 30 Sep 2021 07:31:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=UIkM47FXs5P7Z/Da3Mn+8DCMex1zjJveQjNKvVhGh4k=; b=HAsc0xpvY/J7ECsq2WRyYU/gr+1qYLJdXXgcW7LBuP6Ck7/Ze+a0P8y/kFcZ9s4sBG CwKWrt/wRlgYhSyE8RDbsVvBXDwWX3j3zpgu3zaWiQbEL57hwBFqnzhPBbq/ZXYEa9X6 OTV8mpwqGNCSOvXYIvWJjjiVqAb255gIiz0u656noA+ql4bIpfbuLR/FvM+oxIDWJYxC Ovp2KOoR3zDAdTJ8kp0W1Doa1VICEloK8cfOUhjbl9VsuR0lbeMoG8zNTtpiVfDxuEAD XQQHdoPSdene+TR0RbeUo3xCYIjqnjPoTuNAvD4reELA4S2ULq+Z1lbpIp9kkqH0jtVV hnXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UIkM47FXs5P7Z/Da3Mn+8DCMex1zjJveQjNKvVhGh4k=; b=uCqGXD3wLPqEsLc+sv4Dd04xVSCuGSG6ajS+kh/X03YekzrHNIoj8SG0AoET3fHoyt JwwBDNLb5CPyPnwG1nCA4coK1Jd7QOb2wb5mGiSOkbUPMTXsdoKOqpb1KtWK9VAU/qju pKRAWakNebjKGKdgRwy7G1A6o98LIGgwGe+ghfRaK93xnBdPj31Bq08la2i6ofxJbYIB 33Zuo2KeQsMHygaxnQ6YJhi7TXVckjPhN0LNM5KZPsAb/Y4253szXYPOvIJZ5buvoLSZ 9k0DGU6qcrG9rEXt0Pgdap6zeCNEWmQ6iPj76trq3yQ+iNPiURHZ1bjc5QH5I0sL0YvZ vRog== X-Gm-Message-State: AOAM5311e35F7kTYMqaoG7MFu6AApSxZMN7C8BduXe2FW5Jt9h1tp9X/ +1gCZJcc2dGTduUa1i63D5Rcmg== X-Google-Smtp-Source: ABdhPJz+yokIGfxLu3ulCxnz5lUdNKG81GcyHkZE4+G8gOjEmo4IXXWq+pXgLlUabAlOLSxioSB5Qg== X-Received: by 2002:a37:993:: with SMTP id 141mr4982102qkj.488.1633012281915; Thu, 30 Sep 2021 07:31:21 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:21 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 06/15] arm64: kexec: Use dcache ops macros instead of open-coding Date: Thu, 30 Sep 2021 14:31:04 +0000 Message-Id: <20210930143113.1502553-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073123_706121_31052AFE X-CRM114-Status: GOOD ( 13.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitized DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/assembler.h | 30 ++++++++++++++++++++++------- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 26 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index bfa58409a4d4..d5281f75a58d 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -405,19 +405,19 @@ alternative_endif /* * Macro to perform a data cache maintenance for the interval - * [start, end) + * [start, end) with dcache line size explicitly provided. * * op: operation passed to dc instruction * domain: domain used in dsb instruciton * start: starting virtual address of the region * end: end virtual address of the region + * linesz: dcache line size * fixup: optional label to branch to on user fault - * Corrupts: start, end, tmp1, tmp2 + * Corrupts: start, end, tmp */ - .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup - dcache_line_size \tmp1, \tmp2 - sub \tmp2, \tmp1, #1 - bic \start, \start, \tmp2 + .macro dcache_by_myline_op op, domain, start, end, linesz, tmp, fixup + sub \tmp, \linesz, #1 + bic \start, \start, \tmp .Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \start @@ -436,7 +436,7 @@ alternative_endif .endif .endif .endif - add \start, \start, \tmp1 + add \start, \start, \linesz cmp \start, \end b.lo .Ldcache_op\@ dsb \domain @@ -444,6 +444,22 @@ alternative_endif _cond_extable .Ldcache_op\@, \fixup .endm +/* + * Macro to perform a data cache maintenance for the interval + * [start, end) + * + * op: operation passed to dc instruction + * domain: domain used in dsb instruciton + * start: starting virtual address of the region + * end: end virtual address of the region + * fixup: optional label to branch to on user fault + * Corrupts: start, end, tmp1, tmp2 + */ + .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \start, \end, \tmp1, \tmp2, \fixup + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..8c43779e8cc6 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + add x1, x2, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Thu Sep 30 14:31:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F269C433F5 for ; Thu, 30 Sep 2021 14:38:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4DB9D61A07 for ; Thu, 30 Sep 2021 14:38:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 4DB9D61A07 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zcROrQW9PHOD7hTfffP+7NZcTZuZPVZ7Rckqmvi26Qo=; b=VoZ+XFD9ejS/SV hbLGvnvBc4PWxtidhK+yjKRxetrIu09lEtBotWlxLATvms1YMr1BOhpkNfK6qtDqRNGBembE1dTOf vdwEQ22/zNJkVi1Ygk2/whm9HTVLlmmoMH6Oid0iC2YHg+eAhs8EE7Q5+XX0uzXdjpCYT1w4OgeG2 PHA8d27Ozsc7FcuuSq2R95pZsiEv+XqKcD2qUPiHa0wmrDMtwLl4lZiaNQpHFEZ/WieJfSde+rXeb ZzI8zt3oKWyCi4ZwQu9EhPg2kaFPYQ/YjAq4EOpzv9G5XEIzYlpzcceVGgalKyuy/ByWWF7REFu7Y ucKNdoq+7f/sr+SdDivQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx9G-00Ej8U-Pw; Thu, 30 Sep 2021 14:35:00 +0000 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5o-00EhRa-Dj for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:28 +0000 Received: by mail-qt1-x831.google.com with SMTP id e16so5846866qts.4 for ; Thu, 30 Sep 2021 07:31:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5tAiOQNbsPwYQy1X9AyljFW/jKh8PdcAlWcG2o1mwWI=; b=XCzx76XQyqYzRm1ibea0WoCNtcgtXq1gejadijr7/Tsm8GnPNst3oyZj3R9I9AnbJG lrBnaIGN9DLqClY6Iu3a0cPk49EhlCdK/rDuxQQYANtGNfP8HFek80IgFaJdn1NGVLDJ 27ChNmLpSzUczm8ERB32kFy54GWlk57lRkLWF4zD1P65qWvy/Iad8ko3u57FmlRhAxP9 hho1onk8yY5gm6Iw+NSNJ1q5UqXxrZlGyOQYSWjs498DAm/VFk2pIkmf5qWtWOKGMdgF ygnW+Ww/+cSezwh8z/XfxMhB2qr8HyapKNi3SW/qFQ+y1Vy8tWorf86vcsOyv9ttqucp +12Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5tAiOQNbsPwYQy1X9AyljFW/jKh8PdcAlWcG2o1mwWI=; b=o77nTaO1wV+0uhFVDIQ28j8ojGrEIzXpoE0GxQrYfWXoitfyZNJE/LKpcMkg4h2sV+ M1o4CCWORGyf0f4OeuLFU7JTVgg7aBh+QYlblM62hw2x4DqepDwnQOk4fGNyE2aD9SGt QvyHrALPcdmYefbqww8nS5v4ahb32RzxAlrO7uxF3zj+u8yneqFEPJx9XxIjmD7C/S5s OKTbWvikIHEUOMtT6JS02kmXSlyg0h6XsPLamsUi+ikZ3Y5PLlXmxpCB8W7dsMQgOwON 74yGSb4LTG4SSAEJIsHzgcpulJum5dYnN+ogYVJzQxICmXZwBwwTmIC5VNGcNr+7R7yv lYcw== X-Gm-Message-State: AOAM533v1SG9R8+HvLQ9W01ooTmMA90DNYUv0V/cCxrcEZ2hcJgZij1F VBvhIWWr35rCPDEN711M4NeEvw== X-Google-Smtp-Source: ABdhPJy8KG5Co2knAgWKxrqUz+zTSFnivorQOkpsDEWq0y8x/CqeHR1bJLHAsbozgKtfArAnLhmX4Q== X-Received: by 2002:ac8:4e8f:: with SMTP id 15mr6748304qtp.189.1633012282856; Thu, 30 Sep 2021 07:31:22 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:22 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 07/15] arm64: kexec: pass kimage as the only argument to relocation function Date: Thu, 30 Sep 2021 14:31:05 +0000 Message-Id: <20210930143113.1502553-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073124_544330_7EC4CFBA X-CRM114-Status: GOOD ( 16.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 7 +++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 551427ae8cc5..1d3319c7518e 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -170,6 +171,12 @@ int main(void) DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); #endif BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 7f1cb5a2a463..e210b19592c6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -84,6 +84,9 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + dcache_clean_inval_poc((unsigned long)kimage, + (unsigned long)kimage + sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; unsigned long addr; @@ -206,8 +209,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8c43779e8cc6..63ea19868f63 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Sep 30 14:31:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14213C433F5 for ; Thu, 30 Sep 2021 14:37:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D056C610A2 for ; Thu, 30 Sep 2021 14:37:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org D056C610A2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EZBfoRgDvxW/V7bwS2BRiwXmzDaHgK8klg9vbt53b5c=; b=g71mjqGlvVBodt r7KerLVGvfbkfCV+gb3vYqx2GVj5YdRlmxTVFzQLdaPsEkQ6DwqjhIUmy3d1zH1PfZKKzLs7Eo6hr Jm2TkTZdunjJw+oRFHCz8N94LHoK7dUB8OPPTRBNpXSsvNWPFl97IKO1U/eCbP2W4hRevlRnTLLNF rg83uxJ3/K3goveuOzpghun6G9YFTZXOGVXsurKPZ88bKL6Hvroq/auxhMM1Cm19SwzjSWGDqczn9 tVZBGHFJQPzR+1+EMSHH9+r5yUDeDCl4fTT7K4MQ7rKXTN0XHcAIH1nd8Atgid5EOAieYcDIesEXO xshueZMsQltLeYlGItXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx9j-00EjNZ-UU; Thu, 30 Sep 2021 14:35:28 +0000 Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5o-00EhSJ-Pj for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:29 +0000 Received: by mail-qv1-xf2b.google.com with SMTP id v14so2326631qvl.9 for ; Thu, 30 Sep 2021 07:31:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WoiHQItxXcz2mVRE9xPdmbdljbzjyyQiR7Sfhvuof+4=; b=XkNv35QVeDXA+9ICJFiqcH3hry1Rj1XAj0QdnxsbRz6fwc9oUluTUbenVe9NBN3pnV CUtSC8fikhV2y56qoIbIQdAUA9zdrd5Nxa+TFhd+oYTNqu8c8kf0g+ku6eMcAWK/g19w GKkIGpvfDKObl1kWlsfqY6H6Si+vIw6XJTqX5Dep2sFYe+/Afg1UNtOAlbIuRexpz+QQ pTQo68B12adRqkcXim8b7BbrVZYUDQPxnDXJui5IcH58P0vuYghNKFrgnGhK4AXfsEyX 03B4BBSipxHXtkVuQI16JQ/KClI21eNiaZVTH04d5cDKAa2eEbm/SAqlRsZ3FbyKuoTh ElDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WoiHQItxXcz2mVRE9xPdmbdljbzjyyQiR7Sfhvuof+4=; b=Nsg90zLO9lvMOGusC4XLe+nikasjLAY8+594Xy4c1ApJv3fw/cSz9MACZO0Zr83olL 3JNQPoMa21aAFJikh2wxkFFZytIezXOvWtR9prqFr2/MmP/FNygRiGQZ9MMKS7zyY+3/ /Ac+8cB9fQ55NPrfse+MSh9tA9B7AObTAPsU2q4GT4fnAhovOEqoNEGzKEk1/h9IgUEa dF7WESllDTnfqX9N5Zccc0CEfWab0r3Z2d1ZH+eu6YX5HTBLHOPZjHzVHjo2FuGDGJzi DxrHTggiAsYSY+DmDEVLMaEYqz+5V3iGleuri5Gc8r4yoJX83l7+jz90A4Tj1HpdgEDP Fl3Q== X-Gm-Message-State: AOAM530VQMLUzoAJY0baSKYMyTSIE4sfLEVNp6x2nGVFdwdrfUeTy/fu I+Axn0Ipti11VGOAjb71UoGTnA== X-Google-Smtp-Source: ABdhPJycCwt+koTjzhe5LqBKA+zcsOrf0LEcpfr3AGB09z+yDDgnPiU9k7YZduVSbOerYBZTV4y+2Q== X-Received: by 2002:ad4:4989:: with SMTP id t9mr4126913qvx.29.1633012283934; Thu, 30 Sep 2021 07:31:23 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:23 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 08/15] arm64: kexec: configure EL2 vectors for kexec Date: Thu, 30 Sep 2021 14:31:06 +0000 Message-Id: <20210930143113.1502553-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073124_923372_ECD6532D X-CRM114-Status: GOOD ( 19.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preparation to MMU enabled relocation, configure our EL2 table now. Kexec uses #HVC_SOFT_RESTART to branch to the new world, so extend el1_sync vector that is provided by trans_pgd_copy_el2_vectors() to support this case. Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd-asm.S | 9 ++++++++- 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 5c7ae4c3954b..552a057b40af 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1135,7 +1135,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 00dbcc71aeb2..753a1c398898 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 1d3319c7518e..6a2b8b1a4872 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -174,6 +174,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index e210b19592c6..59a4b4172b68 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -43,7 +44,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -143,9 +146,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -154,6 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_nvhe()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -209,6 +238,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_nvhe()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S index 8c4bffe3089d..021c31573bcb 100644 --- a/arch/arm64/mm/trans_pgd-asm.S +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -24,7 +24,14 @@ SYM_CODE_START_LOCAL(el1_sync) msr vbar_el2, x1 mov x0, xzr eret -1: /* Unexpected argument, set an error */ +1: cmp x0, #HVC_SOFT_RESTART /* Called from kexec */ + b.ne 2f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 +2: /* Unexpected argument, set an error */ mov_q x0, HVC_STUB_ERR eret SYM_CODE_END(el1_sync) From patchwork Thu Sep 30 14:31:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B75ECC433F5 for ; Thu, 30 Sep 2021 14:38:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8869D61A05 for ; Thu, 30 Sep 2021 14:38:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8869D61A05 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kVwNPEafOjAZSstzWnf95oE/p2o4Oedq/O9C42JtM2M=; b=V7sRwfo6gE4iuA JaNfmAUaAJzy0pXlaeRL8OcKW6kHU1gKD4fBAmpAdL8W1F1mgYMFCVz2Clp6nN2naTISsdlRwGY24 fmLJG/gyqlkWS9R2DHVcpzNU6DlGZp/wGOnDa/xZu7mDvhVtZzDgU3cl0JAFBsmyPMDZeKARbI1dp t2/FvLLaOvuHECwnOvvWLfTBX0nE91WzToQUQk6nUhLI7BdaFLUdTAwvsQ1DwL8cdEQ66Q+KogIQ5 q5JrSyw+hZd1xgqQjHNjJ5AFoZlMsVbJEDeQrtiS0x1qo8XvetWj5mnH3YcvzpgTx9TZP2ToItFcf 5SzuuJz2wzu7Ga0vnzdw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxAC-00Ejaz-0m; Thu, 30 Sep 2021 14:35:56 +0000 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5q-00EhTX-Fe for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:30 +0000 Received: by mail-qk1-x736.google.com with SMTP id b65so5922759qkc.13 for ; Thu, 30 Sep 2021 07:31:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8cW/IVlTGR5Obb7nU8rBJjg2uf8tpcGiOc+9iw339z8=; b=LGZ+V3ieH2k3Ak86m2WgJk69Yb4/UpPLxocU8YOxOu6VywnSRkV5tG6QX1CZAFVkjH IdxIVertagbOxiCikHfXeI+gyRxyIK4n4WNp1gD2dM3hVmXHeWY/0MZVIVU7p16dt0EE kQyfCpW5wHlohwk4aa6PjA7uPCBnCZ3aqzRo52eBJjEIs/y+7yHsylM6P6A0lmWxLvdl W4ughsB3UmWmnmLbk1MTeuaS/nugc+CYzDd6T7qr/a1d7xhATy3542wZgxbkBlmDuy4P Vz0qwo0+V3u7WtKUnsS7vgeAv3YUSK4CE8SJAitCqTQ88TYgfnftSHQblwWyCxVgwAFj 5hNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8cW/IVlTGR5Obb7nU8rBJjg2uf8tpcGiOc+9iw339z8=; b=eZvMIh8AC3+vqVhZL/7LrB31FLrVBHsPGH+iN2Pa4XUHTJaIlQUhyeDbYZkV931/fD r4SbjtvBZhrW59PL20kCrE3x9xl6OkgPpuS7K8Mhd4yonllZKSZ41kZ4G3yiMCHW3OlO 06ob+VyYJyCZUjaXQMF2fGErMS2St0pH8No+15VgwVj51fidWmX7Vp5ZR55MgYRKJTNb YapaXPAZOdLHPSlEy3lTpzBBx+mB2V72SGTU6T84euGCqXa0gd6FvVgo/BLOcUnndjvt x3DCQ1txolOOfX+cbIzCk1pUaWr7yjhavgQ8wW+OgO+6Y/2zOYLACNtSyyxE+4k3X3sO OHWw== X-Gm-Message-State: AOAM5317Ftl4s1DEiSckZRGDUWIPkas13f3nTRJCZzJXvaVP/XMeUwO7 fv2c9QMcpoLCQ/8dMx0OOj/Iig== X-Google-Smtp-Source: ABdhPJyVmacE1sVXxqp5kGQ3RLtcUFdTcFqDSjNppAD3P445XkdfyacytvCYjB45IVcqvbFv/tXoJQ== X-Received: by 2002:a37:9e8d:: with SMTP id h135mr5202679qke.189.1633012284930; Thu, 30 Sep 2021 07:31:24 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:24 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 09/15] arm64: kexec: relocate in EL1 mode Date: Thu, 30 Sep 2021 14:31:07 +0000 Message-Id: <20210930143113.1502553-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073126_565811_FDC80292 X-CRM114-Status: GOOD ( 13.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before entering the new world. Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 81b3d0fe7a63..296abbac7192 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 59a4b4172b68..cf5d6f22a041 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -240,8 +240,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 63ea19868f63..b4fb97312a80 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Sep 30 14:31:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD7AEC433EF for ; Thu, 30 Sep 2021 14:39:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FBB661A05 for ; Thu, 30 Sep 2021 14:39:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 8FBB661A05 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=htK3jwV4R2xE8kaFBWQhz0rhBGrqeJT53mycQ53mJ1M=; b=QcGiyo9rlnvw2p 5EzB1cI/Men7VBnPnZ/FDxQA4+hKl881A120ilt++wn+B+nplfwSGglu68NakgRwLy1ImqSDZhU4r FE9gX7jbHV3jEpH+9v8LKgTM5E4TZt0SIHl1NBhBqgxgS+Dh5/iDrLo1PD8eGFucfTkgpGb5KwCSd 9sfR3OadkR51wCVemdj03zTz2Uc92U8RVDRRv4h5gOjfUUSBo2WkY379nNTqNKhOpgpOUAScUHLSp mEr0syt3bU2bGPe42YPlp7M6tK7bdq/wWYiAywMNV/yJvD6bq4DQjdqyqG3AqH27h77x5qHOmFw+d BiCT497aN9lBQGfDPsaA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxAm-00Ejrz-SN; Thu, 30 Sep 2021 14:36:33 +0000 Received: from mail-qk1-x733.google.com ([2607:f8b0:4864:20::733]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5r-00EhUD-8k for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:31 +0000 Received: by mail-qk1-x733.google.com with SMTP id q81so5986551qke.5 for ; Thu, 30 Sep 2021 07:31:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=a4UWBG19ndOi78HFaDxtpOvDb/2uQd3EH24/B6Fqa/w=; b=Ds4z7LD5Z+Mi2pRwCBj8ySV7y8sqlAsz82jDxaarB1CA8ARQGfNTDZKfqF8Sw6E4Wh nq/ud6gBw+F5e/Y2OW/7cL/d/DDweMQR48tmkTJJY5qFyYVoBFrpCziwIyh1lASxIO5x ed4MDzfcQ1jkwRMQPZHI8rnx9767iPUYN830/oSF4GY1gF10sLA5M4FSVxSNqSVAh4Vy 2ItlwMPGrCVYIJfqXFG934ycmp8xkFGhONWCDFfKWRVf2ybRcTQpWx2px7piZpzRGj1A c7CbY+TLQanIe+9leu1vf1U2p22xlSETEb/ACrt8zfvpRqXRIt9cbG/H1bR2Ht39X65y 8Uow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=a4UWBG19ndOi78HFaDxtpOvDb/2uQd3EH24/B6Fqa/w=; b=U/bpJqRAg4TWy6fdgmXpkU2EPWw1Lq2h2LbJFSmhVOYnQVZO8kyEAUa+jMgpJAkaDC e0Akdr6s2myNQ7VAw3072HdQoDO2GipodQWVeR1QJpoiuxsyphUqiM983IN8G3rDxw1V B/dx9UQQNfkszlgvNozbY1tZZm8/axnDVSsVRIxeqdzosMnfDuDKyxElSXAIrvEB/Yql +mUc76f2FLHvSsj7QgOYVR4mg3ddNez0fvUrxYtho2YbLzJUo7+LtkbLFfbGoZGmzuSR H2OsT3UtpcW6MIRlhA+1i/0mG8qloZw6x9KRQqF0IQ8mFRcaR7NB99oVXsIDTbWBL0ub IYCg== X-Gm-Message-State: AOAM533LxUZR2bryRLMyF0olOR81+PQ/PHJgU2Cm19wi9uSuIWkbGB8J gj8cme0YjNIEgOCOYsiGzfX87CCDG59SAU4z X-Google-Smtp-Source: ABdhPJyqI9O+dVJs+vntZTBGZ9pxzbqB0UkHVZ/IqqI1agRY7/nbveVWZblYlDFFkKLclptvtLx9aQ== X-Received: by 2002:a37:9144:: with SMTP id t65mr5175741qkd.170.1633012285793; Thu, 30 Sep 2021 07:31:25 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:25 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 10/15] arm64: kexec: use ld script for relocation function Date: Thu, 30 Sep 2021 14:31:08 +0000 Message-Id: <20210930143113.1502553-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073127_393000_61052A9A X-CRM114-Status: GOOD ( 14.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script, and put relocation function in its own section. Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 16 ++++++---------- arch/arm64/kernel/relocate_kernel.S | 14 +------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 27 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index e4ad9db53af1..152cb35bf9df 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,5 +21,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index cf5d6f22a041..320442d35811 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,14 +21,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -163,6 +160,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -183,17 +181,15 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, - (unsigned long)reloc_code + - arm64_relocate_new_kernel_size); + (unsigned long)reloc_code + reloc_size); icache_inval_pou((uintptr_t)reloc_code, - (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + (uintptr_t)reloc_code + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b4fb97312a80..2227741b96fa 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.section ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,3 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index f6b1a88245db..0760331af85c 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -160,6 +171,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -348,3 +360,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is broken") +#endif From patchwork Thu Sep 30 14:31:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60ACEC433F5 for ; Thu, 30 Sep 2021 14:40:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2AB3861A08 for ; Thu, 30 Sep 2021 14:40:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2AB3861A08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=rlXfCx/77pp9JKm8ind5LBkFpQbt1KrrDDUT09QrXl4=; b=wVlCEOAsAQPmnt WZf2UU22jeEo6dO7qDUxmhA7BVYbF18K152QSCkBOXPxAUq73xhzMcZh1LghSbiRf0HmxxfYZ/lWL fQkm8/YWckZbMwSqTuz3Kw1Xb0EEMuAA/s2OpocvPSNxpINi+MvhLZ5GHKv0wl7whpJe1+mxYpglA SQTZGLFAScySDYOv2n2rKYrAOAd8crr6VpviYuCOz+Z+eGU1zQOJqxz4GvE0ETwxJF8GhT35+inu/ 8eFNDF7MqLesYApPSAaeRtgz6cFZVvAJA540MwgHp+y/OG2xakaRiDetaHIH9ceVixYzvnBhUqZ2S RkJ6mr5aUyDPDB7lo7lg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxBZ-00EkF3-M2; Thu, 30 Sep 2021 14:37:22 +0000 Received: from mail-qk1-x72e.google.com ([2607:f8b0:4864:20::72e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5s-00EhVP-5E for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:33 +0000 Received: by mail-qk1-x72e.google.com with SMTP id d207so6030023qkg.0 for ; Thu, 30 Sep 2021 07:31:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uM7nePTc3Pr5alLDIhJT26d9ASl4ItsEhUSF3SoRlOw=; b=NwQN2f451Un2tZ6PYU+nIoolI85dWRHZ/cijulDx9n/s9hwGzuXkW3IZ70abRtcZr1 UihJG5WIsBZze7Uhc9U6n8ZhFUVV6sjgMxUHqKkALpbBH1aLmHdxIqHLZotMPYuJMUG5 dtROBU6G+v4Hc99a9FIbqRwzucEMLrSQUt4fG/SlKhwvEU0CMmXsDVfXg4sG/MTZVYZw 3VfpxwF+KE06wtbGousa78FexB4Ux4sRoD/m79nmaRw9GPDPwN2Qah0Ucj9D5Wit2dNh pE1NViWkWEz1soNCMqDHhOse8+PSySe2+kn6mCe+F1oMlNpsMhFX0TDirDbZNOpVKxBo mhYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uM7nePTc3Pr5alLDIhJT26d9ASl4ItsEhUSF3SoRlOw=; b=cNuNylDbrvucyk6P6ALsGDRK/6kHa7seclNiCHI/KM3OIrNN3/ihy8Gi6yGXtS/H77 CG3G78wPjiigJwQV36hS3IhTjmqSWFQAU/88kVXJyI6xo/u34kfB3W3ZUbW0bcIl97aF yQPcicc/7uPdQLOJ0ZtBnBJ3PoILzU7MpF4Y/5+/j3ElaNx870MJSe80oKJkAsCcAhEy 9IuzDDg2B3bAYKKIrOx77k4lOrc8fjmQLvLS75EM9l6ehUJKvve86pwsVrUS7Mi01pA3 lfM9D6VUXPsmy8z8pRxjms3WDGbyzfm3M/FRhfnenwxUjQ8s5MYO02S0s/sZtiAQgQ4N aHSQ== X-Gm-Message-State: AOAM5313kLSdsfdEMq40CUqfUBnNJ1Hv9ymzBD2NwhuwVYBGA3kLcTPE HNFRtFIdLhuxBs2Bm6lKbRKvUA== X-Google-Smtp-Source: ABdhPJyqaxYjUoVqqI6Z2WRqe7YV7XGmKu6XQqMsK8o/rIRI06qkcwlSoFfm5Mh/Y4OegOULvC+4kQ== X-Received: by 2002:ae9:eb58:: with SMTP id b85mr4879543qkg.323.1633012287361; Thu, 30 Sep 2021 07:31:27 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:26 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 11/15] arm64: kexec: install a copy of the linear-map Date: Thu, 30 Sep 2021 14:31:09 +0000 Message-Id: <20210930143113.1502553-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073128_271662_260BEE2F X-CRM114-Status: GOOD ( 20.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocation with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index d5281f75a58d..5da176dd37a9 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -483,6 +483,25 @@ alternative_endif _cond_extable .Licache_op\@, \fixup .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 753a1c398898..d678f0ceb7ee 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 6a2b8b1a4872..1f565224dafd 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -175,6 +175,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index a30a2c3f905e..0e1d9c3c6a93 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 320442d35811..fbff545565f1 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -159,6 +159,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -175,12 +177,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_nvhe()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 2227741b96fa..2b80232246f7 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Thu Sep 30 14:31:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA0C4C433EF for ; Thu, 30 Sep 2021 14:40:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A34A861A08 for ; Thu, 30 Sep 2021 14:40:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A34A861A08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=okPjHJAhTFpbUT/dXgFrhGSV/DgX8ufG7RAeQ9yp4q8=; b=VrPY2C7e8AnI1M RhrntRX9HhCFATMRQs/+uDedATtryTUsBXtQOLyAlxRtH+3HGup5nZcAypvEoHKwhYwCTZyBiXwRB IZt/X9hXs3Vj5sCYGQ49tdc1JMiUp/pORUpXAEwVrojS28Wr+dTQ1y3xHLcvXDUYL0OC4If5XCBWL fJD1YJc6+Gw5rDVRi4uvxRMHwX2fEFSvA8bMx90OyRSCRR7H9uQz3E0hystrSxTDltFMtZg20aiF/ i+9mBzwNhwNNI/hcIfb4O33uRHTBgOCjVHNz9wxAUw+4J2JW3x4AohvP0qR0/e0ZE7XWXPgoOUqBq mpVse3Nvp+6yFueM8HPw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxCk-00Ekmk-Ls; Thu, 30 Sep 2021 14:38:36 +0000 Received: from mail-qt1-x833.google.com ([2607:f8b0:4864:20::833]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5t-00EhWC-TA for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:35 +0000 Received: by mail-qt1-x833.google.com with SMTP id t2so5818580qtx.8 for ; Thu, 30 Sep 2021 07:31:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hiDX8HDwZgFFsas4OkEqmNALnkk9CtONfrYzDPONMsg=; b=gbFRTimS7Mkao9xkEPYbRHZaSmf78Aj8pSH6G1/KFohe9hMG6P20jSLuzhJDqeF/Bo tOa6kt9PbJe2vDBc1ovjbpdsqm1TPyx3FgReKagqq4bG9MhZV8T6vLvX0sd/uhkoutJL I9pB7aMGLq2Cua3b0/e+Xf79ceuAeJpM/o8h0KJUdEtVN2ujzbjuExv12XVoKsZqexrD qpXTF95mNaT6dsxe31Y9MmwQDZIVaYs6cCbmfPNU/o8Z9FI42qFo3GcidOPlNqTY8YbM CQmGE1huR7tHEKQ2FAp3ufKRAX2aocQNPrTBnUMWj8KoVckqjmEQiiGSAwjUz+a5g0lG ki9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hiDX8HDwZgFFsas4OkEqmNALnkk9CtONfrYzDPONMsg=; b=JUJtGwmi7RcLOSzCiJxHP8mOu945C37oFtqmczLE0PMLcpZEwq9wY7VgHL9hGY3h7X 9A2a4aoLTCx7hqpgFNXwmorkVydiH7k92mBkYGFahWpJhJGiAVBouoqOk4yT4qsa9Cy1 JMNUH7XbCVAFO3Aii2G8AkDHPN/roEugkmQ1oqi9lyFKnGzOqJ7xWue9f2gzoA2LLiSd 7f5JJ9/uHxXOxLik0+Qsoc75WzEhTuIy+DGbFRvcyWTWTpkEeceiwy2Na/x4HhvNk7XO OAtLwqX4b9o48rpnRiwuHEjUQqmQ30l5b4gb0RGS9NBHaqDJYWwYtkrtNW89QL4ERt8Z cl9w== X-Gm-Message-State: AOAM5304EO7coKtbz/nshetJJ6FNcc7oaYacTzFld4NGJ3FWVEt0QiYZ 4fNxqLpR6jPzufYlyvmVoSrmUg== X-Google-Smtp-Source: ABdhPJyfgNR+pR4HEDq+pM1gsVASNpdjnBQkyGm2HuLMBJBmfbiE+cFDJ7JjKeGzrdgbRSvxiuKivA== X-Received: by 2002:ac8:4c8d:: with SMTP id j13mr6612858qtv.155.1633012288410; Thu, 30 Sep 2021 07:31:28 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:27 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 12/15] arm64: kexec: keep MMU enabled during kexec relocation Date: Thu, 30 Sep 2021 14:31:10 +0000 Message-Id: <20210930143113.1502553-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073130_017675_7D63FBF3 X-CRM114-Status: GOOD ( 18.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pasha Tatashin Tested-by: Pingfan Liu Acked-by: Catalin Marinas --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 +++++++++++---- arch/arm64/kernel/relocate_kernel.S | 31 +++++++++++++++++++---------- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d678f0ceb7ee..dca6dedc3b25 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 1f565224dafd..2124357c2075 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -176,6 +176,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fbff545565f1..1e9a2a45e016 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -196,6 +196,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, @@ -230,9 +235,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -246,10 +251,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 2b80232246f7..f0a3df9e18a3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2021, Microsoft Corporation. + * Pasha Tatashin */ #include @@ -15,6 +17,13 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .section ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +41,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x1, x2, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + add x1, x19, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +72,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Thu Sep 30 14:31:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7291C433EF for ; Thu, 30 Sep 2021 14:42:56 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7FF1A61A0A for ; Thu, 30 Sep 2021 14:42:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7FF1A61A0A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YfA6s5WHeBEKjlQiuOb7RHoKy4Rt9yMFV4C/YqUtM/c=; b=sMHgIo68FAvYcs +By2QzTmevh/2VniFvVkdG3Tlftqzl19xl+tMl24LmwXS1rH0FzcJZkgUI706hFmMB+xJY2IMwAaD 7TVuOby0n7MkEn6dnX2ugDkXGRSfqSru1gPLbU60NXwnBNOceNcScyayCIwDcsYYPpoLUbiDk+Swy OyQRer/SBTte2QLKVe0WV9QTO7dB8ACJ7RE9vQkWzXVIectuAY9VL4xAlwx371Srh8MrU9yy5MpwP 6z1q09g6PQKR85OYNtBiJsQi4LFOqTWH9lwPKxMkkhIYyOEsmOTRCkQAM86uDuiDQjPBZnJVeQFam I6JWkBTv9meiF7J/rTeA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxE5-00ElPl-Qs; Thu, 30 Sep 2021 14:39:59 +0000 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5u-00EhWb-Vq for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:37 +0000 Received: by mail-qv1-xf30.google.com with SMTP id a14so3684195qvb.6 for ; Thu, 30 Sep 2021 07:31:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5Swv6lplDzjvDpp+CqlYUWQ4CihY4pr+PdPAAcX6X0k=; b=QIl+s6AgZb/E9V+qFkaJv28iDbmNKtriCkBKOedA+vTjWGPQUHwOWsSN82vUhYFDPa aKGX8pbht4XajcOxe9AsUPnPHyFeMsy53kNmxOp3NUiw7gCTOFbJHD2xFmkobfsYc6tc +a3sdvCpBBLPEhOdR01yAiMVZgRcqbM5QUcmBf0qkoXY2cJCdwNF4UEA2JfG0IDurSJL wCqpUukwDUB5zPuu1BM5g/ZOGHHARNjAM79AbZES71bLfS3MSsmn1x/xQsLYN7rTb6s9 X7JRzwlBTy1hpr+G/1SwuZkqioteecdFDqEFHGew47hr8gs54RBmUJ+WmmZ3E1UZGl1E MNFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5Swv6lplDzjvDpp+CqlYUWQ4CihY4pr+PdPAAcX6X0k=; b=SU4XQEHgObEYphqBbpMa6PQP19k7YCr8ezUrd+r29UlZNch44DaG6CDi/miTGGlhuU jjLv8IEewRtAcXkfE95P/TL6grGrwXxww40MGOALfov94GTdxh1mGs4AYnIN7wCv2/ul 4iaVon33OGHSMqiMi859d1zjpyeqCx7/3LLueq9GmRs8Gx271DPGfPrR0sEy1eeHH8Ss e8vUqK5UQEJD1qpaQCdqWvta1xDdxzkc5WV8I3kwEeCsCN1fjrpo1+eMmO8dG3Brb30+ OCYb5pTHyeZy4j8i8qg8lGL9dYSVwsv9RtSGG5DXMQGbkISR1FYOi9YRccUBJ61jnJKY Y0fg== X-Gm-Message-State: AOAM531I895WXzHXsg8krZe0E1J3VQwdOzuLgxCBs2c0Mge9y3NPcOS9 yRGGRpIczdiFTLrbxPeGFU1RTg== X-Google-Smtp-Source: ABdhPJxh6s/187JmlvOIcH5ROC+cTxyh2kXDCoA5lkqv5O44XaWCE6Q0LOi1e6FSLI9Y930/MBRD8A== X-Received: by 2002:a05:6214:1427:: with SMTP id o7mr5627954qvx.45.1633012289388; Thu, 30 Sep 2021 07:31:29 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:28 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 13/15] arm64: kexec: remove the pre-kexec PoC maintenance Date: Thu, 30 Sep 2021 14:31:11 +0000 Message-Id: <20210930143113.1502553-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073131_066396_F13035A1 X-CRM114-Status: GOOD ( 11.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/kernel/machine_kexec.c | 43 ------------------------------- 1 file changed, 43 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 1e9a2a45e016..559d47a3c59c 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -77,48 +77,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - dcache_clean_inval_poc((unsigned long)kimage, - (unsigned long)kimage + sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - unsigned long addr; - - /* flush the list entries. */ - dcache_clean_inval_poc((unsigned long)entry, - (unsigned long)entry + - sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = (unsigned long)phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - dcache_clean_inval_poc(addr, addr + PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -207,7 +165,6 @@ int machine_kexec_post_load(struct kimage *kimage) (unsigned long)reloc_code + reloc_size); icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Thu Sep 30 14:31:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3B37C433EF for ; Thu, 30 Sep 2021 14:43:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B28E561A0D for ; Thu, 30 Sep 2021 14:43:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B28E561A0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HHqLie2xPRX8f2WpQV1Y0Ny3yFQpBiz81EHfL/PItmU=; b=FGJGaWl9wi+Sc7 N/sy1rExqqzK7RD5VmINnvItaSV/etN3scbOYyaLCsf69V2ovSUf2kdXy7JX5gxIHMq99BC5md17d xr843LJOd2RSxpaFsQEx06zBjPJ0250KtUSD2i3Sx8zhriDm/GBFkQKcqgL5B6uIg7DV9rSRwLu/W xlo9SV4NSruNu3WAHrefJlTbWbSxYQLQ764VujvtavmHd6zaxs/5mOwgowSVY6GtNg6S1Sx48TY2s 6K3Q7PfF0iOSSoa5Yx0wqIGSwvXiPZqVJGiRtTNNJK99S491f9DqGH89tzUrM+1oLw9ji1miiIO2f VXXtVtezWuhyJMWVe5OA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxF3-00Eloz-C6; Thu, 30 Sep 2021 14:40:59 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5v-00EhXk-U1 for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:37 +0000 Received: by mail-qk1-x730.google.com with SMTP id i132so6000240qke.1 for ; Thu, 30 Sep 2021 07:31:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+AOkYJOnQvYKR/38NvELzatiZDcWOHgVXKELSMlb5SM=; b=J5UQ7UZyfmWWA8PGgeNyC7jbgKspoR4nC8w8x68ivVY8HpNv+1af8uvYuEZzW7fTQT JW0pPONsRs0dngUFtIDqKg3T4bcpIwi6oVM+aCnbkzDE5Di4J7g/yTcWL7Gu2lQbh+Zm qg+s5ZIK4LCaICSgJO9NMMahZfisKm+U5Tzzbo2ep7S11va1mhIojJHfvKTJq4OocfLr XB80LgfKaKVLrEsgV+kcLXqnC35HNyCkz/OrUln0LzZBCP4N+1sG2ROW9TSujy+hUpJ5 GVJv9mLRsSKM2ypF2prF1I1gIh9AnkvoN8SFFYqEfooatv7xr4Pv+NuWlrcZ5U4ihEAw A5OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+AOkYJOnQvYKR/38NvELzatiZDcWOHgVXKELSMlb5SM=; b=mus7B1bJ6GPPgQMwSrX8YPEdtg4VqIklBA7aa5P3+4yCv+HQpGsmvjg4il5aMCtYPo k/4XOt98kGBgRxPBmx1YeTOlBRGBUKE9NSFpkrhqDlTsR1uWByPrx1zhZcsrbZzK+IeY 3n+VvuofwaI90Y4vCp23mDkS/89DjEulKKmI0FNdb6o4FbVAqx0EzSxixryluA7YMH8/ wpWLTFbliUUNnrs2CD32adfdCvkKFOEPsZRxE9uGL+DCilwUAOjUepcDTlzLAoCmB7O0 K9Z+OQ0phhN8j+KKnGnu7WE1W6ckWG1RONKAolD3mPRZ6jllTK4yRiU/5YRKIl6ASZGn Nqzw== X-Gm-Message-State: AOAM530e5ifpnIBxaQepZZsf7gM1P2i+MOYNUUOdp3eoxRU8OMZBi7Fz 30DI4BSt3mSqP0AqAgEyKuI5nw== X-Google-Smtp-Source: ABdhPJwcDK47/5GmWQlsLpv+of/CmJ8uJnwVzFmIbGlzIotuh9aCabUvIq/XQKbO0rOygE1OJAfqqg== X-Received: by 2002:a37:8883:: with SMTP id k125mr4898086qkd.458.1633012290355; Thu, 30 Sep 2021 07:31:30 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:29 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 14/15] arm64: kexec: remove cpu-reset.h Date: Thu, 30 Sep 2021 14:31:12 +0000 Message-Id: <20210930143113.1502553-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073131_998041_83D44C19 X-CRM114-Status: GOOD ( 15.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index dca6dedc3b25..9839bfc163d7 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index d47ff63a5b66..48a8af97faa9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) mov_q x12, INIT_SCTLR_EL1_MMU_OFF pre_disable_mmu_workaround /* @@ -48,6 +47,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index 296abbac7192..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 559d47a3c59c..1038494135c8 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -24,8 +24,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -201,10 +199,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart = (void *)__pa_symbol(function_nocfi(cpu_soft_restart)); restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Thu Sep 30 14:31:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12528591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00D54C433F5 for ; Thu, 30 Sep 2021 14:44:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C494A61A0D for ; Thu, 30 Sep 2021 14:44:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org C494A61A0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UGIrrDuaTmvwREm/s26wE33GNDsL/BpnUgOLUnqMOEo=; b=u3g5/NfJAbE11i 9a2FBi85NPtEmmiMMo1pUrP1emCRz59a1qmPX3x6Q6BrpbROGxVYTOoRheIhXOhON9GhkFqSf17hl NW084jXj0E+2CLwI6KARQTCAI+skh9JfdVSGPMzzZKDXCtyusVb7yDsUC3qG3tkm7i2ACJtVWCmDQ 8XoDDC6WXFCum80XvOuXA+NWpoi9NwGHIKEtDL2ntE97IPchppy90OVsTS8QkTDs8jMQnpNDEtKUk NU9uzRuM9sTkFqjyjGr6vF4LsKZnjvwEtK7Du1VM0Dxd37Lx4kUu3U87lcY8Zo4fyy1ohW4jfgNb8 FPXFTEK/hFURGFXqADoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVxG8-00EmIP-N0; Thu, 30 Sep 2021 14:42:06 +0000 Received: from mail-qt1-x829.google.com ([2607:f8b0:4864:20::829]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mVx5x-00EhYa-7W for linux-arm-kernel@lists.infradead.org; Thu, 30 Sep 2021 14:31:39 +0000 Received: by mail-qt1-x829.google.com with SMTP id e16so5847389qts.4 for ; Thu, 30 Sep 2021 07:31:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XxlQwO+zvTPvOSLEI+q8JmpmwLvHiOqSq9H/D1w+4mk=; b=aOdtcLsSMWhdmH/KKMmZnRau19vAPajS02V43ANM6ifhO/tnr/ulMkrIJTQmX7vAlq YADiIG39r9Vb8SZLAASMRqAQkCynKyOg0JkQwWMSIFTPKBulzt9tXQmn3UQ4nHZ3Vec4 voVkB4YK0EgaS0YEkK4o633fgSi4CFFC1DAx3UaBZ+BZVZx8Bh8Rv4FppGd+Ii7Uv5Ii gccTAgM1+8aZq5FXtWSip+4iu1L56aFhwQS1FUhvvzUT1IpJKlNrC0qCJuQm/EWJhkSX yeK0+vcszokE9LKwJMyqxq71n5JxLS0hdUyuJ15uNVFeMWMaLUwrC8M2NJ6UH9kOWleh WTtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XxlQwO+zvTPvOSLEI+q8JmpmwLvHiOqSq9H/D1w+4mk=; b=KlO2Fj1ogBq4TobJHe0P5WTIwzsVZ7hwuXaOIxmx2YNa+UwOnY8lNYuq6TwpnUJGbV 6ok955Ngx0Y6eYaFJKaPn/nWDG5vArNJYIWJbMIOuU8wsv0J+4YZT1ro5gPFTvPaRump k4ne8rBbYvohfv5ZvUdMOqS/oByZ9qu/axiJbbYWpRl2OsbtwryTMmDreXt+TZg8Y6U1 G+YBC6iBKhVuTijJiJizqxx7PgjShsSWkGrC29egqXD4eouf+soya6l5aWhNyKFS8Nel nQDT20FTGeliBnyHTR9bgsqmp84OU0PacbUYrHw0YjwsFmiQyFB9lYjpEGR99hoUvOIH eEUA== X-Gm-Message-State: AOAM533EBE1YdlRsCOZN/xf7z2oRPpc8u2FMFpIRUfaFcsBU3STHZWHS vRAH4pqYcPzJwA2BMWN0DXTfZw== X-Google-Smtp-Source: ABdhPJzecZm5xCqX2NKvPHbAfDt/pCmj5N9m76pk7Eb3EnDGgwSGvIf0RbDVFg/Bu7rhjM9gly/kXw== X-Received: by 2002:ac8:138b:: with SMTP id h11mr6860629qtj.163.1633012291650; Thu, 30 Sep 2021 07:31:31 -0700 (PDT) Received: from soleen.c.googlers.com.com (189.216.85.34.bc.googleusercontent.com. [34.85.216.189]) by smtp.gmail.com with ESMTPSA id l195sm1528731qke.98.2021.09.30.07.31.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 30 Sep 2021 07:31:30 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, kernelfans@gmail.com, akpm@linux-foundation.org Subject: [PATCH v18 15/15] arm64: trans_pgd: remove trans_pgd_map_page() Date: Thu, 30 Sep 2021 14:31:13 +0000 Message-Id: <20210930143113.1502553-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.33.0.800.g4c38ced690-goog In-Reply-To: <20210930143113.1502553-1-pasha.tatashin@soleen.com> References: <20210930143113.1502553-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210930_073133_320010_157EFF2E X-CRM114-Status: GOOD ( 13.56 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The intend of trans_pgd_map_page() was to map contiguous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contiguous range this function is not needed Suggested-by: Pingfan Liu Signed-off-by: Pasha Tatashin Acked-by: Catalin Marinas --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 7b04d32b102c..033d400a4ea4 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 26bd8f2d95af..d7da8ca40d2e 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a