From patchwork Wed Jun 9 00:44:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4518C4743D for ; Wed, 9 Jun 2021 00:53:59 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 89A866124B for ; Wed, 9 Jun 2021 00:53:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89A866124B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NYePaami7VdhPbmZMO8/Q4qOir2s1qN9EKca5OzFPUA=; b=TrJeGNEW4OMIRq HobL9npMSKAJczBG+T5F5JPm8Z/SH00VnBuCGYWDoRnbZIPvWOX6+p3cWj/L8KjJoKo48kR+/CUbC erzkSeDBAeQzHi++gOHKCFjxWVCnoDuNNNrHMMIAmkxY/WsmZxJIbARWuredzKmGwqY7lyIgKMCRG 0VHKyEEaTMbO7Gaty8zcInYbhRtKqVdcsyQJ3GxbCPmsb7WhQjgyklZk/1SLn0EqLo6ES4aImEVDY jZXA544DpXAwpzV0S3HAUv1gf7BP1R80dkbGZ8xr7czBk5PcoaYfEQ0zC2iL8hkXi668uMwJjPb9N ojGW8UFuqo7+DeeJLu9g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmRV-00Ayu1-Fg; Wed, 09 Jun 2021 00:51:38 +0000 Received: from mail-qt1-f176.google.com ([209.85.160.176]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLU-00AvUs-9c for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:45:26 +0000 Received: by mail-qt1-f176.google.com with SMTP id r20so1430672qtp.3 for ; Tue, 08 Jun 2021 17:45:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=gWxay37gJuUJcx+nxRsc6EljZGbCG+r7QOiRM4oSXXw=; b=XVTcMoxKQNpHAERlWFtpAWIkpNZdrKfflATOF4mJVjZ28fHCHXALrlRZKavWPEu/T4 F7rPyv0tFMiwq5YDEuNhRpmMxfXTtMrvxs9Uw7ZJhVRfssebxr9BrJIPhEcOVRmmTI8U sl62Z/+mIL4M2AC9o0ZMC/nj4a1udl7Qs6wM1kkgMmVvCApnCJBmSS1SJHjhs2ScpdXV 7A3Qe/Hs3AtbOFDmrbaym/wcQ10LG5aVf43LkrUVamE0ldR7HN/v1QUOgAsD6qAkKC/7 JzYTXc0mMucVZfhRyf6Qglt3tT+VeDNLE7o5/g3BtEPK7m+NwtIi2r2sWspJut84Gwlm ex2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gWxay37gJuUJcx+nxRsc6EljZGbCG+r7QOiRM4oSXXw=; b=dxn22o2i6IS12F7c+RUYMSDyOZkZY/D6Mfw3r10Hl/o/ScYps4+fdVC+/DeVWFG0ZA 0qFpV90X5WjLI0A3SDm2XwN6Db4qTcpzvzSyri/yjN2Qt54F5H5cOoz08qq985JMj+sp gbJtUfB0cKLt/RLY8aYU6YDDVtKYKCEM+hx5yJmNb7twlGZBRcZxQBrUGjcnbVKixiqT sfCYdZclPcDoN3k2spUNuVTUIumaKFp7WyeX/0QMV3iVfFKELaAjXoWGap5TejqfcHrn fI8W93cr2unI7iDUX2+3erRXS6q5lEwpK2SSEEgAUuz+LvcaCLbf+19BwiP9EEF0cuG2 wYWQ== X-Gm-Message-State: AOAM533cFwNxlqc4hOtSTuHbIuxtt5rUVTsn2FlDLD9+kJuIWBD9rWVv 3ax4jy5JjSDfKPQGTLVnKiUJag== X-Google-Smtp-Source: ABdhPJzeGFOwUM6YKiMqoMb6gKBWjIc1V8rc2G8wqLoy7ugeIsvgOGfXCBZV0ZKPy+a1WBOPcxSuAQ== X-Received: by 2002:ac8:684:: with SMTP id f4mr23920164qth.79.1623199463270; Tue, 08 Jun 2021 17:44:23 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:22 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 01/15] arm64: kernel: add helper for booted at EL2 and not VHE Date: Tue, 8 Jun 2021 20:44:05 -0400 Message-Id: <20210609004419.936873-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174524_357655_0FBF9C25 X-CRM114-Status: GOOD ( 15.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_nvhe(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 2 +- arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..a9457e96203c 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_nvhe(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 9a7b1262ef17..81b3d0fe7a63 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..42b5d8105ab9 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -49,7 +49,7 @@ extern int in_suspend; /* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) +#define el2_reset_needed() (is_hyp_nvhe()) /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..cf06929c44e3 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -200,7 +200,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_nvhe()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Wed Jun 9 00:44:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA2A2C48BCF for ; Wed, 9 Jun 2021 00:49:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A29516124B for ; Wed, 9 Jun 2021 00:49:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A29516124B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4Ozm8NAPy8OUf0nGAoDCzHQCxkXYtcFEN+WYCMk8a7s=; b=w+QN/qQ/TpBYgQ 606rEfAQ+xoBsJ7s6CiFsl0NjDonAawYJHdvF4yr/2DOdQh1oQjAYLsAcG/5rS9z4cb2iAGu2jNS4 qiynwaoCRQS3dMeaZBBkvMnnXX+XLlEU5ywYhfJ655+9IVXsznxVm24dRE5cc3J9TGDhycGJTqCYH +MaHeyHWgFpmggqWjbv7qZvSkO0Nje5A2nN8fBRWgCgq77HMWoQkuRB48i+3h0JI2KvlSdRZP3J+z PKchEwLtlV5ZmmyfZrcgFTtGPQuT5L8AVoljmm9Ca0RWqBgMLQQDkVQ6Tih6NDclp5WYnudot+cy5 n+ikgdwXgCJARE+LIExQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKi-00Av8g-EL; Wed, 09 Jun 2021 00:44:36 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKX-00Av4Q-PI for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:28 +0000 Received: by mail-qk1-x730.google.com with SMTP id c18so7070037qkc.11 for ; Tue, 08 Jun 2021 17:44:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ty+UClaOpOrJ9qBY9R0mQjMrJWAQ6ePXvWjy+fR64ec=; b=RR8QUSqJxkab4n7fmJSXo/jzTn32bDpgPv2YsNY/5axM1O6sP1S6nPBHJVZBPF65BJ fsAZqSkI/THGc5On0vujnM3SC3SdUHU4Ra7CwQ6fZhMIHhKrJBAAE0mZStEfZAKFqSXT c2Kivn4pMHh5Ktn32vJ1Dgcv0/1phq4V31x7KIDAEqnPqpxIFA0h1wtW9XO4p4xMngyQ 99NKOTeMQ4KiTyHVUHhf2VO8YNI0OBapKZHarsx++HvNdJ8tZ3kFjn4QV9llfCqSo3gt 1OX4f/fLH1+pA9fgDDiF6L3wl+IDbvcYjhenfAI/N2j5lA8N3y8H1Hq6JfMZU7GZ3Vpb 3+mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ty+UClaOpOrJ9qBY9R0mQjMrJWAQ6ePXvWjy+fR64ec=; b=ZEvKoJDSToAG7TjBET/1m+j9sX2CVE+1IlGQQoCiiLfI0CnXR8uXFfopbkzJ5ajQJ3 7RuKLnigTl+C3bbAMb3+A44/PFmg+qpWsSVps8hMlvovcZlxT3fnvwYOPxvCZkHtikRx 1Pn9Zp406lv1zfQzMjgjoELFQ8KELerBtl7DS/9HUTOtequ4of/rOQopHXJcphCKjnvg vtdI8C21QfHsh3Hcay9oiRo0+C0AizEuRP7yKyJZQknGYOraQHoMTm/+dinVkg968alr z/NmgEqiBKOgq2it9ueFocuj7H0sJkt01GxUzXkSENLgLyVuhvcmRelR8xkpFXGa/aTc 4f3A== X-Gm-Message-State: AOAM530QC+XIanNwURVZhO0b4PojPHMlgEPCJjrORD/zEtqbL3ekBP58 hPO87SLg9p4akTMMjJQE7F/Pcg== X-Google-Smtp-Source: ABdhPJwWZhdJj60QwViLmZW1EwFSY+KG2N/540VAQfq1U7H7vM7bK4gGZ4OWOvVNVTGROjphnexwIw== X-Received: by 2002:a05:620a:214b:: with SMTP id m11mr21044160qkm.53.1623199464633; Tue, 08 Jun 2021 17:44:24 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:24 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 02/15] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Tue, 8 Jun 2021 20:44:06 -0400 Message-Id: <20210609004419.936873-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174426_226100_ABBC8D3E X-CRM114-Status: GOOD ( 24.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 7 +++- arch/arm64/include/asm/virt.h | 2 ++ arch/arm64/kernel/hibernate-asm.S | 52 --------------------------- arch/arm64/kernel/hibernate.c | 25 ++++++------- arch/arm64/mm/Makefile | 1 + arch/arm64/mm/trans_pgd-asm.S | 58 ++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd.c | 22 +++++++++++- 7 files changed, 98 insertions(+), 69 deletions(-) create mode 100644 arch/arm64/mm/trans_pgd-asm.S diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..411323bda7a4 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -1,7 +1,7 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Copyright (c) 2020, Microsoft Corporation. + * Copyright (c) 2021, Microsoft Corporation. * Pavel Tatashin */ @@ -36,4 +36,9 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + +extern char trans_pgd_stub_vectors[]; + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index a9457e96203c..3c8af033a997 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,8 @@ */ extern u32 __boot_cpu_mode[2]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..e4b0de990dec 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -111,56 +111,4 @@ SYM_CODE_START(swsusp_arch_suspend_exit) hvc #0 3: ret SYM_CODE_END(swsusp_arch_suspend_exit) - -/* - * Restore the hyp stub. - * This must be done before the hibernate page is unmapped by _cpu_resume(), - * but happens before any of the hyp-stub's code is cleaned to PoC. - * - * x24: The physical address of __hyp_stub_vectors - */ -SYM_CODE_START_LOCAL(el1_sync) - msr vbar_el2, x24 - eret -SYM_CODE_END(el1_sync) - -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - -/* el2 vectors - switch el2 here while we restore the memory image. */ - .align 11 -SYM_CODE_START(hibernate_el2_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t - - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h - - ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 -SYM_CODE_END(hibernate_el2_vectors) - .popsection diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 42b5d8105ab9..fa2d1cb844a7 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -51,9 +51,6 @@ extern int in_suspend; /* Do we need to reset el2? */ #define el2_reset_needed() (is_hyp_nvhe()) -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - /* hyp-stub vectors, used to restore el2 during resume from hibernate. */ extern char __hyp_stub_vectors[]; @@ -431,6 +428,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -458,6 +456,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (el2_reset_needed()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -470,25 +476,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - __flush_dcache_area(hibernate_exit, exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (el2_reset_needed()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index f188c9092696..ff1e800ba7a1 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o +obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o KASAN_SANITIZE_physaddr.o += n diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S new file mode 100644 index 000000000000..831d6369494e --- /dev/null +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +/* + * Copyright (c) 2021, Microsoft Corporation. + * Pavel Tatashin + */ + +#include +#include +#include + +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + +.macro el1_sync_vector +SYM_CODE_START_LOCAL(el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS /* Called from hibernate */ + b.ne 1f + msr vbar_el2, x1 + mov x0, xzr + eret +1: /* Unexpected argument, set an error */ + mov_q x0, HVC_STUB_ERR + eret +SYM_CODE_END(el1_sync) +.endm + +SYM_CODE_START(trans_pgd_stub_vectors) + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t + + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h + + el1_sync_vector // Synchronous 64-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__trans_pgd_stub_vectors_end, SYM_L_LOCAL) +SYM_CODE_END(trans_pgd_stub_vectors) + +# Check the trans_pgd_stub_vectors didn't overflow +.org . - (__trans_pgd_stub_vectors_end - trans_pgd_stub_vectors) + SZ_2K diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..d4f0a6492d8f 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -5,7 +5,7 @@ * * This file derived from: arch/arm64/kernel/hibernate.c * - * Copyright (c) 2020, Microsoft Corporation. + * Copyright (c) 2021, Microsoft Corporation. * Pavel Tatashin * */ @@ -322,3 +322,23 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &trans_pgd_stub_vectors, ARM64_VECTOR_TABLE_LEN); + __flush_icache_range((unsigned long)hyp_stub, + (unsigned long)hyp_stub + ARM64_VECTOR_TABLE_LEN); + __flush_dcache_area(hyp_stub, ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Wed Jun 9 00:44:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0B74C4743D for ; Wed, 9 Jun 2021 00:54:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 88A2F61003 for ; Wed, 9 Jun 2021 00:54:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 88A2F61003 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PvvH4eLpvvMy5UDHknPMPoP6IQ5x5eq83dQOwgl/iqc=; b=NxlH57Pu0p3t1d pfmkPZuU6iOdZ/QiEdJqXxzxHh9MzF9usmlM48jtk4aGqtTn5O5nWhA+V+xy9QJ4Mly0+wueaBUPT BXwOw4nblKGVp54Ec7VJ5J60BPlkTZkDWS0YmDPKOaL65PGhOuAJLkjw3Zn9ZN/6CjnGPPBsth/fm lljLOUmrDzysPyl9zBQ03jCKysROcWdyG3efLXNXcnCq3IbE6fmRZLFcYq9ll3MMzHxNW0mizjRbM EYKwTAQMDTH3WiODla80P6Oo4KbENVi5DXh3G3BKbfNHCH7d85JlA+m6ZKz8aDlf7kkzrTePVeNr7 K0EJ2UmIr9K9NsVy6j2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmSV-00AzF6-GR; Wed, 09 Jun 2021 00:52:40 +0000 Received: from mail-qk1-f173.google.com ([209.85.222.173]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLX-00AvWb-1l for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:45:28 +0000 Received: by mail-qk1-f173.google.com with SMTP id j189so22200040qkf.2 for ; Tue, 08 Jun 2021 17:45:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=h7/0+77ndabR+mxgeXAoHMyf1o3rde7WHj0V90p1QaI=; b=IVvfQXHZK2mgQNL2VKhZdBH2cFdFVnvi67WIksJs+EJTPZ80DCLgf6x7V0AY/DP9n5 KPgwdo1s2mo3JDGJ5fuatqamvtcwaG4T8fEJK/Bw5QsiOjn1lOzxUTZKZAxJNUd24r8e A/aqF+NmNP+nGi30Dk5FMvJrh3Wz9//HGGRNayhIhz52mo5L9fGeviQ6LlXK55xH/WgF sUM6DamMUuZvFX8BQzFtBwq9U07uUGNbw1p4n9xg0w6r3BeQihTedZ0KERAwA+nav6oV +7aiaCkIUJGK6A8RasaLwtoBEykds2pKFqqfqW1vv0VeIfpvk+epR+C56PAYWG+NdO61 laPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h7/0+77ndabR+mxgeXAoHMyf1o3rde7WHj0V90p1QaI=; b=VUNt6WyBQuZNE22un2UrHDzBRpW9PuFvxuao/t7uBZ5wizxt1/uLgtcuQFjP4cNjRU OfCgkLluAxeGEjzzj1BdFrPeyEgl0ClDXKAyCswzbswYzsiBEwt3cn9MdrPHic3uq7Dz xFax+IHHqTVPR+vFjMNMfl0oC/sT0KdJcJopnxiT3Oqbxk2H7d84l/yvLkuQT70WdXBR 2xBya1rRqaOXW4Se3sB27oIGzymvxf5mZzCcgzM+OTINQAs9dFeGx1RQCk+uDDAkCtWt 9DFi37jsTwsiAEMOw0HhW4BW330XvUVFOm1eHFfYYqofrzZxHBW1A3vuzPZ5pK6Lb9Rf FB1w== X-Gm-Message-State: AOAM5316cOiYMb3GF0K94ci1hz+xOpBAwVc2FSf9CTWLR4rTPNb9zA90 TvRcQNMTy6QDSjIFGDUdl3KeTQ== X-Google-Smtp-Source: ABdhPJziHWaxeJQaB1odr8nTuQIw3SApIPPJKWycUP6hFCST1MTMEPuGtiIWcoxNijhifWrDO7tqkg== X-Received: by 2002:ae9:ed91:: with SMTP id c139mr1724055qkg.454.1623199466063; Tue, 08 Jun 2021 17:44:26 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:25 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 03/15] arm64: hibernate: abstract ttrb0 setup function Date: Tue, 8 Jun 2021 20:44:07 -0400 Message-Id: <20210609004419.936873-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174527_117981_4735D820 X-CRM114-Status: GOOD ( 15.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functionality when relocation code is going to be idmapped. Move the setup sequence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index d3cef9133539..85eb92a2ffcb 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index fa2d1cb844a7..39f0da125876 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -212,26 +212,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Wed Jun 9 00:44:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2DA2C4743D for ; Wed, 9 Jun 2021 00:56:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6E8AD61003 for ; Wed, 9 Jun 2021 00:56:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E8AD61003 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L9vS0tAUgnK695+xMNOhNTPthc35T9698qK9uRNN4rs=; b=2ZBcx+3AYMhtEA 7zp7oYVBvcdWOPjgHcwUnj40LQNwUU4MrR6H5tj+UX62uyLhj+QPhvysHGKFb7T33kANhkIpEule0 s8M+JN3eFldW/EM2Ad7M3EhUsw9U7E+dUJ3gNV6kyJ9zRuGwMhTswspKObmPz055ziOMyZS29qenR bELt4KAhPvn9TRcjFk9QXRoYPe5uxscwjgpK4H/2610keN9FUY11W83YxZ/t6bfKxjHTeC3lfgbTz i4lkHR7bbnOdVmc0ow9PQ8Pw3OPBtKLA1v6V8aSOTxDUfszJRvjAJMJc6Z4mQiAxQnsdK7hEpi9eB qF/sf8Vn9OalaWTwGGrg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmTe-00AzlZ-3t; Wed, 09 Jun 2021 00:53:51 +0000 Received: from mail-qk1-f172.google.com ([209.85.222.172]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLY-00AvXJ-Dk for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:45:30 +0000 Received: by mail-qk1-f172.google.com with SMTP id c138so9282055qkg.5 for ; Tue, 08 Jun 2021 17:45:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=QeK9i9pdmaa5nrGaHB0x+uBs1hguggMbvBuQFu3ngSXQDb/b76nSpsoWSwBEz1ZACF 8ZX9NJFTN6TDfgF/DKMPtvjN3Ytjy+7x9eRS9YkXaGNzJPpop9/fOLqR31ZeGJJ2e7mA +esyIRxbARxZLUdDBflzEqprasBY+fb3LOQqwfPL1M7t0uOBFj8ptNfgn7d6wbFX6nH6 sU/95ZCHJH2gX/SZiMrYBybdiyc7mOapL8IpuTw4ayFFpfIhy7lwlU/pCpKcMBKMIY5l GmNfsInXoZvl4nLA4vJg/YckdTiEwOA7CGMnn5e7ZmQg6RlxfdZk83AjSftF1dPIm3j6 5m0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=m0pP+dww1b8KFXomCqH4x7RsyI+oYQVHZfOibBSuWmhQt8GOHW1MjD4VRru024BoGK gPaJhtX8zu03BZjRSP1oxnvSa/2+DUHvx5lqL4ryJ+gmnRZrdbeQjv7g5ZJdMzZXWjbM C6mWvPcqbLs0yu8UghK/gjOVW4iW/fNaO8MMvP0DIGut83zwqn3mdhADRhAygZAw6iB5 4YnSEeq2QIWyK+da+X6GdGGqX4IPYA03N6k0GtxuCxhtJVFx1fgVUDelmA4tCSXorsvX oC88mlVf5xf5vXuyEkJos21cmW7L5aMYp49uy5T+9PcMzXYB4g/LCzGDZt+GK3LlskO6 jKtA== X-Gm-Message-State: AOAM531IUV4YzbkcCw/rl9kzN967pfwN60i9FMcMggDiOmptWHceEYmg ytw5XygX4I2quYnt9Rg71cVDHQ== X-Google-Smtp-Source: ABdhPJzgsboPEWF+K2hwsvp2uYA2cq6gLqfyoPJs2qRhodktFoeQ6z2ViDG81uQldPhedYPiyLuRrw== X-Received: by 2002:a37:5dc4:: with SMTP id r187mr23745072qkb.174.1623199467407; Tue, 08 Jun 2021 17:44:27 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:27 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 04/15] arm64: kexec: flush image and lists during kexec load time Date: Tue, 8 Jun 2021 20:44:08 -0400 Message-Id: <20210609004419.936873-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174528_484511_C7550754 X-CRM114-Status: GOOD ( 15.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++++++---------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..3a034bc25709 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -59,23 +59,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -152,6 +135,29 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -169,13 +175,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -250,8 +249,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Wed Jun 9 00:44:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65B0EC4743D for ; Wed, 9 Jun 2021 00:49:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 33FEE610A5 for ; Wed, 9 Jun 2021 00:49:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 33FEE610A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=oxzeKb2pXGss0P/nY1ko8cNK9QxGXOnwbx7iKhOcvbU=; b=P+AJvnBc80jcwU k4xUTE9OWqOvnTAgSSBq1WRVECDIPL1xj6NLvO46KgHEHMY2BbfHwWZc+EON0BWecc0fhlgtSVBks fd+ebjWu6N6xpXBH0xqtqoIy0nwLuPZG//jzIuU0O3iZyPVcZ6iV9bs7mx/qVT0A5+wWjHTTL2N1v 28J2GkQCvVd0CIK1nL44RuHK8n/nYJ9mjcWNN8rAgFTyjHMRQb4q77d2ttesH+9iN66YmaUuo8uUx SSKK1O5mM6xr3pi6XCejk+0xu9ONcFjS3fygVn7NgRAgcEyr5muHWHR8mjAFwMCwhzFPTpalhIfpH T60HehEe7bBg/P67hIMg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmL7-00AvME-QO; Wed, 09 Jun 2021 00:45:02 +0000 Received: from mail-qt1-x835.google.com ([2607:f8b0:4864:20::835]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKc-00Av67-05 for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:32 +0000 Received: by mail-qt1-x835.google.com with SMTP id a15so2501652qtx.13 for ; Tue, 08 Jun 2021 17:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+rQtvXCRxKOTIz/EZlF+eoC/3R9xTGrhR6gwWFr/7DU=; b=Cw6JYKkMN7njo+Mj8aYXX7ELOe/Z0YqxT5vMM3ZTvVFAbWJGnKRohD82vrII8WNg4F D0I9z+YHniSs564yAlOnK/rUlpDhOCLCJeshp6T9I7pMZhPxTtdgQ2RtrebnjPTIwrRO U5lczcMyVea7KnzBmWqcD0vZiyDjy1rhL8h1g0rnbObUGqG6AQPptj4dHEvi77ttrPc6 gLPMKkk8/Ag2InWRefSkOs9Nmh046fkcd90KZQH8EXO4GFYwZYy73NE5ePutErBiz0In udElGn51BM4wZXIiyQhPp/p77yro83tlcnfVP9KOW5ZMFxbUY+9cYeaPPubpQTpUF22I ov4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+rQtvXCRxKOTIz/EZlF+eoC/3R9xTGrhR6gwWFr/7DU=; b=L8e5h78sdXelKX/DE5jPv951Px9TN32Ycg+WiXHiuv1NJb2992rB17YroH5ikg8tuK fiziHK11mdxHclu7ZXFeWyRCBl+T7yXwC7Q489RJwvn98z0/O9W5ifn7dY/FDhV4cOSf Kg7BobWhC16fIeac3K++xQvaMCP8Qh4xWAaC9EdHaWKcVwU6l9fU7HVR2C7ML4y5zlNr 1FZouEXO4rX/2t8DkfO6eEH8biNddm63paiD5V5NvLGGoAg0exVSN67JNVKP1DJs5EB4 oQjHsZLL+dec4VOXkcU6mgJY27MPt3FQfEwFkcZQP/CBcsaZv+gpAOMtP7sDkBjSh2R8 046w== X-Gm-Message-State: AOAM5312wIqpVjosb2dbOJYd7BMhPwUpCXmWp7wPKBlpItJc2AwDivcH Y9yGv2NvQxls5CgXUJ3n8wddig== X-Google-Smtp-Source: ABdhPJxXVHxnf1PDup+eZ76FBqOdGwSPuYwAKL25Z/mZFDlwB7zQWfVE5rSutxxPSstYdHMEj3yKxQ== X-Received: by 2002:ac8:4997:: with SMTP id f23mr5521694qtq.94.1623199468970; Tue, 08 Jun 2021 17:44:28 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:28 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 05/15] arm64: kexec: skip relocation code for inplace kexec Date: Tue, 8 Jun 2021 20:44:09 -0400 Message-Id: <20210609004419.936873-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174430_115095_F6803D54 X-CRM114-Status: GOOD ( 19.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3a034bc25709..15c126e09686 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -139,21 +139,23 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -180,19 +182,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Wed Jun 9 00:44:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF13AC4743D for ; Wed, 9 Jun 2021 00:49:28 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F497610A5 for ; Wed, 9 Jun 2021 00:49:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F497610A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=j3JNL8gKEAsXvkjLu7lFKC+grC/beK0+cA4JebFvT6I=; b=spp9TgKvbHc3dE Mj4Y7HTLhl5rXdIQUEc3+GHgmMa3LVrII6orhKhDo8EiSgL8lCgGtkxupl1H8OlBAASjCVwydJH+l /pKa2zVBkw5bQoteBV78D5IeqJPKdIyDmyMHhI4IKqNL5Nx9hq1fa8bNUXLu/q8PhD+oMD6CRNxCO IaF1HDcDP1rjaEiFG5A40OKgfWDWqINKRNacYKG/ZVOzgnJPgsYD5GIlXj0CVnc2rXSUJtDses7Ef C2Eb4CGcyRMB7dBwNik8JWZUz20BXjK2NGETA/6H2fbiV+1C+NbsDKUUtN13Yp1g+z/+DO9VF6KwC 6OKNj3VVzT4Nm6PLdBJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLM-00AvS6-O7; Wed, 09 Jun 2021 00:45:16 +0000 Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKd-00Av6p-DX for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:32 +0000 Received: by mail-qk1-x734.google.com with SMTP id d196so17025961qkg.12 for ; Tue, 08 Jun 2021 17:44:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GQ5BgIaJviFIJ4FCU2ORrVnn57SMRpQ9H71Ow8Vj6m4=; b=Y7E3HUKBWsfN3xCB81SMmwmoPU+UXzi7iX+gQK3HhBmGGLkxpTawLrk0L9WbhmKQcD PH5qlwzeky/C+Se+XOhgLii5iyYIw5w/4Eu+6fTt7CT0z2pngUyDoX/2/zRg9hoYxd4c geMyJmmU7GUUUTJcXBr556Gauh36IKBJ8XpAZeYrRUkn4R6mDaADzFE2koFCW2tbY9LH +1mFKAebl8I/Zh1RiYzMWGa/+8MmuwHqL055hGUjSzSqf3vu7CbQF3qYVbTXJfkgjpr/ V79O61DQjzqNgeE0YVPXFc3CCfVA8dK1PJ93zuKermcSdgc4S4GA+r5dgwtJQLDak4kQ KfLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GQ5BgIaJviFIJ4FCU2ORrVnn57SMRpQ9H71Ow8Vj6m4=; b=c31SyhgZ7ISgnapaXZfxr0LTEcs1h8fILZ0q8zkZJzWAUV35g4SFMxZWZ5rJLKCbz0 WN+5QSTFwWIPdh95ySX3cLkhfVDbuxvJV5AvDqW7C5Q2wHYSa4Ir4BMDXpXTEsl2H/Ot 3H9A4jPtWbr1oT91pl6u4D0uiknaXi2wduSBYLzRihpEoXWaPkLNi1kSPVKWEyP3m9N2 6GK/g8PTJ8yDIsvoBYE52hEM4A469msPwvOiUsEF3Kd7Sl9BHbt1Bd+5z0/sU/MEP2OH sHkLBY9O0WBxZAWNk11UJb/45WzEhERM7Ht/9IU+0dEAC6EQ9G6V7/iwO8+6E530EG1W sAdQ== X-Gm-Message-State: AOAM530N10thKmgWgOCdwTmRRsQKz07I61j7xxBvxuWbE7NwYrLmS7Yc /2yLMgqO0GqH40BriieFzLbKGg== X-Google-Smtp-Source: ABdhPJxJHulb7L09eWqLRuz1AoAZSBaTc/m3jSYUkhZOutuErP2+QhJiIPXDPhjGVQPaqZVif1ou/g== X-Received: by 2002:a37:7145:: with SMTP id m66mr24028223qkc.379.1623199470340; Tue, 08 Jun 2021 17:44:30 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:29 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 06/15] arm64: kexec: Use dcache ops macros instead of open-coding Date: Tue, 8 Jun 2021 20:44:10 -0400 Message-Id: <20210609004419.936873-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174431_507851_C5D2F042 X-CRM114-Status: GOOD ( 12.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitized DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 12 ++++++++---- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8418c1bd8f04..f1367863d995 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -393,10 +393,9 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 - dcache_line_size \tmp1, \tmp2 + .macro dcache_by_myline_op op, domain, kaddr, size, linesz, tmp2 add \size, \kaddr, \size - sub \tmp2, \tmp1, #1 + sub \tmp2, \linesz, #1 bic \kaddr, \kaddr, \tmp2 9998: .ifc \op, cvau @@ -416,12 +415,17 @@ alternative_endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 + add \kaddr, \kaddr, \linesz cmp \kaddr, \size b.lo 9998b dsb \domain .endm + .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \kaddr, \size, \tmp1, \tmp2 + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..718037bef560 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Wed Jun 9 00:44:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47AAAC4743D for ; Wed, 9 Jun 2021 00:50:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 157746124B for ; Wed, 9 Jun 2021 00:50:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 157746124B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=BW75Pb+NUlY7SPdQ/amnQ5xSSj6HCYlNVrMmaQ35Jlg=; b=lNcBysVrH41a1T qU6swKUUICJX7glTgQrOnVnCmvX2FiVL+v5gm56/oc99HzecRD/jUpg58pJuwhI+CCmqmhTZ9yNId dNirswGNlUuCJmdBfQVef+wxkuj2kxefjaxW8OIxwH1E4tXYZ/jXcpyqe9yLOavHhRszh102Icewv PEQXiB7QAc0i6Pw7B/dOOD8gfdoPQ3AhXRBuSX9mwxrmH9y6KeypqMF1wPtaGJpH3t2EKRXU09ZCP gtMA9jzzH2eHOyErtmF4pr/yVcncl5bdBrVu7NbRJfbhsSsnTemCOXji6hs/J8WJrbhwWULcFyOA5 g236PuqwRnVZd0fbMpbQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLg-00Ava8-7m; Wed, 09 Jun 2021 00:45:36 +0000 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKe-00Av7U-PN for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:34 +0000 Received: by mail-qk1-x729.google.com with SMTP id i68so18573659qke.3 for ; Tue, 08 Jun 2021 17:44:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=47E64LQi31/l0SbhpKLyQRIg6WVPVm2IY96rfzQk1i8=; b=gSI2NZdHfnGG7iF/y/V7CBej6XZmDa58BGu9PceFBJIJisYmwa0gEPxdJo/wOGrV4Q DCw213nxdINz2hzYgbQd98n5EBVTV1tMhLW4bFYfrmdopeY2gyTxYqm9Oo6yZQ57yQ+U MPsNZR+0qm0p/6Je3ijxB5f7MoKWQ9iMkaJbdr07EeTd2QipyJDeDXWlOmgFrKz6/Sb1 qEjoQDpNOrkwmvplw3TgEbIxfbOiBNMdUAqIE6ZutdQu9HtfBp24+4kPE6HkIu8jQTBa GU3OgEqPrnDs44zWvrf3qQiVp8k2jppYVtzZzfv/PSYDMm6Uh5HbefJk2P5nOcaGd2jU 8MfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=47E64LQi31/l0SbhpKLyQRIg6WVPVm2IY96rfzQk1i8=; b=VRH9g3EWcAZxBSOGPxGQEfO0SNhgmE9AG+UeQSWrbzqTJVts+DUM1vJR3lc6rqeqy4 cPs1Qx2W9a+2FynDShEBEloMRakzT4n11aQdQmc58Jm0+5v6LzrbDhX6QjYjl3eIyc7d 3oWLswT3g96O1C1sgD8rQxt7Nq9nA0mKouGoECAawbeqvrh72pmqcLwAlL5/0wGpkppm I0PooKN7Nbz3Sjjm5PJ8XWp1dTIj/uDeLfXPdvsSOjETT4ZxS/Z/RCZ2cQR3AwNktoOd Y48iB3Wd8mPhJm9+pAE+7w2IP6gXX8RMwPDK5fnEPiHhcZFV07j6x4+LhbZpTlinHXXb DAbg== X-Gm-Message-State: AOAM533f2UgY3HkRHH57Yb1dqxGe3aK+0jlsb9cbcFSBdEln7oK7U8f1 acrXgUAvAv5b/jiJOPstCaWTCQ== X-Google-Smtp-Source: ABdhPJwuE8iYBSswaYmBE6s54oqCaDrbIDyDAYyBTNjVtLyhDfoYUnbq5wMn278HggBIcJh4KYl6TQ== X-Received: by 2002:a37:2e07:: with SMTP id u7mr24502829qkh.489.1623199471709; Tue, 08 Jun 2021 17:44:31 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:31 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 07/15] arm64: kexec: pass kimage as the only argument to relocation function Date: Tue, 8 Jun 2021 20:44:11 -0400 Message-Id: <20210609004419.936873-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174432_869526_B4C3BEFF X-CRM114-Status: GOOD ( 15.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 0cb34ccb6e73..92c2ed846730 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -155,6 +156,12 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APIA, offsetof(struct ptrauth_keys_user, apia)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 15c126e09686..e88d5cfbef40 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -83,6 +83,8 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + __flush_dcache_area(kimage, sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; void *addr; @@ -198,8 +200,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 718037bef560..36b4496524c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Wed Jun 9 00:44:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBCDAC4743D for ; Wed, 9 Jun 2021 01:00:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7840B60FF2 for ; Wed, 9 Jun 2021 01:00:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7840B60FF2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=quSOAuHeFxOVoTFGtXxV28dOQc2ofRjvqITZCmYBj58=; b=lSMu2K8kcVI7G8 KjTKpDQmAImcNf19+U8oh0A1eRypgufOgr1QYPsFlOQaUF/YUjQo8pKynhvYFN6rtMQkE1FHKHy7x 9tLb4alR3/9JoCrJ5iufYV1pKI9p/kvFseLk9ElVXAXop9sPHPCZkmTtSpQifBEbQjPaGqzEgUitB C+qp0TmEpj36oqPZhFXUaBcCJH772gpZaP89Wsk1R+QDco2JufunBx+IUqSx+dBT0ii0cr80TuW1f wXQLMzzM2kCVJC8jGo+pmQk73lfe1XxJTjYWc0Rmn2rNv+kFPNI/lJere5sOGFEtHEVZHKqRHKtDU 0pBcfU6sz7fmjIrthP0w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmV2-00B0KZ-S7; Wed, 09 Jun 2021 00:55:18 +0000 Received: from mail-qk1-f172.google.com ([209.85.222.172]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLd-00AvaB-O8 for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:45:35 +0000 Received: by mail-qk1-f172.google.com with SMTP id o27so22134854qkj.9 for ; Tue, 08 Jun 2021 17:45:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Emy7G3R4fnJJtixbqFliehFPYNk4E7oq380r2UBbUTg=; b=YWf0syBZsse8lc/mMTj8DS7PSWN6ra8Xukco5XvSlESCykwl9Vkz4Uhy+rJytLdhAX TsfbCvx+Qf07PG/0Nr25bGgkrY3oUZes58zRrhN1Egyp74h+zKriKvWT1P6QSrpX+Wt9 cCdqQO+3Vu7ZYgpsC50bbrc5Rn8qYOJjeXvu9QlRVsdn/c2iKRfEv94sjnD8Xilu3CJT RIq41RR+cG5Ut8+fzsX1EESUsk8azmnL5AP/DU13BpRy+qv0Mbkco1NhZp5rQX7dYAwj uNPidK7FC8OKN0FBEDUi21YhTZMTNrCelEmFBGTuhg+kU3YNeDwuVf+h4p8QIuNwo817 HKwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Emy7G3R4fnJJtixbqFliehFPYNk4E7oq380r2UBbUTg=; b=cOcz4gKwu4RspwksV/a3R4tkYFcv6d7a8dUpdwpigYbqY3ZqazqofGBsQEFmBU63Zr AZdZKia2kFDnMau/13WYxWRPtdrrZj96h9UFxfSr/dIjIF1sQFKnBJvarTTbZxVDGR8l Qr03X/l6XAiXCKEj2Iwh1Q/gGUvGuPMUXdXMV3P0hoQr1mrH8gXlleWGrdRsxk1yYNfn iI32ILv5OJMwJtvl23mYSm8DPjW35e2yzVD7jH8426YXJOrGub/iIDKj8UCYpQnircID 9rSOsjTWLao8uNick8ui4vvkoPFwxTOAZP50rgDWwYLlIwKq/azd101lz6dhVNk5SdjD HgGQ== X-Gm-Message-State: AOAM530Tijcgd2MjqZJZoncHaXUu0USA5S3IcIftATCRBbxurCgip6Rj C9mqT2rrfOKmo1WYPusnq08GLQ== X-Google-Smtp-Source: ABdhPJzmM3elcFvZ3NQoPrFQ0alk3E3chSyV6enV46/ICZNwkelqZouWMSAEtqgxxJQAPybADdxVXg== X-Received: by 2002:a37:2ec1:: with SMTP id u184mr2736258qkh.500.1623199473091; Tue, 08 Jun 2021 17:44:33 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:32 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 08/15] arm64: kexec: configure EL2 vectors for kexec Date: Tue, 8 Jun 2021 20:44:12 -0400 Message-Id: <20210609004419.936873-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174533_856208_0D86437E X-CRM114-Status: GOOD ( 19.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preparation to MMU enabled relocation, configure our EL2 table now. Kexec uses #HVC_SOFT_RESTART to branch to the new world, so extend el1_sync vector that is provided by trans_pgd_copy_el2_vectors() to support this case. Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd-asm.S | 9 ++++++++- 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 9f1d8566bbf9..e775e6fb03c5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1148,7 +1148,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 00dbcc71aeb2..753a1c398898 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 92c2ed846730..ee37d0e0fb6d 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index e88d5cfbef40..e0ef88cc57d2 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -42,7 +43,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -137,9 +140,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -148,6 +169,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_nvhe()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -200,6 +229,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_nvhe()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S index 831d6369494e..c1f2ed1be6de 100644 --- a/arch/arm64/mm/trans_pgd-asm.S +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -24,7 +24,14 @@ SYM_CODE_START_LOCAL(el1_sync) msr vbar_el2, x1 mov x0, xzr eret -1: /* Unexpected argument, set an error */ +1: cmp x0, #HVC_SOFT_RESTART /* Called from kexec */ + b.ne 2f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 +2: /* Unexpected argument, set an error */ mov_q x0, HVC_STUB_ERR eret SYM_CODE_END(el1_sync) From patchwork Wed Jun 9 00:44:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFF00C4743D for ; Wed, 9 Jun 2021 01:01:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7BED7608FE for ; Wed, 9 Jun 2021 01:01:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7BED7608FE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aHHdcyh3NLwkr8tBPLLH6npc8Fq/lshntEf9CzF7U8Q=; b=jnToGLEJUcpSbT OYhHPBAazDyxNr5fWPT2IVJqoPkn1NjpxOhiDARSV2kr4Ci434IoyALuCG/vgJHWbGrdZ6M5T66Ic 30bS+twIClovRcdnTpzaAZazmKtEgzZK6wSl7u5Tzm5/JHbd/vXgijlRiCvFrA3u8O8wRK8Hqx3ch 1AvTeF4d4SyiOFUjcMv4tmr5SCKJzoCl5j3YVrGQdrWfmeRKUzdWLi+Ip8jibO7NqUyQOQvkxwlSR 6bXRoEc/mQFr1I4jHjQiR0HGPHZ1p6cF+FVT+1oNsTXAuaTx3sbyGEZek8edjV0T2BlJev0PVMJUO ZPnLUm1f3LJJl6BVfUZQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmWO-00B1D8-4j; Wed, 09 Jun 2021 00:56:41 +0000 Received: from mail-qv1-f46.google.com ([209.85.219.46]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLf-00Avao-Bh for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:45:37 +0000 Received: by mail-qv1-f46.google.com with SMTP id w4so4483530qvr.11 for ; Tue, 08 Jun 2021 17:45:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZPSVbTnhn1kzrM4k6yG1hZdalcOX13NxNhzKc5GCg3M=; b=LGgj1DSeN2dBtzalXLyd/X7N/RCTiZa3EUru9uNeWdpVQJnvQ9bVsOe+hpBq10aBS+ aY1IEnfFr3j+uNpDjtIRX15IWRmDkCAmORZVFue5KNHoK9ciqWHqDq8PCBfaG/bKQ9PX H4wnXAYOwb47RzvJL85RhAzTQ4swYnk5uLiqv1RevSgZBxDenYt+PNKEP6Xg8h9NdGQ/ 3DLpUIt2K7tm2CzWfhFt6+tTbTdPRHX38gq3dsPGJUfHALsxCr1UcjF9n5kDaIFRgbym rbYrqPpH4MczVux3IZZxQkckEORpurxOSkjx4dSeN6myWV528RbdrQVhFzdB/rchvYEp Mpzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZPSVbTnhn1kzrM4k6yG1hZdalcOX13NxNhzKc5GCg3M=; b=aipLYIxPJwbp2NOFE9eOflrODdVd1DWYqjwKqEftmVE/Y21ZhpSkFS/l9JLCGKulIh VPsBxNopfpux6MFOY+zRU6FwA+yeX+/UiJwYp0bHr8bfr8imkihj0pnpXckUoPHCE+Wu 7QAlZmbN0xypmNl6rK6os8EENQhxkjAK+AIIyboXIXhDS5+tACrBYHiPUxM29LDWa0mY hWiATbhEdlyRkO8Hzowd/3sC7lDo/YSqkril2wqXPGLw7Ut9zNPQVp9YppIdGl/x3WK5 tRp1eNU0SflVta9EI1PORP19r8gYg0eNmlOVCdR0CjQzjSzeIA42RzHpDX/TdpN2Nx+Q uktg== X-Gm-Message-State: AOAM5312PUehojzjWqDfkge1wEBpZ/SsT5j5n50fRlLhFwfHSz4Brp11 go2Y6Oa4X1HNgExG+zFT+sy52A== X-Google-Smtp-Source: ABdhPJx9mZ6UlFMdNKDraoLwthZ4OWu6MUyQRTHx+1rf8HWW0BNaBOPk307SDiklwhYTyx05MvoNSA== X-Received: by 2002:ad4:4713:: with SMTP id k19mr3379354qvz.26.1623199474423; Tue, 08 Jun 2021 17:44:34 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:34 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 09/15] arm64: kexec: relocate in EL1 mode Date: Tue, 8 Jun 2021 20:44:13 -0400 Message-Id: <20210609004419.936873-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174535_434325_678EC706 X-CRM114-Status: GOOD ( 13.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before entering the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 81b3d0fe7a63..296abbac7192 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index e0ef88cc57d2..152fb00bcccc 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -231,8 +231,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 36b4496524c3..df023b82544b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Wed Jun 9 00:44:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCB8CC48BCD for ; Wed, 9 Jun 2021 00:50:38 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 834346124B for ; Wed, 9 Jun 2021 00:50:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 834346124B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uWVqfDb4BgkH9lniZRgqI7FGMZFmM5mrghO7+QyAPig=; b=XqXUDB1zQmZG4L QPyCr1tPh79aEEXhuNe+TdMQQWjiL7bilwNAZwInPgP8FMY+aQq1BMr+GApH1fvOg4EaTDsINgfae WtnqheP/6pxgpj4dL1YozIhadvEMFROGMZsMjwdnfMiodS7MG7ZCw+TkK46N9OAwgWyy4RwL92/f3 p6o5k96y2DnJZzJ0XJigJW4uGuNFM6tUwhB69h8CHXzXR3rJxBewERdefgEQlT7y68oRDMbE5RlLO d2U6JbdyOjjEwInmrpxQkH+f3CFE/g6/Yrs1jtHoJrW2pvXpCUchyzzkMnk9NwFDeuXcX5XpIwB/r 65wI84EePEYXGzIu3kjA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmMA-00Avpu-LO; Wed, 09 Jun 2021 00:46:06 +0000 Received: from mail-qv1-xf35.google.com ([2607:f8b0:4864:20::f35]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKi-00Av9f-Qk for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:38 +0000 Received: by mail-qv1-xf35.google.com with SMTP id e18so11903609qvm.10 for ; Tue, 08 Jun 2021 17:44:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cxMV479WeipZl06+Qftp6M3RITOKuO1NRSl+w91dGp0=; b=DNFqGimbi2ZaAFtofJSMKKPVBs1tzUQLkbDhvbbZUywzcV7oKfcUtxdnkD4sjmSwaw +STNFwi5yQJopON0UttmSlCfgEDkzbBR93NpO0ypppawz8CePzWHnB79E8HX5l0RFW5Z Q+sB+MKmv+wAiNO+esmUHrN81Hu3PO6ospoQrNVaWRfaIkgwnuqSsRBn6ZcnjEV/0zRg gKSYjjYl8OwnacGRuMHjo3VTrxb1De8u/dufrI/KVxm/ekhudrg2P68y161pWPBqsIaA cTMn7xhKT/eb40APGLpnPNnmEbVizNk+fu9jdSoOANOmI+VuXUN02IOmm1J0OzHWFHbe LipQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cxMV479WeipZl06+Qftp6M3RITOKuO1NRSl+w91dGp0=; b=D3b5g14fPqmUm7h9z/upsM+Vsqfdpf+TOeUC9zpo63ow6LtAmCxdNx0cLqkZQzNeHE vOJrl3vKCf4RcZ/bygwNth6SfNCNMOp3KulN193bnqJywTW6VPbgdXW2a/M6cJPDbMCB BFb/iRT9hRYfaL9HbMtMgGGSgFcGUVDOvUWD61/Ak9l7pKmWsx0N0L/dJQZnpSGwlh8l lzEel0NtfluJJiPJcOeXw/4Z5373QK2GzR37/JJKm+r7HPeSD5+jfUQ7Lodif4pD/rcX X8lSoSBX0mcZKvlIT1Kf/Lsz//94fWsyxi4y+5l5x0rPbLtSxexH7UgizYTSvxaVK4bA 1v/g== X-Gm-Message-State: AOAM531Zy28ZOL4n70/4D/Qjuta6ap3dOHM6vcLL7qr6XGFTWu0D98HZ wWNwqWzTW1EQ5t9IMbfD5mooyg== X-Google-Smtp-Source: ABdhPJw7gmNUdJRW/ie9/w+A4qzgwiXpfa5N55uOpMPstvYCCOGoeKLjFY2iWDtA3rymDdhU6kxUAg== X-Received: by 2002:ad4:566f:: with SMTP id bm15mr3224450qvb.54.1623199475802; Tue, 08 Jun 2021 17:44:35 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:35 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 10/15] arm64: kexec: use ld script for relocation function Date: Tue, 8 Jun 2021 20:44:14 -0400 Message-Id: <20210609004419.936873-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174436_914896_CC512C04 X-CRM114-Status: GOOD ( 14.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 14 ++++++-------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index e4ad9db53af1..152cb35bf9df 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,5 +21,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 152fb00bcccc..f6b8ed04455b 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,14 +20,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -157,6 +154,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -177,14 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index df023b82544b..7a600ba33ae1 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 709d2c433c5e..9479d66f03bf 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -160,6 +171,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -348,3 +360,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Wed Jun 9 00:44:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07141C48BCF for ; Wed, 9 Jun 2021 01:00:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7F9B60FEA for ; Wed, 9 Jun 2021 01:00:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7F9B60FEA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WQ0nRu6SpBiAIor3gfvHXsNPEHjCNLFeAaGOI/2901c=; b=uMNuL4iJPuiLQ/ uFtTDQFp1LW0WeegEki/E7kEMIIM8AFM22uOgfiwc/TO0j9SwajJjfoefJfITcTUFkLTJJvYy8kax fiBJ/j17yeb1H4iXkUv3DkyOvN6nnZ6ExodncKhHecWsSHvdrYY/8jamX+58N97KAI/udvGpLciwm /ASbDQ/hX1FHf0SRsH5c9PrYqenaTAg3YRH9jWlncauwseVqv8NbULy7ri+9NxQ8EoV/QZqSTBQS2 ktHcsrwkUOJ6kKJOOImQLElCaC9a+oN7xtbBGCbAm1+4scrze07FqtL/n1QsHM3dnvN74ADUrVEkD DtU9yKMUIC5TrernrtnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmXr-00B2Qo-T4; Wed, 09 Jun 2021 00:58:15 +0000 Received: from mail-qk1-f179.google.com ([209.85.222.179]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmLi-00AvcN-Bf for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:45:40 +0000 Received: by mail-qk1-f179.google.com with SMTP id c124so22156085qkd.8 for ; Tue, 08 Jun 2021 17:45:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3eWeW4Y4iQSGTtocCXM4wkdh/+aIjk6bq3fG2ZZ6bS0=; b=FTVcVluVcfoqgmfRksxkpFIjKC8oplnrgu/Lw0adpEMSMj7nNKxh8VvY94kuyL8ktV OO2LERDmFUy4f48VnjnHqmsf7sdiWg6L4cG1engfKDYZgwS+1wHLQUOOLhmWvSIvPSY5 d/lRoYvyaV4lp7+WVsu/qg2W8X2xPu03ITby9nb+qnE96lHVMJbK3rDXOM081DaaBOxy zEv+TLG5gYpdovZAR0hJRFNNAf45bKGDLSbc/a2hTgo52BLDkKFWh5HWUzz8r1ikP1A9 NYsXiWyXOFct8OSC2EXYYMYfktezfNPve7Vf5pzFuZ6ogdVh68dxBc1j/HNXqtczk39+ 1bAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3eWeW4Y4iQSGTtocCXM4wkdh/+aIjk6bq3fG2ZZ6bS0=; b=QsfOVgq55m3PDlBIJEJ50/Ng/0R9pEmjD/Xpj0rCPxOOLVSiJWeWt39RzA7dqvD0mw zYSUF9NQWegIvLthPUH0DzAUzOQqMbNKxBj/4e8gblJNqG4mVgWmEEBC9dmDTn/xeYcZ 8sJv7a8+bwL6l6mWNpuL04g6LhyESCak8M9DscsXDEkwBdsUrFEFvhpqPl3ORcBAKRZd rMxxmmr9NY4S4dQUkyEVb3Yau0kQZN6tAwcPkXgbPJ8+1TCkwnpzvUFoEe332rdz7r1n KOvkLFvWuQ37S23fXBQVY352R4x6pNs4Y3EBaxzgE9GJ7qtBjRLVyC08rpYTuhSelQrq goBw== X-Gm-Message-State: AOAM533hifytNemVvhsiGL/w96sVgAXBYZSeCc1KPpyorNE/4vEpJj3Y HvfWTI0QxvSdivpuWT01JdguUQ== X-Google-Smtp-Source: ABdhPJzwDOMQgBDAmqMFpfnESJV4GkKJHC/ch2VZ1hUO9W2Vb2HB6gExRogLx8kmbb8CnNBgePIhRg== X-Received: by 2002:ae9:ed91:: with SMTP id c139mr1724569qkg.454.1623199477301; Tue, 08 Jun 2021 17:44:37 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:36 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 11/15] arm64: kexec: install a copy of the linear-map Date: Tue, 8 Jun 2021 20:44:15 -0400 Message-Id: <20210609004419.936873-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174538_424153_25A4DCB3 X-CRM114-Status: GOOD ( 21.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocation with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index f1367863d995..7f268bfa98c6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -447,6 +447,25 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU isb .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 753a1c398898..d678f0ceb7ee 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index ee37d0e0fb6d..f9794cd95eef 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -160,6 +160,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index e4b0de990dec..f349b3adecc7 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index f6b8ed04455b..4e796e46baa3 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -153,6 +153,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -169,12 +171,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_nvhe()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 7a600ba33ae1..e83b6380907d 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Wed Jun 9 00:44:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32E49C4743D for ; Wed, 9 Jun 2021 00:50:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03A9B610A5 for ; Wed, 9 Jun 2021 00:50:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03A9B610A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xpAGhZv9QZ0doBWraigwFQYTPU8zrTLvKPp/UH+yxnE=; b=rHhil73vOMmGKQ FNYjxg2nd1sQc0sjNtz5zw7JvnYI3R1Ltsxs4r27KAEXRRO4+UZ5epGCAABcDuMHzmWFAJ1JKjtDS ns0rJeTJpCTZQunGETn15ydrlYykiY1AN6PFKN6Re7WwvhpeUKm2AeAcmxYNMnE4iX3iTSYJvvRw0 y/uCBaOFFebI8IPNEC0LmgTUEO1YTCUs/53T1OdWf1lJ7+B3uvgGv4s7c+wzg/XBTmP/SgEy7qzvW c4eZmRQbfBSnEMnIQfyd4xr242/ClL5+KY1izFwBO0svlee6Pgd1pD7G+9k9h7rFP+eVe8toh0u5l FZdIPmeyx06mGFI2TvyA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmMk-00AwMK-3e; Wed, 09 Jun 2021 00:46:42 +0000 Received: from mail-qv1-xf2e.google.com ([2607:f8b0:4864:20::f2e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKl-00AvBA-Sz for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:44 +0000 Received: by mail-qv1-xf2e.google.com with SMTP id im10so11056345qvb.3 for ; Tue, 08 Jun 2021 17:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=iis7jLmMWJqrO4j8bEs42fC5zcjpgKAbVhRTVokGzVo=; b=hIZtdQG0l1hp6VYn3jLkRnYmqOspOKZG6tdUGGFo80N8ec7YlGXxpUBrWDY3B4OxKA iIsr0x85HR2sTdVQw5KeJ3V+HF5QdifZzV49tR9m1uqifviZTd16PwepNMwuRKg4bPl+ HaFll3WSZZlN2qmLLRchlnvwxeTd1zH6v/yR9/NImVmr7XBEPE0NChcfjy7F2jsNxhwr BYpqX+P/rUtSiHlKSe+n3OjO2R1dHiSlwr4FNSqGh4kGKnWigWhCsvx5Hyq5bk0dTaay gQSgXUuM2XiRVZS1G98++2cg+p0YzOaHXrJqxQ4fP9MGSXT+8zQVrLIclkN28L6JaEUK tMUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iis7jLmMWJqrO4j8bEs42fC5zcjpgKAbVhRTVokGzVo=; b=nYROjMXJuj0T4VPtCCr2LmHFQPxVT1B+uaCIj0cuFHZHFbuE3h5A9zfJ4lvKU6Gx1q 92oGKlMG3XVDg0qssTJzhlOEoaai4oj+OLhlQGoDdT8v4dUhkHSiQkJPjOCx8NzzU61Y wWdk7rt1tWBl0Qds8K+CvBf7JPRhSHQR2vmtZbYSL6ciVkRWUww9DcjZi13F8upouB/I cLGp4c3B/o82vFc85EQ+7U41+3CU7SYHektdQuI45A+GTfUzlGTOM6EQ2lkchLf3+VTy WL5NCprecD9N9VNSXsPr8JU0aiKlvLKnSQI4KePa42iHrSgjw+7n3wEMLiX3NCDahlJX WCxw== X-Gm-Message-State: AOAM533Opb2lsbfS9L/HqGLGIu7fadgBx1kOwoLhpr4XmwWIKy+Btsbp hl3xoYCkT79f8VKDHN/zwV6HIg== X-Google-Smtp-Source: ABdhPJznpfHdSaLE7Nj8U/+ZSKzFicXzREzb8C7HiU/eXa3XPqv3tCX/MIKaB+9KROa4cEA0gtbJKg== X-Received: by 2002:ad4:5343:: with SMTP id v3mr3232974qvs.45.1623199478651; Tue, 08 Jun 2021 17:44:38 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:38 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 12/15] arm64: kexec: keep MMU enabled during kexec relocation Date: Tue, 8 Jun 2021 20:44:16 -0400 Message-Id: <20210609004419.936873-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174440_035166_AFD5DDB7 X-CRM114-Status: GOOD ( 18.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 +++++++++++---- arch/arm64/kernel/relocate_kernel.S | 31 +++++++++++++++++++---------- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d678f0ceb7ee..dca6dedc3b25 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index f9794cd95eef..bbdf20930491 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -161,6 +161,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 4e796e46baa3..a2163ea24fc1 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -190,6 +190,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, reloc_size); @@ -223,9 +228,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -239,10 +244,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index e83b6380907d..fc2348584a54 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,13 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +41,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - mov x1, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + mov x1, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +72,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Wed Jun 9 00:44:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1D32C4743D for ; Wed, 9 Jun 2021 00:50:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B263C6124B for ; Wed, 9 Jun 2021 00:50:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B263C6124B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qymwVeS4o3E/1ZRxKI/I3gE4SVJmC8PvSCXxADuufcY=; b=eXWIUR9mMrKr+z Hbaiu7mVR9j6F9Gb0SnNAjZo9G/cWrvkdtX4jkV0pfY5T3lumiE+dy7JYoNPvys3G2oug14LW8AfH 3NnQERljqmSWTRjzjGqv9zr0zgS0G9PTYho9CFUk9uSyJZV9fPDQRE9lUTSukGqkjzPtA7/rLf+K4 uolIjXnAn9fS+SZFpvtHTYXSupkioriZjYKncFi79mMEF1WBHaVPXbKM67kJkA4/RL4xfkiA9QxQt sfXaq/YARA34kSbO6vkLklz1JPLqWv6K3f6lo84h3zP1YT4Q9VUAIzRgndcoh4FUrJ6t4LX55d9jX Z3Z+MnvAopXwC+QhgCsg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmMR-00Aw0o-G3; Wed, 09 Jun 2021 00:46:23 +0000 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKn-00AvCb-0G for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:43 +0000 Received: by mail-qv1-xf30.google.com with SMTP id if15so1077975qvb.2 for ; Tue, 08 Jun 2021 17:44:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vcjYajrJXv/lTpIFROIc3rQPTqnw03xFW9e2NsWvMmk=; b=SgxuxwSh4m1h9hRbLSuPfBWqYm1As0EuKsoebPFcfd02FoFxpOL3mT49jSoUnPLaoY xXYMG+tTO3C+N7O2Oa8tDi6s3ZDHgA3UT3Zd6sJ4IAddJ8riUwdeXtuSRY8OeoY6B9bN Qvg/1asVoH690WnwI+WVu/qNQPr848I0tiZ0Cyc8bOt5OyMCHQO+FBh4aacOcfQj6fDb ImY6AVSlcszk63W/OPJ/U1o/2NdSKg0m/IOcyH415Qun96VY7uwVn4/mZTzN4O/gXnVz aus3TnuWpj+obaFwUe1jmETnpKTJh9r58mmHR1KQwsYmxORiX/pUvrT1lCDV6ddrY1uo LOzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vcjYajrJXv/lTpIFROIc3rQPTqnw03xFW9e2NsWvMmk=; b=W/im84/Z5c6lNZGmNlo9Eb1lTXCWOs+mZ+z3t+37fxn6v4cxkVAzWvHAiR6pTJfQNs 02ErByGmzM6/KuqsDw74o6J3Srpc6Q8JLUR4hXK6FvHyIUsF3oQIWO+fVD0S92ZWUonY 0r4ABLMkRsT07qdKpXNKvmMjGUxXE5MtLM+bnlmNCkTExfrQ33xW3Nku0sVovdrJvkqs 2wDcQESp5XMjigoFFut917lJfKNY7coqKB4FcwlhV71hGXGpD9SE+SGKsXwfW3td0+aI OP0XaT4nZCG7PMYpcthWS15tYWwV7fjhALRPaLCTJ5fHX5fCySjJ7Ovh+mLeLLsrw1XJ 1gLg== X-Gm-Message-State: AOAM5321KiYIW1LwHbAaIi8+o7gWxyLYmZIYa7H4PdRIbIwIXd2bL5mf Illd1wLMsHp5S+RMcDGEuHdXiQ== X-Google-Smtp-Source: ABdhPJwHZbV81EmFC6ZnIEtukl3+Va3nM0ikxHKwWsJt6Ayst+Mc2MnTDU4ZHzbTNSVXncCD8AXVhw== X-Received: by 2002:a05:6214:21c8:: with SMTP id d8mr3294306qvh.35.1623199480026; Tue, 08 Jun 2021 17:44:40 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:39 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 13/15] arm64: kexec: remove the pre-kexec PoC maintenance Date: Tue, 8 Jun 2021 20:44:17 -0400 Message-Id: <20210609004419.936873-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174441_102193_90517649 X-CRM114-Status: GOOD ( 11.33 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 40 ------------------------------- 1 file changed, 40 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a2163ea24fc1..8f8962f53ba2 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -76,45 +76,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - __flush_dcache_area(kimage, sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - void *addr; - - /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -200,7 +161,6 @@ int machine_kexec_post_load(struct kimage *kimage) __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Wed Jun 9 00:44:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1BCDC48BCD for ; Wed, 9 Jun 2021 00:50:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8AE63613AD for ; Wed, 9 Jun 2021 00:50:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AE63613AD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=M4Drjpem78jWAoTzlwrJiuxvVp0QwpdDfRDvebYXCps=; b=tsloq+vumZ/PPB y0D1ktu9zaT/xBXnrCedoe/UsRncQfaxh+WxEK+AUL9XojJLIFYBFy0R4TYjRD0xCs0N5NmwqzswR ENilvkxn04n+2+GERxoHRBo4SDDf6VOLF/48etQRtepT3FJuBfcGksVF88yWH8Ko7DCRVg5U3Hvsa +xhFhOgmA8T+cHpeUQVUDxvbZcW6zy3ggGHIxFr9Z+oJtFXUA0O6QW/mWckCt+UIJz6vMyQXtWkqi 6d6BlxU5x/OnFde2IB7HQvB0ILJ5suczf8Iig31PiSuYn+jlSQRlnH3QwnceZR8GIZeo244QniRM1 lSD6Yb65fi/BnVJOalNw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmN2-00AwTs-N5; Wed, 09 Jun 2021 00:47:00 +0000 Received: from mail-qt1-x82d.google.com ([2607:f8b0:4864:20::82d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKo-00AvDU-EA for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:46 +0000 Received: by mail-qt1-x82d.google.com with SMTP id r20so1431135qtp.3 for ; Tue, 08 Jun 2021 17:44:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=dD+TD8PMfklj/Tm2LUNfDWuVRT2HSdSCWIOTmpPI7yw=; b=O+Ohn+hKzZXmnA/HVMExoBkeGSArjSolNHHV6X/vsjYGljOGAoFHyyvRDFaJ1DzgS4 8e8OBeZprYopNPRUeT+Bfr1O+raND6QEAo4AEaCR8Gno0GqMSYcxPtHKelB8fjRPHhtS 3L7U5JO+ulsYXlO+7H/ajE0iH92A4aHuQeoGyR3lemsxcLfVPzxsO1f9ysCQEG3hK4T8 kn7Tp8P+u8BcIGknsAkyqVaCKwsBskqTAldMeML+LwCDonzqBGulDV6mCx3ZKapWTPKd aH1f4Xq9eLhBCe/0ICwW/IpaG6wdeHjOskxrCj/mfAxIdvewjNsfVjhcSXGO3PsCdLIX x4bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dD+TD8PMfklj/Tm2LUNfDWuVRT2HSdSCWIOTmpPI7yw=; b=GoS7spH0WqqVFfOiu+a82xu9mUV6N5KDZ/lEOHAgXPxZ3niNSgOk1xzJBSgVSv8SQT KBUJQhAm5HQXio2Qgg0eUJb54AMDEXDs7CY52feg1F82aOcbl4j49bw5DOJyBEmeqgQn XqHqlGb6nqTl2fXqpqslRBWMt+Pp3h15EMsk2GaEl5RvN4nqN+HvkG4Kl1v84soiHwDD VGtpzPyLs4GIhao/fEXWQ9Zc1vEvdFHoT7hOS1MYX/q8BJj504/lRinxwncaPK+axuCu /RXkREumalVHBky/uceb5SSG2vCKJKLP87wsQsiR3s0jVyXjLt6fLpiwWdxEUFdEsHqv Y9UQ== X-Gm-Message-State: AOAM53134Hkyxt/cOJ2MCSIkZnkRz1R5FLMCM9in9CsNgihh7bdqhjLU khc/y4cHamCB3FUVLGX0h8u6sQ== X-Google-Smtp-Source: ABdhPJwYtcKFj/+EEy4UoJHm9+fpf+9phFDeQ7OyT+OER+lXujNZukaA4hrJWqqp0WZ3jMZ2lqx4CA== X-Received: by 2002:ac8:6998:: with SMTP id o24mr20140603qtq.113.1623199481339; Tue, 08 Jun 2021 17:44:41 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:40 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 14/15] arm64: kexec: remove cpu-reset.h Date: Tue, 8 Jun 2021 20:44:18 -0400 Message-Id: <20210609004419.936873-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174442_510324_EDE2C398 X-CRM114-Status: GOOD ( 15.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index dca6dedc3b25..9839bfc163d7 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index d47ff63a5b66..48a8af97faa9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) mov_q x12, INIT_SCTLR_EL1_MMU_OFF pre_disable_mmu_workaround /* @@ -48,6 +47,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index 296abbac7192..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 8f8962f53ba2..a1d98f42b741 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -23,8 +23,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -197,10 +195,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart = (void *)__pa_symbol(function_nocfi(cpu_soft_restart)); restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Wed Jun 9 00:44:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12308489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F4A3C4743D for ; Wed, 9 Jun 2021 00:49:56 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E81F0610A5 for ; Wed, 9 Jun 2021 00:49:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E81F0610A5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HZ5ypUCZAsBY+uJ49S99GU88u8IF1FORZUXl+znI6yc=; b=1DnQkA4VapzE7v 8118BBF1QGoEI1SOJcJqFlA/bmjUkm2yr9iqZSfAOd6XRCxNojTK1J4tergNu4vOPoT+FLrPhianR YAvbDUGrH7lgX6byhLS1cC4jzCFNtCywJtU0p9VyjEsVDaIvy5tDuxgOau7eJKg/QMgIeTsPuSWg1 40fSUK8ziVdIIxXVoQcbceZUEZWS9xWJNBrCbnpJB4AelxM5S1T1mTqQjeouIAmjhePVQqOOzi6al 0X0wM8DTdYDEtS/lEtB1dXTGkQgO5yAgvNYT4/tNEFO7RX6Gv/TXEIpDHG2sfQEcZrxDs9z5AYwL7 kyfJVA9hIs3vsxVkiCWw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmNP-00AweV-CI; Wed, 09 Jun 2021 00:47:23 +0000 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqmKp-00AvEB-MC for linux-arm-kernel@lists.infradead.org; Wed, 09 Jun 2021 00:44:46 +0000 Received: by mail-qt1-x82a.google.com with SMTP id o20so1346244qtr.8 for ; Tue, 08 Jun 2021 17:44:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GjSDlwdpWFRYVlnzm94wTdyVwlXZM6Svb6WtI86k1kg=; b=ZuuVM+oG7LEN8BGkaNklXAgBWK1R83XggxUaKuqIhawxrVr4hkcQbeGaRstDd0MhxZ 2y8rk1EO5extgzrNLXpnzxFc67giCQF2xjOgbwQTWSnVPm+j9YjSZJ/GLxXfqRPxsgRe 1LBBED3bhIDcYR53xtZBI+0Cx3RQE/za9PCCrkva6V0qwlF1d+kmT0iuEhX/bitmVS19 h0wh1vmTJ92P1Pdf5Lb51FdfztXyzfhtFKAcd7CO0KdxsDFA/yl/ug3nPS2bVJ/AG9GG 8rhd4otd8vwBPylUMO+VsdTRspg1bncZtbWNoZhZHQfvLPHMDXQgXmLx/wKAvsjUahOe xO/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GjSDlwdpWFRYVlnzm94wTdyVwlXZM6Svb6WtI86k1kg=; b=JPVo/mMKgppohViTaNgVYN2/hIdo0WZBvHItLVoLecX88ZUpqGJtxlZbVpAjk052lp +l/vk2dzVoFU4qepC0dMFPleN7kjyPJpQ9oULCJ7PuXV6wdBuNhhhBJqyloLArsbWvBx FfRaUTltGIeygxh1OPOiiNq94O7/KcZSHkRS6P808tGH7YjY9t+Xxzck6Lsp2dmCjJwI xf9JEeocAzyUGy1PV87J8kjn78o/xWD3fWYYFn5Zbiu1XY6G6GlP9A81fa9J8rB38JkV 2bQoxs5sfnLP6KyBgRePVRc8DSy8ezq4CP15DEhJFgRbxBL9pglUzwoFuRvf0I3ZqEw9 tTPQ== X-Gm-Message-State: AOAM532WTLeNp3MTDZ1Kb1E9QGXAI2QeFZo5BgoBCwAf7SfrldJnFY5p 66KQwYk/KPybT5T11psN3RG4fg== X-Google-Smtp-Source: ABdhPJzqtRWOkclgaHjthbeRbJ9Nm1ps+o1K7YT9wm8f0oWLxYvA3vqVz21USCX+0GNStAMjm0lKdw== X-Received: by 2002:ac8:5550:: with SMTP id o16mr11967741qtr.387.1623199482673; Tue, 08 Jun 2021 17:44:42 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n194sm12869011qka.66.2021.06.08.17.44.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 17:44:42 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v15 15/15] arm64: trans_pgd: remove trans_pgd_map_page() Date: Tue, 8 Jun 2021 20:44:19 -0400 Message-Id: <20210609004419.936873-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210609004419.936873-1-pasha.tatashin@soleen.com> References: <20210609004419.936873-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_174443_779015_BEE7041A X-CRM114-Status: GOOD ( 14.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The intend of trans_pgd_map_page() was to map contiguous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contiguous range this function is not needed Suggested-by: Pingfan Liu Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 411323bda7a4..c373d43c07ec 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index d4f0a6492d8f..7ae27a948145 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a