From patchwork Thu Sep 16 23:13:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5FD3C433F5 for ; Thu, 16 Sep 2021 23:15:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9263D611EE for ; Thu, 16 Sep 2021 23:15:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9263D611EE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aRIJWFBPXom/IUO0SnbXCaole1RPALtsTqkwZR9mI+4=; b=V1ATpmlvq/Hioa MzVWY6GNFJFdG7R01lymIPNSALie7T7jJOBtGDB6uqDh1/rHwE52vSKnpCCp2QciWR9rsX43jtrJ1 h0cYNnkkms3+hGfTRF977fqwsqf9GcXX1CBupm85opIjmtacG+IRy0fEk8AYoMM/Cm71vJiEC9Ill sPs/QXmZ+eXkaufo0YoSI9CX5m06ofBB5zY3FljL7iITnD3l7JpPe4jcSGc/6n7oXFhbXrQ/zywvG ieEU50xsCCZ3QplgtkUZvb41x++9yv3CXIQzs/Q2oqlz0CbCL8qsKISjcw2H8oEq75E5ThOwBog8U zgTIMGpuhgVfCEpZqOJQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Zm-00CaJs-Vo; Thu, 16 Sep 2021 23:13:55 +0000 Received: from mail-qv1-xf2f.google.com ([2607:f8b0:4864:20::f2f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZQ-00Ca3r-L3 for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:34 +0000 Received: by mail-qv1-xf2f.google.com with SMTP id cf2so5289029qvb.10 for ; Thu, 16 Sep 2021 16:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=izAVFpEvWmgGrg5dDdCTfojtmVMYt432wfvBnpESnjE=; b=GAeeTsTiMWgZ/mrEbLarF+Ykm+dqSe+pmGVZ0WDN8xE7kyy69Iebgu6cvmgXJVOXLm OGd58KGOD9rxyk1SsSDQJgtTAkcl6mqQQwubDE/olaD+l/gSVM2Wnvx6NobhbR5cGsCl CUv0H3cFXUKMbB3hSV4cpYfzViaySxUwAJhOlc61BhZYlJgBhcEqcGJ7KUf2Eq20YCH7 ui60w/IyJvPR7AEBxxUAqRAzMaJ5cTHKnGzHeP37FdyFBv9Q21yICC48ly/GiELsytCC jRkHtnnUDubxibZXVu3V6xMFzbfqa/xPs4N0smZdPjhJcrI83zk4WG7Kfskl7omyw6/M FJIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=izAVFpEvWmgGrg5dDdCTfojtmVMYt432wfvBnpESnjE=; b=dRtTo5uWEdkCQZOAj1tOspDy/BPJ7J8scfc8IfIBEnMPfmSnaDQSS+6TZfmwxBKCl3 H9DolyivU1X6ZEwNRZE1OquPYZi7TryG4UnI7MDqwPb+0uSkUPAVEZjTbUy29Lms+VSB JRRiR6lUj68bvaBHq042ZoIs4on70O0+iqxOP3p81RRaYZiE0pGgrd5lK6lm8rHYt4jA z7T4NjzjByR39p4htgCT0XJqrkCTKL2qjMfQI0RS/WJZv6oLyOKeXyXG72/pT0Zuw2KT +Xxpwu8b0KjT5Iks+3Lx2hk2WwHCFe3iAaWr+v9PdyvA9pECb7u5EYkoVWor5MVeTrlW MkAQ== X-Gm-Message-State: AOAM533vOZByYzdvDx3KN1RgqiwQuQLFcEbrZBYOz92F03+2WKESo1aB J4EZh+RjejiGd+GJEP8QewwXBg== X-Google-Smtp-Source: ABdhPJxDA+gY3OZD9CV/MvCWYphxLHZFgApqCPwPjnAgRNs7iMjwV+PWgCoWyHRjGBLVUarVMeiG2A== X-Received: by 2002:ad4:466a:: with SMTP id z10mr8048091qvv.7.1631834009592; Thu, 16 Sep 2021 16:13:29 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:29 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 01/15] arm64: kernel: add helper for booted at EL2 and not VHE Date: Thu, 16 Sep 2021 19:13:11 -0400 Message-Id: <20210916231325.125533-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161332_737856_798C8D26 X-CRM114-Status: GOOD ( 15.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_nvhe(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 2 +- arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 8 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..a9457e96203c 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_nvhe(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 9a7b1262ef17..81b3d0fe7a63 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 46a0b4d6e251..4c9533f4c0c4 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -49,7 +49,7 @@ extern int in_suspend; /* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) +#define el2_reset_needed() (is_hyp_nvhe()) /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 47f77d1234cb..d20620a1c51a 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -202,7 +202,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_nvhe()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Thu Sep 16 23:13:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7C61C433EF for ; Thu, 16 Sep 2021 23:15:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A273C6103C for ; Thu, 16 Sep 2021 23:15:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A273C6103C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/IqWtnrgVMebbBsyDSpN7KpIK0DqkoeKjzb8Tn+D+4M=; b=APUEjsRrx9ZX4o /iyXlH4odHIAFzaWm/8G3OAqy5Fx6Gn/sJ2NWY6v+cmJPduQSevN5d+zgrQQhL2esc6APk0MbWWtR /sFl4m8qFocoB2yxu7XiUPohv6zP/FztJNLIIKlhI4ionzFKiQKP9i9Rq2sG8rs7TT2Y1F3UqgxzA OVPWA7mAmdmxU5igFvRIn1wcyWhUlQlUlm+jNioNCP20Z9gdtydFkBcSACYgrfx4Fkgc1P2m+Zvwu evm8csZDfAPH5uhIMmpgskZmO6QWUL2ltcVxdt2cD/PDCHC9mPspxVxVSMhS3R1+/qPvOcSb5Ehy6 Ote/YKD6krAUhRslkOWQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0a0-00CaN6-DY; Thu, 16 Sep 2021 23:14:08 +0000 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZQ-00Ca49-Iu for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:35 +0000 Received: by mail-qt1-x831.google.com with SMTP id g11so7172068qtk.5 for ; Thu, 16 Sep 2021 16:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=rKX/fIe3nU0g8gcrpvpSLWOlJ7PIrw+sk/25DNY+zbg=; b=JYeS9elwctyC4RXuKoev66TQ0cE9rJLK3JYjNnI4WDAjP+HTa877mSRi89ECV4fzFg h2U21sdz7Q6QV8aKgI+p7a9nU09cFSH/JAN1A/cRtO/C0JkASS4MI9QXhwjL5C+isKmH b91pr27As8Np04zLuVMh6opOMWl2PqI6b6nzMcDMtULN0RdMtehjiEZiNuodgEFmorxa f4Zuam41IbbLw/oOUkQ16FCoohWM9wofICT1JTNXS3TWBrr9B2Rs1G0QTCmmhge3zWcg QK37lPgO0Kh6+sZ1xg8XYTKAa57/oWCjWTzkfV8PBUsj0+vKezvo9DIeuSGihA4XBsz3 XE2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rKX/fIe3nU0g8gcrpvpSLWOlJ7PIrw+sk/25DNY+zbg=; b=7LjwcvLq9k31xNhlF9W4MrDgn/epQOay3Yg1nmDlwWzQJC3ouVMzlo8WZmvF8arR2h TqBtL/ZGOEGOQQ/ZfpaPPIcPoFSt3tBniTMO+frKA6RfabPp2Xnt6JLqMNHaatNHwP8d 1ecVFzS+DY4FkCoWJqpYJrq091Lmk66+qeMmnn1bw2+n3U3C5oKratMemmKbu8kQ/yvB rLFn06Jm0Y62cHubDs6M2mXOJtiWbnnFp6nwKnV1SNt0sudnh0KSm9CpUSy8IzIVkqBC aDSBUc+ZuNRAdU97ns6Idgqrhfedl2oFyQiDxwZGhMlWpJuBM8Z9BHlG41GCq88HqIcq sYhw== X-Gm-Message-State: AOAM533MUHo5aStd0fVAXBXSRft/gGCcHXHwzr7mw0V+fPjSpjFkyCOS yR6hpvyqk5n4YMG/1XLrQyu9Ow== X-Google-Smtp-Source: ABdhPJwjEj9m/CIgRx9txAhuquXbm5DxVwMv4OtomzrP5spzFFScdOT/yz1BgQBF6JdnzQjo+Lk4Ig== X-Received: by 2002:ac8:7c52:: with SMTP id o18mr4782438qtv.168.1631834011035; Thu, 16 Sep 2021 16:13:31 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:30 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 02/15] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Thu, 16 Sep 2021 19:13:12 -0400 Message-Id: <20210916231325.125533-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161332_675213_C5C0B14D X-CRM114-Status: GOOD ( 23.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/trans_pgd.h | 9 +++-- arch/arm64/include/asm/virt.h | 2 ++ arch/arm64/kernel/hibernate-asm.S | 52 --------------------------- arch/arm64/kernel/hibernate.c | 26 ++++++-------- arch/arm64/mm/Makefile | 1 + arch/arm64/mm/trans_pgd-asm.S | 58 ++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd.c | 27 ++++++++++++-- 7 files changed, 103 insertions(+), 72 deletions(-) create mode 100644 arch/arm64/mm/trans_pgd-asm.S diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..7b04d32b102c 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -1,8 +1,8 @@ /* SPDX-License-Identifier: GPL-2.0 */ /* - * Copyright (c) 2020, Microsoft Corporation. - * Pavel Tatashin + * Copyright (c) 2021, Microsoft Corporation. + * Pasha Tatashin */ #ifndef _ASM_TRANS_TABLE_H @@ -36,4 +36,9 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + +extern char trans_pgd_stub_vectors[]; + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index a9457e96203c..3c8af033a997 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,8 @@ */ extern u32 __boot_cpu_mode[2]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 81c0186a5e32..a30a2c3f905e 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -112,56 +112,4 @@ alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE hvc #0 3: ret SYM_CODE_END(swsusp_arch_suspend_exit) - -/* - * Restore the hyp stub. - * This must be done before the hibernate page is unmapped by _cpu_resume(), - * but happens before any of the hyp-stub's code is cleaned to PoC. - * - * x24: The physical address of __hyp_stub_vectors - */ -SYM_CODE_START_LOCAL(el1_sync) - msr vbar_el2, x24 - eret -SYM_CODE_END(el1_sync) - -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - -/* el2 vectors - switch el2 here while we restore the memory image. */ - .align 11 -SYM_CODE_START(hibernate_el2_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t - - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h - - ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 -SYM_CODE_END(hibernate_el2_vectors) - .popsection diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 4c9533f4c0c4..b96ef9060e4c 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -51,9 +51,6 @@ extern int in_suspend; /* Do we need to reset el2? */ #define el2_reset_needed() (is_hyp_nvhe()) -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - /* hyp-stub vectors, used to restore el2 during resume from hibernate. */ extern char __hyp_stub_vectors[]; @@ -434,6 +431,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -461,6 +459,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (el2_reset_needed()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -473,26 +479,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - dcache_clean_inval_poc((unsigned long)hibernate_exit, - (unsigned long)hibernate_exit + exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (el2_reset_needed()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index f188c9092696..ff1e800ba7a1 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o +obj-$(CONFIG_TRANS_TABLE) += trans_pgd-asm.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o KASAN_SANITIZE_physaddr.o += n diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S new file mode 100644 index 000000000000..8c4bffe3089d --- /dev/null +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +/* + * Copyright (c) 2021, Microsoft Corporation. + * Pasha Tatashin + */ + +#include +#include +#include + +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + +.macro el1_sync_vector +SYM_CODE_START_LOCAL(el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS /* Called from hibernate */ + b.ne 1f + msr vbar_el2, x1 + mov x0, xzr + eret +1: /* Unexpected argument, set an error */ + mov_q x0, HVC_STUB_ERR + eret +SYM_CODE_END(el1_sync) +.endm + +SYM_CODE_START(trans_pgd_stub_vectors) + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t + + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h + + el1_sync_vector // Synchronous 64-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__trans_pgd_stub_vectors_end, SYM_L_LOCAL) +SYM_CODE_END(trans_pgd_stub_vectors) + +# Check the trans_pgd_stub_vectors didn't overflow +.org . - (__trans_pgd_stub_vectors_end - trans_pgd_stub_vectors) + SZ_2K diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..26bd8f2d95af 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -5,8 +5,8 @@ * * This file derived from: arch/arm64/kernel/hibernate.c * - * Copyright (c) 2020, Microsoft Corporation. - * Pavel Tatashin + * Copyright (c) 2021, Microsoft Corporation. + * Pasha Tatashin * */ @@ -322,3 +322,26 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &trans_pgd_stub_vectors, ARM64_VECTOR_TABLE_LEN); + caches_clean_inval_pou((unsigned long)hyp_stub, + (unsigned long)hyp_stub + + ARM64_VECTOR_TABLE_LEN); + dcache_clean_inval_poc((unsigned long)hyp_stub, + (unsigned long)hyp_stub + + ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Thu Sep 16 23:13:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 836B0C433F5 for ; Thu, 16 Sep 2021 23:17:50 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 500036103C for ; Thu, 16 Sep 2021 23:17:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 500036103C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ic8fEhvSNX7zwK/taJa+p12JEuSOQ9NCIn7RCvRVSi4=; b=sA05s2eN37YF6y ZRXqF6IUCi0Jgq2trxiktJ3McMGGX+DwENWBIQsmfGb9mu6AQ72udhN2ppqYhUjN9g8uSL5bKqhUP S8St/vpwppxhY80HlSujns7uzEEgtY3G5QGR5viVRvYMChCG2ZCeWU54rrBRtLKeKQzy2RhDB0yqS FkP2h0s8CPCY/+ei+nvoHgQdXKDyXnhOTYf9zCJ1w5ak3JQgFqXCTwVKD6TU+giOZL3H3MAZbrdBp mE6NHeSKmjYhLwYgV+8OydM+3imJLUro4val2iDQJVd9D5mn4rzgPWaF1DDRkiTOxgu8XOplnHpmn dp1TstCMZTKsY8989aqA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0bj-00Cb9S-8S; Thu, 16 Sep 2021 23:15:56 +0000 Received: from mail-qt1-x829.google.com ([2607:f8b0:4864:20::829]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZW-00Ca5q-NL for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:41 +0000 Received: by mail-qt1-x829.google.com with SMTP id s15so7139373qta.10 for ; Thu, 16 Sep 2021 16:13:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=vATAyacEN1RBZ4iY9UvfQznDv8XKXbprxPlso1XrO3s=; b=gZB9fgDcFd48xLYIrxyh+JGpbXTeFbm0ji1GBX7tm82vd7HFkYtLoMTXTT4IhnMlZ6 5bLx5mqq+ycUoQPIfP+lE4fs9sQqeYiIx3OdOmj1BVHNmMoSVaFjRhoDV9x7oDyQH0Ly EcUgOsZW+ExaRa+Ifjsyu9ljpRI7YFva8jH4Y/B14osyeDOz1EojnubioGqn4HrfXVmN QQ1EoGD+L/mRut1G/X5rQcThsMTcwmlq9t87oSm8rHbaGaEq0T6MgYJQ2PXCCfnE4KBC EzalqM8ViXMhxDwIfHp2282OL0rNPf1h1YEXmGPzTHQS7OWvI6nbW/Ow5/taEGoYgX5P jbqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vATAyacEN1RBZ4iY9UvfQznDv8XKXbprxPlso1XrO3s=; b=j0xH+TUQCyv+lR1KHaxX4Mwk+dY33baklSERlmf9SP/YFtLa+joOVF5XgfdMZi7aDr Vj1SIC+a9hSujDUjhQArOfkqC2Ml2elFd4y4hZjAj5lEPz8w0Z29+1dgixpM9ICflVpS QiR+ycHShd/ZSJa6tAXhBC8wXR38YQ8r4lrtY6TSLG+Lm2HiKrrSLGRZzc509FmHlQYK CNhRy88LdKrTcXaJYqFOrs0FzobErS0rZNm2wAzhzHuxyj5W6CKOL1w/qXk9Dj8ArZIS 6Vx2PkvY6awyBwWkcVAanfmu1jcqGSsCMoT+fukOEN47LKxB1eeBVkOaFy4tSCKny1t2 0Uzw== X-Gm-Message-State: AOAM5301Vd+5K0HxTSvb5l71axEq5mzeQyz5/PbJDgzoxKuP4Tcy3Aih pCRho3JBl+Jk2FkB2k2MRmdjnA== X-Google-Smtp-Source: ABdhPJysMPROcVY9uVmXIYZZutZ8Tz1xrsOad/eRErKKPAsu10LVzfxhmRCZ46+JETnnHyeGrvmtDQ== X-Received: by 2002:a05:622a:199d:: with SMTP id u29mr7515959qtc.357.1631834012463; Thu, 16 Sep 2021 16:13:32 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:32 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 03/15] arm64: hibernate: abstract ttrb0 setup function Date: Thu, 16 Sep 2021 19:13:13 -0400 Message-Id: <20210916231325.125533-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161338_836312_B67EDF86 X-CRM114-Status: GOOD ( 15.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functionality when relocation code is going to be idmapped. Move the setup sequence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index f4ba93d4ffeb..6770667b34a3 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b96ef9060e4c..2758f75d6809 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -212,26 +212,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Thu Sep 16 23:13:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 294CFC433F5 for ; Thu, 16 Sep 2021 23:17:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DD96260F6E for ; Thu, 16 Sep 2021 23:16:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DD96260F6E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=P82JWHu5Lh6XKhrKSiNVT9hvdDp4jsXHp0Ncer/LdYw=; b=s2ZlMBplhSDTl8 f+It30Fw8V+7KIrEdpnvMfFZHH26RBbEw9aFh89wdCXER39rM8zMuUh1RZlvudbr8vpbgNr3sXQaa P9kfzJaatVuT4goe8OPoNkyLV1Z1K2tGZrwgLBewZhbg2eKf/Mm1tw5uYcFScD3EuE8wzSJ169Kzr W6JFLiyPZJWbclLqFEcOBfpIVd0gZmmXSPcuitRkRwAMa6fQxEYBJBW+EyQldt1b98dbyGUzY9Nle g1+GaUbdaEf6dMidToxVlmLY6WVU6o0mMS0JL3zCm/YCXNF+71XSa2fw4uNlw92T8GDqJqef03usT 5KouoQM88W211uNzE4pA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0an-00CahU-JU; Thu, 16 Sep 2021 23:14:57 +0000 Received: from mail-qk1-x732.google.com ([2607:f8b0:4864:20::732]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZV-00Ca61-Op for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:39 +0000 Received: by mail-qk1-x732.google.com with SMTP id f22so11662821qkm.5 for ; Thu, 16 Sep 2021 16:13:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Gd/bGj4Nwzpg1oqGDK3dKyYdSLy7Idt++VqAt+4ZWi4=; b=PdS3WIztg0/FRRtRRTtPPxtFIldbe++n6HCVMM6V0yGOpBq5XsN2nTiu23qMmZdl8q UA5t8YTZtjM0INsMm1j5XpUeJQCvNSXi4Rq+hRSx8IZ2tKHgAeE6k1wOiFqeKPRpYTGj zRK1b3KP5CjtP6keZb3NU4vaQeXPW1AEYLAEn2/dz1SITPu6y8Td5DSwFsE+M2382rk0 cfy3A1wF4r2QdsnvixiMLxjZsAm3BnZjF692O3QVY4jYrCvOzUBT/aeQWJUMtPNwhut0 mSdS9tjivQ9eqe5Ke36xs36CO+CdTVkXk7rAL7kWFu+iMdYsA/i/iS9kUia9rdNF6KX3 BPOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Gd/bGj4Nwzpg1oqGDK3dKyYdSLy7Idt++VqAt+4ZWi4=; b=dZ5eW5gBzpPpNDfg21aL7fOxhlUaCEVchkS+5uaVznt6kSWx3Nu4kBcXKQS+JKT0Vk PEKnrNoMfYL22yYgpdU/4NeTGoag9WM12kPe6jvBQQKAsvHwiZ0Jln9JhS5EUyKu2yTm 9AdnerCv+9qqFCvYOiAuLWEXooKpmuBDEkS1nJWeHXdaucjzNWaqOHrWq1pNzcVR5qBh dz/kqawaVRCQM+N3YSUp/2lN3MIedLCulKxajiGAKCBCwWSLtc40OB7oqKMeKHVE4YXR oBVjFcA8bfwjM2+6VQ94oFkkjMwWT94yyvwtGQkFraWWuvRSyQ9Lm3EbQIdGFwefqJJa pyVA== X-Gm-Message-State: AOAM531VtpB6nitmZvje5/gFLeanCi2HqVt9sP0WoVq27+HtofrZ8Zrs F2HgRkkNr5AgjBBw+AnoCtaIBw== X-Google-Smtp-Source: ABdhPJxpsmcX5Y/ALpzZd3SrKNGuslXcDYHnRWX9AFYJaIjAKYK0WUYvvI7zcdvzBL3J+s6eb2VL7w== X-Received: by 2002:a05:620a:1998:: with SMTP id bm24mr7898451qkb.444.1631834013914; Thu, 16 Sep 2021 16:13:33 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:33 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 04/15] arm64: kexec: flush image and lists during kexec load time Date: Thu, 16 Sep 2021 19:13:14 -0400 Message-Id: <20210916231325.125533-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161337_852113_C106241C X-CRM114-Status: GOOD ( 16.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pasha Tatashin --- arch/arm64/kernel/machine_kexec.c | 58 ++++++++++++++----------------- 1 file changed, 26 insertions(+), 32 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 213d56c14f60..b6d5a02cba2e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -60,29 +60,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* - * For execution with the MMU off, reloc_code needs to be cleaned to the - * PoC and invalidated from the I-cache. - */ - dcache_clean_inval_poc((unsigned long)reloc_code, - (unsigned long)reloc_code + - arm64_relocate_new_kernel_size); - icache_inval_pou((uintptr_t)reloc_code, - (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -163,6 +140,32 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + dcache_clean_inval_poc((unsigned long)reloc_code, + (unsigned long)reloc_code + + arm64_relocate_new_kernel_size); + icache_inval_pou((uintptr_t)reloc_code, + (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -180,13 +183,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -261,8 +257,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Thu Sep 16 23:13:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEF8BC433F5 for ; Thu, 16 Sep 2021 23:16:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6D4DD60F58 for ; Thu, 16 Sep 2021 23:16:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6D4DD60F58 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AWjbxIHjeXO3XOCv1SKGdbhcjADnF7oHV1rBI5c+6P8=; b=JM9+lktdpULHxH 0kdTdp9dGI4OIEsVLmMwy5/yfFmCEiSIbWS8Ts48lL1jYnPWVuIcVeqyCOcciniFt6v0MiyMa9ji5 LrsKyoSCU6xClGrWkw81C76Yv+WNSvvAqihCNa5RSbqUUeAjfTx0UZ/gy1zr31By1oxYe92MK/GC9 Y57m6G3uAQ+9bjIkaBItQnvL0JzPtWYUKPNAkogfVW1QDVcZpbEtIWC24PyUyGByJhf9bTMrPPZg6 1JcoJFFUweJRrJuVPbf0HVhpDqeb2Zq5MKe34CDhy/iLD8ULng6wwyNedT0vWaWGkxL8tRH0wj4cu mjiNXcYjjKsFFJ+PS1Dg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0aO-00CaVe-Eg; Thu, 16 Sep 2021 23:14:32 +0000 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZU-00Ca6o-33 for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:38 +0000 Received: by mail-qt1-x831.google.com with SMTP id s15so7139448qta.10 for ; Thu, 16 Sep 2021 16:13:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Q0X9wptyBwxDna881tcar0OeMQAxIOwfXu68NQUnafI=; b=CC4algdnAtsH9amZNnT+KVVlnw33h5Jdh3h05XmPiUN9DejKe9AvFtHTA396RlfRui B5H1ETmA2DzpAANiNzlbZU4YZ2uJo876rTr+aMD6WhKXS15ifxbdelf3URW4ex4Ox7cw Z6Io6u8SGm1V0sTmcBg5MWGbWmana7higW3HVbquVxsXVtlPrZr3pd6tBsjrnK0fVO2W HrOZ/as1QBTwiNvsRNV0xyuI6xADyVZvxMmCL9NCErppS6GyuiICDGm/Bj5+zqxHUX3o I3YV5awOsn9yp4YAEgotCZ5q/YoPRoTEGPRO0jQmxdlY+wST4bGalHWc4TvU2F5QpDs+ Eqng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Q0X9wptyBwxDna881tcar0OeMQAxIOwfXu68NQUnafI=; b=ul3oodqlU3uzPi5bhRbiyAm9tWgItsc2wehhjvqD48h3vKV9F0ksIAsyPvFvuoeW82 vJ9L62A5+ORBCpXOtve30STBXKcndqA73OWyUjxBChqvY4aV7yhHt0sL29xHdb0aE347 K3rv/Y3eSePjkY2HbTNfjE8U8fRf+3AZ7TfwswBrWPZMPG7WSdOzSRPbqnKs2vKShiWe WT9RvmTlXftIWmU14xUijLBXjQWkw6nFGV5AA0VfteQHBYJfxQGb7JmeVzIfWP2d13hD uNo1YClnan4ABYb8nXn4g6fOW4N5orblze+Rk/qOI386jjHHGjWj6o6GfvSzK8S3NH9k oYZw== X-Gm-Message-State: AOAM531aahFqwUsfZNX47zb2zQFT0d28OVdx5wZhLGaKfp+Wvt8vwhMT m1Cl9aEozbnYMk6EeSVw0oMYWA== X-Google-Smtp-Source: ABdhPJw9vRYrw4iWZuL7eob/6c1iVPoPfda6H/N97kzq3jld260E01UPVSLZmJD5zdsp5ECEl7t4/Q== X-Received: by 2002:a05:622a:18c:: with SMTP id s12mr4045444qtw.62.1631834015346; Thu, 16 Sep 2021 16:13:35 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:34 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 05/15] arm64: kexec: skip relocation code for inplace kexec Date: Thu, 16 Sep 2021 19:13:15 -0400 Message-Id: <20210916231325.125533-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161336_217536_A0606083 X-CRM114-Status: GOOD ( 18.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pasha Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b6d5a02cba2e..7f1cb5a2a463 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -144,16 +144,16 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, @@ -162,6 +162,8 @@ int machine_kexec_post_load(struct kimage *kimage) icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -188,19 +190,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Thu Sep 16 23:13:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8CCBC433F5 for ; Thu, 16 Sep 2021 23:17:12 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9781B61029 for ; Thu, 16 Sep 2021 23:17:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9781B61029 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jgHIHbWJ2cvvQGn0uKl6D5EdLkS3kwi+5UXx8pQPyTw=; b=cPSSJ4Sl8Qkk6l 4qH63GcSInxTLEpuv+5pEJx4fntt4IFvmlQJ6JoJOk1i0TutGRLVnnu8aQb+lvr8Ol2rOXT/pG60n dcbNATdvFQphgrZH0VUl0J/vZ27DZiMMimsiWBT9iev8qsKTMvlJv4Sd249c1AbAQdAT80IIcB/gq 9ontAL8QhkWmaS6dNJLQ/MtAjczN/WjoaPgX4mdYD8iZPWommtJBpOg5fwR/ZnRySUkXxfJls4Hrd 7A6kIc8vdt+ScebYY5UV/48kIqlrRp3eMlrgqXgYOra78KU1pkSQoJMQEvdVXkJdgM/mm84ljUUFE jYae8iBgn6Yp28U5vlnQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0bC-00Catv-Vd; Thu, 16 Sep 2021 23:15:23 +0000 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZW-00Ca7g-7t for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:40 +0000 Received: by mail-qk1-x729.google.com with SMTP id ay33so11583825qkb.10 for ; Thu, 16 Sep 2021 16:13:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=exeHdjIsxyGc1OZ7fPc2yLNva0RUnn224iKjfeO/Tps=; b=csh+0NjNjXq4xVbHcgfV0tkZaYVDDxnR+KyTpl/TMfJf1Bcqjh8iGnlSYcecebap8G 6k/safqs9PVQDTztgaPl40nThhrfZi5ed4IK8xePB0T8Sa8MYs+aXBV//x3Fck3k2iV9 6a5XjOPT7Lwu25MAGF1c0z2M3Lw/EqJOfJm5d0CoIPTkZth5NctzNRz9VHjYtnSocyrL j31/z+Tl0HcwZVo7n036pWZYjyzxJP7gLbRfZBIkFGRKsYORdyu9Krb9oPdMxFVUemfr nJA+7UTbw0/PsRDqMlZmUc/nfCSJ196huH92bsDYJpqgAJN94ges86+cyx1CJDz6CLvi fijg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=exeHdjIsxyGc1OZ7fPc2yLNva0RUnn224iKjfeO/Tps=; b=CQTjCDDW+BWwM/rmkjF/7zxsKtRM1sOGgI/SdLo6/xjcep/h+T+lf3vl45LfXjOke2 5+yolimmCG00bflX21WUsQ6/4UluooLqTZ4iuI2ew9LpjI5Z10fvYDnC/rk5LfuxBewD 0X5YKEmhYbn02/JRZiDSe9hV5d/bZj/750q0eKCMj5+x7I8JOXi6z1M56HlLc+EKnhNw jMyMkh7sFGiCjrcRmZlfqA4H/nBMV1hSJ616TJUx2GZDFWzpohZinj8ss+uTPjjfAgoC 5IiVq4N/hhE/NnSqlZTSkPB1yZphZsWKJ3NjkqMevPysRcjhn5JdauHBWd4oK+A0d9MZ MTTQ== X-Gm-Message-State: AOAM533N229Br07EvWQDLgEpEFvpiH01zSbW4LW07YtNLnuTsbMPNslp DNdGGBiu3ZAdP4ihJWM6ivYUUQ== X-Google-Smtp-Source: ABdhPJyjBUpS0VkoacHlSf0TOrxrLfXY2aEqHlMPIXlkIZ7kOCY/abeaLzU3ssMdOIXsPkRjrkDvCQ== X-Received: by 2002:a37:68d0:: with SMTP id d199mr7411119qkc.96.1631834016835; Thu, 16 Sep 2021 16:13:36 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:36 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 06/15] arm64: kexec: Use dcache ops macros instead of open-coding Date: Thu, 16 Sep 2021 19:13:16 -0400 Message-Id: <20210916231325.125533-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161338_439441_379812DD X-CRM114-Status: GOOD ( 14.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitized DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/assembler.h | 30 ++++++++++++++++++++++------- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 26 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 89faca0e740d..71999a325055 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -405,19 +405,19 @@ alternative_endif /* * Macro to perform a data cache maintenance for the interval - * [start, end) + * [start, end) with dcache line size explicitly provided. * * op: operation passed to dc instruction * domain: domain used in dsb instruciton * start: starting virtual address of the region * end: end virtual address of the region + * linesz: dcache line size * fixup: optional label to branch to on user fault - * Corrupts: start, end, tmp1, tmp2 + * Corrupts: start, end, tmp */ - .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup - dcache_line_size \tmp1, \tmp2 - sub \tmp2, \tmp1, #1 - bic \start, \start, \tmp2 + .macro dcache_by_myline_op op, domain, start, end, linesz, tmp, fixup + sub \tmp, \linesz, #1 + bic \start, \start, \tmp .Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \start @@ -436,7 +436,7 @@ alternative_endif .endif .endif .endif - add \start, \start, \tmp1 + add \start, \start, \linesz cmp \start, \end b.lo .Ldcache_op\@ dsb \domain @@ -444,6 +444,22 @@ alternative_endif _cond_extable .Ldcache_op\@, \fixup .endm +/* + * Macro to perform a data cache maintenance for the interval + * [start, end) + * + * op: operation passed to dc instruction + * domain: domain used in dsb instruciton + * start: starting virtual address of the region + * end: end virtual address of the region + * fixup: optional label to branch to on user fault + * Corrupts: start, end, tmp1, tmp2 + */ + .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \start, \end, \tmp1, \tmp2, \fixup + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..8c43779e8cc6 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + add x1, x2, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Thu Sep 16 23:13:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5DC9C433EF for ; Thu, 16 Sep 2021 23:19:24 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A904461029 for ; Thu, 16 Sep 2021 23:19:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org A904461029 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SfGAvWsn75a3RlTyhYff53uuJ+IMe4WXDVlauURmJNo=; b=ZjkiR+rEXk56+b HharrCqaUt19hyFeem8pfKOHlgsY1tLnI8k+Czgyj1q7f9zY0FpRAOWQYuZil0uH0XRGRXP6/UQhi Tj8RHOT0FhnsyzorZe/QNIbL/VGKR8gpuCljcdw2l593UxpAqtnvZ43cTIG2onRCLPwCAym/cIluB lrs37K8beF5VFzeVN67iktoyi097lzXpWAbCFMEQX8cfngenX3xTYHvnY+OYkVNWa3RCcEvJSV/zd nfxIi9YAZr4hZRSeLkADMuTJxbNEvp6mgxFmBs1GnEbgd9vAqo+6QEEMHTiHDtbjykJtHkQ0QbQpY z42FXKB5v027QX74jtqA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0cR-00Cba1-SM; Thu, 16 Sep 2021 23:16:40 +0000 Received: from mail-qk1-x72f.google.com ([2607:f8b0:4864:20::72f]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZX-00Ca97-Ne for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:42 +0000 Received: by mail-qk1-x72f.google.com with SMTP id t190so11653169qke.7 for ; Thu, 16 Sep 2021 16:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BQux9XsHP7dRxDZt08eGPFQfNHY8adQiq/RCf4mKhCc=; b=NoOj5f6yoU9HGkB256tuDMAQ+oQg0DyOgXuLeSabqfzpjeS+7uNGL9VmF19o+suZG4 P+8q6IFIevz5R0FrLUfKl2PXuke2OJ7YsQ7gpaXcP6cRzKJIExHQXG4MwsYuJ7zM3+R/ yzzoHDGEEmvUq6h4MWsssOfat3nUtPdglWdytijV1K8C2dr/UMZKEwJRMOgLTI/ZpKfk 82Rv/1ai89aWXENaXB5FEw2ZvywuM0DeJ7M2PTD9CaUVeEJtoSfs3fB6scvNcVby04xC 6vWJyN3x6L+UQepyd9BfZQEO/m9Pil40d4cq+NnZepTUfaz12E/t8QGCIQ2RHj9+BsL+ 7NoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BQux9XsHP7dRxDZt08eGPFQfNHY8adQiq/RCf4mKhCc=; b=BHN/sL71fcCsnf280eXW3xCuUhoowLdN4HZzz6FwNb/POrsIUYo/dlPq0Yq6oBR9mW xGlNjQL0WPlj6t14hsDvsh7ZyVdWoYzQvxbKBcI779T3klvFZDl5YVC3827FFSBgVsJs MNyLkCa9aTrH3JnggstYSzORv4fmgTLiyqb7d/Eja7jW1FyiIOytfPfGc/dxtRpMV/kP FBaf034TYifbiXUX7Xxzsy/N8tJEMB4Zpa11jq1DbGhzOIKxgHV6WQj2ipi1EnM14W1V GjI3RLQtLye41upF+mRIgbpVjNpRvZIqtyzOa4I6FstEN4yhIOM3lBfcIcMk3JORjxJA 5FNg== X-Gm-Message-State: AOAM533rArP0wxbuIe3HstgZw08glvSBy8dDZplLYpoQkSqirNqNEijz hqilIJXPtAnRUNCJOGAlpVbjvTltGnbigA== X-Google-Smtp-Source: ABdhPJx+ngjPujbpkT75YECKrfmJxJOEJsaePp8x2rrmF2+9A4AswChr/Jh1n8ne7MSEYUPNLgftWA== X-Received: by 2002:a37:ab0d:: with SMTP id u13mr7574675qke.521.1631834018299; Thu, 16 Sep 2021 16:13:38 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:37 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 07/15] arm64: kexec: pass kimage as the only argument to relocation function Date: Thu, 16 Sep 2021 19:13:17 -0400 Message-Id: <20210916231325.125533-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161339_867501_01581C47 X-CRM114-Status: GOOD ( 16.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Signed-off-by: Pasha Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 7 +++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 16 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 551427ae8cc5..1d3319c7518e 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -170,6 +171,12 @@ int main(void) DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); #endif BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 7f1cb5a2a463..e210b19592c6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -84,6 +84,9 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + dcache_clean_inval_poc((unsigned long)kimage, + (unsigned long)kimage + sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; unsigned long addr; @@ -206,8 +209,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8c43779e8cc6..63ea19868f63 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Sep 16 23:13:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A3E0C433EF for ; Thu, 16 Sep 2021 23:19:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 450F261029 for ; Thu, 16 Sep 2021 23:19:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 450F261029 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=n5rZZv6/OVRvOKIxVtrp6KkbnXV9d+5mHW0jcEcAXw4=; b=WhFqe6ELAeyO3R L2uiBIpQ8Gf8XsTwrzdjfLiAlUVL1jUCkEBv3Q0H0dWjtV007VZFhiHndd5dwkti1527Qv/TQVioc TSGRpyDjpfxcE0WlnxMkcDuJZfDI3UJG2aMOwronyuLnymlLBSmKT/Lmae/NdZ2w4sbYIMV+8igEm RMvBihPCjh7r/EC3QtRwqJvopy/Xsbcn17E1V6mPuiGvfrGU1636dr4aqVRpA98TMr3xID1L46Jzz f0z1Wg8v5ERJv9MiAcxXsn2YLHR2pAJAYsk7ibz6Fg0wwMICqmwmrIc5VWnyf7XJdNJ4BVDsJV+vj OvGXP2yX9NZnPjD3scSQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0da-00CcBD-QQ; Thu, 16 Sep 2021 23:17:51 +0000 Received: from mail-qk1-x72c.google.com ([2607:f8b0:4864:20::72c]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ZZ-00CaAQ-92 for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:44 +0000 Received: by mail-qk1-x72c.google.com with SMTP id a10so11619112qka.12 for ; Thu, 16 Sep 2021 16:13:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8FUJFzqt29N5Ss1p+4onoJEfNw1xT2RtYvA0/vFG3Ko=; b=hIV5fEKR5jcZGhwihAmoefv0odw9Na+zUKVUUo9eozF8S5Cey8VtRfJueopds858/v GlOm8Lt7FcxWc4iCjmNQ5KznCqnwh1ljctkQUiPtu+2vHeMJcWEOA/SEw+mR9ydMYlcw P2ZHM1wrXPXtsWAaFtK+SwEPjfYUnyAd0z2FJFmu+pFdP1onrdi59DFa9SLnEQB8mkDg LNNW2/pqZpqajvVin9+qS+JM7faSMtUfjQbyVS7CprFbhXTySDMZYP65bN/t+BhjJM7H sTS7i7mRyZ4s+GhlVZq3khzp9NYeb78WcgkW/SWd/8yx5RmtJGon+lmIu9v8NDeRMBoc hdPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8FUJFzqt29N5Ss1p+4onoJEfNw1xT2RtYvA0/vFG3Ko=; b=cjsGGz3LjLQ1MPL9dAJuRKbajg6258Nr5Gj3e+3fe93XTOui98lAFoV2/M7WV/DNcu NVMrrB7CI4ufV9aS6w4I43NnWcPvDQH20cMA91K7dZsCPbMswvgW+Es4N5v42VsDMZ3u zRP4L2tB4pR+t9x1AgjYRLcwLYmpVXeLg4JC0o41nA3chejB5joG4A1LhmgkBR7YdFHE 33Pvgi5FVX/gQfjaiRd+i7tT0L0x1xoviaDpMgoKpialLGh5WU9S0DobSUPNjD10/Bia XLfYrQutPPdELoGtFw61AQr6v2x9vWia8IxQg3HmCKApB07ToHb+JTjf74qWFxXSCBTs eT/w== X-Gm-Message-State: AOAM533jB7mgN//fXFr/SLz4/QFsmeTV+6IAg9TGwT8RKSv+qZknqNxc dhuTtE1ne6phd3Im/beXij3zNQ== X-Google-Smtp-Source: ABdhPJyfnlWIRJFsUYimyJ5KPZVFQyCX1ni6ObAwNp+ZlPmmtFtbIgx0LIG/c7E+o26ReAwVoiM4Tw== X-Received: by 2002:a05:620a:66d:: with SMTP id a13mr7792659qkh.102.1631834019717; Thu, 16 Sep 2021 16:13:39 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:39 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 08/15] arm64: kexec: configure EL2 vectors for kexec Date: Thu, 16 Sep 2021 19:13:18 -0400 Message-Id: <20210916231325.125533-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161341_368124_1164D471 X-CRM114-Status: GOOD ( 19.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preparation to MMU enabled relocation, configure our EL2 table now. Kexec uses #HVC_SOFT_RESTART to branch to the new world, so extend el1_sync vector that is provided by trans_pgd_copy_el2_vectors() to support this case. Signed-off-by: Pasha Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ arch/arm64/mm/trans_pgd-asm.S | 9 ++++++++- 5 files changed, 42 insertions(+), 2 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 5c7ae4c3954b..552a057b40af 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1135,7 +1135,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 00dbcc71aeb2..753a1c398898 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 1d3319c7518e..6a2b8b1a4872 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -174,6 +174,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index e210b19592c6..59a4b4172b68 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -43,7 +44,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -143,9 +146,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -154,6 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_nvhe()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -209,6 +238,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_nvhe()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } diff --git a/arch/arm64/mm/trans_pgd-asm.S b/arch/arm64/mm/trans_pgd-asm.S index 8c4bffe3089d..021c31573bcb 100644 --- a/arch/arm64/mm/trans_pgd-asm.S +++ b/arch/arm64/mm/trans_pgd-asm.S @@ -24,7 +24,14 @@ SYM_CODE_START_LOCAL(el1_sync) msr vbar_el2, x1 mov x0, xzr eret -1: /* Unexpected argument, set an error */ +1: cmp x0, #HVC_SOFT_RESTART /* Called from kexec */ + b.ne 2f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 +2: /* Unexpected argument, set an error */ mov_q x0, HVC_STUB_ERR eret SYM_CODE_END(el1_sync) From patchwork Thu Sep 16 23:13:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 080EFC433EF for ; Thu, 16 Sep 2021 23:21:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD75561029 for ; Thu, 16 Sep 2021 23:21:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BD75561029 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eEkDGl4IFEzru7xTqPxtQnVSLxboeO3cJr5Ged2WN6c=; b=fqc+cSUiGE4Lb7 dpXmX+LUnXKZIymFwsvFPQeOc3LVMFY2iAoFINYzLSu+q83WuV5mvnoBdj6MDdVrMdISuwuKbanMH hvNgnNrGzawxlUM5Z5Mu9o+UQdy2u8Y9gAj2TWPlI0uf0howS2Ws/JxhWSLcGZLWBQg96wRthVMEw nvSdWMu2frWrzXnGN1e+HCaJmphmQXopHGruaGqmcfiycmvjo4gP8DhH3iLRJ8uoDf0E831QTrpMi TBPiAuShcmCRoWUzjnBoQWWXEnr2gZX098HW+SEKs5I00mm9aASsmkQ2+kX5R10sRZ1FBqKF5WgAb 1m1j7SVSSD3Cb51cfEhg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0eH-00CcWS-Lo; Thu, 16 Sep 2021 23:18:34 +0000 Received: from mail-qk1-x72e.google.com ([2607:f8b0:4864:20::72e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Za-00CaBg-L5 for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:45 +0000 Received: by mail-qk1-x72e.google.com with SMTP id ay33so11584274qkb.10 for ; Thu, 16 Sep 2021 16:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NWsMjir7JLvcnjD8xhnB6n3wImzlZNO2jGCGH+AGQL4=; b=cB/9gAHcI9xMOZMGhUHMmoaZ4eX9ZrPDip4z8OjM0AZioqvYvr8GL+MRm6GnudDDNx G5BDmCaZBnOmiXHQZk8ZBd932wVBTMnFyT8sauWkYq6K9u/7xeWnNj8UhO7OJYvjYftR uc/o15lKpVu1WgNZg5Qm9ouL/178MWWpFwiC+fHOfQy1x4IVEQBtoEaqHHBM516RKyYH RSsyXSosIpPE7JwRtT7ENWoaill1to/kSd/F+/gXXDAlRgQqU32S8HRHng/VS5zAm+1S TT4vhAi8GfrQabgkyNPRnNTMUNO/wj4iLrCQ2E8RZ6KTRgbOVjA1adfYyHqkmFYaTIde K33A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NWsMjir7JLvcnjD8xhnB6n3wImzlZNO2jGCGH+AGQL4=; b=N8j2FDLWcT+hHhQG3k9Sg9KtJsXSThBCGCmo4tk4kW90PB0O/gm/xdQC1U/+radD3L chjWHh3+rxdeiUxwGNSwHEV4NwqM2Wm/nmobs44Vd9WbXpVtu18MyD93e63K6TjboziF GX30ALvSQPJZrEt772sMzo8k3DHb6M9JV8njqBjD1qLXRQEObpv8QBcc9uqfWHPi5URm Hz8kZQR6vvCQFyRBbkR9lpwsyeTc10e0MqGxlVqK3BWJ9z7aNsL0UvDiJXsxoFjZEMlM t3WRPgY3vAZZ9TUaHKM62Y6iXrYL4RJ7Ciw2XPag6bUiwRwLdJeofwDh0gXS6BEoXVMG n8pA== X-Gm-Message-State: AOAM530V9BDpooSkwcr53d5odn/fKV3iALs1HawvJav6U8em7v6JAx9g zfqKXsT6jH91L15BT86QF+RHCQ== X-Google-Smtp-Source: ABdhPJyr/w2wgeq1H7NcfK32k0qrGhT2vhypkOOkKoRwzsQC6ihtdXUgyVzeBL/bHBxHZgee8ex8RA== X-Received: by 2002:a05:620a:4404:: with SMTP id v4mr7806522qkp.344.1631834021164; Thu, 16 Sep 2021 16:13:41 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:40 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 09/15] arm64: kexec: relocate in EL1 mode Date: Thu, 16 Sep 2021 19:13:19 -0400 Message-Id: <20210916231325.125533-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161342_772607_407F7FE6 X-CRM114-Status: GOOD ( 12.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before entering the new world. Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 81b3d0fe7a63..296abbac7192 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_nvhe(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 59a4b4172b68..cf5d6f22a041 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -240,8 +240,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 63ea19868f63..b4fb97312a80 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Sep 16 23:13:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D163FC433EF for ; Thu, 16 Sep 2021 23:21:58 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 96D1161074 for ; Thu, 16 Sep 2021 23:21:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 96D1161074 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=q6dDEuZozDcj/XYtW71FotAjM//6/1pXoJdjiEWUzXs=; b=1Z1bmpqO3lrsyf vWuoDJOjG+ggWhFCIfd742SXJEI1JtcOGey4ewYq5TlwsXJG3M2AZGJJkD6I3meaU2NW4ZxcWddHf uZjHe7S1hiZprzU/LW5ahzYoZzEHxFDYyDziTuQnM4pY03jsRVFnvAsDJLG4pMYSHNU9iAOHOcOzK DCyuXln6M34OlwGQb8ATV7V01YwS3rt6i5ZQNtdMIKNUlokQ+JINJBjGVUPItmuM1KK4ftnY1X3+l EoxWgKe66ixVLpqTFrfZi0akr41D8/kt1qdIhDPprRUc/ibdS8EKwXZMpVlknVlfR2KtTWiGDFukA gYxr5cPeWZjnIffs2bkQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0et-00CcqH-TI; Thu, 16 Sep 2021 23:19:12 +0000 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Zc-00CaCf-0o for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:46 +0000 Received: by mail-qv1-xf30.google.com with SMTP id w9so5278254qvs.12 for ; Thu, 16 Sep 2021 16:13:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Ax+khFLMo4ZiuU9FVutUfzu+NTMgGKavPhnStp8tBHw=; b=PbkZVHKG1lzCk69ODfdNdcgHyqZquLzFuzVT6KKZbcORyICTzwb+CxRxFl+5dybvEe z5P1PAlMuXnW8gDxKtYN+y0VFpBI/4JZD836aPq+Utuovdu67emLZ1VcVCUaMdzpSVz+ jfj3a0cCtAqkUVzTY2rw6CsM66A1xZIFxR2KsV+OEVGs9vK9lgMNVT10JUej/4xxxlV+ MkUCl6eEpYQ6ZYv2h2qH8N3xLG7l1HWudXMoAu5ghDCXRUhezxdJzB9CtCgdQJo7XP9Q PyP3+Hp2uq7may6hJyp/yc2W8oIDztRnQWLXlPgBjWkE8QqDyv86IlK99GHbmABoXA1f Gl8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ax+khFLMo4ZiuU9FVutUfzu+NTMgGKavPhnStp8tBHw=; b=eRAiGdtQ/QOhnrEI8fDY1LWRxlJpQvA5qccQXO+zWiRNUeSGatMHBvDp2QFsN3ArIU v1Q8wwsYlnVAthp9hBeJXqRsto1SwGYqFCHmFoE3Ox2BtRoO0y1tvkG48Bo93bwP1qL5 dQjEjnsGoZavONEQjgK12FvmWm2964A4WvkGczKky4EK44DOVH8KeSbc/ZM/k1oU8Te0 o6XsY9eZdN20fwK9n7S7Q61bWdCAMprUqWKFU7TQOofhZynf08DdSDyAfcYiBcBdWWw7 tuXBkQ3wKM0g7Qv+IalmSZxdlMZYmvlvpmtgYGZMtXfai5UnqB+yqCpZZw8tqUZWBL6t 5Rgw== X-Gm-Message-State: AOAM531azLC5hajdcz9ob4X9LgveLnpHXbuARbBfPQBZt7D7p/UdMhwd bpanE+77821ymMv1ZOH6q2prJw== X-Google-Smtp-Source: ABdhPJxdt0Gno/JIaUtNR1NRx65yQhLTr3uYqpdqpaJPl0uNatREDKubCc3nBMJAWCg6/G4DMYc9CQ== X-Received: by 2002:ad4:5583:: with SMTP id e3mr8239009qvx.42.1631834022594; Thu, 16 Sep 2021 16:13:42 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:42 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 10/15] arm64: kexec: use ld script for relocation function Date: Thu, 16 Sep 2021 19:13:20 -0400 Message-Id: <20210916231325.125533-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161344_137642_28ED2CB2 X-CRM114-Status: GOOD ( 15.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 16 ++++++---------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index e4ad9db53af1..152cb35bf9df 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,5 +21,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index cf5d6f22a041..83da6045cd45 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,14 +21,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -163,6 +160,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -183,17 +181,15 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, - (unsigned long)reloc_code + - arm64_relocate_new_kernel_size); + (unsigned long)reloc_code + reloc_size); icache_inval_pou((uintptr_t)reloc_code, - (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + (uintptr_t)reloc_code + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b4fb97312a80..9d2400855ee4 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index f6b1a88245db..ab457b609e69 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -160,6 +171,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -348,3 +360,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Thu Sep 16 23:13:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F145C433EF for ; Thu, 16 Sep 2021 23:22:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF4D0611CA for ; Thu, 16 Sep 2021 23:22:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org DF4D0611CA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7AlY9bFA9zcCpvKZ3be+rxI/dg5agaBsLrgThQn7PEg=; b=pEL7OcHwohFfru 5ybvIzLKyH4DPDEqsd9oPPsq0qQKAmO877DSzWih4u3jwDViV7lcGIgrOMaiyBqWm1OSQ1Y4ZqYoB RnF4NOsGoJRr3+ghvPZt8M5NRRQzntlJl39qWjySsHp2UOuwm286x62AJKBu21D+GxOk0WK+RgTaH LpeaKKr9BOVQ1Eqnl5EZ+KkaCvMi39issFSZtZbWJRh6QGUPY4shJvSq4FdVSyqAxztk77tuQExNh 14dxWBbwLCsscCFrd2CeN/gZcX07Qezu0U2EfZuSATW61kkGEBxRX8PHhX4rw1gUbphxhsFxU0DSt YUdC5kGcBy/fly4aB4+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ff-00CdBx-II; Thu, 16 Sep 2021 23:20:00 +0000 Received: from mail-qt1-x834.google.com ([2607:f8b0:4864:20::834]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Zd-00CaDe-IL for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:49 +0000 Received: by mail-qt1-x834.google.com with SMTP id g11so7172433qtk.5 for ; Thu, 16 Sep 2021 16:13:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=JDZrEF/sCDnWcKlUfW5zJu5PS6tO5sZA6oFNkHkJDRY=; b=Rc5YN7zxFvRFzl/O1PJTtlOE7q/0aG1hbqNxGTnAQUocHjgAVlL+c1Z9F6T4TFbyqN tIxqNmujTPV7vP1tXrI4hw5+sWHwwhkj7dkO5YBMrjibcMQF1SY6d0/gA30xprUHSPrK 5bpHgtcy+M0Ye9xVi8oMNq+EF0EjcZrGrcSbBmLGy5IOxg3vxoFxOexVk9DeSdp4u9QC xLiw2O7dzu6wSmtrhSiw9JLowsz8FLRaDa05whhIw6IOJD0D6leHB78kf3Fg0Rp0abSG XNEY2vIkJAVE4G/uzSkQ5NjNprXzQ4XgO7WVHPmZFRQx7hi0H61+bdlqSJx3HodcOiZH DV/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JDZrEF/sCDnWcKlUfW5zJu5PS6tO5sZA6oFNkHkJDRY=; b=KWjORbsAukR+ltS43/40JzMulpQP8d6qhTWF137+uTuyiwtRW+PcCwkwIycF2Dzujx jebJlQ4rFP14H3PH6KSOhtk08P0BpWF86dWnDJ3e7M6nA5q/wMF06LVszgNQgHo445P/ LRBhipI++UaNql3sw3HD329ftf3f9fW8GAT9iCIbEkMD0foKwDt2Vu4LEFyC6OmLWiyL YQcQmcYdhhZbutQS4LHVRIyAoOTp5wphZJv+YXb/nmnqzyvBMMUY7TaAXkoFY9kn5JzF GPidWn72wjmTNgXZOsung3dfqe3p2rmYTiss/6aBrEjw1fDGlNux7EYbWCEeYxkfqDIQ Tgqg== X-Gm-Message-State: AOAM530mOe7L29rln74yOgEqVmYd1B0DBpE0h4hXXg7yWXS4JkICNdyb MuwnhvJp4KKCkDx0b7F20Mc4OQ== X-Google-Smtp-Source: ABdhPJysH2s+WIL4w7gagdK4FBh+E6tL7ZSWKhm5aEaAVpwtx+nGBt3pKhCQPXRi/rFizb+cLcakOg== X-Received: by 2002:ac8:7354:: with SMTP id q20mr7634239qtp.329.1631834024082; Thu, 16 Sep 2021 16:13:44 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:43 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 11/15] arm64: kexec: install a copy of the linear-map Date: Thu, 16 Sep 2021 19:13:21 -0400 Message-Id: <20210916231325.125533-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161345_761572_186949FA X-CRM114-Status: GOOD ( 20.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocation with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 71999a325055..4289c4e1c2a3 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -483,6 +483,25 @@ alternative_endif _cond_extable .Licache_op\@, \fixup .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 753a1c398898..d678f0ceb7ee 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 6a2b8b1a4872..1f565224dafd 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -175,6 +175,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index a30a2c3f905e..0e1d9c3c6a93 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 83da6045cd45..50bc0a265c86 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -159,6 +159,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -175,12 +177,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_nvhe()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 9d2400855ee4..a07b737533c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Thu Sep 16 23:13:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A5A7C433EF for ; Thu, 16 Sep 2021 23:23:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 023E66120D for ; Thu, 16 Sep 2021 23:23:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 023E66120D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KLT9EH+VDyQq2AQbj0fncOw1UIj1quqeaOA8jD33Pko=; b=0j7ozSB6kzzkML wfL0cSNuYL/PZFDihtPXYVx3WexePLTwAXA7oGiTlFbLhATzNXU8k1d8d5PlelGIiX5pzMl4NLr7Q 2YS49aP4hS8/9e2xwdCtufKAYvC92bI5Us5+y5fUtIE7UEeFDRVW04+EHr9fDFEihRti4wNflG+QD ab42AbeKd50qSjldpe1Y6Rl7VZdhtDf4Ttteea+yA9D+p0POs9eBxgk6GZUtFzFS1+gcvpYrj+Jo+ sd7SxONK/hPY6UW6z5O2pJRMDztdVMin3vyknZSkuf7/7eg3mkWTCamiOg3Zzu0G6Bj7DTDRM+C3E 0p42A6eM4qPPNRZY5lkw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0gP-00CdTc-1C; Thu, 16 Sep 2021 23:20:47 +0000 Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Ze-00CaF4-De for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:49 +0000 Received: by mail-qt1-x836.google.com with SMTP id c19so7152279qte.7 for ; Thu, 16 Sep 2021 16:13:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=yMGg0Xfp2pZ14g18eEM9KhKltW20J0qZ6gJK+wfsCCg=; b=CDQRBv6RvkIPz7g0exzMSrPJb1aTTyhukfT9426Na/M20vICxvuR0EtgqiXleG1bS5 Z0w79ZypnkSuSaRt4pU0V+2OigHjC24vTtN8mdL45eMK9Rvm6S8MTwmvH6Af6hwDnWYy jBKPyNw1RuwcfE+qMbCJS+xZuSN/uP9zVXuk/f1NRT5vVpndbkBfRzUD4pfAh4CIT4X6 Q4H1l5pUACNKdq+0P0leHe09UmLP/CxQFALFD8qJVIwWM0Xck1Rs0C2cijCwH8rV8gr0 ApsjxdJBJVhG9AT4MigNjrtynI2THzcAOdMhbdnA0GorOleyYDqQ51n1X0/PBwcn0LRB L38Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yMGg0Xfp2pZ14g18eEM9KhKltW20J0qZ6gJK+wfsCCg=; b=0j4Ga6t4sgZycM3bvk3HTDXLLvF8b1jcriNooL7h7fsFon+h0cHaKeTKc6IVb73Otb Q+RZQt52zFH4sM+aQO2B8mjgddd+yAyTlJ6bRAhuQgsmlawTNzrBeJyTd5MwlWX5dESv BIqnPnirGzW9NBPb2j208sVGALKrsNupZpJYoJBxNB66cNrZTu75uBPtBdMQA+AB3oRa jelgDXAw4A1ao8gSILLemgm0TxCRqa0zZewCwskpe9HE1O5Kpu8vcXp+A6btHUTKn/n1 aE1OyecBfA/5u14PY2OKWm/bAAGZZSmhzD26Fd08k9pbmXBQS7R3BqdP/ktUdtRV8Jm6 zN6A== X-Gm-Message-State: AOAM5326d2M2elnpKROAnwUjiaGH2pglp9zvFBiAOTtSNkYzXxMv0MIC +CUzX4SG+9Dqh6IsUOSqkwYzYA== X-Google-Smtp-Source: ABdhPJwFHP3dox0FIdAIZI1L1iarZ99wM48APM5ODIljbr26Mhd9BWYCw3e1zfbpOFYRGIWFgQOMhw== X-Received: by 2002:ac8:4159:: with SMTP id e25mr7666220qtm.69.1631834025548; Thu, 16 Sep 2021 16:13:45 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:45 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 12/15] arm64: kexec: keep MMU enabled during kexec relocation Date: Thu, 16 Sep 2021 19:13:22 -0400 Message-Id: <20210916231325.125533-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161346_612199_A2E33E49 X-CRM114-Status: GOOD ( 18.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pasha Tatashin Tested-by: Pingfan Liu --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 +++++++++++---- arch/arm64/kernel/relocate_kernel.S | 31 +++++++++++++++++++---------- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d678f0ceb7ee..dca6dedc3b25 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 1f565224dafd..2124357c2075 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -176,6 +176,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 50bc0a265c86..b43195a7fcb6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -196,6 +196,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ dcache_clean_inval_poc((unsigned long)reloc_code, @@ -230,9 +235,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -246,10 +251,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_nvhe()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index a07b737533c3..cdba96928401 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2021, Microsoft Corporation. + * Pasha Tatashin */ #include @@ -15,6 +17,13 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +41,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x1, x2, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + add x1, x19, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +72,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Thu Sep 16 23:13:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAF3EC433EF for ; Thu, 16 Sep 2021 23:25:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B2902611EE for ; Thu, 16 Sep 2021 23:25:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org B2902611EE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0ygblUODVOJH8FcqACjyq+38rImzmF+MjRbNCSoTtdg=; b=z4KGyT41z075ru GDK2Bt3NRT+yz9yBCj9KrPkkKJlJz0nxgyrtZ8suygGSTVPLZT1cwwzZ0FGrMJ+kQBPJ/H3fALePJ ixpIPW7IsviQ78+UMk8FNVjhCwebM1oZat2bp4Z5JiKRZqL/75WOk4DWxr9KdcUKCx0A9zJY96MLV fdXtmeaJSIEATU1owJVPXfRunxJAewUtGuvl5e1ZENLAoSpCI3/C+zZDdMDi7MMXMEbmQmGyrS202 Nzjj+m/qZxV8yRUB4VbHM9M6JYfArNEqxbvTJJNj45PHcBL8Gkp+tjZnd2jXPazrYQueyWmePb0fH fpjfpX0GXpfy8FEX6fTA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0ho-00Ce4L-2Y; Thu, 16 Sep 2021 23:22:13 +0000 Received: from mail-qt1-x82d.google.com ([2607:f8b0:4864:20::82d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Zg-00CaG7-EW for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:50 +0000 Received: by mail-qt1-x82d.google.com with SMTP id w17so7138303qta.9 for ; Thu, 16 Sep 2021 16:13:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GN2IcTbnQy/l+AFlkLqiV8UzoxXsj+8ZLQih7s8aKx8=; b=M5CTfZo1Z3z8nnNhnSFnT5bZBoCyHcCSNQY5IAC70LR1ZVyh+O5AiCH+3JYzT54AO6 cogP5W8eYZVIR2HOkx271Tj9VX8PBKz8/eNdidBdVg6QXW33cttuUdaqwjaZ70DOvRyQ urh5mgNU8A9pRPwtSyQnngaEuDTYYjvPzcqsyq7+Zt33I/s9aPLeGoYRv0xCRcZPFjAg rw5EHVkik9ia9CQVkBGQ0DyL76YQByOc8x2ktd7fEnxBHGxeswnzAPZNf/nDH85hlAML 5haphvkFDZ3lgxc1HuLEOZ4EcyLLdUqdW35viL7w2GR0UsaCv1Z1s8rbSdWn4NYvDQh6 vtig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GN2IcTbnQy/l+AFlkLqiV8UzoxXsj+8ZLQih7s8aKx8=; b=XBF1oTjZuVowcBLrzTOXrxGK7M+2BmV4VUSyL92tqXxpE4nOaMYq8uh1NW3Fdpr8B2 dFjDDPhLz9gg9WRYNxNGkauAfTDc9M8peGXBvKLAJ5C+jh2gebQeqbOMCu1aK+5+FcaB BhKc1NCi729UtmbQhg1ALc4AcqHP2JS2Ltut3VIPRVYG0qR902YTjZweR7oZ2vEZlJ8f cXP83l/DXrPbt446TpUUO5ZGjBl+kLYCmr0P4gc/v+UwJoHL9UqDllDk4CTSp4wlyYOg 4gGe9LmC9SCHILEHnQv5SFS7gcgjGfiMPQyIHfbZAxVMNZ6dOvfsEMZmOoQGBGEzb7AI RDbw== X-Gm-Message-State: AOAM531BAq8mU5d16aB2t30vbSTs2H0/A8/JaD3z5qG+S1KiUdcTqvam PHPQ/Jj+W7MbjgTDcmNhDwaVWg== X-Google-Smtp-Source: ABdhPJwIqEQH++TUi7ZgjSc7lOCNqi10ZjOdZ+AKQdZt+gJhzCLC515q41J5bOiRYru4kSW7phljFQ== X-Received: by 2002:ac8:4149:: with SMTP id e9mr7629536qtm.249.1631834026997; Thu, 16 Sep 2021 16:13:46 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:46 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 13/15] arm64: kexec: remove the pre-kexec PoC maintenance Date: Thu, 16 Sep 2021 19:13:23 -0400 Message-Id: <20210916231325.125533-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161348_530567_D06D4B4E X-CRM114-Status: GOOD ( 11.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/kernel/machine_kexec.c | 43 ------------------------------- 1 file changed, 43 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b43195a7fcb6..b1856f34cc51 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -77,48 +77,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - dcache_clean_inval_poc((unsigned long)kimage, - (unsigned long)kimage + sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - unsigned long addr; - - /* flush the list entries. */ - dcache_clean_inval_poc((unsigned long)entry, - (unsigned long)entry + - sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = (unsigned long)phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - dcache_clean_inval_poc(addr, addr + PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -207,7 +165,6 @@ int machine_kexec_post_load(struct kimage *kimage) (unsigned long)reloc_code + reloc_size); icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Thu Sep 16 23:13:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FDBAC433EF for ; Thu, 16 Sep 2021 23:26:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 52E22604E9 for ; Thu, 16 Sep 2021 23:26:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 52E22604E9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0mb4LRU2lqWP/uoTDw1tbJIijuWtvKnK8/mV68vU0QQ=; b=WcVE4ypFSrbRMK rgWJyB1JBuR8UPeJik8dxb6/BXI21jxSIe7FErlTXdjJvFZiI4wsRcG9mvK8P+Gw6l9d4kxibm1xz 3hntfwreYZCVmmBnRgkhrMPYjTkqXtpX8U1THLu/IXEOD3VbHlQFjqYaf2WbQe2eJ9pNYm4d1Aswn n0/+oE66NZvMmApzg3rBiTOTXMf9wBhNAnIrLwc3wftk7Afz0largOZJBS5d6AaFgKF7vvnACD/pV 7ZJNoWatOcjqUgVpM/9Kwwiaw5NqTRPrYosmoSEFYRJnmbtk9pO8UOmrIcFOV8lwuF92J9pirFaOj ez02f7E1OF5X+/agWWnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0jA-00CebQ-Oi; Thu, 16 Sep 2021 23:23:38 +0000 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Zh-00CaH4-7E for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:51 +0000 Received: by mail-qt1-x831.google.com with SMTP id r21so7139005qtw.11 for ; Thu, 16 Sep 2021 16:13:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=7N5nVHzB6dwVIUX0EUd0g/YbIEPJbmQC1LjYPlojIKk=; b=mLpqe1KaYQMF0cLBRzJ45xIIqNHIVBadFlMTXXvcP/DL6J6XCYhX74U44EBDufy4ce oOPKIl0jWuW1B972+mQbVW2HKYLnhuAhozgdt9HJo3N8FWne8glStby4hooCqfP00nXI Bl2MSKg9JOu/Yo5Nu/D1S3uDZqwkKKm2I4bRQEHqCbttLbtiNn7fnjRHbSDW9wFOBBSU EpZkNHivP47/YGJ5UIUnfekArrWeqIRb2ShLmYRLM5agEzscVgXk/6bEyEgcQ0iNwapC yYPa5RjNJ1eCkZVuiwIRs3cBm2bo17ClU8EaAZjpXBNzS2zWCk2goeGXX9MtEHeHBZIg ntcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7N5nVHzB6dwVIUX0EUd0g/YbIEPJbmQC1LjYPlojIKk=; b=kDptGzM04PwykYy7r6FaKLwjwWjq5JPIboaSXHTiftBLZFOhWOknIrnW9JxUoUd8sx xu4Q591n9/7ry5Rf5vEiuDMEeCO/dDf/w9kn3PZkmsAPkVYyU/OIkPnkmME0UwIaMW9S eEtdC/UAjaXUsaOR3CuUwcr8lZte9GWHGdURixnyzDzjknvVNYhs8J7KIoeSl9aAT9yJ /VAaXAGvQmOYW0UgHUzaNp1SVxUPMsHnq11DyeYtC4V+cyHyIuy2MLVFD0kPZY7WJXhA FkPk7MvVFhS4LYtOR3nwZhDsGOE4xEGQ1N69J9QojvKA5WL3+CKVDJsNqEWQjnns5GGh zphg== X-Gm-Message-State: AOAM533GidY9KetfLT7/V9hsO0bHeVLY1TNyBYXM7n8H/mFgIaeK93Ii sF9KI4L4jd0DK6s3sh/WChy0Vw== X-Google-Smtp-Source: ABdhPJyIuGt330ZUOfC/l/iZXTFzzUZyl4ceVHMabmenoyk7fIVAEKa2yGLWG9gwg8yBpK8tkWOCBQ== X-Received: by 2002:ac8:4818:: with SMTP id g24mr7438906qtq.328.1631834028420; Thu, 16 Sep 2021 16:13:48 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:48 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 14/15] arm64: kexec: remove cpu-reset.h Date: Thu, 16 Sep 2021 19:13:24 -0400 Message-Id: <20210916231325.125533-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161349_350662_C8C18066 X-CRM114-Status: GOOD ( 15.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index dca6dedc3b25..9839bfc163d7 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index d47ff63a5b66..48a8af97faa9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) mov_q x12, INIT_SCTLR_EL1_MMU_OFF pre_disable_mmu_workaround /* @@ -48,6 +47,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index 296abbac7192..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b1856f34cc51..6f1da3fd5320 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -24,8 +24,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -201,10 +199,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart = (void *)__pa_symbol(function_nocfi(cpu_soft_restart)); restart(is_hyp_nvhe(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Thu Sep 16 23:13:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12500515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7ED4BC433F5 for ; Thu, 16 Sep 2021 23:27:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 49459604E9 for ; Thu, 16 Sep 2021 23:27:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 49459604E9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=o9/j+kCgptvNeVCI9nYFsbIeb482t1j+GlhSUrVzrXw=; b=rRKHPmdnidbli+ wkJX4ilSJ0rv3KNChAVrKDSyva7mmqk86rMkcWRUHa4KoCtcDkjbNZRl3oxXigewwFTOnoD6B7zgp rXZ7Y45xtvNYj4vI+SYCqRRnX0icVReFP8mxw80BD/bsfj6pBI6QTmxmSuN63h/5I7pSYIWJcQNjQ PTr9DUljKs6t6I+eSKCJWPNuvc7bPexaHa7cDIJIrywtFExxAVKttnUXAvkYYLeXbdnMR4ptwBnyn AadZOSown3uZNq12UWjRgpf4JS2SUgBpMG09TKcp/dOvS1/QF2ueLRlf4Gco5I2/KnOnzVdX2pn6D YpyxeVXOUb47L1NMZhgQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0k8-00Cey6-Px; Thu, 16 Sep 2021 23:24:37 +0000 Received: from mail-qv1-xf2d.google.com ([2607:f8b0:4864:20::f2d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mR0Zj-00CaI4-7R for linux-arm-kernel@lists.infradead.org; Thu, 16 Sep 2021 23:13:53 +0000 Received: by mail-qv1-xf2d.google.com with SMTP id di6so5319671qvb.1 for ; Thu, 16 Sep 2021 16:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Y1Uk63vpkrh4HSEXHQNbtcSjOksblIyIRxi91WKbEAM=; b=ZX6T1oIM4R3goFT9v/2cf0FQGFBs9qoQ1Dv0TkKRDmuJXkp0fJ4WJqes6t87z1iuLS 2dvwca0W8lx+DFNgimo62pYY1v2NlFLAvt6vo2HthGJRqDkA1Ogsd1ivXC/NEzdln8KQ kC/vjnO88YyNTPWT214IP97mzvlq7sozyzRAtDw78VAOk43svmZFJuicv3ozQSnczCBf Uy+BK2ojCcJhc8bdfYtDAvjzrxlx/H7zJkRdVeS8tHN5S5i3ZVsCxPzLB+zVPcxwCzuZ ygQjb4HPG6I7UqyVUggGHNtedtZTV6stzxnmQ6pDF1ArDvMDsiNABNm17COwOLbtS/gz pAfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y1Uk63vpkrh4HSEXHQNbtcSjOksblIyIRxi91WKbEAM=; b=Roh+mjPQZ3iWMaiXfVQdvqCWVqcjfvKgyQaXZlFO9Bfr+6EQqWAaUwjvLtbbAEO3J2 ODdzyEN3MWlB9EZTZn3dq7L2jhtPUiwOKa2gQhZ2Gh/qb4eCediyD79x/4oMsFf9WkpS TAJZhqcAmwURvcbMQOW7MmF72pBKmh20NZviBxvyZ5/yhBEszQTcQn/wFZF08hKiWVEr NmAy6MGgHFxlKTElOtjx+aMTU+40Xkx7OOACWLCFKCoH5Iavva5bjAQLywvz4FEvvdrl Qyh2F4+zVErFg+DNHGQFsL7czZF19pRDMQkaRtZ6/tustuItKLT77tN/c0txDJMX4+hr WhTw== X-Gm-Message-State: AOAM530bUBJQITtJoouzC+XPMu/8BAl+ZF3pS6/6+Fu5tyDlGZ22PYnK 1Y/IKtI1USF+dzu5yiYW2nm4NA== X-Google-Smtp-Source: ABdhPJzANQuYEdGyFPzM2h5cBvhaMadksnrGaWSms5XeNGsc2+WLwnbrziOAQEnBWBa9YCXfDzrokQ== X-Received: by 2002:ad4:54b2:: with SMTP id r18mr8169426qvy.6.1631834029839; Thu, 16 Sep 2021 16:13:49 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id az6sm3312891qkb.70.2021.09.16.16.13.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 16 Sep 2021 16:13:49 -0700 (PDT) From: Pasha Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH v17 15/15] arm64: trans_pgd: remove trans_pgd_map_page() Date: Thu, 16 Sep 2021 19:13:25 -0400 Message-Id: <20210916231325.125533-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210916231325.125533-1-pasha.tatashin@soleen.com> References: <20210916231325.125533-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210916_161351_411532_81D8A1C5 X-CRM114-Status: GOOD ( 13.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The intend of trans_pgd_map_page() was to map contiguous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contiguous range this function is not needed Suggested-by: Pingfan Liu Signed-off-by: Pasha Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 7b04d32b102c..033d400a4ea4 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 26bd8f2d95af..d7da8ca40d2e 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a