From patchwork Wed Mar 3 00:22:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBFABC433E0 for ; Wed, 3 Mar 2021 19:38:57 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3286A601FB for ; Wed, 3 Mar 2021 19:38:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3286A601FB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jf4fJUJSA9KF2U1nNQDZaCT9B1ALjOLGT9H4Hag4XfM=; b=KR+CGUUU5v2ifJWRWb8qMsI7b N2wrj2GLXv53QAv6Viq4LLps43KYkezvqLzI74B1ICjsCizCrmmtsy3lwBlaEcm9poGQSDLNxgbki 1h9qLjCh0Wy61GAwsiOBsMuXYjkrb/vohfN7S/d5oqQ74zVqJH9AgUJO8NDbOJduhOmD6IMdPxq+7 G9buhjhm9tu31RWSqjBKZ9P9MlgVkDfrNkko4omdTVzhltGLJ0XmKbyTzJUJKbhgiSXr60LNdPkxf /a9R5VOYpit7ZKe0HtFAFbdElA/dgi4vA8NVk4rn/A2Dh3U8S/+n9EEhjaT2oawc0M2Xea+wYSMlr ray6gULHw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHXI8-006F8G-Hl; Wed, 03 Mar 2021 19:36:16 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT3Y-005I6u-Q9 for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:04:56 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=FYGPnhLj6niG8ng7uI2M0ZSiAf /S6F+vitVqjVPeCpPzde1GrWzVMLFDY3v3/zfaQJXvf88vIoGGIoKkHNPmZ1Zh5H52QXBwHLqyNhZ hXJzMbiSWzWlflxqxX0zfd5I5mmiNqozYfpvVyahp3bzC+kLiOiidTuBSiN+sPkypPVKpHjIqlD21 qqd4YcNk4XlTuQJyZMEGr7OTY42dMkCwapMYGcWCdvM/H7v9jeIBaHep3zWsxoNKBg+Fbix0/2+jZ CJHKjWkvQUyyJXtDqlhsJhxg2wcQ4afh2OR9jMfumuICs0mw66PQEAOnLgn2TgFXoP69uzG7U3Zns Ir4Cl9BQ==; Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIr-000jc4-PL for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:51 +0000 Received: by mail-qk1-x72d.google.com with SMTP id l4so7558064qkl.0 for ; Tue, 02 Mar 2021 16:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=HFHYvchu6CpdqexxuDSsNYf6zc2J6VEKWIBzKSjPVt7Oj5tn2/oRKB1PtdVtAhgBcu 8QtnWiz/7P8/LqX5wFb1aqUQju3puj7mBA1eY2moIHPzNRpZr6tHqSrfP6M19tFtCndA Dw7uvs/8X/wLRsL+Y2oTYcwh4VUTm1gb0QoWI27HLxniuFGEj0KhR+xLWCeoEvbTnu+7 iDZO6hGQaznymI4Z7Kti3JjhODZC9KYEwl0lPssNr6T7Q8YrqF4+AmY67n+7+DzGelJs WYrCxrVf4ENesVrOPXLzb1mOiHh9nKU3H9yCHxRR06iUcT7IO02sj3oVqitQee/e+0NO HyTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=RA9uvPs6N4WAMjFmwib6ZXuguuNXivPKYTQOasTzkgxSzDk8O3e0gSWGPGebTZ57BW lamwC6ps/sOMrxCnbmqte0JKSv4UTmVxNzq6kU7p6swQCAjm2UuFHd29uC0jPR8IqlNP b24W4Dyod+nyLpaBSUbMFVr1yDUnp68N/skv83PfheiV/DiwR0CIP9X0/vq6hcProkip py6QCM5O8OSdDPWkxf4kxggPxwW02db9TL2WHF88DN6i/pcgW/BuppFKeWLIaxkTrJNj wn0a/JNRL7kime4rDkPnNlrFNj7O2f64VGvA+IjAbkf8AzOE/ccnH3DGuMNWhVRJv8wK kY7A== X-Gm-Message-State: AOAM530OXFzh3d1nHXgA03BZJ50JjwRcuFBpwPpIb1HZgmSHBmIyi+NE Kg4ltBVuAXwfohYK2NgjqMUo+A== X-Google-Smtp-Source: ABdhPJwsTtNwrwyfksF0JxhDL724PEtC95EO+MO9/7fXbkyN8TnG2jdsy9RcmwXwPeG1L2V0hRnmfQ== X-Received: by 2002:a37:4743:: with SMTP id u64mr16003296qka.350.1614730954363; Tue, 02 Mar 2021 16:22:34 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:34 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 01/17] arm64: hyp-stub: Check the size of the HYP stub's vectors Date: Tue, 2 Mar 2021 19:22:14 -0500 Message-Id: <20210303002230.1083176-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002351_858896_1DAE4817 X-CRM114-Status: GOOD ( 11.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Hibernate contains a set of temporary EL2 vectors used to 'park' EL2 somewhere safe while all the memory is thrown in the air. Making kexec do its relocations with the MMU on means they have to be done at EL1, so EL2 has to be parked. This means yet another set of vectors. All these things do is HVC_SET_VECTORS and HVC_SOFT_RESTART, both of which are implemented by the hyp-stub. Lets copy it instead of re-inventing it. To do this the hyp-stub's entrails need to be packed neatly inside its 2K vectors. Start by moving the final 2K alignment inside the end marker, and add a build check that we didn't overflow 2K. Signed-off-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 5eccbd62fec8..572b28646005 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -41,9 +41,13 @@ SYM_CODE_START(__hyp_stub_vectors) ventry el1_irq_invalid // IRQ 32-bit EL1 ventry el1_fiq_invalid // FIQ 32-bit EL1 ventry el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) - .align 11 +# Check the __hyp_stub_vectors didn't overflow +.org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K + SYM_CODE_START_LOCAL(el1_sync) cmp x0, #HVC_SET_VECTORS From patchwork Wed Mar 3 00:22:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0706FC433DB for ; Wed, 3 Mar 2021 14:14:50 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5018F64E4A for ; Wed, 3 Mar 2021 14:14:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5018F64E4A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aDrNGcQsUWYMwonvnQfh90LVXq+vVHy7hn8P1biowAU=; b=NTSz7qdmtjwN30WDIot3iXOBt ByjuohD7SOU3MczFxWLJkuoYmBWk/l0qL/Q0//Nqx9wx+Hjsa01wgjiEjPJnkqYSXZDLEgMSYWmK1 r/jIF7NECfoESKzED2UXqiQVIghpiNgbFgt0l8chiz/I6yv/aRS7QjViFBkFGiLVUo2ydxhHbiTXW AYIYM6EctkHtaEZV9sHVXOtf6fbrfkqkEUWgotnjsHEAHmXDPbAFKE354gJXlltnbAhI+x8eT7Afg 6GDEJq2DpVi+rrsl+UlSW3MRAkzBbWTGukl7jnJzvzAhX/EWbarvCoLqLwZrIY12OYYbURuort+mb C9oGRmHqw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHSDa-00553h-Md; Wed, 03 Mar 2021 14:11:17 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHS5f-0052JF-Fz for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:03:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=fglQOCa5In2LSmlzep/H1c9B7e 3d7IDXKU+JaE7Gyb99363LZdN+Bca5VmoYMeqnQbqxQaJEnIhhHfKNysDM8FnDebEG4M4HQipXe7F qfweUZbhNeZG3XRduz+7zw7/wXwgJ9faMGny/WTylJKsm+Q1i3TtHqJBp/n99RXiKfB/Gg2ZP2pLT K0/UTNknK3cx+cuo3SnTeAgVyn8BSZ+eL4U7/+r0NNd4RS7yOQoUpWhYmbpAWT3ked0AAJSYZ56pw L323DG9/KlLqX0KEYWuGQgQtQVuTE5HJy60rSKWe09h83a/YsejTDE3x6efuss8SdACl11U8wju3+ MGu46LaQ==; Received: from mail-qt1-x833.google.com ([2607:f8b0:4864:20::833]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIt-000jbw-Qz for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:52 +0000 Received: by mail-qt1-x833.google.com with SMTP id j3so577989qtj.12 for ; Tue, 02 Mar 2021 16:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=TexeuK1LzjZUxl+qx3qQULwT87vQYc1MD6Bj68BOVjP3cGLOT2rRwv1i74chPgGfYO UbwIjulEE5WjuUopzaIzxJPEuV7hmvshndmHJBpW/eW8BnrLt1Cr+b2u4gSWJdSJ3OGP JwY+w0YB4TdGP9WGlGzIaL5gqIKtQQYXEyCZUEkKWc4+LfQjREDeMeDJf799nFNdtcYa 3loS2FGH6MBJ5vfpVkn8MnukQ54Se5vlVbHC6yPt9D67uZmJrJJh8vJDHEUtq80ftZPG l026k6HCjmsCE6HHevk/8lCVPEc5jzpfn6+0cF6QpQLBswS06CStNq8tM/bnGM6z0lET sIMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=hR9zK0ExaUSZ4+x7qnOj6GCY/QoXT+Idf0aEM0JnL406q58eTrHgtN6TiFgx3WmT0H WBuPsZgTuHhSD2VMAcGVeqlkn0URnoRnE2IqsCmojNb6DZyrK0Aq9Iq5R3/Tzn8eB+yk gLLSqn9kRnAvWNzRLR9Jct+lCO5xVLrYRAK3qbB4XoGex926EGmzHnKILjYL9P7My7da ZsARxVHZKIllmgjuOTOwREJ2FTeVuTYi/8hG6IiFRflFfghb88+tuQRWAUpiksgb0kgb xZPhiYBcPU9Q4znKqJRONlhrIMCDIQUs9KpxhWdJunsUaT+lvb6GyWwppvTSz5OUBMNY S2ww== X-Gm-Message-State: AOAM531mr2SgJSgt2NYnNEbUtybJLO49zGtqOk8VB0KHGLaNkfB7qkSW s0p5zz9cf0GSAH2LvJk+MZTT1Q== X-Google-Smtp-Source: ABdhPJwug5BfAFqjtQc6bxUz5+jervNuBn1c/vyjkzVFvBEuDqyurtMGznEMcQrgeoHuudD6UVUuOA== X-Received: by 2002:ac8:544:: with SMTP id c4mr20060173qth.245.1614730955693; Tue, 02 Mar 2021 16:22:35 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:35 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 02/17] arm64: hyp-stub: Move invalid vector entries into the vectors Date: Tue, 2 Mar 2021 19:22:15 -0500 Message-Id: <20210303002230.1083176-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002352_392990_4D1EB0C5 X-CRM114-Status: GOOD ( 10.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Most of the hyp-stub's vector entries are invalid. These are each a unique function that branches to itself. To move these into the vectors, merge the ventry and invalid_vector macros and give each one a unique name. This means we can copy the hyp-stub as it is self contained within its vectors. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 56 +++++++++++++++--------------------- 1 file changed, 23 insertions(+), 33 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 572b28646005..ff329c5c074d 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -16,31 +16,38 @@ #include #include +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + .text .pushsection .hyp.text, "ax" .align 11 SYM_CODE_START(__hyp_stub_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 .align 11 SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) @@ -173,23 +180,6 @@ SYM_CODE_END(enter_vhe) .popsection -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - - .popsection - /* * __hyp_set_vectors: Call this after boot to set the initial hypervisor * vectors as part of hypervisor installation. On an SMP system, this should From patchwork Wed Mar 3 00:22:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F1A5C433E0 for ; Wed, 3 Mar 2021 18:45:22 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5361864ECF for ; Wed, 3 Mar 2021 18:45:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5361864ECF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VEi1qaleTRDAmZPEG9yNTBux/nw1x+assWfgyTJJl4g=; b=CiHgTuGawcuO7dAELCXeujtac nA0LwZe5LVZS7aCOhv5bKPG4iikrA14ox3Y+ysxL7YKV3ypIYe/G52FDJfnoppvlfL70cF/YDnzcR Tj3d+ptejondUZ/cuxMR31xXjn6aZBbctOikR6UaOxZ+y+CWp3zn7K9zdKeGiyZrE8MPrXIS16nzW P+h99R/Nm2J24voaRqAkxKHGzbP5WyCh4ELto9DuEYmmLqLwwnuB3oFDWymb6FrBLdS5a8Q1UjWCc o9qu8gw6XpOK+V5fjNkGSpKAwqQOn8vSVuq8nRdZs8MAma7qKai17V1XqwZcAOV2Z6Unc8WsLEW0j 9aTK0cSaQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHWS5-0064BL-Qh; Wed, 03 Mar 2021 18:42:29 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT1L-005HJg-2g for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:02:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=YzcvcO9MlK7Ogd9RYV6F+Vv2tq eI5fIDn5hb82nKqCb8pi/zA/gYDY6qI7Non2/ipufQ0JeLNSVd7RVcgwtmdnxahnayN4aph0UnpNH r/U99+PEY7lzla3EkZyb76uNbxFqyrZRxlAtrIGl+1sZ0hYTteH9xt798AbpUpaeWqM/6P3BANcyW zCkmrWXIviNYwsgIC+tXOh9e2MrUIkpk9ykj5bpQViEwKPGARiH7+p750fjc7X3xKkzDwp3bN5qg6 BjW+yD7Q/p5Ii0SFkd9Wi8rxuI1gfTUV+jOkiShvlPuUQtBFIWkoeKLVsHtyDeUyvo2nUf6G1WCOT uybIryMQ==; Received: from mail-qk1-x72f.google.com ([2607:f8b0:4864:20::72f]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIm-000jbx-2B for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:44 +0000 Received: by mail-qk1-x72f.google.com with SMTP id g185so3058292qkf.6 for ; Tue, 02 Mar 2021 16:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=dcWduAuTpEnsZ23w4q/nGHITQ9VeRVbopJtAQolCLmSGYvvi8NkPc5iT5pqOtLEI+X /sN64YP6ApTGMwT4gN0aCuIbX2Wf5v4YZFTXQwBOPHm5src59LNHJDHloF+tQBkzuW6H W24eGwHiIzWRAaystxYlL0kVLtHGNHklPZtuwO+YFXmkTv8YCN+qaNaRdJ574xHDHN7n Xl5EbudHJckK44CfwjCJvtC80arwVKn0HT4+PXXkI8IRd+qhia0D+KE2nv8hEhlvqYWx osSeISMrNU39E2zAoOMWs+dLxNpFl9AVxtMy9JdcplGBSucizAVfKfRAnp8EPZLiGRQv reLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=E0mK0+Eq/aA+zuYwDKW8A5m/ixdC6gsqV7g+b+cgrCnPpOTTyAp5LLhI0YNnV6/xly gIQTl5hmJb/mVuOhuSCvTd3I/yUW9UowkFUr6qhoClBTGRckFM4Tpmsi2v5RhyLiYNC+ qEyybulHU2CuyVBQpsARt+HQynRqyug8gvVZzeSGVjf4zn3+wePuFIv+QWijkDGpXwpG pNWmWwZvgvmR+l7kZAMnzbtI6L3gdXdoOWFE3gx4VcwJ2/wgvYU6RnnfBTX75vXgrwQf pCUy7sKu6PMCHLMGv6ngHeaUHlN/6iiZYv1TvdAKXDE5ocyI21nZSRg/k9XFTbXLDYo/ eRIQ== X-Gm-Message-State: AOAM533iJAew91cgEzjY0dEEfH/jxKLmPOk40wEQmrMQXR1NL5YkPeTu fmeoacr7LssPvpRhUJJAA+6ZqQ== X-Google-Smtp-Source: ABdhPJy9cSXd6XyxnY9AZClyXZ4f6veKnYkkla2SRvqY4v7p7SiG4DwUFuL6Yj3oldPgCXfdo+UiPw== X-Received: by 2002:a05:620a:c0e:: with SMTP id l14mr13557520qki.91.1614730957001; Tue, 02 Mar 2021 16:22:37 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:36 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 03/17] arm64: hyp-stub: Move el1_sync into the vectors Date: Tue, 2 Mar 2021 19:22:16 -0500 Message-Id: <20210303002230.1083176-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002344_571586_1C4B9F3B X-CRM114-Status: UNSURE ( 9.94 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse The hyp-stub's el1_sync code doesn't do very much, this can easily fit in the vectors. With this, all of the hyp-stubs behaviour is contained in its vectors. This lets kexec and hibernate copy the hyp-stub when they need its behaviour, instead of re-implementing it. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 59 ++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index ff329c5c074d..d1a73d0f74e0 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -21,6 +21,34 @@ SYM_CODE_START_LOCAL(\label) .align 7 b \label SYM_CODE_END(\label) +.endm + +.macro hyp_stub_el1_sync +SYM_CODE_START_LOCAL(hyp_stub_el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS + b.ne 2f + msr vbar_el2, x1 + b 9f + +2: cmp x0, #HVC_SOFT_RESTART + b.ne 3f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 // no return + +3: cmp x0, #HVC_RESET_VECTORS + beq 9f // Nothing to reset! + + /* Someone called kvm_call_hyp() against the hyp-stub... */ + mov_q x0, HVC_STUB_ERR + eret + +9: mov x0, xzr + eret +SYM_CODE_END(hyp_stub_el1_sync) .endm .text @@ -39,7 +67,7 @@ SYM_CODE_START(__hyp_stub_vectors) invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h invalid_vector hyp_stub_el2h_error_invalid // Error EL2h - ventry el1_sync // Synchronous 64-bit EL1 + hyp_stub_el1_sync // Synchronous 64-bit EL1 invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 @@ -55,35 +83,6 @@ SYM_CODE_END(__hyp_stub_vectors) # Check the __hyp_stub_vectors didn't overflow .org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K - -SYM_CODE_START_LOCAL(el1_sync) - cmp x0, #HVC_SET_VECTORS - b.ne 1f - msr vbar_el2, x1 - b 9f - -1: cmp x0, #HVC_VHE_RESTART - b.eq mutate_to_vhe - -2: cmp x0, #HVC_SOFT_RESTART - b.ne 3f - mov x0, x2 - mov x2, x4 - mov x4, x1 - mov x1, x3 - br x4 // no return - -3: cmp x0, #HVC_RESET_VECTORS - beq 9f // Nothing to reset! - - /* Someone called kvm_call_hyp() against the hyp-stub... */ - mov_q x0, HVC_STUB_ERR - eret - -9: mov x0, xzr - eret -SYM_CODE_END(el1_sync) - // nVHE? No way! Give me the real thing! SYM_CODE_START_LOCAL(mutate_to_vhe) // Sanity check: MMU *must* be off From patchwork Wed Mar 3 00:22:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91530C433E0 for ; Wed, 3 Mar 2021 18:11:33 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 09B8664E6F for ; Wed, 3 Mar 2021 18:11:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09B8664E6F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RHC26DOCaEh0UGZGzsmH+fP0EsSVIBH3J7+XFneg7Oc=; b=K6+acTAq+KPAgqGCQoaeXvODy 8dLhkDmlE5Cn98scLfIcmCNAI2dJklFEGITtBHAZf5zaPxzMZAcvDRNPqrFyA9gzAjYVes5I2KDcr QQeMIyxASR+zRWKGWCFlW41EgbE3GkVm5yMcr2gL+thFXU/s4MPze+DCi5568TWneRUNlJHfhr92v jX5Tqo2qrbiydnyiPTBRR+CRP6s+sEH3RhjzDu4TBl/1FTR2AA6Bg/7uAEKN0Mg2kFOXKKJtzzEsK c0SpLfhBE8KwxcyZjBws3WbgrSyK0aMEq4sMPTEYp/+rXrcA+8FqPWbQWJzowdMgtC6+hRP4oWJFX KUVR7mzWQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHVva-005xFx-M7; Wed, 03 Mar 2021 18:08:56 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSyo-005GVM-1O for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:00:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=uEJddvtEUaHs7HHMGU75TsmDko YBQM+Bl4Vf4xJ9K4CgLiQ9S8SF8+ZqiCwFas2l0JHoEV7IPQsNRg+jTtW2r8QSFEgTPYGEjVl3+3A UEGY7s21CLD0RfhNOJf6oQQyjGyOA3wlj4zD2/x9bFpZJe5fTPAp/bjzUPhtom3bXMqISn8e9NGj/ oglXYuy7n5YKyHbjYNhfb7McsCXTC/ejPlTMZ5VFgeFvgWft1hTX0CHmef8gFmTEDF6k0zmwGbk9d WOgQtN0geCB8UOFk2kWJ0a4pTUOASi9IxJ2n/+cqsDmbaYd1MDB0viHRG1J63Vc4qESdOvah/22L9 /zp9AeFA==; Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIm-000jc1-Fs for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:46 +0000 Received: by mail-qt1-x836.google.com with SMTP id s15so16291321qtq.0 for ; Tue, 02 Mar 2021 16:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=SwKTL4eCvkIvgxU6lGAV//PLN7/clkFuLLU3DXUzK1tQP8M45hnpri8l5cR3f1+Ylp sMpfdEfEz+Et5fTiJRblDRcFcdDME1BtQOImc3LBxmY2ee/EnHXeyesUtIl0vD64exx+ VcRzv+DTWT/hDHKT9/XTJeTMiBflR4d/CWf1PPm9mvSFV7yUEEIVgbmppDsdR6CWOocz V0blw/3FCGwiuLgjQvSQ/B5NmZjurWUX6kzX83Lt5n0pSVAHFZPPAh+CUnOKE143JiH9 EaWCHQ+48dXFnyzY6WGGo0svwQnyOB1kX9/+fl90p2Ln1YIFMmaJnFRL2kdEe5kov6l2 jBiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=NBK4EF89GX0aTvGpkYq7ZMZ0thG+vbCaAP++hPjmoZP61RBWDPD8h27R2ptlak49u7 uREpLGB3VEr/1fR85bH1nEGmam/D+v6FiZvuz+57+ie6w3Pdg3TEKDMiQ6ZdHUyZYi/9 a9/RU2xHgksc4A0B6p51OuSVZaMRBqKYrH/kiFi+5ihaEyy5vxl60jpGuovh1q91fN8D FResCEtAT70Z9VYMa0k5/0enK9X2dCx+BzQR4amL0BtBsal4SBKDaNWJTPr5B/t9Iq9t +IGcb9WxCDiA31v7GdPRImeIotMvXKoxi22o2pFQ5cHn3Xu8D1LCezEGsTfAUFHmORxw 1GZA== X-Gm-Message-State: AOAM530+3DzXVrIt5cbwvmVEOw6IRj2w8wYOGNNbH/f3RBRqwW3wKlFy yJr3zJPbdzQliBkwWxU9T/GpoA== X-Google-Smtp-Source: ABdhPJzO9otEd6iJTRYqtDhi+XwWHgyDvgnqwPpNWgOkK7byt0RQkwHr9BNXQF6YdGG3LDA/I4eZOQ== X-Received: by 2002:ac8:d44:: with SMTP id r4mr19942609qti.7.1614730958326; Tue, 02 Mar 2021 16:22:38 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:37 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 04/17] arm64: kernel: add helper for booted at EL2 and not VHE Date: Tue, 2 Mar 2021 19:22:17 -0500 Message-Id: <20210303002230.1083176-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002345_949968_57A511A4 X-CRM114-Status: GOOD ( 15.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_callable(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 9 +++------ arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..4216c8623538 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_callable(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..1922e7a690f8 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..c764574a1acb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,9 +48,6 @@ */ extern int in_suspend; -/* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) - /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; @@ -125,7 +122,7 @@ int arch_hibernation_header_save(void *addr, unsigned int max_size) hdr->reenter_kernel = _cpu_resume; /* We can't use __hyp_get_vectors() because kvm may still be loaded */ - if (el2_reset_needed()) + if (is_hyp_callable()) hdr->__hyp_stub_vectors = __pa_symbol(__hyp_stub_vectors); else hdr->__hyp_stub_vectors = 0; @@ -387,7 +384,7 @@ int swsusp_arch_suspend(void) dcache_clean_range(__idmap_text_start, __idmap_text_end); /* Clean kvm setup code to PoC? */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); dcache_clean_range(__hyp_text_start, __hyp_text_end); } @@ -482,7 +479,7 @@ int swsusp_arch_resume(void) * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; el2_vectors += hibernate_el2_vectors - __hibernate_exit_text_start; /* offset */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..af0ac2f920cf 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -200,7 +200,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_callable()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Wed Mar 3 00:22:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB191C433E0 for ; Wed, 3 Mar 2021 15:39:32 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A492464EBD for ; Wed, 3 Mar 2021 15:39:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A492464EBD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Z5P2vBPyqwsUFuvLrRMg53ubUWKyHW6IoDHjYwlhRYw=; b=ZtL2/0VLBqXHRXDKoDhiLZZkY 3uZI273NVcymhNlGp9wEvjWH7hlUz/IRVL0gWF/Fmo2hcLJgLT/ly81ntcMdRIdM4by670kMBwiV4 VOsWgn82YTMXVAYcY1MNw/ecWBDKFWTH4F9Qqcu5A2mrDZLo2PVhNQKY2YcW5WDYmqZR5+sBAnSKC LKeNpcHnjzb+v06XElT17h1z2RjNMhge3oI6la3o6wo/tUO7epRGFFINMjxBMaxnJ3LDP4BhoTqkN CxX7nC/iUohpvmApRmsdEiM1mn4k1GYZ/TH92JFO4Ls2KhLips8srSuyfWpVmX4JIZ7a/ZAmTqe/0 D1LiwGbdQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHTYv-005Pum-5t; Wed, 03 Mar 2021 15:37:21 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSWB-0057rq-T3 for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:30:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=RWy4W2HOIvc9UEVJGl+l3rA8EP w+wrD6YYRxLUfkCXfLGSQtgddyUxeCviUt2Bjhc1u7p6xaksRpjweKmEKymdNQHiFZQDwMaJ8PQtW eYyu+z9RYa+Yfwue0t5AwU564N8qg/D2vaKYkXuWYL/6WV5Vvcq9PbOjjX7ndEs++OaVV68jVgwjS IR1Mm0/AFj6dkHAac8clfB/wU7b5hozhuNhsHyF233sP2VviPWZ8jQH4SMoCD0BJVj6P3Qe0YQ6tR rqRrzQih1LucIoqYNir5ghIDDkVkKDYOkL89gNfEelyWfbuTFzUqh5pZ9FmQsPt/w/rmwyf9QNdIB h7tSAb9w==; Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIn-000jcK-SW for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:48 +0000 Received: by mail-qk1-x730.google.com with SMTP id l4so7558231qkl.0 for ; Tue, 02 Mar 2021 16:23:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=iC7nX8qZIRxsD5OhB+90MycRKzn7baBcfRaKmKM42+Rfyoo/TzKiZgG5k29FGJj22A 3IRibCdly/NeZnW6O+0v5jPgsqfvXNS1Ri+Wt6eQSXzKy3TbGQQV4zSr6bMCrDJuT7CZ wbYECoGcv5wZ7z7t/QWfP/9Xvt9vqJltxAJ86LAPyLkWWe6VZP61mFU9hqtH0rfIll3j yqgpLmU5q7CK4YhI+VLTcek0xOakTRKi1EyHf/AWlhklroigd/U3mJP42p7NVtYd41ss E/16iSCm4wnaioO47C7o9vp4jWJnN/N+Wd8sno0i9qIHZvkE+SUfGDtxCGR/W6bqKUJ3 pAFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=EDP0wUQb3OgnOkt1+md49IxU4AUNkkR51sWlJ7xgsT9ykZ7YxapPJ9f4iFcOAh76eS /GbFXamUR6oo2N0U8qKGc2++KzSvLM+halGauYc1QFut7U/z3FuQKxv1dbG17iVE+Nnx y1iqDg2RMAUIZ/PkAYXMZtJgRFEF8azC4+cwbyRin6fOP0sRHgkeyDjinHJk2M4TXz4Z u/pwSraVy7VNqC7JARSBPKCPqDMVM39qO1SoYqk80VAJwluDw/noYGc10NCJbkiu0fs4 gMTBBqI9UH+M1HrL73VFog28ef7VKYnWSdqyPSw4J78hFe6CV75O6xq5jlNpDqR7pAg3 buVw== X-Gm-Message-State: AOAM530hltrCDInIjNKChltVEuASm0eXIKfaGQCKB5tIBmYHh/ZT/B1Z 67oKNIZ1/0t9THA+ajxxvHUhkQ== X-Google-Smtp-Source: ABdhPJzAuGpf1um8csVUUBUG3gga/6awbhEJ2kTfUdApasxiQpu4oFmL5IVED6iXYZoNv2iCme9QRw== X-Received: by 2002:a37:6115:: with SMTP id v21mr14220445qkb.239.1614730959621; Tue, 02 Mar 2021 16:22:39 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:39 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 05/17] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Tue, 2 Mar 2021 19:22:18 -0500 Message-Id: <20210303002230.1083176-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002348_884568_76F07253 X-CRM114-Status: GOOD ( 15.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 3 +++ arch/arm64/include/asm/virt.h | 3 +++ arch/arm64/kernel/hibernate.c | 28 ++++++++++------------------ arch/arm64/mm/trans_pgd.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..e0760e52d36d 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -36,4 +36,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 4216c8623538..bfbb66018114 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,9 @@ */ extern u32 __boot_cpu_mode[2]; +extern char __hyp_stub_vectors[]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index c764574a1acb..0b8bad8bb6eb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,12 +48,6 @@ */ extern int in_suspend; -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - -/* hyp-stub vectors, used to restore el2 during resume from hibernate. */ -extern char __hyp_stub_vectors[]; - /* * The logical cpu number we should resume on, initialised to a non-cpu * number. @@ -428,6 +422,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -455,6 +450,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (is_hyp_callable()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -467,25 +470,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - __flush_dcache_area(hibernate_exit, exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (is_hyp_callable()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (is_hyp_callable()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..61549451ed3a 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -322,3 +322,23 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &__hyp_stub_vectors, ARM64_VECTOR_TABLE_LEN); + __flush_icache_range((unsigned long)hyp_stub, + (unsigned long)hyp_stub + ARM64_VECTOR_TABLE_LEN); + __flush_dcache_area(hyp_stub, ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Wed Mar 3 00:22:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113849 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50211C433E0 for ; Wed, 3 Mar 2021 16:33:24 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 87E7A614A7 for ; Wed, 3 Mar 2021 16:33:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 87E7A614A7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=waz80f1KmrB1QOOAfkG8AGv+ji/sGDm7NLtMTFa1jBM=; b=gul9rWIoeQ7Bf5lSNJJ/GNRvT Jn2/PsD4g0FkON1BLon5oxH8J2gvJdFqcMFXOiIjaS/A1rz9XcTTAytL6XF/3wpYltJxk3pcgYvUo YVB9Gm7HBXCaocRMc1fwDI2CzhM2RTAvB2cFLUf7agQIqjN8NtB4JtDRa3ixiwXb2nVZ3iGt7lJGv BK22+uyhzMXcP0q/APDdJsTm0+UvaRDAJJXG/XIc5btB4BStLk6kZeNU4r7TGavntnlzuj9YUDpXL Quku7cm9rNTyqmVlu717KbtB8r/ioHfh3il72sDADAuS5RJIG6A2QJ3nwYYhwyifIAHb8zdeiHrBx U+KhoieRA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHUOy-005c32-B2; Wed, 03 Mar 2021 16:31:08 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSpn-005E2a-7y for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:50:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=PfFJiiZEssC2qRvy3KJXm7j/Y4Op5RWRizR2E1HngfE=; b=ZijYn5tru+2JPtvRCUYsSG9m2A I1+v3LA3qvric6v0UjLrS+cp+7Lzc8muO9sRJNU7UPPlZP2dCKFLwf9g684+FAg8EspKnd3FBdEFl efd+BoARMHkhjxVxiUWY73sJUcKzgcWuiB6qRT2XVvxsxblN0+H2szK6p2t8YDEyEojalQCYMzJy0 vIsMrOS/DHohr2Ya/hbAO5kRnAousOdtxNxQSWqu2BviGFdG9w5Im1dKh+qIrie3MmiX9Aq0f6gpM urZhkjI0JaMHX90xvUs/h035tRp892OM0gIGmnURczemlWFxRRtXDmQPGY152cdroufD8WtmLrLe5 ZhyGiGiw==; Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIv-000jcI-5E for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:53 +0000 Received: by mail-qk1-x731.google.com with SMTP id f124so4158664qkj.5 for ; Tue, 02 Mar 2021 16:23:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PfFJiiZEssC2qRvy3KJXm7j/Y4Op5RWRizR2E1HngfE=; b=jMDvuz/Zgmq1B1MpVMXtPYeQ4yIQVYunhOd0cuqlnPu73G25rI6A8bbQBy7YIsq+XL QLFaC6VqVzMJKOhhn6eYQS9l2TbV/YMXjSMgRvzpw+kAe19ABWobfO0KCffKshahDMdI bUlq1RwNGtmC7j7E3GIqW51fgJ+qLsCRs7WDT07CRyQw/7RmxffV2qQ5/UKUDv4FWBAd rEakxu/EEIwCQfy3eqO5Nq7V6xiG538snw78opWM2Sc0e/ol/nnofnFY7H9ogH44hG/Y QWBHfv7b8Hjc3HJ2evECDpilQuTYVfBG/XhchNb4WN/jfozGm70uSgAosPcJBkyAtyS7 tJWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PfFJiiZEssC2qRvy3KJXm7j/Y4Op5RWRizR2E1HngfE=; b=aQ4D6VRyumi5roYTBZOD7dq3HroL27NYmkvF/RS2CmS9/Ycf1Fw1uWuQoWsW62AB4/ FPUYYD9L+YQju2zvNMNDfFaG9bgh0uhP7+Kg5O4+/I9U61e0pVFrbkkQnLnSNpgb+MU/ nCsDNGy61QCaKUhD1t2h75eNI/cMJmLkqf9kbFSh+4GNBWGd+CKg7DVOBbpZJ9K56FID L6ubIdtS7BPd8KwLyAUxbNLIi5YH3QyT6E9W1xIRo8kGRLLwU/M2bdlZXGeg6SwvvQT/ RlYKc4tBmmuI+thvy3LAsezfck+HugnMS74skq8XhaDd4H7luEVV67gpsrLHcCvoLBJk Fh3w== X-Gm-Message-State: AOAM531EKh2rXyeDjVagYTFyvhoAHjT4U4kFT6GTYKAOY7t1cnxcpe1P weK4ynCQCc6JaY68gYNa481Q7g== X-Google-Smtp-Source: ABdhPJz+uVEaSRAvPV9aeu5i6C2Vj6mlXfoQyiLGMJfrnW+9tKudvptZdZIYZEjfRvwkDzISw2eE+Q== X-Received: by 2002:a37:4e01:: with SMTP id c1mr10344514qkb.16.1614730960877; Tue, 02 Mar 2021 16:22:40 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:40 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 06/17] arm64: hibernate: abstract ttrb0 setup function Date: Tue, 2 Mar 2021 19:22:19 -0500 Message-Id: <20210303002230.1083176-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002353_735331_54777A08 X-CRM114-Status: GOOD ( 13.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functinality when relocation code is going to be idmapped. Move the setup seqeuence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 70ce8c1d2b07..c6521c8c06ac 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -132,6 +132,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 0b8bad8bb6eb..ded5115bcb63 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -206,26 +206,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Wed Mar 3 00:22:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D629C433E0 for ; Wed, 3 Mar 2021 20:37:42 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0855F64E90 for ; Wed, 3 Mar 2021 20:37:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0855F64E90 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=L9vS0tAUgnK695+xMNOhNTPthc35T9698qK9uRNN4rs=; b=j3eLHfJZpB/GpkeeyoIt2Po3e zdajcioKPz4bPPGaEoneGFzyOpQ9ue4MXOwiFyp8A5TF8bRyhC5DsX7ubS0CgyFK2B6EOVpEjjUv2 +eZmq3S4klI26SWa2NKlOJM3hPRXkBDjD39tEU8CHEP+E2pVSVdfzG2WCsOg0qzFk9fTT9724kgDi rVE1UYTb8j9un+y9xwcTY0Qpe+miDP+IDc2ccQOEWv/dYpMpn/NEPXcYKgxvtPLuwFT5L251xX+AQ dlc8PHQh18z7/8ruVuWRiQVTrANqD5te7SkUmDEkh5mSVU46rnRSMcyzFB5mDcgI45g3JkIaFLKk/ ZcKPNlmnA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHYCB-006RCV-3N; Wed, 03 Mar 2021 20:34:12 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT6D-005IzU-GR for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:07:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=rEcZvNlVNYcHlsevKKjxkcTfUQ nAXA0ILmZPVNTsSqHHZ7SPNirWKQUxvrNjWdLfIcMnny1fdsz//fhBgKtzC1ifON81BDwc+Xt8XZ+ UwJBJ64qNSJbkKxxHThjvo1iO0zrPxQ86LG3aRrzRa8pga0LEiFBp3cqznP5FZ7EwJwLlBubGSDOJ 0iHJjAGw56NlBkMae1sqd9GGBckLJDWQqRmNMcPESgNzTawV/Kgf0aaPVEpAwvRpfvn2hCgm9cERf 4UwdAInGUP1V0weuXVGz7cWVejGLfw7yrztRqe3s5l2oAOTHiSEtwa62olKSTs7S0cpbMSyWPfE8T pLw0IN3Q==; Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFPc-000k7y-DP for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:30:49 +0000 Received: by mail-qt1-x836.google.com with SMTP id v64so16288950qtd.5 for ; Tue, 02 Mar 2021 16:30:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=O7cPe49tVOcGseTiLHnzDyJZBZLGwQ8Mqe5Dn8mcQ7k5iBMuCobmbyqA2O4I+bELpB zHLp06elBMsXF5GXqnhfiFgVl8HzIeeAxXZanZh9XuGoGveIfu8Logb3klD20cg4Lajg camd8RfSe3wGmyeRjz3ajPwHXzU4RxSIsDAkaf3k/Ym8xBkcBk/VVx0mn/4+26u0Jk9y E9oPu8omML486M7rrPC5Fi8osaWQbS4+CuhExOmrALnpB9CgOvT1M9JrxHC17qiWLnOl lJtLtc7A4zTfzbV0WqzqanyzS2CyWzkHN3ltfjyEQs54FPOEDCa4pJ3trCc+b9GEUq4a XcFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=Z3EafucQ9hvVfap0VC+HfiYrCY2YPiJp64tbDsV0IOwETb/VUShZTXAPPBc1hgFB4Q Ex5hcCS2rLtnJYQGA1giTcntXrND1uctVGJYJiboEzhJkMFWXucs+RsByBVw1PBov/35 qeXnZdaAH9k6zQh2bwmrn+QcA7QxxLV2C0gNNqUQ2EnOGvBA0rOku/CQzHgSFLkWSI6G 218WNYmSLrBU/Wkisv9XSlv0Eu/NK55R0nKQ4y8phLf9nC+q0Hgd+JRJjmCAkoWVBnUI 5gJY55jAhe1fSf9F+F54tt7/GQxiUnDPE7r93BndVg0VF+CkGW94EG96NQYTo55isBcb EsIQ== X-Gm-Message-State: AOAM530IraTCQQlkEi4LJhIxzm0tkHC1nHhlEWiMOdkimGm4BCtGQp4g EJlbfqslCkLGE+uWcuujJxHD80elS2Kn2g== X-Google-Smtp-Source: ABdhPJwfuxp+t9/gww6zzmUbohUN1pBfabKCa8Gw5J69qpWws0xk/h6jJgeII7h77ZjFF8nEckCeZQ== X-Received: by 2002:a37:2795:: with SMTP id n143mr21781313qkn.292.1614730962187; Tue, 02 Mar 2021 16:22:42 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:41 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 07/17] arm64: kexec: flush image and lists during kexec load time Date: Tue, 2 Mar 2021 19:22:20 -0500 Message-Id: <20210303002230.1083176-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_003049_446925_AC2CE496 X-CRM114-Status: GOOD ( 14.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++++++---------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..3a034bc25709 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -59,23 +59,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -152,6 +135,29 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -169,13 +175,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -250,8 +249,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Wed Mar 3 00:22:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3592EC433DB for ; Wed, 3 Mar 2021 16:59:56 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 482A464ECF for ; Wed, 3 Mar 2021 16:59:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 482A464ECF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/fn+ZhrZS9uefso0j1dWPH2tztmDB9rkpEALYq3wuk4=; b=q5/+Fy36hTUcrFQ0ovdbkhkgA sLEkR2cDwwQMKGtxaoxKsdQcpsUjFfmjr7NA7U7Jn8cUD8xE6Qwxibq6TNFwUW7mHkAjJMJlZS9yV meKk1yI8le3HCZNzcqTfxaHF23APYPPm0FesxVc9aNy9+Wvd6kj/0waCfJtAUzIr9/cUbNqAJdqcO ho+58kl6yFCZb4DBHW9iGgNzfSYIFWoFS0rPGDI95p35uIdzEcF3nEzpRMaFgXdLTM3Dfv+k1hCi1 Vv+EcruwXis5uaGUcuK64H9gV/b9G3bxrM2t8U/q4KL1HHU9snFBNfZbp01/DNXfjETy35OczjLcV 5d3ZLJP4w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHUnU-005h2g-9w; Wed, 03 Mar 2021 16:56:29 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSrg-005EY7-SJ for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:52:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=quaEZ+HwMhRYGVCgE4qYjrGXd6 kmCQ38Z6fo6tn6sfVn2ENvArnu3mPSGDdyq4d7Us8+KrBdr6O1IKF93ypFdnVI8v3YQMYxMdnmeLT +sd/KBr13QVixy+MGQ+0WQpDCMOrp1x3YEuzUX1RtUsGFeNsXpXhVGr2dATCcasMAcVebVq4fhDoa 7TTR2kxK4JehDqIApqr8BDX+gPMAGPWW4Hfu1ifp1CmddoMO/gYpqfmnqjPPnyTInnXhvJ9aViq7A TQf+7mtYzMC09kQwqqxA3GwY8Bqv32FzDUDKSyNfPNo1c84y1lgK9ltdvizWvdaOgAzqfOAKRBZJN y8iKsmfg==; Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFRa-000kvG-44 for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:32:54 +0000 Received: by mail-qk1-x731.google.com with SMTP id s7so12252466qkg.4 for ; Tue, 02 Mar 2021 16:32:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=UueDkZWeVdt0ZVUeFCaAS95OcbctawuVt2jRXcfNleEQHioaIPsafjb3iN3wPEqMdt J9stLwiFQM/z9W3CKU1wRsTGAe+nD318y5F9IIMtfpYl9ooGJOj9fq3oWyPfDl8EafIQ QkbzJVXwjyMJ3s+WeBV5VvNz7u8LvjFGWo2A0yPGEvKqoLl2EWQ4TA6kdAzhcd0pCOxi bNZKhhZ2PTEsy6gzqf5i+WWcAhW6s6kBywZl2oPrlqd186gvUOmG2SJ5qEzykeb+bgo7 zNWjDcSlC0bR7+IcSuyKZ2OqdH/81WqWH4B4D8mKrZgAcxAnonHq8pCXd9qjhhgHBr97 fnAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=YcI2XKOLf3rnA0/6vKOVI6oL+yzqWYMqactpLKIXpxFoRPXSJr6IWJ134EyfjzHaFz wWPhqXnqbC+5dqVqpxZih9Zh0bEIAt4aJqlhigTqtyge53xESUfN0d062r6iXbrnwC/o JcpVSq8o520xsYCZvScMBPZwZTiuMdGXRngPUbf5pHRjfH6UuuI8vI44awskAeqzMBY0 iHQurdP+xgPjQ6/u8cKtIjFTYQuTVJuTLONYT8obxIHgGhDquXKYqo7SzoJx3QP2A6Lj MS0lJFKofyS8CtxK+qJe4QschmwDXAcUS6ABAgVR152whqTb/NYqFjB+O+pLI/jwLwaw KxPQ== X-Gm-Message-State: AOAM531CgQKQgkT4lqHJ9ASACMspsRo3RLbClKtZ1pkRxHREqxgBS/43 2ETIOb7ebQylgny0+gM6ckKpAzqvl0uJ7A== X-Google-Smtp-Source: ABdhPJx73Id09SRSbyQ7yttHUhQ1XFsUCayEKdjf5Um64FVS0lPYLKHoeNVlJXLwSpbmAURdNlizvQ== X-Received: by 2002:a37:2785:: with SMTP id n127mr12916932qkn.320.1614730963459; Tue, 02 Mar 2021 16:22:43 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:43 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 08/17] arm64: kexec: skip relocation code for inplace kexec Date: Tue, 2 Mar 2021 19:22:21 -0500 Message-Id: <20210303002230.1083176-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_003254_213898_BC506122 X-CRM114-Status: GOOD ( 17.42 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3a034bc25709..b150b65f0b84 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -139,21 +139,23 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -180,19 +182,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(__cpu_soft_restart); + restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Wed Mar 3 00:22:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4184BC433E0 for ; Wed, 3 Mar 2021 16:44:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D470F64EBB for ; Wed, 3 Mar 2021 16:44:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D470F64EBB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=79Glskrpq9wLpPHrIrocDP7p5BHgF+r7llzQIidIOFA=; b=KmpzUbMa/YI29ssTuBxav39LH tGIFTv3troClHoWsc+1qx7roKAVGQrbXaFlX9+/6Vxs2olgvEe5V1f7LBQvcDT7u7i0ZA/uNmY4iT us56cz/RzaLyXUQsxlGl+NYBVs63xu9TOohdb9InTBbXZti/keqligUhYK07WxyDOysU0qrgLAlva 7ZgIqpISLWi3kj7rClj0lCBhvBFlFJ0TUjWNm05PzwMvx+qN0BXPLXDhU/kYXHcm0mqnriOVHYbOV lz9QFiXBfRSS2TN28gSzJlhWClYi/GXS1QmnxzaLzhPQ8xEnEppyGH2YRbyHqm8ARckwAlxK0zv4A S9ffef5aw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHUZN-005e5N-Js; Wed, 03 Mar 2021 16:41:54 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSqZ-005EGf-1k for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:51:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=HDVvT0PqJTTMLrcqcuF84fWmLl TUyqEzl7/mBWvIC8oK08wXc7y45dIjCQ88s7aMAVwW2yeOfkIlt6KgzuTuwfJn8UyO42UrwgQ/Uaf 3n1sdpKR7+iKSS3VYKAMaXf5rS+VOcD8CRHwdY9tSnnV4MLsyhnxmQvKuhYOGLeW/rjoPDoHRGAFU geX72fNI5QkyfMeuP909hvJors+ZOITa8V1WbtQcSYGISw0FzA96IUWMs8Z/qjIIGQwZRx1xliw3h JPYXtLL+8566V0zcQ79mC+w2MpvRJRBZ9AVpEH4xA8OUN+ORYbymvc3UH9z4tUi6XogpQ753TWl6H uBiVpazw==; Received: from mail-qk1-x72f.google.com ([2607:f8b0:4864:20::72f]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIv-000jcb-ES for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:54 +0000 Received: by mail-qk1-x72f.google.com with SMTP id s7so12234008qkg.4 for ; Tue, 02 Mar 2021 16:23:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=J384aoqFXPkOEjGgXRUpI8bPGle2xZ/d/8Mv1cPGwzPmmCqCRjHEm8PmZPVcr9NZpT Z/KgtFrb4DGavOcxweTEDqhVwUSVb1KHU+Z037e6LKv3ESLtvkWkFGCvdLtPmlCQZTxT RCBUz0Wn/BU+REqHmXxhFqO6OJhLWdMwlbEFdOvgxGxtN5Z2/RSnuvKm9x2xboFy4ck6 ZaSRuztXVrXoPZDDSlAgvE9ZvAUzTrAYXdlp1fhgAD/cLkj2PII4I8mvog8wI+/OCKQF SpzPPSl9SYrpcKF3EJVTlYtPkA5IMuj47TmLjStHmcLs5ptke7rHEW6NWdgao9sd7QOO Z+FQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=JeF+27Vrg+LgE6kwgLwyGRARpvexae362RU0k6Iroi7voNFC4s6auY72sMrFNuQ2+l eHiS7bnOVmP/FpD6MTO/pJIg8xpLR1hUUegmFA8W7AhL2C5BCygdAbAww3Ni5udWssa8 QYw6j/QBrndckQwqcAGzPOuEfv+/9N7sT/+q6yDFBbTM/+VVMt6YYFFeE8o61J8X81Ov iwSV3CIgovadiYLAT4DXxwaWjD5N3xtH7FHxdBFkkPJS99FPvAoX3jAvTuni8r2etlz0 GsZGviqJk4P+zudN1i3a7kre6PPqohvfvfBEGW8Xc8cDfPbMel38iifrsDYRQpeZsStF 7x7g== X-Gm-Message-State: AOAM5323tujIO2YbajqnqiSGLU6VEkySWy2lh+RCNia2e5yymNnSiN9l gbo4Pkr3Mosl0WfdBEvJoUCslw== X-Google-Smtp-Source: ABdhPJzrhYCjfrgecXKXC5DvGpsZHuFJheu6IGy3c5MWfmYc4aXycDNvYiyhxoSY+nCVc07vrEcfIw== X-Received: by 2002:a37:a48a:: with SMTP id n132mr13712448qke.359.1614730964775; Tue, 02 Mar 2021 16:22:44 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:44 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 09/17] arm64: kexec: Use dcache ops macros instead of open-coding Date: Tue, 2 Mar 2021 19:22:22 -0500 Message-Id: <20210303002230.1083176-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002354_693593_38587E91 X-CRM114-Status: GOOD ( 11.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitised DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 12 ++++++++---- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ca31594d3d6c..29061b76aab6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -371,10 +371,9 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 - dcache_line_size \tmp1, \tmp2 + .macro dcache_by_myline_op op, domain, kaddr, size, linesz, tmp2 add \size, \kaddr, \size - sub \tmp2, \tmp1, #1 + sub \tmp2, \linesz, #1 bic \kaddr, \kaddr, \tmp2 9998: .ifc \op, cvau @@ -394,12 +393,17 @@ alternative_endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 + add \kaddr, \kaddr, \linesz cmp \kaddr, \size b.lo 9998b dsb \domain .endm + .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \kaddr, \size, \tmp1, \tmp2 + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..718037bef560 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Wed Mar 3 00:22:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF336C433DB for ; Wed, 3 Mar 2021 19:23:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EF601601FB for ; Wed, 3 Mar 2021 19:23:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF601601FB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vyymY05vH7RqsRKuKQoYiIt3jnphnFyU8lJil/qCQZU=; b=gSOrJcozd+jtGsfxaffRFqnbf mqD7NKsT+q1AHtKcoMm6bzqe/X+qz0hRj2U2xclDlJcbsJECQjYPd1g+l3kReiImaWQN03f5wiBjR ZuMoYD/u72f2mvBxvFxTTEmsXxS5XkqilQ/gwSUWE01NEvOar5m06w/76mbMgDM4MQKldy5+V2F4F sBHCaMZ81eGMLKwkEldHIO3GXqyshHV5ltwVCzhCem4NRn0yKLfoEVvMLHmUv6M4Lau9kpw6kwHQx 0wHAxr1B1JitJsdkGkknK+Cok6+dDG1l7V2HsbchpR+LessjOXNcgaKgo/0aqfqgSJj5gq4VdXj85 X7jjdqxvA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHX3O-006Bnd-B3; Wed, 03 Mar 2021 19:21:02 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT32-005HvB-JT for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:04:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=OPH3ilIIo1QVRT5mlIK+FYVb+e dq+tnNygHL6B8mlGcAmJKPndJm6vRhOGLzDJNPyj3wvb4H4/btoc1u0rZggT3O+OyHWHHJkd0BQD4 LsgcELVNGqKcE8HJY7BmOHVoyJUXHbU9H+Qnst9hFAeF00Eie0MZUmB0cCStxG5/EDwSRi5JhKCkW hWwS0LJ0DqkHQG0bVjwJaN/Gpy/Zogo06gSG+VpJDEbJ8xeLwWtD/HP+VPzKraiglAZBCxj9N6Tw5 +Uh/CFibkCb7ih0AkS72TEo+eFWHm79Xb2fF1koF1jVdTVkrLk1w9iqhbB24fExzP4K8GCgdtu368 9xHTG2rw==; Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFRd-000kvK-9M for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:32:54 +0000 Received: by mail-qt1-x836.google.com with SMTP id h9so6423697qtq.7 for ; Tue, 02 Mar 2021 16:32:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=DhKQLAF9gGWy+VQ4vEFv8vyOX67wEL5g+Ibv7M6YGg2GyOsWVpcLrO8l/KN0L5IZAc RxUn6cVCLaJEnn1GP49dKDyfXS2S5pAiSCfkfdBs3217gnmAIZ5KJF9kQwUjn08WVXoD eVb/xAgyG6HCeSJF560IMqYfvfngZgwPC0Gmto4sET2VI1otYwBlS0xkYWwsZi+5Y5wu CiGi9fY2dPpyuaQ0XGJefCQyLBo5uJyBd2Zt5atz0jeIcn/r/0MV3W88wPds72buWKfc OID9WDzMy8wulcxcHzZaME1ROXYkHzAQzPpJnO8T7pARrDaLZCrx6HDCoBT9CIwWJebN jArg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=oGxp25uKno8NPyqKDRiVuqJHdIf5u8LCeVwuzpm01JNYNe3JhQboJ4mmij42H7sw+D Jga85ZeFzrfU0WIInZ1HYjYlL1TdZ5eNPU1HIaQIQzNeSwVtahTkEX5PHj8IY1DUCOKb 32UOhxyzjyoW7GsSdsETmmT+qSZML+IaAoIyBaZMajmPXonHo2DrByw2TbUXh3DI0gm3 7oedTdlry7bQZNuEGVc3GVVXINEKDjPJXzMm27o7Aly7RMqDJWhSAUw0uzHzjJIubv3/ 8m6aVUh2pHt1a8Uv8e8prnFFnCoS6h7jSBDuit4aKArxyf9e4/b/u7h0/hiQh3wQ743l XmgA== X-Gm-Message-State: AOAM532lMG6bziJqmS2kUtGBFbwetlS5PG99MysD/AC5i3vFMaP4KX/U vrFPA5yevm8ClalZ/nPsriu1T5nzZpgImQ== X-Google-Smtp-Source: ABdhPJz8rw5a4BVvRDYMXLHg55FhSEneNbuZ0yxoLq7h3VikY6X4yYzXWKIMomKucbL+q9Rc23LaTA== X-Received: by 2002:a05:620a:13a5:: with SMTP id m5mr2243290qki.498.1614730966050; Tue, 02 Mar 2021 16:22:46 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:45 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 10/17] arm64: kexec: pass kimage as the only argument to relocation function Date: Tue, 2 Mar 2021 19:22:23 -0500 Message-Id: <20210303002230.1083176-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_003254_759385_D7BD776F X-CRM114-Status: GOOD ( 15.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index a36e2fc330d4..0c92e193f866 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -153,6 +154,12 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APGA, offsetof(struct ptrauth_keys_user, apga)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b150b65f0b84..2e734e4ae12e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -83,6 +83,8 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + __flush_dcache_area(kimage, sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; void *addr; @@ -198,8 +200,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 718037bef560..36b4496524c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Wed Mar 3 00:22:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36DF5C433E0 for ; Wed, 3 Mar 2021 20:33:19 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3773C64F02 for ; Wed, 3 Mar 2021 20:33:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3773C64F02 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=advOwH8ZLIQWOjainb8O1aM2XfhFt/rcTLC+awhz6I0=; b=AOgLZJJol7DgF/5fy0LIMfcas ezWpsezEEhhfc5FoJFJLr6K0WyHE2WHhaBgxkSEUv0B8ZQCuTVj3vhcHmE/4JZ77AwNIzdW2W4d5k aMvry76ngv1SDNO6ssj4oee0YLzcqaEVGEaCfRh2EL/1L4cZ91HZDNvNX1gQm8I8t9aWbSC6/dM5c a7i4jh3j2sRTtRGTdnkgpawk4cVoGPh9zf+BtxRXGeTIjPq1kDtGpg1tvXX6CetpcjdkhwQ7rwwnk 3RTZCCW9xlY6O69KwSBs3IcX/r35NLf6xW+tr8FmqJaXFqh1gyIjzaFZjmJHMwOCFnmq5POd8GnAY +IkYI/eeA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHY7h-006Q6L-Jw; Wed, 03 Mar 2021 20:29:33 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT5f-005IpM-D4 for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:07:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=CkOHd4Ubs6QxUZPpaIRvKfllHz4QaZDE7Cb9NwMEurE=; b=dOGn17Ljpu8dKn/REohe48yr3M mFb5yx7TZXJMUi+2TluzI0ZyllLu1VoLdXAS54m6ltJyO2zkwZUhpRXy+tnpR0C02aPt++ZWVItZv UhWWe8E8VNfRZ29dGwSZ3nFQYmBB74g9KLJ0LXUME9CtyrnagA5foVnqpqgNNuIfZzVgD08riHkZb s1XjzJV6s9D/ur1wrfNoqA+ryYSOGuCBBkCYXHly43vTjrsIZDBRhvkjsSazaYUFajjbhlo8Yz1nB TZnOgsOLbVZEB5Yx4GQ1fdT/ZpqiqX+Vripxg74NpmZle50J+rQ0zzVVVpVFJPiBpWMwxlnDIMGsF f7C6a2uQ==; Received: from mail-qt1-x832.google.com ([2607:f8b0:4864:20::832]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFQD-000kUq-0w for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:31:25 +0000 Received: by mail-qt1-x832.google.com with SMTP id j3so589192qtj.12 for ; Tue, 02 Mar 2021 16:31:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=CkOHd4Ubs6QxUZPpaIRvKfllHz4QaZDE7Cb9NwMEurE=; b=kv+djcD5Dn8tl5qEcKKg9wUfgrKRtIeHMDiyEGXGQs7F/ZmuhWe//JnMzxmKu3F7AH 88SbhKfkE78CfHCAU44Ln2BV0iQYNwXNNODQZqJTwJvNp3lruyG1V2IxzYqcxJbNeIZ6 girfcwSI1nPDGrreCNMXLSEJQko02mF+3Yihp7JjLbeVQdjBz1rF1xU7FCCpI1MADl1I pOeZtuW+wh2ejaP/7dgGE0DRh69BVBAUaJ+0RDEenZ88YSFdaBOLm7hTX3jYfk39tjpV 9gPTcFqCKqbCgpdfet/cTN4tW9o3BcjhJkKgOcD4wkV5+2WV0S3t0adzPBj5dTCa3CUU saSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CkOHd4Ubs6QxUZPpaIRvKfllHz4QaZDE7Cb9NwMEurE=; b=LGnQQNhURvdGAOi/fjGsiZhDDsOc7Vkf9kO2ciRHvCtGnPaQASBhftvf+PoeBNF5my plVutyxUa1c6j700uJdwB9kW1G2Bru3OLJjU28XRCy4p9jqXYUB2bpNTCmZqXgtK/lRN NLImskIKPX5Q/HJJQppB3drDLIs+WBqq7Vll3W/jb/zMXrnkPpk8ETXsfiqttL8qQtsa R4ZV2L7Df/QDQBYUmaLCULlgTz+bt7iuuc96g6T6/fsosrj/w604JyXpF5pEfR1fj5AC +fgOaHcJ4smxEfS+CL6GdkNHEX7K4omCG84N8hUC4VEjyjoVvh3XvuDHUohbROnRipMP 4Qog== X-Gm-Message-State: AOAM530BBh+pij3nGs1nD+l6p/V3DN4ab9hgVgdGySOED9Y9ZZmf0hXe qsyo5Sjc45hrHYWQIRNxC5NE0NWCh4jXGQ== X-Google-Smtp-Source: ABdhPJzh2Sttz/3luOleODmtp3yKBF23rhlq0H6VUfZnw3KEBQCZesiL+EeFQlA4puGBiExt3HvrBQ== X-Received: by 2002:ac8:5947:: with SMTP id 7mr8369297qtz.60.1614730967312; Tue, 02 Mar 2021 16:22:47 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:46 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 11/17] arm64: kexec: kexec may require EL2 vectors Date: Tue, 2 Mar 2021 19:22:24 -0500 Message-Id: <20210303002230.1083176-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_003125_338646_C8393A0A X-CRM114-Status: GOOD ( 18.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preporation to MMU enabled relocation, configure our EL2 table now. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1f212b47a48a..825fe88b7c08 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1141,7 +1141,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 9befcd87e9a8..305cf0840ed3 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 0c92e193f866..2e3278df1fc3 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -157,6 +157,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 2e734e4ae12e..fb03b6676fb9 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -42,7 +43,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -137,9 +140,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -148,6 +169,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_callable()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -200,6 +229,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_callable()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } From patchwork Wed Mar 3 00:22:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC179C433E0 for ; Wed, 3 Mar 2021 16:48:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 34F64614A7 for ; Wed, 3 Mar 2021 16:48:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34F64614A7 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=w4tjLuT88XI7esg9mATGYdwrDDwW4QIa1+frpU2SCPY=; b=S2uyEpEJ42GyZFxkPpayYG2O/ sR45IPPmsnOhqgriU4VQ8vThByvwVhpteH8pCREpd8fGQ1bS7694vGe/1DWNWgcJPeB7CBZojdWip 3pfQHiakLAFJn7LcZTgmOpE779KrdyS33LCHweUB2kSWpK/aGLccSGmxg0tv7zIR6epbDQ7xbiEwG 8FuhGvXCJ0zHV0O3nJqYZ95LTJXDqVENeSUSrJp7B/2mTHI9G7Jh+nYueP9cmG9vxNOTX+hfZ4i/I 4lgvsfXO/Yc15YHlqirlBjQQTRH9ZWXpBxD1UuU6PZIpd8Hib9H50bQU90ugL3975oa7aPUmrfE65 rvza3lfaA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHUd6-005euB-1l; Wed, 03 Mar 2021 16:45:44 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSqu-005EME-BK for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:51:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=wWRoz+XbR8tPPhasmzh2AGk+yB EuRAy3IBDuC86ZLeI+cQjlxKytX8HF57PjgEsAUoFFDWCWUje72RleQ+CoyQJDf6Rg0zSYhelLkMo ufYY4FKB/KQS+QVpJcfqRjb5KOusS98Ulk/zDjZazN5EeVu+jFj4RxgO5qAcPJ85A86KQxp5P4FE1 EzC4Xkw+L9QFP1YM7gjGpDI3nuyjAq+tukPOCe72VYXNpp8kaub77/u8ex6T/Lgim0gtJ6oKZld1Y sSbIX50RY4wOne81dL0Ld9wpoJ3/tBrIdpNR7cC437rK5DLUKrN/4l6o4fyA99P4i7y2hg0t1cXSz N2j7Bo9Q==; Received: from mail-qt1-x82b.google.com ([2607:f8b0:4864:20::82b]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIv-000jdk-EW for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:54 +0000 Received: by mail-qt1-x82b.google.com with SMTP id d11so15555696qtx.9 for ; Tue, 02 Mar 2021 16:23:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=FBdyqZaIcDyDYLlxElrZkuvpRkRku/nV1NAZj9UA7Kl0fuBORRQply+OelppUmA5/X YdOr7jgb6xCxZMVvxMAcCH+UB76E2UqPr8+Unv7yNzOfN0g/mRQJU850K0bipeneaRdC 0WfXYfl0XdrewOTEEVkNkr4Lc/IGmFcluOa3uHtSXXXbq7gKZ5vwgyBe/7c9xGU1wiFK Q38hSdvjT0G2vbcJ1e+1tLkWB8lGsOtC1XnvGT+Bg/yC/a1whgpNW0yKjwt/kUY1FC93 zcHgH6K4aNPWKbx2D1PifjppE8zXUC4vlX60NN8LCSNtkLWhJOCrvpiJkGwJyeoZ6GOV CuTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=A3quC9GajXSfJNTbigFlhqjEAGHbFLOa26NGSETz3j9nahhPEi9p4JpiRgjZJVRiZb 2el1W7X0Vc3s2eHOplVHLm6kq5b3uOOr+I+Ei/jI2h+kDJ+1r4IW8BFLBYWeW89jTDkz P+oeKNLx8GjHxBzOKdnnWcrzUA7ijSPViDl6ppvRp3En/Y8JlZYed7vLsn9zbTP5SjvM 4QrizbA5MTBZuG2HU51A4IoykH29dubz3Rn8gaR0RmQIsejusVqKkIzJJD0+eo5R7Ld+ rAd0JYY+su0zJ2b5wYTJnb7yYY20xQDQXm+wK8wMIyWmWTuiQGq7lsRLyztNXD9cgOJh EZpg== X-Gm-Message-State: AOAM5314/mKo/inMb3E6MH0EvX8nj2rOcwjYx0rcZYh2d0/3qL8mWHUx uWOoKyli9uSGJQSYIr75RGjWxg== X-Google-Smtp-Source: ABdhPJyyN8JxnvqauMS4QIYOtU4flOLUY/VfPIcIZzPJfu3L3JbZXHRoVShtyq5YtQyvU4C/7Kg4+A== X-Received: by 2002:ac8:5953:: with SMTP id 19mr20290245qtz.263.1614730968594; Tue, 02 Mar 2021 16:22:48 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:48 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 12/17] arm64: kexec: relocate in EL1 mode Date: Tue, 2 Mar 2021 19:22:25 -0500 Message-Id: <20210303002230.1083176-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002354_705125_DB7B86EB X-CRM114-Status: GOOD ( 12.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before enterying the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 1922e7a690f8..f6d95512fec6 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fb03b6676fb9..d5940b7889f8 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -231,8 +231,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 36b4496524c3..df023b82544b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Wed Mar 3 00:22:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5B8EC433DB for ; Wed, 3 Mar 2021 22:04:02 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DFAF861494 for ; Wed, 3 Mar 2021 22:04:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFAF861494 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=io/cWx7BdiEjCX7N1wxtt7K0qpmh/VjBgr7vmckYv4Y=; b=fTyQ8UY79VenUwMHknOcLQPfW yPf4C8LzB421UHpTVK5O2nCsKfilggkfP6KviJgtsowuUY7fGTdiUvf1S8SFIIYcpkfXOWPoZt9U1 DRcAWnadrBuP8P/t+mf1XWWKjxx03KiU1INH17wYSANuks99KlcglcOJlv7h8hT1lilt9i16MW1Q1 f0v7oBvDW0FNlazzWHp9o+zJU0PTruOSo2as/xur5QT1bnUlLr/wQGdAvVau97w0aqJHPcJuK6hPO EmoitQ91OSzZVEj2CO8w3uNSRqKFWhIRi2gevmG3HBx+3DpDbJpOKX2yBZRa2X+oCLQiKJacdDfDI QbzhIncqA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHZVh-006g3P-VB; Wed, 03 Mar 2021 21:58:32 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHTAN-005KWR-TY for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:12:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=s/Jg8GhVx+itY5wURoLis7aa3G nXHB7njYcqW0CLs2F70nEoGhUiuuL8ZT0G+BexlchLPkJE80tHtFeRjlZ5P8kJn339umsyVR/aWgp aRsmRzeSl1OrcvQLR7920d7M6yw2ozbKRD5BuzFAtfJJ8sOFt8v2skC4rSksggynIDArahaYE5yUK XnoL3k8vVJyHuJj9imIrYoHdUd84Tkb9LeYVSbY1TtiKgtGgK05fgA8qEbxaS2fXbI8yCxKWxWKEs nqfikMeFiyAXkIRO4rGKWOwMKJfB/tOQR20nsyzX87L3qRV6p3DPeNfxlmzXSNAQLVn4KkHPE31Cp 90Z2tzeQ==; Received: from mail-qv1-xf33.google.com ([2607:f8b0:4864:20::f33]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIx-000jfk-7M for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:57 +0000 Received: by mail-qv1-xf33.google.com with SMTP id 2so10894022qvd.0 for ; Tue, 02 Mar 2021 16:23:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=bKq3DOfy+ZZ2Br+ZXPwnr++JLsk2BOWy0QyrfsLY6aZS5OS4WSDGnNHfH8Icr0Z8I+ HOk/5OYLaspLekY1uItNjP9HFswk0cGO5JyOgS4Xbrfa9H7cSB4LdBlvUxuJhK4H+cib ru04FyGfJRo+sC3ysELsdQ2o9h1TMC4Jld9NJUUtDG6UjhaCyL1ezmZMZh5s6WU0PGc4 8bQe9pI4CW2cns5OQYOpxkOSPbubGDDyxQeJyQ9wZ3fHz7qOj0xd3HFKiaLqOqqwWAfP tbh9FZQ+wJC+teNKGy/f19ohcQicVlGPjTjHkfLWWPH8aSxnj73rrhZgLmKrV3EOd8Z5 f6Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=MXFT0o1IcxQTWxratkywKm3meuJAZf8qTWwAe+Bdm/UtM5DxRNA2SHwYiU2jQGAevw E18PrWB8L//4ppym/mis6nrsmvvl0kiMEXDKZECOM/yv9yKEi1wOOKTXaF1LrR8A1rWb H93Yo8PDkK1GPThYtSBSvoA/XJJmcgrSqeVUF/3p55g8Pv5CsrsVL9XQE7tYSkP+tlHF VyksFEU2pwUfbyu3XYxaL/4Q6ZCsAXJXzZaGNWhi2Dxi4JCVhkdSDLRpm7dlsy18WUe2 BvmdDqEh5o9al56uWuwfmqg9QdTOtmw6Y9aRQd0VBmg9RGe/gpM5bPvCMvxDfAZEFP7X LkYQ== X-Gm-Message-State: AOAM5312lyFQPxT+VQzNXNolLXhk9wdVUQh5KYNHAUbmIvxoiFBHV4dt rrzrZrhUTnxvLxS+Ea2m3QK8eA== X-Google-Smtp-Source: ABdhPJxvNv+/84aSaILV/bO4Xb7Cban1AhQ63vws8517ykCszMsFD3aYB5zfOTZP/wFR41IfZGWghg== X-Received: by 2002:a0c:8f1a:: with SMTP id z26mr350314qvd.51.1614730969894; Tue, 02 Mar 2021 16:22:49 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:49 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 13/17] arm64: kexec: use ld script for relocation function Date: Tue, 2 Mar 2021 19:22:26 -0500 Message-Id: <20210303002230.1083176-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002357_822879_595E6C06 X-CRM114-Status: GOOD ( 14.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 14 ++++++-------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 2f36b16a5b5d..31e459af89f6 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -20,5 +20,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index d5940b7889f8..f1451d807708 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,14 +20,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -157,6 +154,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -177,14 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index df023b82544b..7a600ba33ae1 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7eea7888bb02..0d9d5e6af66f 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -92,6 +93,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -152,6 +163,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -336,3 +348,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Wed Mar 3 00:22:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3FC1C433E0 for ; Wed, 3 Mar 2021 22:05:35 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7E12661494 for ; Wed, 3 Mar 2021 22:05:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E12661494 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=qYG/BVk2e7Ror8xd1wWHZOQYmQXiS+AdwbsWZXIqchI=; b=ZSlvM9uaW7VxW/Gt1EYN69pqA 9GuvnxM0BK9nfEpOFtS9IpDhUsj9eHKgCRO7NDM+9ejKqGO9oZtqkTPY+xCxnpHnwv1zg7K6hp0/2 bZewBZZDAyLGYJnaYaC0DrKnYMup0Q3oqp5OFAZCQklbenuBxEJa0BaUbWLggcS9LHUAcEGLHrOGF FhGTCimu1jXIXuv4wgYv1ICHFAmrWuApmYeAmkH0bKujq8rWoKsAaPeH9zhM+1MTdN+Tbp2OsJfwl 469NQH9C1z/48IOwtqwfiGH/VkLsvBbo3US+LENjzWT7xtaxjBDUAvZuczK1wCXZlTnE0UiMv8zos A7sYO5P5Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHZXw-006gUO-Tb; Wed, 03 Mar 2021 22:00:54 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHTAS-005KYA-Bv for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:12:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=Olm+fYXYbuVgp6bWsix/kMAWKOrqJlCkAHr9xCmjIpk=; b=jPmUKU/+Rj+vIMaAS7caPSj8DG KlQlTf6rJTtB6/pwK6YSr+cGpEGf7RbdtR0KBZ3p1uYrcrTz588ohAd7zV2qe1oWkOOEe2YwCsLbQ KSe+ea6GiwGxuskrBvAuuq4rNNbfaULuhiMTCbHiDHgeOBzK50xiOj6yA5DNnN0TGzsb+jx4mEkzX +9didgcLdOlOSD1Z5TWF+aS1WUUC2ZGQIKHbDLlsgvZpGY05Yc5K3MX/Uv1OQE1sguV92X/sPXwbN bSp0+IZGmMBT92ObJvQcmmKB3fyM8GYj+RYHvr3wJzx0mZsKAEGaVBR+J+yWX+RzXjew6W2ApSK+b c0qE7H4Q==; Received: from mail-qt1-x82f.google.com ([2607:f8b0:4864:20::82f]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIx-000jfg-8w for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:57 +0000 Received: by mail-qt1-x82f.google.com with SMTP id r24so16258277qtt.8 for ; Tue, 02 Mar 2021 16:23:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Olm+fYXYbuVgp6bWsix/kMAWKOrqJlCkAHr9xCmjIpk=; b=K8kaTNPYcS3zquvbMQ0dtlLepeH8HKg/xCkwC1qj6nsKzbvTfQ63+Ks/RZEV3bUkmE y+QkmQaX6vcgEtn1YnwU7oJSM7haYdmSxw2yqC1Qf3YNDgEFTj1vUr6PvuQRH/KJFtHo JD59CklExk570Vv9Pr1/M82SYFlrf93ssmow3dcobqItrZ9QMnpounC4Sh2DTaIcy3oc CvvomkmEgsTKwkBVp9BC0SmnLqI5CuktCtdt20kzsgz1kO8YMSQGsgMHC9E+2iXhK6As rp6QpILmBre+MfVCQrotSwc9ZOttqOwTTzIP/JkW/2P0pHt12j8jhRTB6iZgS2quUD27 /56w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Olm+fYXYbuVgp6bWsix/kMAWKOrqJlCkAHr9xCmjIpk=; b=cBPH/pWXxHtM/+cPNzXzMs08jrFQoXA4xWah5p6/E+i43ZQyTHZzgq0+u1XP9tZ0fC AxP8FLZUVlYG8JLUCdsWb3O5Q1ARCN+dZKBM9cPtkpDUZYvCeJvB1bib7OYbr0n5zYso A2Fl3KSvO+crU9MCNaUiccCUqDukhCF89yKl8dsFYrUEWKJ9CmXV4cYmiXvOi9LOkCgm VySaTvxL4OUqZJGK5Y20MyI0GxaQm+JztZPe1dotLYypLSx9yGvRp70TYOeBK9fYDkCy GBS/p5rlmCkS/WuWNpOppt869xEYDlHrd25CgzgEEZp5NSCD4dQOfGY2LhIhbxQmntog z8Mg== X-Gm-Message-State: AOAM532DtXZ53/5kWzarOSpOIUckWnPG05I3xevBewToJG8yh1+lybfD TsShZa81+gthbf51kpA4DNXPOg== X-Google-Smtp-Source: ABdhPJzanxLZpfCCAQZtVgjL5PevxfgLuq3u4oQDxWDQ5ASJZdnQkNQpbYmMikL7M6jXu/HxEyd+hA== X-Received: by 2002:ac8:75d4:: with SMTP id z20mr20660664qtq.61.1614730971180; Tue, 02 Mar 2021 16:22:51 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:50 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 14/17] arm64: kexec: install a copy of the linear-map Date: Tue, 2 Mar 2021 19:22:27 -0500 Message-Id: <20210303002230.1083176-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002357_631775_ADF3A8B6 X-CRM114-Status: GOOD ( 18.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocations with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 29061b76aab6..3ce8131ad660 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -425,6 +425,25 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU isb .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 305cf0840ed3..59ac166daf53 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 2e3278df1fc3..609362b5aa76 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -158,6 +158,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..a31e621ba867 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index f1451d807708..c875ef522e53 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -153,6 +153,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -169,12 +171,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_callable()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 7a600ba33ae1..e83b6380907d 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Wed Mar 3 00:22:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 057A2C433DB for ; Wed, 3 Mar 2021 19:27:53 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C420364ED0 for ; Wed, 3 Mar 2021 19:27:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C420364ED0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mKhgK4EtUu32rnJQv0bDEEpaVWXVT0hj/Ul/6N8AKQI=; b=JuduijANCnW6bhcFXi30zH9ME /71bdr2K850Fq9zq8Efv6mn0wSfQi4SSG+av0jfScIi/ufPxeSYXh6HkEy4VOqYVTZdPONti2obN1 ZwO9Dcc/LricTJ5u8flSpsAOH2IhTFTTzJulD212aPO/JrB0SisHpipPKAmfC9XsIG/TPT8EQ4VQk 7RSuP7lNeX8Q1a/YAIX4QZCB9kGtDPRXf9kh0cLnqpHBR1/LDp3OGAbryJ531nfBvouVfKFJLnpBX 1N3XyGxOT9ssuUFNlvygLnzA4WQx9Fd9Dq/mdvjTz2bXnesxCeLbRyjphnK6I3Ibov/Joh6P2E0zi /pc+qy72Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHX82-006CtH-4T; Wed, 03 Mar 2021 19:25:50 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT3O-005I2K-OC for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:04:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=UKjtjEDCRzQ4WDzfqfuqgp2BgU7fhP5QVy8I5s6gkYg=; b=oEw5HqJsbF4gs1gf/MIt4YWdBi bWRQ4TvZu3+rVCuXg1B49cOTGem4IUa9I5H/uhTH2X18TQC4O/7IrPwlFnjh3ayQRb7l2Sr2utEdH rouArI5pIePvhP3vQZJa8tSQh1nhtx2zWIca7KFizb2PTksiY4vwiCKMY9EPELKN9ibDtMJKrt9w4 tKcqIT5QgFIQ6DM+JZ4ALsw5Mdv6DsSKCNjBMTAUGzq9O9o3ry+XUB9mNBVB2NCWzZWQuAbm+LaDk pmA3MMycKyot3i3i6o2XAsMCjYHcHQ+JA0zklKHDNhod6BTdJARHR4pHJlw00wBXhWeAFZK/8lluz HZo4K7Dw==; Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFIw-000jff-W2 for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:57 +0000 Received: by mail-qk1-x72d.google.com with SMTP id l4so7558617qkl.0 for ; Tue, 02 Mar 2021 16:23:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=UKjtjEDCRzQ4WDzfqfuqgp2BgU7fhP5QVy8I5s6gkYg=; b=fSLQuV82Jj2QUma3YmUBsnpeEsIRTLafxYwB28IFsK6PJ2gWAr7BlPSOvdbgCDrY0V 7HXoOxavXo2LYem8r3aWvfCujfpeTOoQ23KaKeI7QTwWr+XBRtc9A+OLPcoKhyscI06N uIGWARRpGBQZaGjaHiw/VVaw/7yLctzYfwfMwMbH/polx4wQEMsB/johD9Y3JLxmUZVz aZA24DlpuLA+BHJcNwtQl10b5bntN1CLS/hFgv6E0o7lLCbLeeoW1elkT8BymHot13xH 0zF/IAF12YxFrhHx+fREOYaZzOCleNBzrBSYZsJZm6LrwYV6MWmBlqfc8CNjPuVFotY8 NcIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UKjtjEDCRzQ4WDzfqfuqgp2BgU7fhP5QVy8I5s6gkYg=; b=oavaTEjxH/Zic7W21m1pYEEqVVKKWvtEihURbHS2ot3uWPt8ozECoZZ8K424JusT4p 54fdF8ucQ0Duyaqg/VNKNDvWytMh9eRCAN/5/FfIAqmUMKMqtC+Nn+RT9ShX3aQ4WfCN Ghr0lPEIjDTAt5FlldkkM0iMmMOhgRcd3ZCm6LYBXAHQDYi5DMFKc1XuqN3sEdZ+XtHG r9RNfiCW1swS6t/y2GB0ZOTf657Xvg9kull9m/th0dR12p5rlT+iy2+i0J6BRlGBZNdr S3PjldwkY3i25HmJZuBaZsWFAyDcZdod6z5wuO1oJbfMxx5NnRSMEBk/lRzNJhi4KiF/ TD9g== X-Gm-Message-State: AOAM531skPyxIU2artnA89+Nwxxdkh+h4ATo4ANKaAhZLohR3uaySzUf yVn05u8zmMk/nUQSghwm2WekPw== X-Google-Smtp-Source: ABdhPJyKPWdMLDIUss61XZukaIXsHhimeP40sTJYFttLXHFVC8t+0q7qOi4TzBxxIkgsER4FQ5kv4g== X-Received: by 2002:a37:4743:: with SMTP id u64mr16004437qka.350.1614730972452; Tue, 02 Mar 2021 16:22:52 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:52 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 15/17] arm64: kexec: keep MMU enabled during kexec relocation Date: Tue, 2 Mar 2021 19:22:28 -0500 Message-Id: <20210303002230.1083176-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002357_631145_60368442 X-CRM114-Status: GOOD ( 16.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 ++++++++++---- arch/arm64/kernel/relocate_kernel.S | 33 +++++++++++++++++++---------- 4 files changed, 38 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 59ac166daf53..5fc87b51f8a9 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 609362b5aa76..ec7bb80aedc8 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index c875ef522e53..d5c8aefc66f3 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -190,6 +190,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, reloc_size); @@ -223,9 +228,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -239,10 +244,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index e83b6380907d..8ac4b2d7f5e8 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,15 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mrs \tmp1, sctlr_el1 + mov_q \tmp2, SCTLR_ELx_FLAGS + bic \tmp1, \tmp1, \tmp2 + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +43,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - mov x1, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +74,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Wed Mar 3 00:22:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12113659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6124EC433DB for ; Wed, 3 Mar 2021 15:10:32 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DF8E164EE1 for ; Wed, 3 Mar 2021 15:10:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DF8E164EE1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QqqRfWfAnRNsVY6usBaa0NBN3wvGm58U+rzLkKHVabE=; b=HWMraG2Uviphda3izssPX2bzI qkpz78KygZB4TJ+reK+hST313AXpv1jbuIDzj8GYnnGLxzpHmOTbDbFCmI4nJJAsdhMOG78W9Er6F G6XVJlPLoM9mTj1bO0WMsYWaudtIDMTRJa27ykv78qqZIrpZws5HtAIceLzfX6bxwdNbIMskkCaR/ bEeS5tJLVs2+YITYpJSvxx2m20y4FdI//NikCSET+EEWSWKt6F548UpvCLfbc7WtT1QvXqDRG4d3L YwxJtneObsi3dj4xKOZugYcAy3rE0IOL5lmCE8w0Sv+sTZiyMZshjpHZbdDVyPvaSNSoe3YC4uiBU 3yhPmtFrg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHT6K-005J0G-4K; Wed, 03 Mar 2021 15:07:49 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHSBx-0054aJ-Jx for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 14:09:33 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=S1HFPR+Dxfysb6gqJV7/Bfp6t0 FCGQ/RMwq4WcqIBOBZeDiaTyG6T8slYVm8Mx0ItKa5p4v1qlBtsCZ4yx4JXck0bm/jsq1HwDMudhr BGJ1y5x4SoEgCqL49HwjP4ps0iLZnp7G5HXLDa3DxgJ3UOT0jjwFJb4o20y92+qFOgMZPaYg0j0BK 0FRRKYHydTgh3b8S0wB5TZIEpBFRn4iQlV76KrtNOlGCVav+hd2uzc9KrKfJVI0pGqqktIYkq3pEb hSQRfgT/aqePsXJwLbmR+IFuqIRtkOrgzGt+t1LCZiluzw2zSxpuUfxl7xU+CuXccmu0XbRq9IUhf FhLNsFjA==; Received: from mail-qt1-x833.google.com ([2607:f8b0:4864:20::833]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFRn-000kwO-Oi for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:33:06 +0000 Received: by mail-qt1-x833.google.com with SMTP id b3so16255939qtj.10 for ; Tue, 02 Mar 2021 16:33:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=cbwp13Q2ddpME4q7SnIYDmvYSLO9Ch0h6chGS0mJcuKYrVSiyW9RqugBaVOLREGq/3 RaXjQUi+1f2H7Dq7uhH7XbTZd4G8/Xc+rFwYUEEx/AdUSO30WgYG1LUfV2w1G6wCcjc2 /8GAhHNr0K4pH8SgFuD2VradryKueR66b6D+IX/xPN+SlF18R9pC4oMGtJaJMn5wE4eO 2KXKyC3D/rElN2+qfLl+X13RT9wKObgzzMT/qtKkvQmWMY9NClwSB/YVP+mMnf17tCpK BcdzbAKBOoD7Tz3sqa63lxj0/n8gza47BWFQQyLZKmLhuN8xvSXm97fDqDAG6jrXOUvD BH8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=eHRma9Y69psTqhzyD5tAhwKVkgnM7rjQutqGtT+GI+j1nhZuo8VGln5KI4NuF0mOwV 0NYy8wOVkn8o29lOArLZYctlkY9NRfRn36QLL5XgOtDPTgjAyXpCm3PdDfp1IHm2vimb PwFxi5HFzF5er+/l4AjUATZF8e/hTEAnQMcNcCF1ieD1ZmpeAvAXUC/O7NDHuuuqBti3 L7xoy6nmxGCB0EtmmYhmiUbY5LUQaTluu4NYzO8Kyda0veyCsfW2XdIKubtuVXx0uSAI 5MxzYUkj23bxR28kT6+ttu4M1V+yhYnDx13+7yqXXvzIENkhOFIFFwRpUmSTlNb36ma1 YKrw== X-Gm-Message-State: AOAM530xmosVobdCsWb6ZoP0IUTWnI5+P0zDAE7yOnwmn8wyLSfv5+61 zIcLlKieozpukIR//ehtZpaGR8hyxtS2cg== X-Google-Smtp-Source: ABdhPJzDM5pLJqM5foQKA752jyI+AxEsX2Dhp+xecxEILxLFKkHZG6f5Ei6Ery303Y4YtrOKPjg5sg== X-Received: by 2002:ac8:72cf:: with SMTP id o15mr15747863qtp.118.1614730973753; Tue, 02 Mar 2021 16:22:53 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:53 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 16/17] arm64: kexec: remove the pre-kexec PoC maintenance Date: Tue, 2 Mar 2021 19:22:29 -0500 Message-Id: <20210303002230.1083176-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_003306_578485_35E4D4DD X-CRM114-Status: GOOD ( 10.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 40 ------------------------------- 1 file changed, 40 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index d5c8aefc66f3..a1c9bee0cddd 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -76,45 +76,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - __flush_dcache_area(kimage, sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - void *addr; - - /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -200,7 +161,6 @@ int machine_kexec_post_load(struct kimage *kimage) __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Wed Mar 3 00:22:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12114211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B58F5C433DB for ; Wed, 3 Mar 2021 20:46:49 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5B77164EF4 for ; Wed, 3 Mar 2021 20:46:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5B77164EF4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=N1L8QQUtsGz6gXhIWKu7VofoVG3uEimmklfclzGedFM=; b=ipQoooZMb6qmOsY+/MH8RXfxI gOlIJ7EMAkMJNtGbMD0o5VCrw33/v0TPK18oXIDYn2ORY0wtd6w4UhEQHPRGDpOqn6rbart8UCyng m5P5SNQbqNrZf1N0Dkxy7OtyZwHc2sc7eTQnTb+R/A7XUCpgPQf0M+LHSV1Wvqi1+me3ROtldHVmc s+zMA8r8J7KCFyD3afFI1zfeT6ad7A10vbcYlq8wUr4INZwIjcg67viuIB4H7lIiuq3PnymQjBELL eCgs+iC9chHGToKLjaVKIC9ZPNmzlsfnjUyXxwtmzLuwjSVMxl+GyklvSU7ZyRUd1xgBdzUhpd0M0 uSWEZsvEQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHYKA-006SqM-Lu; Wed, 03 Mar 2021 20:42:26 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHT6e-005J7Y-2E for linux-arm-kernel@desiato.infradead.org; Wed, 03 Mar 2021 15:08:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=RGjiQfz1fmzi7CHo80SRyqmJEV zjvfsHcHPql6ICZKfEzwV5nBtsApMATMme/BKFwZueofWXCxA6rBN3OeuKIVUSMRTZIFrUfPvN5+w NIDy/U0ZTcyuHJCg+Iv4YloF7XjAjSTXU5dVRm9vDaT79aHvP7XJRFmEu4PerD6iqdvTnKShXvCCv /jPuLyPgVk2eiEmSMyty+Fxm3pDhg0prDQTOsiEZi7WKjCkx/6euiUJzihPOxv2FI2Olq3K1zPg3C jG3nYphaT3JyN/FTygs8CgNfgD1zoURu/ROHB6csIkQqTfjN9PMZmOmZu3jbl2xdd0V9Hq3VaFDui vm8I93NQ==; Received: from mail-qt1-x82f.google.com ([2607:f8b0:4864:20::82f]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHFJ0-000jhU-19 for linux-arm-kernel@lists.infradead.org; Wed, 03 Mar 2021 00:23:58 +0000 Received: by mail-qt1-x82f.google.com with SMTP id b3so16242392qtj.10 for ; Tue, 02 Mar 2021 16:23:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=oPIjUKZfKiJwU7NlOxCWJ5OPY57CbXoshRc624GxpBnFgpup2P/nV58n0kK9/w2XTx brLP+1+u8lqb56VBhBgco7Y8J95WM0noHIV2xeYo4P7naJMOF8jyTmpN1bSO7ZzqBYkf wLnTE9IuEPsdQ9DhW53OsI8CGals1KWGCUuu9oUKLerRbT0Eefa0xtN6bNdQt8IR07uA 2nsXk3Z0vFKr1jZmw1N75kg8R6bKWnQATRIxJn4n7gJcwpld3SrUrVk/3xub3eHr43iP T9Uf5ZzJh8r4nx+VCew3/uiDHEi1lSTdW1/r3/oOd9UBCwABx/E9DzUpEJ1T7iG2PJER ybYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=IGYbvqV30lK2cAzTlG6tQxzmOZICOzQHcrVuGPDkkYrp7zSRlrhyC3d6iy1pwJCuzt 4lE7UiQdTv36Ilws1zkeirIx+mR2EmBl2jGpZNwsliH6B9QgqLc2vLN2t5m0RfX14dDK qWA4MulGrcey045OreYo9P5fDC0u848esAa0zwViShoyqFQ7NHDAhZR4NaYVyxHeCEGd ElsfPadVTVcu9oAtt/k5jatWWvs/TD06zLrp+o92WizT1Chxjdj15BCDBgFDhpvNZLqP o4lTrdeQ4kdfLvwmLRVYsaX4T7bm7UIJDLJHvE59O3LaI6On8duZ9knaG89okHvSJUye 3n9A== X-Gm-Message-State: AOAM532TzmfAgjwZe/a7XdeWBZpqxzPMkQhJO94tLzBRH2sHG5zB2I60 OvEb3Bu7iU2CzrAxMVJZMeX17g== X-Google-Smtp-Source: ABdhPJx5815Pblab/QccqNxqmgkFadcK+ZfVV2aeyidjFjH4QiHxtUtFGn8129DzTiPluIgZh9HpDw== X-Received: by 2002:ac8:64c:: with SMTP id e12mr20040140qth.103.1614730975050; Tue, 02 Mar 2021 16:22:55 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id r3sm16690512qkm.129.2021.03.02.16.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Mar 2021 16:22:54 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v12 17/17] arm64: kexec: Remove cpu-reset.h Date: Tue, 2 Mar 2021 19:22:30 -0500 Message-Id: <20210303002230.1083176-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210303002230.1083176-1-pasha.tatashin@soleen.com> References: <20210303002230.1083176-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210303_002358_923216_B72AA93D X-CRM114-Status: GOOD ( 15.41 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 5fc87b51f8a9..ee71ae3b93ed 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 37721eb6f9a1..5d47d6c92634 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) /* Clear sctlr_el1 flags. */ mrs x12, sctlr_el1 mov_q x13, SCTLR_ELx_FLAGS @@ -51,6 +50,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index f6d95512fec6..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(__cpu_soft_restart); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a1c9bee0cddd..ef7ba93f2bd6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -23,8 +23,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -197,10 +195,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(__cpu_soft_restart); + restart = (void *)__pa_symbol(cpu_soft_restart); restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else {