From patchwork Thu Apr 8 04:05:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A748C433B4 for ; Thu, 8 Apr 2021 04:07:58 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8E988611BD for ; Thu, 8 Apr 2021 04:07:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E988611BD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jf4fJUJSA9KF2U1nNQDZaCT9B1ALjOLGT9H4Hag4XfM=; b=pwRDMSAmolkx4HDvGwXCjUM2o RKsxVVoNuNHxKP+J3wUK5+bWaOvr6FYGqktqtQERexb1iWZ/UKuh085u1kOkw9Du9no1V2KR7Ztq5 a0OcO2gNFqugZpcI+slvyQGhF2YK8k5+0e3MisrhiDUD01qvaA3l/55grhiTkpZN8Ji/0XzmRTgSm TCDVbVLFH9F4YzA9mID4or+Oj/d8vqKH7arYlMEmskwf+8Qvd347mquHTEsrVFw0phaxCuhi37gf9 6cE9Lw10mi74ecX7+Lml28+vEYVYU6Qs287M6ZHcr/5/loGIhvLhX5K/i5GerVyK2WN8DYdKwehFs ++r9gyr4w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULvx-006lqN-Oj; Thu, 08 Apr 2021 04:06:21 +0000 Received: from mail-qk1-x735.google.com ([2607:f8b0:4864:20::735]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvJ-006lVA-PV for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:47 +0000 Received: by mail-qk1-x735.google.com with SMTP id 7so927418qka.7 for ; Wed, 07 Apr 2021 21:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=X9eAU/eM3040QGUYmUbw7fYBZz3JtF3avBuvT24i1+XpVGvbIi9BNTH2rSlvhxMZ6p vB7bmo+61os1iTwbbNQRR69s0xibH9y2Zh+JxqbT2cEzOMZyXY4XleOC3VU2OQbelf4e qUx64StBUbF+O1txg2vRkmFpmHWn5MioQBG/fANNQ2YzUxTJC8uDaJ4j6/ABFGO80TmL lT7jf5YqjgaPI2Fa4vi35ToQnOb1K/7GhDLWSoB81CHHWcprY9EhPAQcUKCUF69eeTrQ 5rcxFBkWQYTv7dEA+PHoUkVJSA2Bu07e+E9jh6dpYWIsqXcT3kY8zV4f1HpZcTG33b+V 3i2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2ZHo6bqqoFQHDt1XsnxWzUIHllyauSrjIJToDTBiZLY=; b=UlbaO277ZukoPiY/g6oykAW1G/vAjqKs5sG4pgSjYPQAt7xjYzSUV6iQ+mGpskeQec l3ZsxHTRU5mTEc8E/2jv4WQYX/niYo8X9nU6Mu45VDpy9n07zY20ok1kuFHi3FhtSwKX AGQABkSo+kf8TZhRPg8coyW7GhkThyGDerMl0pgIjr3Sjsi6JpWwLVWqFjn1hIqFiAD4 MX26Tvb0gSgL+WF4Foo8CmgH95KgZGUQX55kn3cLdoiOOX22rrHphMUzK+44PPU5lZNF na9a8FZuV7KkB7Kdq0TanojwLlhPlcH8C/YSZ3L+d75OSYMsIvb1Z/w0gDGYEPrxFrOM XGJA== X-Gm-Message-State: AOAM530zf5ZZBW/ho93kwY5bRSfN1UGl9SfPTW/09hKJ/gbfa33lh9nz NKmY2iT3GYHXi5sW9d2685q17A== X-Google-Smtp-Source: ABdhPJxYXvk2FxZ0FX2jOzjDLvti4P2RgEhIeGGR2Vwo3ThVtxJOO1wAhCh/APii2j5zwyiwpg/lrg== X-Received: by 2002:a37:78b:: with SMTP id 133mr6607278qkh.109.1617854740543; Wed, 07 Apr 2021 21:05:40 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:39 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 01/18] arm64: hyp-stub: Check the size of the HYP stub's vectors Date: Thu, 8 Apr 2021 00:05:20 -0400 Message-Id: <20210408040537.2703241-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050541_925975_D878E9D2 X-CRM114-Status: GOOD ( 12.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Hibernate contains a set of temporary EL2 vectors used to 'park' EL2 somewhere safe while all the memory is thrown in the air. Making kexec do its relocations with the MMU on means they have to be done at EL1, so EL2 has to be parked. This means yet another set of vectors. All these things do is HVC_SET_VECTORS and HVC_SOFT_RESTART, both of which are implemented by the hyp-stub. Lets copy it instead of re-inventing it. To do this the hyp-stub's entrails need to be packed neatly inside its 2K vectors. Start by moving the final 2K alignment inside the end marker, and add a build check that we didn't overflow 2K. Signed-off-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 5eccbd62fec8..572b28646005 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -41,9 +41,13 @@ SYM_CODE_START(__hyp_stub_vectors) ventry el1_irq_invalid // IRQ 32-bit EL1 ventry el1_fiq_invalid // FIQ 32-bit EL1 ventry el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) - .align 11 +# Check the __hyp_stub_vectors didn't overflow +.org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K + SYM_CODE_START_LOCAL(el1_sync) cmp x0, #HVC_SET_VECTORS From patchwork Thu Apr 8 04:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78325C433B4 for ; Thu, 8 Apr 2021 04:08:24 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 13BE8611C0 for ; Thu, 8 Apr 2021 04:08:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 13BE8611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=aDrNGcQsUWYMwonvnQfh90LVXq+vVHy7hn8P1biowAU=; b=q8mq+m05AcIXvdJy4sfrwZx1f twI7FI3bSOh6BzUyaH6hUosevYFHwc2BL6knGjb5GBLZlcpMLwTgvWmwXYmrO2URpEJIVieAHP7sk +Fd634zFi7Dhh/RKPBiYQ/a+bqr0PEdkGg/nzRCURGHGli1cC2UhDqiLRFEip8SiYktF3LfrEnPOd lNmdrAFEPHzojYD6bSDu0l6cAloq/8EeBkv3BaKe+WOv1FqEFflv2jAQCNdnGYPtvXi/vTI+KO3CB ugmrjKhlNDchOYzbXwYDCT3Urmfds7Ly0A9Dz97nFqvoFvI0Lcji+8wpDO3SREiMZyRsLg3+eXlTG C+8wBoXaQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULwE-006lwp-G5; Thu, 08 Apr 2021 04:06:39 +0000 Received: from mail-qk1-x734.google.com ([2607:f8b0:4864:20::734]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvK-006lWS-Uz for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:46 +0000 Received: by mail-qk1-x734.google.com with SMTP id y5so918310qkl.9 for ; Wed, 07 Apr 2021 21:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=HrdrsH+/LJvmodXx6qJUKjxjbprurxBWQX7rF7YH/tot181AndUWuV7gbnA0UKrCI7 P33v6LFzUMT/CxhdJRYj39gFLLn8Kl24eYKMWwa5ryYEp8ZR56p76Dx73+juO2TS5fmp 03vZh8F/ou36y+EHeihspp1zs5O7FTxECJIbzCqeYzBohoD8eVUWtj5V6ci9q3xx7aO5 65ZjTbwF8/uVodijBWQC1fYHP0gWkn7GLjbIKlV1CsDmRzTQV9JCjReAOMPviz5wlbsO gGMCoefM2xm2+wWIHCBHlGKMERalPbcFc8KYZkj4p3rp0rYMbNOKjl4AAZ7ZMZnf7Vwn dEWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Me+IJOmpVZMJE6OZITjjNJXxLybVS0PvpkZtNmZgu9s=; b=C3Vc28hmVdUhb3XYrfP4xD75+hmA2hYGY+woXKhA7qQ0KB6jDilXnofslBkM+9AqWi lO5Cec4iJswHN5GebRy0Ybf7LsvHo/nicihXMpvi/lvmQDfHMKBLCu325U/ez9hH6NcN d7EPZhHAbr185tN7MeeZnR4q8bxuiQ0Mkqvizc6zy5koJGfX25J0pTx3by2IJru/3qw6 JEKtdCMhSEwKEJc7L52dTZ3VpyjmVXcT1LCneL5mJcP8UbzAc+M7dRfwQt0nbahDPPrl EumiNwngDN+tx8a070yG4APRjg/p9qyn85RPx47Hj7qa35LUJ8G+ifQi2KbIy6534CS8 r/qw== X-Gm-Message-State: AOAM530yRZ3rXu41KGegmJR5ASp4OwDGgCXzHC/WyQkwCcYCmVV/q5Uq M8tlQ7gayeAwgxcQdtfxj3mt0w== X-Google-Smtp-Source: ABdhPJxYHvA6IkQyz8orCAuu/6NxJtk1GKdC6ouuVl2yzdNWDbtFSj8FFUMuYUgnnPsyTxVVAcEuTw== X-Received: by 2002:a05:620a:10a6:: with SMTP id h6mr6649236qkk.366.1617854741843; Wed, 07 Apr 2021 21:05:41 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:41 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 02/18] arm64: hyp-stub: Move invalid vector entries into the vectors Date: Thu, 8 Apr 2021 00:05:21 -0400 Message-Id: <20210408040537.2703241-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050543_209494_C83909FA X-CRM114-Status: GOOD ( 11.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Most of the hyp-stub's vector entries are invalid. These are each a unique function that branches to itself. To move these into the vectors, merge the ventry and invalid_vector macros and give each one a unique name. This means we can copy the hyp-stub as it is self contained within its vectors. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 56 +++++++++++++++--------------------- 1 file changed, 23 insertions(+), 33 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 572b28646005..ff329c5c074d 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -16,31 +16,38 @@ #include #include +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + .text .pushsection .hyp.text, "ax" .align 11 SYM_CODE_START(__hyp_stub_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t - ventry el2_sync_invalid // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h + invalid_vector hyp_stub_el2h_sync_invalid // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h ventry el1_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 .align 11 SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) @@ -173,23 +180,6 @@ SYM_CODE_END(enter_vhe) .popsection -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - - .popsection - /* * __hyp_set_vectors: Call this after boot to set the initial hypervisor * vectors as part of hypervisor installation. On an SMP system, this should From patchwork Thu Apr 8 04:05:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189979 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 094E8C433ED for ; Thu, 8 Apr 2021 04:08:31 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 80F9A61157 for ; Thu, 8 Apr 2021 04:08:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80F9A61157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=VEi1qaleTRDAmZPEG9yNTBux/nw1x+assWfgyTJJl4g=; b=JPqzNcKTRV1c2xISqz9y77UUo 4apu71BgrA/JwePLwvCw1g8XbZ/aBsP3u0ospDvi0SQSBH8PJRi/qkxljqoYBRpAoRfHop4eUiLMn KqQvXEmySgv1qw1pCm15VPdwEELIFvUFHEjWKV+/aVzHw4SEBoHr09siHC1d2q6itH3ljqe8nOob0 A3E9LoezFlS0iP4ysOI+WzgyOqS/j6yDu6yYeWt+4039Bv6gAN2PIyAW2zumG3WnwtsykDvc+kgVX hexIrHCIDvHVWGdnK8DDa1fEA5mjkffCAEkCUpnNEeVimwO1qBqo3+y432g5Q9Ln0X3PWFfHljtq0 8TCIcYbfw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULwY-006m3s-Ah; Thu, 08 Apr 2021 04:06:58 +0000 Received: from mail-qv1-xf29.google.com ([2607:f8b0:4864:20::f29]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvM-006lXB-8o for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:48 +0000 Received: by mail-qv1-xf29.google.com with SMTP id j1so260592qvp.6 for ; Wed, 07 Apr 2021 21:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=juqdJ1+rPty+mZcJWzuSbk/vYifDHS3s3mojAmWOO+G3qlz7jgNSEEr4f0hG/M5Jpz aw5IKBcea62xB54/m6w+NyeqowI5FwwiwHyTaw6lq6VK5IRdOjzT1vIcWu1vBaBc9Dks LEMgkcImNq1vT7jsoJmB0yPzqkmdvjISYyHKlIglDEXSTmTN+bl3hCkpinJlw9owJSjZ Pt6Guc7V+JIgF6rOLn73KQv2wGXWNbNlCqzxYe7k45r8vnwcZ6SJRPIUBV6qpZjfCqPo bRG1IhRT2MZYsMPCE9cDpsb9zqKsMxgemCJo0vnGi5U5P+iz5i0DiYjYyDasJIOjkdUI XXOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ys4BurbzE8f3xOtcZckwDgg+eZ/2ibCIBaYK/PrLc5I=; b=uiypyTmV1RhBS7E8XEbp0c8n2xwTDHYvcqluu3/2D0Sw34z76u4B4Oe0ftCuiqGzUt sCf2j6OIh1y5RTbgy2Y6iSL1hSM5UFONgz+8BfUouRSD+JyLya1IoFfkg5NJG7HYBhdv 2SikkjH3IzmcDDQF56XgvEHgNEwItsH2oEk8qaJJbz9ftS2/9SASTKdEIEVj8qEZB6O3 bXuZ0OCoAtOor/E5ytpFXSGKuKTI3Buyvtlr5gWZVz9z/djfCucgDM7ENc8EA/QmTlWA zZ2xCgblUAPsVhcEwPFGI9gg42oBnSjgEV0gkDxrZXPUscCrZ+dp3UeBGCP5Swrgqi4G C7VA== X-Gm-Message-State: AOAM530zkShXCZbzA7v7PMOqQr7GDX2XPFNO+jiPGj29ZQXKKyiPAYWe 3KPg6IPS1xJvsgYCkA5BrExzVA== X-Google-Smtp-Source: ABdhPJwMb53rZZbQdfaU1akeMPvCbZqVXP8rhlkM/XM+uDXitGdEDxjjUj6mbyKR2RxS/SJppBRTmQ== X-Received: by 2002:a0c:9e0f:: with SMTP id p15mr7047700qve.27.1617854743132; Wed, 07 Apr 2021 21:05:43 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:42 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 03/18] arm64: hyp-stub: Move el1_sync into the vectors Date: Thu, 8 Apr 2021 00:05:22 -0400 Message-Id: <20210408040537.2703241-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050547_111548_53B878E7 X-CRM114-Status: GOOD ( 10.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse The hyp-stub's el1_sync code doesn't do very much, this can easily fit in the vectors. With this, all of the hyp-stubs behaviour is contained in its vectors. This lets kexec and hibernate copy the hyp-stub when they need its behaviour, instead of re-implementing it. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 59 ++++++++++++++++++------------------ 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index ff329c5c074d..d1a73d0f74e0 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -21,6 +21,34 @@ SYM_CODE_START_LOCAL(\label) .align 7 b \label SYM_CODE_END(\label) +.endm + +.macro hyp_stub_el1_sync +SYM_CODE_START_LOCAL(hyp_stub_el1_sync) + .align 7 + cmp x0, #HVC_SET_VECTORS + b.ne 2f + msr vbar_el2, x1 + b 9f + +2: cmp x0, #HVC_SOFT_RESTART + b.ne 3f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 // no return + +3: cmp x0, #HVC_RESET_VECTORS + beq 9f // Nothing to reset! + + /* Someone called kvm_call_hyp() against the hyp-stub... */ + mov_q x0, HVC_STUB_ERR + eret + +9: mov x0, xzr + eret +SYM_CODE_END(hyp_stub_el1_sync) .endm .text @@ -39,7 +67,7 @@ SYM_CODE_START(__hyp_stub_vectors) invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h invalid_vector hyp_stub_el2h_error_invalid // Error EL2h - ventry el1_sync // Synchronous 64-bit EL1 + hyp_stub_el1_sync // Synchronous 64-bit EL1 invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 @@ -55,35 +83,6 @@ SYM_CODE_END(__hyp_stub_vectors) # Check the __hyp_stub_vectors didn't overflow .org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K - -SYM_CODE_START_LOCAL(el1_sync) - cmp x0, #HVC_SET_VECTORS - b.ne 1f - msr vbar_el2, x1 - b 9f - -1: cmp x0, #HVC_VHE_RESTART - b.eq mutate_to_vhe - -2: cmp x0, #HVC_SOFT_RESTART - b.ne 3f - mov x0, x2 - mov x2, x4 - mov x4, x1 - mov x1, x3 - br x4 // no return - -3: cmp x0, #HVC_RESET_VECTORS - beq 9f // Nothing to reset! - - /* Someone called kvm_call_hyp() against the hyp-stub... */ - mov_q x0, HVC_STUB_ERR - eret - -9: mov x0, xzr - eret -SYM_CODE_END(el1_sync) - // nVHE? No way! Give me the real thing! SYM_CODE_START_LOCAL(mutate_to_vhe) // Sanity check: MMU *must* be off From patchwork Thu Apr 8 04:05:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FD75C433ED for ; Thu, 8 Apr 2021 04:09:15 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CB188611BE for ; Thu, 8 Apr 2021 04:09:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB188611BE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RHC26DOCaEh0UGZGzsmH+fP0EsSVIBH3J7+XFneg7Oc=; b=ARHr2/PZ6eT8Mjq40hxhERBoO cgNA5vSq1CR4AUOgxQYVXrlqpWyNQNX2Tkhqc35jvELLf85jA3T2mO3RNQFiIbndjPUcCCz+UydM+ DXNSsvRsdzBTLmD/SpJDKHv5p1Frg0zPUvyAUC/sLQi0I12iKeL8vsrkRZ4spIfUTLDEhuimmOR8R 0zG/QqG1YWwduuBaivvIR0NZGyogHXiHlbtI7wfu9v++c6y4IF+RJDfnw7KPIiWASPDDS3RUxto4C 83ygib/By3hppX5ScRDFWadsV/6GXFES/bh+VyvI3pZPH110/dr6IGWwn4p9vWjOAiZmYcYxNetwf 9PvA2HcSg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULx7-006mGY-1l; Thu, 08 Apr 2021 04:07:33 +0000 Received: from mail-qk1-x732.google.com ([2607:f8b0:4864:20::732]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvN-006lXL-Kn for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:51 +0000 Received: by mail-qk1-x732.google.com with SMTP id c3so941340qkc.5 for ; Wed, 07 Apr 2021 21:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=fNSvTQaCMuUW4WM7DW+/ylfwvq1Nd8GFZI5DXNtogEMS5GfxQgFo8HftVUvgQ2WW+L XjYfVrYBzVm7gymnWJVpRO+ECRA4CD2OPDTfrymbiIxKBrxQUd5XMEqjZ/iypUUsTFIQ 6X50BQCwfYfbVyhawd2DSswD4JiqBLSqjvnv2Rwzd4aXRn5JR/NNKZikJ7Aegbs8viUH jv4Sndd4hZAC5D8ilmQK3HkGGnaU5EB21eHXJMc2E0f2wAioMpgG8cnLIMFcOM1pGRSF OFnz738dO1OxUG42H7ZbsgXsFE1Ly4cb5QLHOxHHiRFqW+fFwulj1I3sI5tzncrV63RB bWNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t5r+nQTDiOBjAcARZmxYD2C/XCnn3pF+1dbE/Rdt76g=; b=esHz+buNu0qV5vgkVIKnEHyWMUF5gF+hQ8GXypHdsc4hqBjrSNBIpZArbruH3/v8rn grp3d5jQZrnQ3dv94YX3XHo5LbjDa81/dECBsjp6H8sjZnaAiH7bw1cquMWjOiUMOqUL nqyy/SLz4gMXflkNwBHegw3bdsAkMkoPNO/AaIUZyTe8ME2W+0zdUXJu/Au9iC1gWYPN YrINoIUGbQd5qDeE76GVS8rqxVcEcCx86wx08y5NX2j7xoQ+iluD8U6yxFGvN82Tux1I epADkV+HzRHgmVmGBdr4n6M045NUpfqsb30SZ76yflPo5rAsQz/8aGKHPKDwqidNBe9Y PywQ== X-Gm-Message-State: AOAM532Y/uW9L3f9FuJG6ikhgRcM8GaF2zmqpLl901kivVrL2Uggf5jI Ff7YJaEQHtYAKrnJauttLixmxA== X-Google-Smtp-Source: ABdhPJwdX428kgXB8v1FqFOp3RPgJRd9YNsFO19DyJ2SqHXbzbyljk/FZrQI44bhTtoFuQP0Gs4fUA== X-Received: by 2002:a05:620a:1497:: with SMTP id w23mr6610916qkj.260.1617854744436; Wed, 07 Apr 2021 21:05:44 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:44 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 04/18] arm64: kernel: add helper for booted at EL2 and not VHE Date: Thu, 8 Apr 2021 00:05:23 -0400 Message-Id: <20210408040537.2703241-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050547_275086_2F870C89 X-CRM114-Status: GOOD ( 16.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_callable(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 9 +++------ arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..4216c8623538 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_callable(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..1922e7a690f8 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..c764574a1acb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,9 +48,6 @@ */ extern int in_suspend; -/* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) - /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; @@ -125,7 +122,7 @@ int arch_hibernation_header_save(void *addr, unsigned int max_size) hdr->reenter_kernel = _cpu_resume; /* We can't use __hyp_get_vectors() because kvm may still be loaded */ - if (el2_reset_needed()) + if (is_hyp_callable()) hdr->__hyp_stub_vectors = __pa_symbol(__hyp_stub_vectors); else hdr->__hyp_stub_vectors = 0; @@ -387,7 +384,7 @@ int swsusp_arch_suspend(void) dcache_clean_range(__idmap_text_start, __idmap_text_end); /* Clean kvm setup code to PoC? */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); dcache_clean_range(__hyp_text_start, __hyp_text_end); } @@ -482,7 +479,7 @@ int swsusp_arch_resume(void) * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; el2_vectors += hibernate_el2_vectors - __hibernate_exit_text_start; /* offset */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..af0ac2f920cf 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -200,7 +200,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_callable()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Thu Apr 8 04:05:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 732C2C433ED for ; Thu, 8 Apr 2021 04:09:00 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CAEE461157 for ; Thu, 8 Apr 2021 04:08:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CAEE461157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Z5P2vBPyqwsUFuvLrRMg53ubUWKyHW6IoDHjYwlhRYw=; b=Y/r1k+Sq5bj1jFz6/0dcApeDI fhgzotXVZW70ACOJCSaNZaB3FvVKXvewcmZYoSpfuR9j9wGM13TmwCFC5UYjOMRcGdeDATYmeXJbO hal6PaM/olbQHlQkh7GKkcBtkjI1uhgNodWsAhbBHCJ+J6qDUuX3yznY2ByWT+Wm4A0ga+ZKiOoO3 fiEeOnm/BTWC+veIT5Jt3LaOeJSJyAXGsbnXHkWoC6XgHlcDDNyXECu9SQHJZkW1A5SkR63zIaosW 4KtCNK6yFCspqRIomHw7oH1NLqm+QYmAulSFwwI9TuVoRymMw8PNsw5usArsOelyAgJSoUiFokylJ ERdyN4WrQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULwu-006mCR-TG; Thu, 08 Apr 2021 04:07:21 +0000 Received: from mail-qt1-x82e.google.com ([2607:f8b0:4864:20::82e]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvO-006lXY-SK for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:51 +0000 Received: by mail-qt1-x82e.google.com with SMTP id f12so472049qtf.2 for ; Wed, 07 Apr 2021 21:05:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=myDJXaOpUVpA8LV8lwPukm1qxzV0VccCr/iZiryb0p12CJdK/BjjSAtNz9PVey5TCf 8TB0NuSbtbOYBJrkUo9jVJKavGXOlXWR9hK+QT9YGnRup03aSNMWNC9/+eMqxXc1jNuU 1Mw7X69o6DYWe3DDKL8oxGUnIbe+GnAioeb1FbKTOqKkZFAKP49DVNBua4GaBwFzgFaY lYqWKxE7CY3d8hs0diMbZU15Aj7eerDOX8d+nL9hWgPwPTGVNnoL8Xr/JMDsTCBLi+Mz 1BU8lY4fuTkGmDFXAAuURPRIt+ItCnjtZpaDBZyjhAoSEA5NpMXRTMSPE4o/BGIISO8k CgTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=llPAHNmh5la6nNqtjYJN/a+JYN2nusk7GBxiZX0ZKEdQxgF/6nm/NsUZUFfSMTgvqx 4g3q6Me0IE98rS9D/psMUeLhcSm7lzK7B7iTwUPRqo5t7blzIF/EjLE4fuCuI9kBEr1t jRugUnqXlDARMFq7PfY5MsWjAEIzUI37maKp4kdtxHpHVnYmSNMeq15nh4wZ+LMc4WIG dXOqsKYtgCgK02m6QJz1PZdP2YjvJas36cZU3Y2fFFCLfzLS07nQSYCVXSmMCobzcIew gHkKNgPdVyotdDexn7e6vK5Q5HQ3E3WtdJp7Py/Wscmgywy5MNEaQ7uPC2I6JbFydf3X aaHw== X-Gm-Message-State: AOAM531225b4Sy/KlQCtBOMm7DlZZ6sVP8Hlo/AtOFNBwRF+240LIPNu 8gyS0B+ZLYmsVEn+2NbQVF4UJw== X-Google-Smtp-Source: ABdhPJxd8U7/h/RVyIw++phyu/2DejR7jUeXg4mgQQ8MV4AUR6f/v61HdCxcTfAWRPU9DA+Gu7EhSA== X-Received: by 2002:a05:622a:14c:: with SMTP id v12mr5688069qtw.46.1617854745725; Wed, 07 Apr 2021 21:05:45 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:45 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 05/18] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Thu, 8 Apr 2021 00:05:24 -0400 Message-Id: <20210408040537.2703241-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050547_255853_F255B8B8 X-CRM114-Status: GOOD ( 17.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 3 +++ arch/arm64/include/asm/virt.h | 3 +++ arch/arm64/kernel/hibernate.c | 28 ++++++++++------------------ arch/arm64/mm/trans_pgd.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..e0760e52d36d 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -36,4 +36,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 4216c8623538..bfbb66018114 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,9 @@ */ extern u32 __boot_cpu_mode[2]; +extern char __hyp_stub_vectors[]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index c764574a1acb..0b8bad8bb6eb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,12 +48,6 @@ */ extern int in_suspend; -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - -/* hyp-stub vectors, used to restore el2 during resume from hibernate. */ -extern char __hyp_stub_vectors[]; - /* * The logical cpu number we should resume on, initialised to a non-cpu * number. @@ -428,6 +422,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -455,6 +450,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (is_hyp_callable()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -467,25 +470,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - __flush_dcache_area(hibernate_exit, exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (is_hyp_callable()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (is_hyp_callable()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..61549451ed3a 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -322,3 +322,23 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &__hyp_stub_vectors, ARM64_VECTOR_TABLE_LEN); + __flush_icache_range((unsigned long)hyp_stub, + (unsigned long)hyp_stub + ARM64_VECTOR_TABLE_LEN); + __flush_dcache_area(hyp_stub, ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Thu Apr 8 04:05:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A528C433B4 for ; Thu, 8 Apr 2021 04:09:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB677611BE for ; Thu, 8 Apr 2021 04:09:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB677611BE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3CQxR8o2l4l3dKHlUiIKRPzy2Fxr6iUPcYMSH2YYpWw=; b=K96hpSvYZcHFLjFl3tF/Ek0zn SZjzBXIu78Zo9PO4Dy3RZP+aP866QRI9yqtd5p87/2XQn1K4zQEipCp21KFA+uI5ANMnh5VqNN8yE ucbe3MZLbnCZ96Ak1iXTc1ilIsnDp+bRwV11LjB9KPc6a8YIPOH/a5vc7JMjbQQ6ka5vIJUfDExwC fQztUudBPy9dHxZBHyVawS1g3Y8NhCeLesBTqQtBWqzGKon9xoZXT+jw5MaG83qe0wUoGRwmBtJ8G pBAXmZ1hJfPSiVytZLogXROJUK5u0EMsO2qQmzpVml2MnMtf6Sf6T+/PyUMe4PRffF5VgX0S0SS4j qndf5tcIA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULxX-006mPa-G1; Thu, 08 Apr 2021 04:08:00 +0000 Received: from mail-qt1-x82f.google.com ([2607:f8b0:4864:20::82f]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvQ-006lZZ-AZ for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:52 +0000 Received: by mail-qt1-x82f.google.com with SMTP id l13so453169qtu.9 for ; Wed, 07 Apr 2021 21:05:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3ttZQ3gfSf54pfSUBx1LDfQsXE0pAbLQUWMLzhdR74s=; b=WNp1gMdsw3Tm4wv5ofO098L7+rqsXN0+xuPMLvxyhfRqO2E2ZKhDEGaGLLpQQCVLp6 kRWKAxhxmJ30NKIqBld7gTpjAwpz32kIBNExJ2Et0HZWEE+ivhSVeMWtXqp7UcIZyBCs X/LNtk7ga4MpNqn+3v94T7MU9nnrxtlJ/zYiq0277ls+/aEP16FB1VizUFYUquLa7aw9 GiGHeJD31iKWwANH8mQHHBOHgqAlwmaUPPJC5gCo6msF/l+C4Y8nVRzmZMqDdNYh/mJM 69+TMIvHTm4pme+kPif1x+MlMxbSFgJaabSY0wDQ4Fxs8xJDRaaUL9ndnx4mfcqONRfT nELQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3ttZQ3gfSf54pfSUBx1LDfQsXE0pAbLQUWMLzhdR74s=; b=IsR8oNGw6WQ3Avd1f9GrnMgU7q1Uq/coLn1ouPSuJXs6LmykB4d3YJrIL2Rr0AF6ak UEmgvI56I666PhayKJq7m1Rd0dHtU0BghwGcj2mehImkipGWhEzkX5JO4VB7gxALM1tU Uh3JsVEzCe2CrhyeKQGzPNdT5azpX577XM9CvddcbxekNAN1RHmJl9sqt6vGwxCXKe15 jeoa89+rG1W428uQanFrPzE6g6N1wxtC/RZCo1srVQzk1z8fLLCZeox/wU01VZIxDIVS aENDt3UefhzSkfNToWmFFRDHLLs17xnnLQSvrhwtBJNBtb/pxVivlWY9/RK0PZuZx+F8 8N7A== X-Gm-Message-State: AOAM533xkZmhXbHdmJxgzCOPZk8XM+DJp6tf5uZkTo7McHPX0X2MM/3x eKGiEsFOQffIYA6s2Fymg5NXbw== X-Google-Smtp-Source: ABdhPJzXmtqxtUQTEJT0lxRGeT3Nceyt+CBZeKYInLcPaN+AnmjXc07TTgCf3AOXx1AjM6gvwPz9Hw== X-Received: by 2002:ac8:6684:: with SMTP id d4mr5596938qtp.34.1617854747044; Wed, 07 Apr 2021 21:05:47 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:46 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 06/18] arm64: hibernate: abstract ttrb0 setup function Date: Thu, 8 Apr 2021 00:05:25 -0400 Message-Id: <20210408040537.2703241-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050548_444109_5F5B413A X-CRM114-Status: GOOD ( 15.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functinality when relocation code is going to be idmapped. Move the setup seqeuence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index bd02e99b1a4c..f64d0d5e1b1f 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 0b8bad8bb6eb..ded5115bcb63 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -206,26 +206,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Thu Apr 8 04:05:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58C0FC433ED for ; Thu, 8 Apr 2021 04:10:34 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3F71D611BD for ; Thu, 8 Apr 2021 04:10:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F71D611BD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=L9vS0tAUgnK695+xMNOhNTPthc35T9698qK9uRNN4rs=; b=P/PUXz4hpvAW0JsltLA6oJ0sq iz/milMzLsqcde3W1EU+Q0CLwrWkSclCks8OoXiSr7aCbA5QnpIeGQJ7xFOJqMeMX0EQ+DXhZn7pO aKge80AqISW528YJiXw8Ueop6ovj3hnHIsBTLsIscENQvYJ0Vqfi1h0VQXL8AIepQUei8+25QoFwE QJ4ldyUy/R/MsMMwFn7JeUNb6Fmcg8GQwbcERlmZxQwy8LdPh83VwkRES8Ne0mphHA7w0sI89+xX7 4XLA4R0Gbyw8tbi0mZesYiZPNZl3oSWUksN33ojXLTRSWCeQC6akR2MAnD28amGOs9pJNU2OnKj3e T6HL7au/w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULyL-006mlX-1N; Thu, 08 Apr 2021 04:08:49 +0000 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvR-006lad-Jb for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:53 +0000 Received: by mail-qt1-x830.google.com with SMTP id c6so476582qtc.1 for ; Wed, 07 Apr 2021 21:05:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=K1NsbXiptJWAFjDZB69MTcxq+sVd357Lm9DKIwBQhxbGV3HftSKBeUqa23Wmi5KKNU zvjcqW4fvbuFxiDUEdiWh3Mcf9hsDD6xZvknyHE0ctG5bfVF8q4qEwPS0v1jcEKtMcbA 8nKTdc1FLRCDhKH+sT2n8fMhr1u+4HPurVK8/xjKe/Z9Cy3im6HxbQ+zh7vl4AwQ97/T ZODphpRxpUyYHVXtt6c71a1yPBcTrl3qHz4ut24V8AGbtwoYkqrh2tIBNNyfurAIblR4 QypVLYj0CXuJ3SBllkLS4gaJWPnDY2g4n0RpZpXaLWcaMPu4cnP4N2D+arBerwRT+OoS Yx6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=LF4oIyWuxLLu8HTH0iJqyMV5jvQ0BWWc+wYxi7TbvMzhFO7IeVecXTV5kGrIJXp3fd EqrD6LIg59XOHQMX4RrYFG9mCysPQyLeElNO5aJdiVCJwLtRMog7PF8Bu9hdPvMvKkrU xx8UZNLeU0snoX246CEg6L3/igAIOp2ujXW1NNxLjAYmbqjZtLG5vWyNiSNcLa9bBxs6 jj5pBOsacDb9hUCw/od3j1TGoEDBGKiqVYtJN8gtzHHKsDMI+RkOP5jtk2lEmBGj5i+7 YHGZXh8mdGZP4XWYrJ3AT24lZ56F1EFBZLySDx43QSM5DI5qNxzO+xVyDWkg7knXmO0s 5f4A== X-Gm-Message-State: AOAM532hJbfjxXh42hMaUSsoEYfp5ynJcMOtrh7bH3335BTX2yY7dYHc KB43kiEzuCPcbnSOguMFAeWuGg== X-Google-Smtp-Source: ABdhPJz+49F5sFnvxO3bcbTDIuYVduCfSWrd4A61z5Tt8rvcCW5QOx1alJlSaDZm0kDJxf714A6VuA== X-Received: by 2002:ac8:5313:: with SMTP id t19mr5657090qtn.148.1617854748347; Wed, 07 Apr 2021 21:05:48 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:47 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 07/18] arm64: kexec: flush image and lists during kexec load time Date: Thu, 8 Apr 2021 00:05:26 -0400 Message-Id: <20210408040537.2703241-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050549_995032_650011ED X-CRM114-Status: GOOD ( 15.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++++++---------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..3a034bc25709 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -59,23 +59,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -152,6 +135,29 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -169,13 +175,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -250,8 +249,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Thu Apr 8 04:05:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189987 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49753C433B4 for ; Thu, 8 Apr 2021 04:10:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 737CB61157 for ; Thu, 8 Apr 2021 04:10:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 737CB61157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/fn+ZhrZS9uefso0j1dWPH2tztmDB9rkpEALYq3wuk4=; b=h9lreTikxgpdwrrgkdPBL8hK8 p8LCIG8unnF4iqq8xW1iQ79bW2IqtTnS+SD+7oRfEd/sRdN2xQfbIRmr/6MrtXCToRQj2ajOxLVbv lhtAsaTVLogBtiMGdnaO8sjpciqz3Aiihkzp2/kLvC4za2Ld9vtQWlEY09w+p1YhoTnBQbb1ES/X3 1aQO8K045yks7SXrsi7gz4UoQ0f7DIT+uw57tW/AfbvDIwzqBdVObGHcRUhxh9oRt0Qzmyyfe0TVx /fW7HS88nZhCT2uJTmRFXZbDhReYNqt4DNHceJHvckFuINp1qJEJhHY+MV7Qbprrdnk/CJNw8TlwJ v6j3mkCpg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULy1-006mcC-9Q; Thu, 08 Apr 2021 04:08:29 +0000 Received: from mail-qt1-x834.google.com ([2607:f8b0:4864:20::834]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvS-006lba-Rl for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:53 +0000 Received: by mail-qt1-x834.google.com with SMTP id s2so446924qtx.10 for ; Wed, 07 Apr 2021 21:05:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=Vd61oEBV7Vn5WTuBeGbCdJHrwIWCibHMtCk++64snK/ZPDtTyhHe0Nq3a+pjo8uswX Et4Le8VT1I2fif7zEsz3e8Gdw81BQpEQ9IblPTxGPS2Jfqhz9lwSe307RzNh+knkCGtX U+Oiqfv/zT5hYamo3zUvH1r9kESLRvQIuF1PEyslShJ+OFm6TrqzjnvBF5TTn7ibNGl+ +48FhVVIo0tE5For8Dxkpf9z9605OZzVjFN6r/o6N+nQ3g+/M/fHSa90RaVvzZAH8MJk /mj0LHL4p3vKmQFIi9y8+DOJA3AnhqcHU1QgRDtJvhbvjntNHQ7XKTYVbQ8++2ZQu+bc eYSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hiQbfer3Cy+5MCvp23I2bat0V/tLiwOPOJLrW/6qX9o=; b=puUGSIREVEdA5nb5H/VajtXNbDiuXCOadcPcWzkJj5Q9veeU/vqAUG99oYL4WD/GYq oGHLEkQ5BV7zhtCQgZwrh8rnV2geqO4HEAxDdWQbcUzbJcWvTl64y7kKK5YkpDUVPOrJ HJwvrp2f/xP/JK4b/cJLZBw5Gav27XSVOBWC54ywySrM0nMovaerKdSvRLoMC0sjxwLg 2FWWbyTmJ0TF1yfrML4VWqVhnHyBo4TTy7s9SfxyLNzQyf5m9xiET9oel3UMP35BR3LC aBOP2Rl44SOhGOKHz8Ga9u8pHJNzKQ8nlZJZkvGpmoVFQgFbZKdp3dVOzpN/l7RymapI vx/w== X-Gm-Message-State: AOAM531AlISeyiv0KenB+EQnwjkqB32wu2LVgL9i9t+KafAUtD8zuKuV Iq9HomxVnS5wcbBwLGqUycXy870Fe9zDsA== X-Google-Smtp-Source: ABdhPJwaCeykglUkEXFAvBa1BzVKFnTCf1ecW8WT2k77yBhsz+NHJ2cCh2ZdKNS2R+jDbwph0oQBwQ== X-Received: by 2002:ac8:7611:: with SMTP id t17mr5459416qtq.361.1617854749679; Wed, 07 Apr 2021 21:05:49 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:49 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 08/18] arm64: kexec: skip relocation code for inplace kexec Date: Thu, 8 Apr 2021 00:05:27 -0400 Message-Id: <20210408040537.2703241-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050552_033140_D2C4379E X-CRM114-Status: GOOD ( 19.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3a034bc25709..b150b65f0b84 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -139,21 +139,23 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -180,19 +182,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(__cpu_soft_restart); + restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Thu Apr 8 04:05:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFD22C433B4 for ; Thu, 8 Apr 2021 04:11:22 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 84A03611BD for ; Thu, 8 Apr 2021 04:11:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84A03611BD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=79Glskrpq9wLpPHrIrocDP7p5BHgF+r7llzQIidIOFA=; b=DNvx97I/m/rvKEVaICr9pMszD jVfgonkTXSMxZFPyi/9DgG5pSi7WEjabajVlEdJV0tSTjObGJfUc8k1727P76fkAEaQqk9mZBHdgM vWmv0s3ihWPD0UE9AQnvifRj2afpQxBHoNWHN8kZ6cMD68EXFsEMON8UDF+stTEaJ90stRhlFfct4 1MolWqJN1vLh0xLIZNgRV97OrU6WD8YtvZckwk7lgzN4jyl2S1OvuBpPr4DT9C07f1M/15fCrewhR PZR+TpFncuRKerFija4piOkoNWl0FlV4A8ScgKzGG00Qj+a6DlMbKOy5feqb5fTOJKtSyI2rgq/ij fSffW/Sdg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULz5-006n4i-5R; Thu, 08 Apr 2021 04:09:36 +0000 Received: from mail-qt1-x82b.google.com ([2607:f8b0:4864:20::82b]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvU-006lbm-1l for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:55 +0000 Received: by mail-qt1-x82b.google.com with SMTP id j7so462687qtx.5 for ; Wed, 07 Apr 2021 21:05:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=CKGSSrZOId5j0LqLgbjZCuXMtuV5OgJo1NkH2oQVBjJKC13+u8jXgkVJP0kzp7RP5Z bbOoQmxt7BF6OWd8qL4HSp0IXWbu3tLCeV1Ishweb4EUIuyoem0zSaoWmRQozHihC3Dm VP7NzQQApfjnGgbM1PHOjz/MnPFTiYWNEro2KMw6pbbZnvbU2wP2Cf3WL9WLSSE647dX 3vI7VBAznYsKmkKOYlE4YXmamMCrIkzG58sFWyB5aImgCIRJKwz4PLiRd9lFW+HZJxaf JmP8CmnjAJdpBp4yxQBnhzmCcNYrzOYbPkaqyPYlB0Es2Sx6yeje04DjJ4yrsO4h+R0T bolA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=d9prc1UZUFN5dwsf86cy1HbKMQhioOGgsdfiYkk/1yQ=; b=hpwBXv6SSxMYq46H30bprdgIOrmThLyb1EUMgOXtzwy5EchQ3N9hBOCZzTRMWXYFDJ SX+RG/4rg25NDEYVkGjO1Vs6Hi2IuhprBlKGIZflDd6y74hO5Ck3H8+MXGbggV3ic0JI 3CaGtLBL9m4U4fJmN3FDz4iXvIX1BQwWYFPqSHVCcbv/JAs0AiTBu3SOIZKA0Eeultqb OF/IiQ7idyyU2JSekrOvtLgnPpsDH1+UHNC1bnk1uvr13ZrYqDkQm1aCrocjfJNk2/xX 6E8wuAwTDKzmszF8Eq1znvEtdCXgc5DwaUROMvYu13S3enfhrhyVsBow1XCyCytDBpdd QVBA== X-Gm-Message-State: AOAM5330u0yYbDhWMgOpe3i9WttDfLmEVzOnsfd+Jv5sXW9Fir5mgacQ AI9AsnS39zUvXjLo0jWFin/0oA== X-Google-Smtp-Source: ABdhPJy2yjiHv6dpdlu+pGlHHabbICyMKqXxlvfx119ga4c5aETs6NQg38h9Fs3h1H/OOkERem7iRQ== X-Received: by 2002:a05:622a:46:: with SMTP id y6mr5573543qtw.154.1617854750976; Wed, 07 Apr 2021 21:05:50 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:50 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 09/18] arm64: kexec: Use dcache ops macros instead of open-coding Date: Thu, 8 Apr 2021 00:05:28 -0400 Message-Id: <20210408040537.2703241-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050552_302982_0B6C5036 X-CRM114-Status: GOOD ( 12.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitised DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 12 ++++++++---- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ca31594d3d6c..29061b76aab6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -371,10 +371,9 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 - dcache_line_size \tmp1, \tmp2 + .macro dcache_by_myline_op op, domain, kaddr, size, linesz, tmp2 add \size, \kaddr, \size - sub \tmp2, \tmp1, #1 + sub \tmp2, \linesz, #1 bic \kaddr, \kaddr, \tmp2 9998: .ifc \op, cvau @@ -394,12 +393,17 @@ alternative_endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 + add \kaddr, \kaddr, \linesz cmp \kaddr, \size b.lo 9998b dsb \domain .endm + .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \kaddr, \size, \tmp1, \tmp2 + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..718037bef560 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Thu Apr 8 04:05:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5DE7C433B4 for ; Thu, 8 Apr 2021 04:11:49 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E515D611BE for ; Thu, 8 Apr 2021 04:11:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E515D611BE Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vyymY05vH7RqsRKuKQoYiIt3jnphnFyU8lJil/qCQZU=; b=auznI2XmOtPLFn7F+mTe3GhM/ gFHQMPKSk8KD0gaNQ351R2MViRj+A05UWAo5QLyJnk23QMMHhZoSVZUk/kLGMv5j2bc7xhlEjiMBE b8IRBG0yKH1dOk2YIZQbbdzPo0suOlc4bhqnSok+lOGxTa5O3MwOsURueZwRMQejea1COjoOZqHnt 5Nrlqnzm2+fJVPYZPSNmZfOo9bjBOjyiWPipMfcoPOQaUklFdRwm0lzvi8+X0axFpIzsaWOROCWdl q5lh/XTVC22NMVLU7JcCt+56XBQ1cKqqUXuMaAoocNBwQBlRItjAngb/LA2MuiaQ1rQAyoQlU+Cob CiVFVYaQg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lULzf-006nJU-CE; Thu, 08 Apr 2021 04:10:13 +0000 Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvV-006ldU-Cf for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:57 +0000 Received: by mail-qv1-xf2c.google.com with SMTP id x27so265230qvd.2 for ; Wed, 07 Apr 2021 21:05:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=F6JZkU80674Q1VkhfFm80eRIhlT4LTp68/RErf1725v9DYSxSAh29UWw3UDtiBURG4 u0n2ebEhoDt5HtuTO3kincM93W8gSFAACO87UbFvfg+TKnnlfjDa1iPq5U8D9ey2nSYN W4fhKnt/bDMPuHnnAp6P4c6OLXcnEd6kMmzkgMOyEPsn0UhQt+5PjB55hUmuN0Nzpzap +0rOYzqOJLSwje/mdtbOjaL0CrjS19OmEbMqcXf4HNwEkMAZ64+V0SJ6fNu52Frq1oIh KlDteFSrD5AR4g5pbiIAHi0PC+tohcS908vNFFXwicB+eWCfa4xnX2RXlCj6/FZxVMFm scaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tCKT8dQy3j0JXLfUAkLHptP8jUCHy5Mtlem3XeT2uzw=; b=krWkH2kYutw/aMyjYwL29PUhWsM5QtUYORiJIV4tENcCcGdUYleR1Ov8/cjuW/SiMm Hmk2ohbFtfMuc5ZurPB3iEsCThZ58p6LxTxgg91IQnkJrPq7l5bEECf7UoJBI+ykwEEF +D/SXS0ul7ZO2iKSm6pLeH+m2P/G3KeL3Dj7tqPjsxDw7TurwIaRBYb47uC19SJ+SYjF nG3rjIHdstVIqcfjYF1F1+YbQBv1iQ+Do16T6+I/LRckGsM4+XLI7EjOU0HUjEqsKuGy aY86O+iE7B95htP9xU8w37oJk9icFksRId18tapBmGQJHF8NLewB+XmYylqGKO3+19pc kpvw== X-Gm-Message-State: AOAM533Vpa3eA51q3MNaFuqR7+HuUTru5ps77B9mXXX2+A06rRYgyEg9 +LPM3czsf3q3CFqxB95k/WcDzQ== X-Google-Smtp-Source: ABdhPJy34xVqJvda6lNplCUNEP3hwQatijEVRlmosHFcsSuFccjv0kKbw6AfhQKTmbRewTzaiAP2bw== X-Received: by 2002:a0c:80e1:: with SMTP id 88mr6471448qvb.43.1617854752265; Wed, 07 Apr 2021 21:05:52 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:51 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 10/18] arm64: kexec: pass kimage as the only argument to relocation function Date: Thu, 8 Apr 2021 00:05:29 -0400 Message-Id: <20210408040537.2703241-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050553_833643_6F8071A5 X-CRM114-Status: GOOD ( 15.94 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index a36e2fc330d4..0c92e193f866 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -153,6 +154,12 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APGA, offsetof(struct ptrauth_keys_user, apga)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index b150b65f0b84..2e734e4ae12e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -83,6 +83,8 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + __flush_dcache_area(kimage, sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; void *addr; @@ -198,8 +200,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 718037bef560..36b4496524c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Apr 8 04:05:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDC88C433B4 for ; Thu, 8 Apr 2021 04:14:15 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 15620611C0 for ; Thu, 8 Apr 2021 04:14:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15620611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=a1UxKW2be8F4gH/qgiBU1ss9ahKarrK/D2P0Hv9t5Dg=; b=IAGH0bMMP/mDOCMDtXSoJFwKx JBKsobAZ9PJZ3CVyDiBxoijsxPkcNeAGqytjJIwjfcXhJS0ETqU6GZvpM0iiXitO7Lh5LrYoTJOtv 0HL1pzuTq8obTECz8nqlX+p6V+4UJjW7RnBCvl065j54nTOz6725Y8KILyLrRNUwwvDCbOJw86/Z5 Abw3m6epEWR2AFaCSxCJQb10Bnth1iR/umuqVmSdRNGes5TW4gmyD50b1PqD7phJ39aOxL60Bh4di Z60VaMzQnEdOs+xPlE2wgM7pz3orNZ9nvw1l/fnBrW8vTjAnJvkPqnvanuWaCaU+Iaer2d8gFAnxZ QsyY8hCcg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM14-006nmw-SZ; Thu, 08 Apr 2021 04:11:39 +0000 Received: from mail-qv1-xf2a.google.com ([2607:f8b0:4864:20::f2a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvX-006le5-3y for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:01 +0000 Received: by mail-qv1-xf2a.google.com with SMTP id o11so249693qvh.11 for ; Wed, 07 Apr 2021 21:05:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=SSVgN/Udo7O/9r5aLt7E06oIBMSmNcVup1eFBvvBBwQ=; b=W7mAMlhvgMQZA1u+YpBSTaa4U2PlAJoqUUKJLnnt92hGehD3XuH/up8FwYAQYvl2sg VA4MzjvZYDTmJc9ylxGHI507Po+vcc8YmZt7tyU4dCv+DCcECJTPoJsTgCFn4QYZ24w2 siKERh3P/f71yG5Bel6jiapye65oFGsz7/yYDXuN38N9g9vNQ3Z7/FiCsqzJq9w9s7gU NE4cAbdyjAmBLaNyyl+kN9arrODHeJPnnkGmT+S2KQSKviXPpzO+kQ5wW7qcXjWPjt95 xQBpkjftdwMpIfRUxcRQoiWji3IOD5GYTov/9U8UQAMpvKKxNvGN2xjnHfJlkY+Bcu/u Mumw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SSVgN/Udo7O/9r5aLt7E06oIBMSmNcVup1eFBvvBBwQ=; b=MVLAceH9A4nDKx/C1i+RfjgqStk9vD9Xwv9SAyf/sYIYlnmVqjFMWdzKYDs9EVOwBO QCZ4rX9hdzOVAzDPw2VY3L8ryq2+IntoZPwT+1dbqgO/FWwL8SU272QMAdpzsMBbwNhB SglH/oLKLw/uy36iQhRK2RA5Sn4OSG6hJVI1eWXiD+5sjkzZORzZjm6HfDjizU/h7XmF uGQ94Xuh8oW8YrvWfrKs+CFsq4eDK1g3DgZXMMaH20ZOq7wBgVRUbKvYXmEzHx2czB9e O050UI2ClnfpmiJLatQ0r4nIh8yG+ZOKENrIaqQDigVm3ifr6K9e31GomsTIbX/nnA5F je8A== X-Gm-Message-State: AOAM531WEGmJdAZyUIK42fIglQxb49KzNbjnquiCUwS9C9qOdx70BAuc n1q79DNQcST8ePvqYySsaIZVdw== X-Google-Smtp-Source: ABdhPJy2AuQlIlurSB61GRRFwjVYnlD1FVCWdmqAS1sn8VdIrau1gL7EH0PszcWgCc7MeQT7AcMizQ== X-Received: by 2002:a0c:e84e:: with SMTP id l14mr7070045qvo.39.1617854753531; Wed, 07 Apr 2021 21:05:53 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:53 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 11/18] arm64: kexec: kexec may require EL2 vectors Date: Thu, 8 Apr 2021 00:05:30 -0400 Message-Id: <20210408040537.2703241-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050557_068189_1FF8FE0A X-CRM114-Status: GOOD ( 18.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preporation to MMU enabled relocation, configure our EL2 table now. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index e4e1b6550115..0e876d980a1f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1149,7 +1149,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 9befcd87e9a8..305cf0840ed3 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 0c92e193f866..2e3278df1fc3 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -157,6 +157,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 2e734e4ae12e..fb03b6676fb9 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -42,7 +43,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -137,9 +140,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -148,6 +169,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_callable()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -200,6 +229,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_callable()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } From patchwork Thu Apr 8 04:05:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12189997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A28DEC433B4 for ; Thu, 8 Apr 2021 04:12:27 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2BA0961157 for ; Thu, 8 Apr 2021 04:12:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2BA0961157 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=w4tjLuT88XI7esg9mATGYdwrDDwW4QIa1+frpU2SCPY=; b=dNpV/HCb8Yf7ePnmCRbYa1I5c dResGZU6n719c8eXskcDSiHWVnhUKtg8WGsuD8i75cHtOmJkyfnot1ZeaaxCHCg2ul5mww07qbitb c8TEBDtYVrvUoeRbYxwy8J4H2Jv/x3TML6k3KQ2Ms2jQ096JAD/7FQDXvDpBwZFt+Ykp0ibixVALB eioDA2KY3F/iojwgr8rvIKEfYrDusfLEHvH/snTJoe54WEYh8wSWRroi4iD+JtJYPZeyPtFMmurAE mF/zrNGeieWdlk7QbB5eLCwhG7VA3HxeQX9MUCBhm8g5UyGID0NgmkKfNM26SRvI4ntqPbBS7f2Qk //knCgn6A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM0A-006nVf-SZ; Thu, 08 Apr 2021 04:10:43 +0000 Received: from mail-qv1-xf36.google.com ([2607:f8b0:4864:20::f36]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvY-006leZ-2F for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:05:59 +0000 Received: by mail-qv1-xf36.google.com with SMTP id c9so245959qvv.13 for ; Wed, 07 Apr 2021 21:05:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=cZfP06UdFwUbDXzsBkl1RZmPvCxQFFD555CU5BZSd+vAYM+/Oe+q5LF6b+AeDb3hx2 LFsT7XpzjiKY3Si2YsqeQ/IsmJpOiDmMG/HNxXKVtP9KTMFFrpTt/zWcuY6K4+nYd9LP 9ZKSsnmvV+2+q+Pmt93pKHZ0lgdpVdpMXDll0Wv1hibDF4r0T45CgLAGXvLMQJPDkBFz 9IzjdkP1g6YNXrx/gkwrI/Ofe/tf10DE04Grydz47vkZEbw+ldvrJuhINzXNy4tlgWG7 P+aKh8JBXoF2aRQrMgr5pwbBamx/FerBKZ1ntwmSE4A+0xIGTTmKGW08lsKS3Z2H1qJG Decw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ATHDsIlQRij0i8p3z1/oZchje9DfRT1VSEQgV22ghC4=; b=Ymdu68QMAUYglYl+fPqw2z/QKZGwBnZ78bg92Kdt0mZ39qnwa/0Pcpms8WAkHWiEXn QfIcpWcmxKzDRAvrgwQMzVNP5KMqOE7Dyr1ywA81m+2kT1hdYqcfq7Bzwi7ZvRfXoAfV fP84zCXjwUx+CtDw64rXY64lZHJ8E+nzWqsj0pw/uuq7HZ33qAfz6FDK8EFz5MEJPRrY hz7AKlv2ElnURx9+l7QykNizOhAsCLjj01MgfA/xaNqY5YMVPB8ru7R7dkUi/sCa55xp 6bRQgmbgsh5QmLLut8gww3kBN0S+56Ll1+rLtp1c+B2tHgCn0B/0L/ng4LeTcWsEZyja 8f6Q== X-Gm-Message-State: AOAM532b4TKV2pNvtwKiwpiUGkCccMQzBEnqCKb8vB8qQagNyPjvjyeL Z4z1zGlPMpSQHFdzo+fAl4xxTg== X-Google-Smtp-Source: ABdhPJyQo5a7wag6qg1KfAN2lYACfcQS04kFQ7QD+1uKAF30/6EW8kWHUAmbTGpYPmRfnaUQw+xfFQ== X-Received: by 2002:a0c:e2cd:: with SMTP id t13mr7015227qvl.21.1617854754850; Wed, 07 Apr 2021 21:05:54 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:54 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 12/18] arm64: kexec: relocate in EL1 mode Date: Thu, 8 Apr 2021 00:05:31 -0400 Message-Id: <20210408040537.2703241-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050557_070305_D08D80B2 X-CRM114-Status: GOOD ( 13.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before enterying the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 1922e7a690f8..f6d95512fec6 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fb03b6676fb9..d5940b7889f8 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -231,8 +231,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 36b4496524c3..df023b82544b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu Apr 8 04:05:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12190005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2248C433B4 for ; Thu, 8 Apr 2021 04:16:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 272AB611C9 for ; Thu, 8 Apr 2021 04:16:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 272AB611C9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=io/cWx7BdiEjCX7N1wxtt7K0qpmh/VjBgr7vmckYv4Y=; b=WSGAIjAu9vv3ubUQafqkobGFQ B3Azov20nZKMNfUYCN456OABkvgXtF9g6cfCG+FKOf5NFM96p5ug8OH+k9PXQz6UZkNmvMo7RKLDt k/LY8MleY/9110Di/cMCQwhE6B+bJdAtgUuES+IALw/ukCdzkoRT7rHo3zM5iDHgm32XBBKvdN4YL o/9GG3ozxTMjC9auifH1Ujzvb9L95KmWztA0bc6jdWmYWs1rMPS2pDLoQlSaIEDX2NMPt6mYJk7Rp Wg06fAjpXbADvaEHkGT/Uowg6nrPxhESRLFftZHI/NbzTl4kuU/kClRnD0Rs9eiAddhSWV2+Vwx89 Hz1RgHW9g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM3X-006ocm-Bj; Thu, 08 Apr 2021 04:14:12 +0000 Received: from mail-qv1-xf36.google.com ([2607:f8b0:4864:20::f36]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvZ-006leg-3m for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:05 +0000 Received: by mail-qv1-xf36.google.com with SMTP id fn8so261951qvb.5 for ; Wed, 07 Apr 2021 21:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=XWa9sziZNgfipw2p+fBnm+a/QR10cQILmaujAFu4l6VA+vlVaDGjksM7UsE3OmENKV VNucczfet1qHKZ/5/akhn3GRx6lbaaSgBaCSZf1ydSx8NjIkndwUoAG5HuRdZkLBbVIE MkqHHktYZswwr8dW1Djt/3BBInIEvh2P8lufEAXfFrn/Rt/5VfbusPDdCIF2C7eq/2L8 2pSPqmCoZMW3PzGChT+fE1dIZHjIOMOOPu1mI6AC5dXeRe+orGOzpTfQwf6KA2FBhZ3R t9p1rvZn1c56KaacZ1E2JNlRU2HvqbI8E1ZTAYwCMHw+AlhL8gIv9J8FfSZdnatgxjtl 97kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IBZj7dYh8lQRd+RkarkK6Bs3S98VDCxIDzsjaOHMyWU=; b=eyGz4q+0GQ0A2B1KzirnIVM2lUGhCH9lZ6iUjQSCEG/K8JSQYBzPNZ9YMIB+1k6t6/ jAgbd0zVEj149KMclq1T6krSPlix6MPYXMOWdg3v5+HBg5CTbB8Kzp1ZdHh7/TRf0Ufo ZjAvQph5TKBBwgBMlJ+Neq2YpA65rF8rhsXb1tLynDTnlL8D0ha1MFrBOIIXjP3nzLT4 2BDZ7kYYoDSUql91X39Y/SR9KH8RQZZsLunay8xvsrMxv8C3JXEtO5yBvv2UC3tngICB zgcISmxCqdJ0zkeBWVD6PTsVtEY0JSxwwNiEQJ4TMI/G5vJBW/GsKjQOyOz2EKd4bYOm 05bw== X-Gm-Message-State: AOAM530EoVU79jlZHnvga2y0NrsxZZXK4CHG1VHqUlNN29G9mCIsiye9 LTCyWq5tLhS7h5FxstHhZSbnzA== X-Google-Smtp-Source: ABdhPJw4OtlQ8Oa4W1fxbuBoRRZz0Ihai6LuM5gpbSIyFOGLtDUBu9w/nYGR7vBoD5f7IcL9Em6JfQ== X-Received: by 2002:a05:6214:aa6:: with SMTP id ew6mr6754949qvb.2.1617854756198; Wed, 07 Apr 2021 21:05:56 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:55 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 13/18] arm64: kexec: use ld script for relocation function Date: Thu, 8 Apr 2021 00:05:32 -0400 Message-Id: <20210408040537.2703241-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050557_865754_57C67D3D X-CRM114-Status: GOOD ( 14.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 14 ++++++-------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 2f36b16a5b5d..31e459af89f6 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -20,5 +20,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index d5940b7889f8..f1451d807708 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,14 +20,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -157,6 +154,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -177,14 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index df023b82544b..7a600ba33ae1 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7eea7888bb02..0d9d5e6af66f 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -92,6 +93,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -152,6 +163,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -336,3 +348,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Thu Apr 8 04:05:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12190001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78609C433ED for ; Thu, 8 Apr 2021 04:14:27 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 22BCC611C0 for ; Thu, 8 Apr 2021 04:14:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 22BCC611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=qYG/BVk2e7Ror8xd1wWHZOQYmQXiS+AdwbsWZXIqchI=; b=OTIHtIxunCfhMtM8lxV8cBSid obbYs8i3AsghVZiAhWsGKWYAQMpGNaUYh1tqfCA0fFWFgSKOKN1K3U2dzkqlVwDEMkHcXYv2YMT/E OlixL8+GQzpZwDFoljOL7DyBmRHzltkjzUXgVFeQrtWBYaVsKr1BxuArAzhgOWUeAwij/iDBPqwtv YowkKkNIBVcx8UtiTqj9crPYBVZpSTaBD+9otAVEr/+CccBKjh2NrjA4CNagMBJKP8DfdgTLAzePv LNNTl9MDxEyFmTcnwkBR+rn6I6hmPMFpEmdLxewwD3J2tHq3/iRT51rRmSxnnInZyeBKKFrxzW1lf cDzdJwPuw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM1s-006o4X-LT; Thu, 08 Apr 2021 04:12:29 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULva-006lgx-TD for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:03 +0000 Received: by mail-qk1-x730.google.com with SMTP id v70so922435qkb.8 for ; Wed, 07 Apr 2021 21:05:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Olm+fYXYbuVgp6bWsix/kMAWKOrqJlCkAHr9xCmjIpk=; b=Ja74be0u5lOgyjL5ukqrO0VzL48CYNlfDj91JpnAEjnkoyN416LTT1tIEj30DKJcpf Ai9oRBcIbMYLQ2oisclkbXG1+0rdbZE+VVnkVlgBsHIr2iyi6ss7QFoVcI1kBO0JKzfz rI95EdUlnw+Y1SInEE4X89iYGz8I+BULoYPnbzA4BXxdupnmOcNJhDfz7caMuT93hq3p tW6FtoWfwwu8CWNbXaFpZFnAPRKRjmZH0Xre624JmLVDgPZ//DBFf6KEWtw5Kd6Gr/Mp Zsu9aoUMoElMIAm3NMXWEJOtC87NzuRbQiH5SOj1fyOSQHYuGPl4v/KtjpYYdo++sILt YvlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Olm+fYXYbuVgp6bWsix/kMAWKOrqJlCkAHr9xCmjIpk=; b=UUpykQanMTsF5r5YA2FYG5NxOFxZ1mrkYCJ1HepUqgOD+hitYi++0vcJ+bBvTCKke9 GfyeZazxCt7DE0ma7kkAhizcUI7szNLeDVv4uMiO5H2zpa59+Il/gt+c0ijkZj+TILbD XXGv42zfo/NqspWeO0+rnsjLDp2X2XC7ef0M9tr0spOlyDu77z7iOT6bf8uecsvAtEfI 4maM4F42yuuFpvG47ixXNeRw2uuWhp5FelxdkhfmJ/y+vYPbKu7SNP8sFVmXNbabaZ+3 U2sMxSgAi6DN6gbowRwVxnr9sXqblX4Uy6FB9slwMQaKJCA1tPbAO4hWSJ++jv2XJlPM ATyQ== X-Gm-Message-State: AOAM533OcGyDpVIe6z5/Z0/sejJS90sf8Qr2ut6SzEHSQxBH8dyeIG/5 bKIdUmvYoOzx6S5ER7ugqyO1Iw== X-Google-Smtp-Source: ABdhPJxH2ZBM/1PaSQCcOMDUpkoTsv/4l1erGBcWQZtAEwuVDDfcl3l2izGRncnix3SDHCHnnzsigA== X-Received: by 2002:a05:620a:142b:: with SMTP id k11mr6548405qkj.46.1617854757440; Wed, 07 Apr 2021 21:05:57 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:57 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 14/18] arm64: kexec: install a copy of the linear-map Date: Thu, 8 Apr 2021 00:05:33 -0400 Message-Id: <20210408040537.2703241-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050559_359789_B79CBF24 X-CRM114-Status: GOOD ( 20.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocations with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 29061b76aab6..3ce8131ad660 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -425,6 +425,25 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU isb .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 305cf0840ed3..59ac166daf53 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 2e3278df1fc3..609362b5aa76 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -158,6 +158,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..a31e621ba867 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index f1451d807708..c875ef522e53 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -153,6 +153,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -169,12 +171,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_callable()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 7a600ba33ae1..e83b6380907d 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Thu Apr 8 04:05:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12190003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25F6FC433ED for ; Thu, 8 Apr 2021 04:15:09 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C250611C0 for ; Thu, 8 Apr 2021 04:15:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C250611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XhDskcd6XdY2/5pODn/0c5enk2qAQk7W964X/lyXYjM=; b=VAPd3B3ObCHEQPkKaxxKQOORw jTYVtjrw9OcfIJdSAz/PWmE0ne/z9vAPhb7ZbwWcq8oS+3359Zx/7Be3VYzvLQ2Ou+3fK/VAcFWjT FPxNYthxCl3ipZUPdR727cp6+V+2cQIDZCjMLrIoVD2q7FzK0KnEgRX2o68jSx2fdLwfsAZh+a9Vm S3YwoC0ofdUYXu6oUGCgGgyAaQl5U4DJnZFN3DPo7TtFtaSL7/eqWz6IkR+Qoi0texhDu5mEIvXop AoeZM4A2EG5ivQrRWo4P94Qyq4lf+dGZqrckkLneY1i9b4NwtOidNACcW4XQEIBZGpFTTwMOvuNo0 XZtzHv4Qg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM2X-006oIK-TV; Thu, 08 Apr 2021 04:13:13 +0000 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvb-006lhf-RV for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:03 +0000 Received: by mail-qt1-x830.google.com with SMTP id h7so471214qtx.3 for ; Wed, 07 Apr 2021 21:05:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2MwMRNHUNcBKpMF1hToORmuQtXmVKMCROcbjizX1Pao=; b=fpo5AOYCGm3rnQMF4k4Kep/dfrnGm0E++/dJ0VrKIb2VHU6OudZV1VTAwgm8LinIqD yxDQzUb6g4ZO7W1zZuk5Xy1KCgLYe0bDuxlE1I85vxW8d11idj7Koq+4BwzP1LqXvRkT 2dMKemIvcTZ0HaQwfZtVjjuH9s1RAs2kq4RBntnYZxjJOey8rqC25tRwXrVjQzgjG1tZ ukpM0SxnmPuiC/eHb6fb/k9eolxaVTtBTwI9qiJ1Ol/j2GzD4T9XnqADIa0xrddwtB1Z tx2k9hhtZcvliRagOHSKFqvfv1hCu+4aDq7lVINevfwTKrZ0jmPaYk9rLS/UjlnbLCLd mwzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2MwMRNHUNcBKpMF1hToORmuQtXmVKMCROcbjizX1Pao=; b=RuUoeiqbqkQ0OUnbrTn3jOo7h4uKTek1S91CM2iaX3/7cdG1KMJsTpEm8xibYDjzP3 ugeONDVeI8G28DMrIO0ZQeidkcoz2lWGQuWhOvmyqpztiRPAHU7ka4ZllRgvmqyIkJ3E f2ECZAMfI56EkME2KpbKSBfPUmAhWb5o1ZN8oxVbv1JQi4eqdqNpHDzmuCIDyXgD3w/f tyeIrnLZy8Eeh1zEXh6/8IOsrdP/V9G0Lkvfa84LE/82fWEm0tT5amlM9GKI9ghoOyAQ OtzURehW/x7rbFA67Y964XfOpCEnlj5E0neNTrr2GjAex2hyolc43RYnFVUceOnaoWAD +jxQ== X-Gm-Message-State: AOAM532uNAw2AanqqU2uqoVdoAUGoPeuTaLknJpgtTIOzgqGg4OGN2kY oah2EKJZjyhYBDebPZs8jEwWEwcb3HmlIQ== X-Google-Smtp-Source: ABdhPJxVaJ3kci0H5jEVPHKEH/dGd4Hj0zfNx1cig37iSok8prwWbLZDkAcKO+o7C9aiRkKogLnWYA== X-Received: by 2002:ac8:5c95:: with SMTP id r21mr5804477qta.347.1617854758731; Wed, 07 Apr 2021 21:05:58 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:58 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 15/18] arm64: kexec: keep MMU enabled during kexec relocation Date: Thu, 8 Apr 2021 00:05:34 -0400 Message-Id: <20210408040537.2703241-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050600_268359_E55F5A08 X-CRM114-Status: GOOD ( 18.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 ++++++++++---- arch/arm64/kernel/relocate_kernel.S | 33 +++++++++++++++++++---------- 4 files changed, 38 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 59ac166daf53..5fc87b51f8a9 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 609362b5aa76..ec7bb80aedc8 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index c875ef522e53..d5c8aefc66f3 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -190,6 +190,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, reloc_size); @@ -223,9 +228,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -239,10 +244,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index e83b6380907d..433a57b3d76e 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,15 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mrs \tmp1, sctlr_el1 + mov_q \tmp2, SCTLR_ELx_FLAGS + bic \tmp1, \tmp1, \tmp2 + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +43,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - mov x1, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + mov x1, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +74,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Thu Apr 8 04:05:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12190007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 636C0C433B4 for ; Thu, 8 Apr 2021 04:17:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 556A9611C0 for ; Thu, 8 Apr 2021 04:17:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 556A9611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QqqRfWfAnRNsVY6usBaa0NBN3wvGm58U+rzLkKHVabE=; b=EaOHwahYEWIW/K60BvFGK0rME TC2fOFpNksAdqjQQw5OaQTERKmfPw4okfgMQzoeLamO/0qW4UPipXjxL1qoWcLgGKp1xn0Ls+YWPd dHmx5yHxL+kpcwT13gTJJhxcGkdhbjis6sNuSpRM611ZM8fkwxBgpJfRh7Ez1269xfn9lSmyQeXr6 UyX3ul7Y+W7GQuaybW3VYvQn3+o60hUCiOGabO2Iosq3pNrMYTp7eW+JPXdphmUXVgkUMtJskNMfT o8ggJCiqXUpHh8pXL/wkOwgrjXvn0vw2PazZ8Yydk9RroboMshufcZk4pWUTyGtN7swbxrjkVRv/U Dve8zE1Yg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM4S-006p2s-FH; Thu, 08 Apr 2021 04:15:12 +0000 Received: from mail-qk1-x72d.google.com ([2607:f8b0:4864:20::72d]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvd-006ljT-FC for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:05 +0000 Received: by mail-qk1-x72d.google.com with SMTP id c3so941715qkc.5 for ; Wed, 07 Apr 2021 21:06:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=elWVu7ElduDgar7M15pPxIjNtGuFk2IxD56SO/LLqYMadH0bLeFPVdDJrrgXHMQQHY 5O+hPoZGbI2/fKrRlolSphZQemU4b8fO04rT4MR4ge8GzoS//g8BcNoHJnDWpxtXN6m6 K4yt0Ll7DaykR8Tyr8MCv2ALdhZZld2Z8v2GxISaLhL5P8n+yqVTAzsr885OTW/r/CaI ZWy3CSy0Jkc5aLPCQK2fFbp0qgOEgsvZVoIRkykT6rD/cRQui4N8LS/GWlBbLALSY4Jx +gOmt9BWnaITKKFW5XLWpNvETX8hDJzB/44NqbVBwH4mg3CMwDb3PBBB0DlOUtBELbj+ 7j3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=mQmK3SsBkKt3YHt5HRugbQ4C32Fw5itH9aDNQf+nmJI=; b=tUn+eQDMvwfKPnkiavobmXowpDKnfGL5VfCRS0MmSsmtkVPsWXL0VsPu2PTQ1XT9yl +/85t2RpaxF2fhYTDiZKYUbhE4NNJv2UbNVcEKyJ5QSbdb9WUMRqyA5YBYgZebtW1UDm ljZ6XAwvc8zHeLRTfIDB4cI/QmhAaYgF/6vapeHx8IC8kh6Wga4+zFL2QOSIcXzxczR7 LMzhVaOo55Iml4sKbuUF5L2SdVztik8StOLPUcUfnwnkK4A6X+9y4yLbKOWqZjW+9Tw2 UJ7nka5dhLDwS01xAZZ2e7tGsVdtYn16IwduwiB3eMiswio5Axq8Xn8iZoAa0utETR4B /bJw== X-Gm-Message-State: AOAM532buqAeMEoXtAqOSP8l6DSgjr+ITDIJaLhRFtr/Y1OP6bnJw83q ZFs3bewDYNgpU3XCgmXj9bggkQ== X-Google-Smtp-Source: ABdhPJw8BJbPk7oxJbt5TiRHY20STpCei8pN80U8MVBJiPqRcb2nWUnW5QpsnWL/n+1r5eSXF613pQ== X-Received: by 2002:a37:c92:: with SMTP id 140mr6714935qkm.177.1617854760170; Wed, 07 Apr 2021 21:06:00 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.05.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:05:59 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 16/18] arm64: kexec: remove the pre-kexec PoC maintenance Date: Thu, 8 Apr 2021 00:05:35 -0400 Message-Id: <20210408040537.2703241-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050603_292084_A2FCF206 X-CRM114-Status: GOOD ( 11.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 40 ------------------------------- 1 file changed, 40 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index d5c8aefc66f3..a1c9bee0cddd 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -76,45 +76,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - __flush_dcache_area(kimage, sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - void *addr; - - /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -200,7 +161,6 @@ int machine_kexec_post_load(struct kimage *kimage) __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Thu Apr 8 04:05:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12190011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3F5BC433B4 for ; Thu, 8 Apr 2021 04:18:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D2CCC611C0 for ; Thu, 8 Apr 2021 04:18:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D2CCC611C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=N1L8QQUtsGz6gXhIWKu7VofoVG3uEimmklfclzGedFM=; b=bet2/vTxjlR+CTOeRQNmbJJye BI+Wyrkz57/wwzdTgbBDHyTj3iOdC7mz4S7Kz74zMr2gKH8cfGLxZIVUaJNrMdjq3uO9A7PSKlgTv 4YcQSlQQw5obe6KnP5qxWZpPhxdBmIVKHt5388GeRXsc8q2w3lkChrqBiSiMDUSJ9MFOzCOm4H83o b+9DX0jrIG8z3WYe2ncXmUDkXnOx3/RgS3/aSWfTHZx0pbVaHe3lOiIUmDAdLl3fUFRPM2I/JCQPq oUBixBTRrwg+7BGPvKm/tBvC+CzGQNYK69Jt6eDahogfVSMtFL4N/wfIINU3luq4Fu6tlZ50yW46K sV2ELf0jQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM6R-006pjL-31; Thu, 08 Apr 2021 04:17:11 +0000 Received: from mail-qk1-x731.google.com ([2607:f8b0:4864:20::731]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULve-006ljV-BQ for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:06 +0000 Received: by mail-qk1-x731.google.com with SMTP id g15so943531qkl.4 for ; Wed, 07 Apr 2021 21:06:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=PQZpDOWvV+YQX3+RRvSpQFtcUJedqJnM7spQTWj55AWeUxKM7SDx4kPAPVhIti6gUy voTZFJ0QpgeKNepc2Wsw2crxkyVzSebcYCpjVmhiUJHU0EF8Cp2eZYaXKcFetCaQ0nbO 9sgz4EynwexL5fBFi5NdfqUX3KBkDnkUvWB2XRI6lXQ6AD58/hESTOEhAo/WhJJNVKKB XRS9V60S3C0yHEY1oJGn5dzDVLybPzxpWkBY5tt1iidF+UOGgqX3qTEnTHcav/dlpF1D 5IWEsW8FOz8Ysa5jWkyRyxS4DJNl79APieTpJKpMGJjApk1t3Y7Drx2OLvk1ksW2Ww8R hdZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xvI3SX7JlP3yoyw6A9a93CvHYXbBacBuW7cN/tDXEXM=; b=sdCn7sxGJXESFymT9gflfZpTIRrXNV08VmUHoAP2ytT2oHcFHjDnJw6ygukC/iN/ki tpgGOsnm2oseNruBsZKDkEeKiHwjtkKPfjKOgvH8o1Y4XW2dv98LXCHUrem49e3Jjd6n eZbXB+Phr/G1WGf9/8n0JiHb8Zq2OdFXq6rRycNnJqF/HxZv2PmOHWNigEYSmocIsOYu b9AvBqyrlNVtmPoxQo12IOMRmSnU5KjpF1MaEui+9+4p5s77MNGjP+0xst3+jfKJFfmw AqJ9/J0Y6NWox+SME6GCJqcODAxy0tkApDSCXu/zGBKgDcY0zASzgrKceP4WHuymB7AE 6olQ== X-Gm-Message-State: AOAM533q0hlry7mW8Rsxi1IM30xQ0h0OIp41x9V6lHpUsEurXrqtUGkm l1EQvgabZ9/eYv9ipoXLNzIGvQ== X-Google-Smtp-Source: ABdhPJyAhGZ8OdBbF6Xwq/FrEvRsf0ZaW/p58HnrVWenn+xQ4Ms5P4taI+EFYNUH/uxAa8e26zbECg== X-Received: by 2002:a37:89c7:: with SMTP id l190mr7002416qkd.361.1617854761446; Wed, 07 Apr 2021 21:06:01 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.06.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:06:01 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 17/18] arm64: kexec: Remove cpu-reset.h Date: Thu, 8 Apr 2021 00:05:36 -0400 Message-Id: <20210408040537.2703241-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050603_500748_F818E110 X-CRM114-Status: GOOD ( 16.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 5fc87b51f8a9..ee71ae3b93ed 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 37721eb6f9a1..5d47d6c92634 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) /* Clear sctlr_el1 flags. */ mrs x12, sctlr_el1 mov_q x13, SCTLR_ELx_FLAGS @@ -51,6 +50,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index f6d95512fec6..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(__cpu_soft_restart); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a1c9bee0cddd..ef7ba93f2bd6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -23,8 +23,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -197,10 +195,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(__cpu_soft_restart); + restart = (void *)__pa_symbol(cpu_soft_restart); restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Thu Apr 8 04:05:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12190009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2A65C433ED for ; Thu, 8 Apr 2021 04:17:57 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D620611C9 for ; Thu, 8 Apr 2021 04:17:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D620611C9 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=gnMygclAlzpdngitI37sF+QFRfuXS5DhT8K63tAqGZ0=; b=b6viFKS4EyXGEqm5MGipz8U1v ToU/SensdmZhrGBNLSoEiSVpy8dNCKkQeWC+R2m/9hjoUtyYo6ZxHj29KFcAuHWJ94PDrVpRFBamr f2Sm6Qb/eMa7bGv3kL8clMwb7fIsyrK7Ab0JxclFPQKtbZkYqGfS0QZr5Jj3RBZZjOgmpl/f5WPJ1 Nk9fHzrwhMlum45lcnMucUiMAmX+uBPvWMHagZ/ve2uhlafR0e98SbUEbgkzyXeyAd3gYi7NV1lJT wMs+g59mkv3/xa9B74AfmiGZ36xODT2dNkCVNx4XEYIDD3w8MnDvWYPgtbWp73mZsJMhsUCiFSGtT Gkyrz1Cgg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lUM5O-006pMG-1N; Thu, 08 Apr 2021 04:16:08 +0000 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lULvf-006lja-Lg for linux-arm-kernel@lists.infradead.org; Thu, 08 Apr 2021 04:06:07 +0000 Received: by mail-qk1-x729.google.com with SMTP id c4so951455qkg.3 for ; Wed, 07 Apr 2021 21:06:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=hEpX6glagkFAuErJ7yQUGxGTKuEwhpiBYJUVY+mF4vPQ9e8cKN4TsDEZ5/+iZBwsv5 iB7RU8pOFfBpYoimzL4WA8WK8J/rXapc5IOLHDQb+zE0bhEap5LZlBQxZuFhwcGMK63S 2DQBTm9SWhcwg2HWWqUv827UrKERhGot15yaSZtIj1uQK6RO2b2ktv7D9pO2jne7A5vv ioXYlqv1fYjtRrxVNgpy9PO4FG3bavAsYW2etVeizyl2QlU2er0E3nb45su4gKMq6UJc NOMZiK2RWQiRXhdFaS78R4rXW+aveD9Z/y6CN3XhfaGVu+dSgOYUiObLZaVkrWU6WsSz c4Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=N9ACVOF+USC5pMneY2PKob3Wzb2Vi68SIT4UqSFM/itKRlswN89YKvfgLycGmbsbQy u6JO/kDWdSTukIcLgpazuSolXYj6jUXMeSb9o6pJpY9+clu+OG3OkUlSfDRZoOmDnuqZ 52Rh0unEt5Y1Hdd+j5y8Acg0KjA4ZMrZXTGAirfAg2bSrk7Cy9OnB3nky3Ykn/5Gt6zk UARW0Yg1KxjT9hTgcRZwFnYxoEE/j4sxWkhg+U7rH5G4lko63viPaF1R8KpJcWQjmNbG bRIblQqJKVVXbQMyU4X+8dKL2p7hcBBWIDfnii3aPOppxEWDVlze4kbSVl4XLdqfJwVc JzEw== X-Gm-Message-State: AOAM53061GBtlXsMwQj2ivKNBynphPCylD/RCCR9YJuEY82ltGUd4RQU 99ZS9JAqr30HuG92FPLWvseLuw== X-Google-Smtp-Source: ABdhPJxRILjhMfqmrTfTbUSlaG1IKDkZR0nrqdNhTJMhzMiqtQkU7L/C22a+USt80/gnZrWOBcd4Zg== X-Received: by 2002:a37:9a05:: with SMTP id c5mr6549116qke.16.1617854762799; Wed, 07 Apr 2021 21:06:02 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id 207sm18177874qkl.125.2021.04.07.21.06.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Apr 2021 21:06:02 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com Subject: [PATCH v13 18/18] arm64/mm: remove useless trans_pgd_map_page() Date: Thu, 8 Apr 2021 00:05:37 -0400 Message-Id: <20210408040537.2703241-19-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210408040537.2703241-1-pasha.tatashin@soleen.com> References: <20210408040537.2703241-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210408_050603_874952_E0AC3AA6 X-CRM114-Status: GOOD ( 14.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Pingfan Liu The intend of trans_pgd_map_page() was to map contigous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contigous range this function is not needed Signed-off-by: Pingfan Liu [Changed commit message] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index e0760e52d36d..234353df2f13 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 61549451ed3a..e24a749013c1 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a