From patchwork Thu May 27 15:05:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 297B9C47089 for ; Thu, 27 May 2021 15:48:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E541C613BF for ; Thu, 27 May 2021 15:48:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E541C613BF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DHLkK2RDAijd+5w1gsIFfsGcAh/A7S1BjHzk2NlDY/A=; b=pl+nnQbK1RpktU jRv+9mp1eDJn7NSO25I+WY/H7wVB9xsQOeoxedj1dupPcm3tqVzUOh+lR2ggfAxi6/pV3a0Am/yVq HmlWkrxGC9yu5xZgBTr5WsVZAwJhyJTjW6Eo6s9bfD11Gg75o2vJjs/9qbrmvqKpJ1aCjQMIRsOxO xbHo6S/cFKlFovBZc9awPEpS39kyjbW+D+ygp/zPJJq7K13PByOqpAz3UW76V5Q3xYdgW5uBQ/r2O r8ruquQljNvb4o94YvCb8o11K9zgH07KJedYlEoDfLKQ77XXLlgdgEdF3KZoQ/76riGDbiHV3yUG8 k47EkHMsc/6N0czuHuSA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmICh-007KVS-H8; Thu, 27 May 2021 15:45:51 +0000 Received: from mail-qk1-x736.google.com ([2607:f8b0:4864:20::736]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZk-0071e3-Bi for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:36 +0000 Received: by mail-qk1-x736.google.com with SMTP id o27so911936qkj.9 for ; Thu, 27 May 2021 08:05:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ghqRky3xeZiHl8S3DiRVTP9pM6BiSKFavllixIkkvd0=; b=caUhVx3J/tqp+Gal3Xi+7VBVenbj1bAKADWn8txk9KZ20D6QNDbgDjYASPogchOfyj MnR9GvDeTVlU/w7h0MDxRGxJcSxJKoV0mjkK+QdLIg1QR5LED8krF6MVh4Dw7m3PScI9 DH5d0pWUvzXOFaty1vK3wY3s2CaSoIxaYfFpwPApeJBhcfwIAViezHl4cLnrFbUJWTEz CxttZKvFn/yQ47HYBvaRhH+Top3dM3Aos+zj8m2STtvk8DprhmBAFJxZS0AcEiFO3Kb7 DrdN7vRSm71g5/nEb3VcE95apeL0RMCTNaPwasAdOu10HhJ2vq1M/c/C1Yn8L1NpF7Rq MkWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ghqRky3xeZiHl8S3DiRVTP9pM6BiSKFavllixIkkvd0=; b=kLgdaNVgG0AzYpLu+FXoVYRBkTFGbKUClOYrVhg4eZryAM0bpVk8efl9526mN+zyzH x7rKncyENBbHXGrtiSa5whaHndrbTQB8XSBwhODB78B10pGt/FLew08VwaCjd2zXuLMc prTuBfriW29O+d4prvLMIO1SP8mByugENeyX/DKrB9cLmiklaLIGwlTLNzkanRyn25t2 VMIQOnYSiWn54bPqZwq2c+u0Ilidsn2BFovM65NG1UlaRDga9ktsact8l06tzwJdF+ne Y4S1vK3Z4iAfIT+eq6geDaID9j3/uKe/lFGiIp2nLVWpRRNOEMD2yDvyW2+osLCKScuV moHQ== X-Gm-Message-State: AOAM531XCief8cFrmHxDJcl6VOMGr0X6cfT0Tc7B8TD7HtuyWzkB3xf1 lYK1jJ0L6h1+0p+2X8uh1SKYNw13aUGq+tSs X-Google-Smtp-Source: ABdhPJzkqRktvQM1YvTQKwrzQq+gZ576cPkKRhONnmbg0z/uIwTpNvyWkip4RNp35WexktltcbHpqw== X-Received: by 2002:a37:c15:: with SMTP id 21mr3945299qkm.180.1622127930853; Thu, 27 May 2021 08:05:30 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:30 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 01/18] arm64: hyp-stub: Check the size of the HYP stub's vectors Date: Thu, 27 May 2021 11:05:09 -0400 Message-Id: <20210527150526.271941-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080532_512420_DB08DEB9 X-CRM114-Status: GOOD ( 12.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Hibernate contains a set of temporary EL2 vectors used to 'park' EL2 somewhere safe while all the memory is thrown in the air. Making kexec do its relocations with the MMU on means they have to be done at EL1, so EL2 has to be parked. This means yet another set of vectors. All these things do is HVC_SET_VECTORS and HVC_SOFT_RESTART, both of which are implemented by the hyp-stub. Lets copy it instead of re-inventing it. To do this the hyp-stub's entrails need to be packed neatly inside its 2K vectors. Start by moving the final 2K alignment inside the end marker, and add a build check that we didn't overflow 2K. Signed-off-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 43d212618834..91aab4121463 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -41,9 +41,13 @@ SYM_CODE_START(__hyp_stub_vectors) ventry el1_irq_invalid // IRQ 32-bit EL1 ventry el1_fiq_invalid // FIQ 32-bit EL1 ventry el1_error_invalid // Error 32-bit EL1 + .align 11 +SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) - .align 11 +# Check the __hyp_stub_vectors didn't overflow +.org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K + SYM_CODE_START_LOCAL(elx_sync) cmp x0, #HVC_SET_VECTORS From patchwork Thu May 27 15:05:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85094C47089 for ; Thu, 27 May 2021 15:45:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4C615613BF for ; Thu, 27 May 2021 15:45:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4C615613BF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dZ2BuRJZb5Jnrgyl8SU5NjCpXGzG/qMaVJYzr2dznxE=; b=uB+MxvuYfVRagx 9iPl1uwPRsCvzFLocJlwGKRxVWF34fY8HuRckYz1hPc5ymM7Pw1vl1fhTDdQoQgCzV95dknrQ2U/T 19sRXaz5g4zuJXf+KkKatmAuE4werud17loFC/PFCufZ0Xlu7KfmFOl33EemIb+ZlqNgaJ3Tkz4vX LcG58/PmVWdxj5jBFa/2a34j3miWj+WFBSbYHhF8OYdwVECfpayGVpKNPr0iGfM+qW7dhxBOJMojO aYiQ4IYySduUGoZCLe/Y0/VcGtg71SEHfaiENlmRRGEAi0X9VE3kS4eLaCjzFt0r+8Pc0QooXaDQb Y8TJR/oksxwS4i6n8LOw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIAH-007JPa-PN; Thu, 27 May 2021 15:43:19 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZl-0071fD-QN for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:36 +0000 Received: by mail-qk1-x730.google.com with SMTP id k4so966123qkd.0 for ; Thu, 27 May 2021 08:05:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=l3SGBPbH3T8MZIM5VPxw+w4kFGUdmwAVrjo+9WKhb6A=; b=lKkn7FEbTutFu0AJibWJbh7gvLM6Up0Orzxu/3QynB6mWeF920eLv/nRIAjn0HTWbz kRhnM1FhPgEgKDcFLc0RU7+8HDyuVrDf1nOdQR+DxW+V/MU79MqDfiJMev3LvByNYJcG mosH1G9Mvi/NBnrEeQoBPbalZSrRc9viFcC8kFho3Oq48p2L9WDorq4FQts5PrzjhPvY PQXufesw6t3klHivvCa41fH7WyDUbQwQhgjcy69LGoTTy5HoW5rSaakGdTqk4RRi31+u qU/1ksTD7uZt2uZRkEseLan5+B9wF27wnlPJAwpkKXvmMj1unG3unZvMI0rv/GW8p4o1 xqPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=l3SGBPbH3T8MZIM5VPxw+w4kFGUdmwAVrjo+9WKhb6A=; b=aIDpcTiI3qW+NCL/3tolLaAS5trczg1vP94X+Ltyui0kRGLtXNKv5uI5cGyoIj4nhb ryD4U22LTOk80HqFrXzpTnBoepb/mlfXuHz/teCtmpoD5VQTiBH72kARSgGl9WQNcuOF x6V6go/bNHxfQw71AsfauLr+GZwaD1OJQCIvZj6K7llZ9I70dfr/kaQJI25P3wTbBSzA NqNRlZVsI2nQ3o1qQ34m9jJrQBiDLuApFfmeQZ2U5wsOqUFDLSVH0VC4TxdYvxP8Texm q1LTseQhg0EyxQ8M36iOqv4dKp1SLlFzPJQByX+z3N0uzPlbQT5b74Cic55FMWF2fNHe SW7w== X-Gm-Message-State: AOAM532r5YCCBQShsTkqjkp7WEroFqNSm/tNgD2PgO299/dzZFfCYied 0vnghBxTalkK3An5xw2qKECAOA== X-Google-Smtp-Source: ABdhPJxOofNUG3149CvRQe7xgHOAYxhjpmqWLXhWUL/moDKhXxMhTLFFTYyF/8yiB65KCQQlF0YIPQ== X-Received: by 2002:a37:f909:: with SMTP id l9mr2053163qkj.428.1622127932312; Thu, 27 May 2021 08:05:32 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:31 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 02/18] arm64: hyp-stub: Move invalid vector entries into the vectors Date: Thu, 27 May 2021 11:05:10 -0400 Message-Id: <20210527150526.271941-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080533_876963_A977CFC3 X-CRM114-Status: GOOD ( 11.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Most of the hyp-stub's vector entries are invalid. These are each a unique function that branches to itself. To move these into the vectors, merge the ventry and invalid_vector macros and give each one a unique name. This means we can copy the hyp-stub as it is self contained within its vectors. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 62 +++++++++++++++--------------------- 1 file changed, 26 insertions(+), 36 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 91aab4121463..18a97bee3779 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -16,31 +16,38 @@ #include #include +.macro invalid_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + b \label +SYM_CODE_END(\label) +.endm + .text .pushsection .hyp.text, "ax" .align 11 SYM_CODE_START(__hyp_stub_vectors) - ventry el2_sync_invalid // Synchronous EL2t - ventry el2_irq_invalid // IRQ EL2t - ventry el2_fiq_invalid // FIQ EL2t - ventry el2_error_invalid // Error EL2t - - ventry elx_sync // Synchronous EL2h - ventry el2_irq_invalid // IRQ EL2h - ventry el2_fiq_invalid // FIQ EL2h - ventry el2_error_invalid // Error EL2h - - ventry elx_sync // Synchronous 64-bit EL1 - ventry el1_irq_invalid // IRQ 64-bit EL1 - ventry el1_fiq_invalid // FIQ 64-bit EL1 - ventry el1_error_invalid // Error 64-bit EL1 - - ventry el1_sync_invalid // Synchronous 32-bit EL1 - ventry el1_irq_invalid // IRQ 32-bit EL1 - ventry el1_fiq_invalid // FIQ 32-bit EL1 - ventry el1_error_invalid // Error 32-bit EL1 + invalid_vector hyp_stub_el2t_sync_invalid // Synchronous EL2t + invalid_vector hyp_stub_el2t_irq_invalid // IRQ EL2t + invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t + invalid_vector hyp_stub_el2t_error_invalid // Error EL2t + + ventry elx_sync // Synchronous EL2h + invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h + invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h + invalid_vector hyp_stub_el2h_error_invalid // Error EL2h + + ventry elx_sync // Synchronous 64-bit EL1 + invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 + invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 + invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 + + invalid_vector hyp_stub_32b_el1_sync_invalid // Synchronous 32-bit EL1 + invalid_vector hyp_stub_32b_el1_irq_invalid // IRQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_fiq_invalid // FIQ 32-bit EL1 + invalid_vector hyp_stub_32b_el1_error_invalid // Error 32-bit EL1 .align 11 SYM_INNER_LABEL(__hyp_stub_vectors_end, SYM_L_LOCAL) SYM_CODE_END(__hyp_stub_vectors) @@ -174,23 +181,6 @@ SYM_CODE_END(enter_vhe) .popsection -.macro invalid_vector label -SYM_CODE_START_LOCAL(\label) - b \label -SYM_CODE_END(\label) -.endm - - invalid_vector el2_sync_invalid - invalid_vector el2_irq_invalid - invalid_vector el2_fiq_invalid - invalid_vector el2_error_invalid - invalid_vector el1_sync_invalid - invalid_vector el1_irq_invalid - invalid_vector el1_fiq_invalid - invalid_vector el1_error_invalid - - .popsection - /* * __hyp_set_vectors: Call this after boot to set the initial hypervisor * vectors as part of hypervisor installation. On an SMP system, this should From patchwork Thu May 27 15:05:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A56E8C47089 for ; Thu, 27 May 2021 15:49:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 61FE1610FC for ; Thu, 27 May 2021 15:49:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61FE1610FC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7AdRbP7ibsHHO/Ub97KB24NEkgxIzRWysFabSQmnGpk=; b=Dz4EM95Pdwtt9V CYHtMT65Yu+bp5qdkbZBEDxVoRPpNsmLIihMhdThaLbsNjyhCFN5sV+gTExg8oMKNJi3f2CJn0Rkw 1kxcw/LNNUaO6sQGx8wpewjqHMPS/aZJa1J1g89jKdBPQDhDMkWwPe8cxnPiJ/nPHlImCyqUXaKth F3CWWnD6AdnBIQwX2lMm6CwsHd9E4QTR/PsYh1ZxXsIrSafUJgO82Oepf1fdgVb51WF7OR9rPLz7G sij+0ST8XWLim/raY+tL7gDGDEZg65Gsgn8jJJ5IcWpcy7h/biOZj1/3LGxlRVA5VoOPPRi/QfARq kF0Mupwt4oUwsllUhK8w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIDx-007L0r-6k; Thu, 27 May 2021 15:47:05 +0000 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZn-0071gI-64 for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:38 +0000 Received: by mail-qt1-x830.google.com with SMTP id c10so407751qtx.10 for ; Thu, 27 May 2021 08:05:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Dd4OYNLmvVEa/N4zByXvuOh+6Xq8z1RHPtKQnceMU+s=; b=PAy8oUO7YX8LhrT2i2a4coviv4EjGLjt9zdp4yLK7smhcEYSnOPuWxth8pQ2sJKm+p 2jg8Z8xa3QAfUFb6YPB7i6dNFCgjUSpCNNTF4y2TBPfNZI+7cCG57PyzHQ9Of9f20W6B R7bkFVD0/i3hf1SwqNdYQsnQuKcFwRo7leOcm3+FWAqSlbSv3tKKbTWkEpxfIENLQcP7 Avke/bt4sGbmFGea4UxrsFvQiaqrhp1VhY7VKD6d9SVV7vKuSzxZ9sKf4SCTFCA3py1E o37jHAYoXineM+R2Or5/XjDQtnkuW/wiU2rUyJdY25VcLfzBGnauKIoXd+B4KclFdTl7 0iSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Dd4OYNLmvVEa/N4zByXvuOh+6Xq8z1RHPtKQnceMU+s=; b=NCYCKywexjLL6sJoQDY0UsByQNGtSjDMOL3THPiivRsEj9DFKJ5gNzCTLh/u7jp6/B mIRK3+Ds8IOA6kiMufn3RqlIFrnwFb9HGL8+BDydv4EUjzBJvapknpC4+c2RHpJcgSa9 DpPOtNSnv3hKnSbV4Tl7duTAP5nCYut8eMKaSOBHfNgaketEyWeR9/gFFtGVAudnW5Ls KVgcOkcvAalWK/oIi5hdls/MgLSH3FtUcgcjs0rqAXrPlXhDXW7M49bDczdawwa5xuDZ bb3JawwNp4D919ilpKTRdKBTAb91u6gedqksRsqfApQRFIX/byIk+HJMBG7Ey2YI2y5l 7hXw== X-Gm-Message-State: AOAM533QgYBVhfOBGgm3ipUzelM66J/I/Po+SR84Xgs1zo7FHnh3qs9i mDiCuu6AedXvIlOJ1foL3C1uW1xbJP51v5Kl X-Google-Smtp-Source: ABdhPJxtuOVFOKAsCuiHh9cOyHIEi9No+IlR64k2qAMP9VTvqHYTpO1ydEtOEgxg9B4D+CYuBIBhhQ== X-Received: by 2002:ac8:74c2:: with SMTP id j2mr3522136qtr.185.1622127933716; Thu, 27 May 2021 08:05:33 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:33 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 03/18] arm64: hyp-stub: Move elx_sync into the vectors Date: Thu, 27 May 2021 11:05:11 -0400 Message-Id: <20210527150526.271941-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080535_276231_D3B4DDE2 X-CRM114-Status: UNSURE ( 9.73 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The hyp-stub's elx_sync code fits in the vector. With this, all of the hyp-stubs behaviour is contained in its vectors. This lets kexec and hibernate copy the hyp-stub when they need its behaviour, instead of re-implementing it. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hyp-stub.S | 64 +++++++++++++++++++----------------- 1 file changed, 33 insertions(+), 31 deletions(-) diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S index 18a97bee3779..86af6c4e52b9 100644 --- a/arch/arm64/kernel/hyp-stub.S +++ b/arch/arm64/kernel/hyp-stub.S @@ -21,6 +21,37 @@ SYM_CODE_START_LOCAL(\label) .align 7 b \label SYM_CODE_END(\label) +.endm + +.macro elx_sync_vector label +SYM_CODE_START_LOCAL(\label) + .align 7 + cmp x0, #HVC_SET_VECTORS + b.ne 1f + msr vbar_el2, x1 + b 9f + +1: cmp x0, #HVC_VHE_RESTART + b.eq mutate_to_vhe + +2: cmp x0, #HVC_SOFT_RESTART + b.ne 3f + mov x0, x2 + mov x2, x4 + mov x4, x1 + mov x1, x3 + br x4 // no return + +3: cmp x0, #HVC_RESET_VECTORS + beq 9f // Nothing to reset! + + /* Someone called kvm_call_hyp() against the hyp-stub... */ + mov_q x0, HVC_STUB_ERR + eret + +9: mov x0, xzr + eret +SYM_CODE_END(\label) .endm .text @@ -34,12 +65,12 @@ SYM_CODE_START(__hyp_stub_vectors) invalid_vector hyp_stub_el2t_fiq_invalid // FIQ EL2t invalid_vector hyp_stub_el2t_error_invalid // Error EL2t - ventry elx_sync // Synchronous EL2h + elx_sync_vector el2h_sync // Synchronous EL2h invalid_vector hyp_stub_el2h_irq_invalid // IRQ EL2h invalid_vector hyp_stub_el2h_fiq_invalid // FIQ EL2h invalid_vector hyp_stub_el2h_error_invalid // Error EL2h - ventry elx_sync // Synchronous 64-bit EL1 + elx_sync_vector el1_sync // Synchronous 64-bit EL1 invalid_vector hyp_stub_el1_irq_invalid // IRQ 64-bit EL1 invalid_vector hyp_stub_el1_fiq_invalid // FIQ 64-bit EL1 invalid_vector hyp_stub_el1_error_invalid // Error 64-bit EL1 @@ -55,35 +86,6 @@ SYM_CODE_END(__hyp_stub_vectors) # Check the __hyp_stub_vectors didn't overflow .org . - (__hyp_stub_vectors_end - __hyp_stub_vectors) + SZ_2K - -SYM_CODE_START_LOCAL(elx_sync) - cmp x0, #HVC_SET_VECTORS - b.ne 1f - msr vbar_el2, x1 - b 9f - -1: cmp x0, #HVC_VHE_RESTART - b.eq mutate_to_vhe - -2: cmp x0, #HVC_SOFT_RESTART - b.ne 3f - mov x0, x2 - mov x2, x4 - mov x4, x1 - mov x1, x3 - br x4 // no return - -3: cmp x0, #HVC_RESET_VECTORS - beq 9f // Nothing to reset! - - /* Someone called kvm_call_hyp() against the hyp-stub... */ - mov_q x0, HVC_STUB_ERR - eret - -9: mov x0, xzr - eret -SYM_CODE_END(elx_sync) - // nVHE? No way! Give me the real thing! SYM_CODE_START_LOCAL(mutate_to_vhe) // Sanity check: MMU *must* be off From patchwork Thu May 27 15:05:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65686C47089 for ; Thu, 27 May 2021 15:51:57 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 11DCC610FC for ; Thu, 27 May 2021 15:51:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11DCC610FC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4kqglnFtaHm9ZE+cWPgljn/8o9XUxhPYEbansdFHvE4=; b=aCKfEXB30tkzCZ S9U+1BJYqhqF4ZDmCqda5CUhBKR/PxlFyfMh0Rdq4gMfC1eD6aAqZKCfMBJgy4SORfQG3Yr5y7zFN QgczF4XOnrTCD3ve7MpeZjsBy/W9iRuMLArzS4kZwr5bhQartZNJfx9j2TqMqIq+gQEohMc2caPV1 Zx35Mj9x/Not4UGPLJRUO73nrDHG9uz9YvrUlydTDAJChlBmkQwh6yEgS9eii0c74IsQEHH2ZMIUf 3ANlSNwMl32tbLaPXndudqnFhGLpnJtbfnOXfj9joMAq/e9Ajb5QleXDo9Ab4Rx+wpjIugyDdmY6a +QZVDw9u2agji9Ktp8hA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIG8-007Lv7-VA; Thu, 27 May 2021 15:49:23 +0000 Received: from mail-qk1-x72b.google.com ([2607:f8b0:4864:20::72b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZo-0071hU-EJ for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:40 +0000 Received: by mail-qk1-x72b.google.com with SMTP id c20so936988qkm.3 for ; Thu, 27 May 2021 08:05:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Wg7p4vPv2NoAb+n17TZqII3/Mw2eE/iFM20fouMRLwI=; b=XrzMTNI74leC845PoHu6NBtRzlE8xBimsUOjXLlm0vqEuD994Nb3Pc9xZT4gWtpHt8 6lnP0+AaY2h28uSAyDh1nu4Z+LTcUlllsx5eiqqj5bxu/ktF78lPtnAclLwfIv9SbkiD mxf6ado+CHp4f52cHMoGtOEQ1QFmpN1GY3q5Su+KiQuZ3j8z40XlQF1SLQjKkxpoYSe4 Qoc1WTi0P9N2l1MX2xW5Que69zUhDpRgbA6a0ISuCBIqhTg95a8x65rSXLfqzWFsJIpb sP6Fsxt+vD5yqAz7mf4NLYOIOZlfCeWr0fuDyYPEX6c3bseja8qsl/UQYn1elCzeeUQ0 b/YQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Wg7p4vPv2NoAb+n17TZqII3/Mw2eE/iFM20fouMRLwI=; b=SlcHu/di8iUhKMpkX5jvvDwDadyw48yZRl/xRPJTCo/9fOB3w/kUFnsup7WkM1cNe1 UZcgxE8kzjHDv+wW0MfO9LnThyJCh/gqIR33MPlQ+sj7R00niCJOck0NYByTMLc8I3c5 PSsJbw2CKziGmyIR2bzvdwg+rVRY8had7W/Wy/GjcuSSjF0bhWLUjUnuulmgB6niuAwd J+BbMCNB17e66L57i3jNHe8yTF1abEW5H1HU3nQdwWF6uWwUvRLpDbAakKbv9AtrmTnx BqEBq+CuN3flPI6T0JCclff04ENJmokHI/uZE9Hr0zt/QxIuh9VACIgWKYTHLCRkQ8A4 jzCg== X-Gm-Message-State: AOAM532TlAJjitVcl3daZBoHFHtN+KUvfLcOoqNhNpo8oYUgZlaTTxDQ gmIcLJqb+htM3C9fA8/Y4YyF1w== X-Google-Smtp-Source: ABdhPJycl8DRfGqs/GI/GT6NLoLRwo6sDRsMxgXrzg8L64wopg0OcAVe2PjXq4CFxxycTq8ansLW+g== X-Received: by 2002:a05:620a:14b9:: with SMTP id x25mr3931435qkj.460.1622127935134; Thu, 27 May 2021 08:05:35 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:34 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 04/18] arm64: kernel: add helper for booted at EL2 and not VHE Date: Thu, 27 May 2021 11:05:12 -0400 Message-Id: <20210527150526.271941-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080536_571731_E137C7D1 X-CRM114-Status: GOOD ( 16.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Replace places that contain logic like this: is_hyp_mode_available() && !is_kernel_in_hyp_mode() With a dedicated boolean function is_hyp_callable(). This will be needed later in kexec in order to sooner switch back to EL2. Suggested-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/virt.h | 5 +++++ arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/hibernate.c | 9 +++------ arch/arm64/kernel/sdei.c | 2 +- 4 files changed, 10 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 7379f35ae2c6..4216c8623538 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -128,6 +128,11 @@ static __always_inline bool is_protected_kvm_enabled(void) return cpus_have_final_cap(ARM64_KVM_PROTECTED_MODE); } +static inline bool is_hyp_callable(void) +{ + return is_hyp_mode_available() && !is_kernel_in_hyp_mode(); +} + #endif /* __ASSEMBLY__ */ #endif /* ! __ASM__VIRT_H */ diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 9a7b1262ef17..48d0ed48c147 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,8 +20,7 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = !is_kernel_in_hyp_mode() && - is_hyp_mode_available(); + unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..c764574a1acb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,9 +48,6 @@ */ extern int in_suspend; -/* Do we need to reset el2? */ -#define el2_reset_needed() (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) - /* temporary el2 vectors in the __hibernate_exit_text section. */ extern char hibernate_el2_vectors[]; @@ -125,7 +122,7 @@ int arch_hibernation_header_save(void *addr, unsigned int max_size) hdr->reenter_kernel = _cpu_resume; /* We can't use __hyp_get_vectors() because kvm may still be loaded */ - if (el2_reset_needed()) + if (is_hyp_callable()) hdr->__hyp_stub_vectors = __pa_symbol(__hyp_stub_vectors); else hdr->__hyp_stub_vectors = 0; @@ -387,7 +384,7 @@ int swsusp_arch_suspend(void) dcache_clean_range(__idmap_text_start, __idmap_text_end); /* Clean kvm setup code to PoC? */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); dcache_clean_range(__hyp_text_start, __hyp_text_end); } @@ -482,7 +479,7 @@ int swsusp_arch_resume(void) * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (el2_reset_needed()) { + if (is_hyp_callable()) { phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; el2_vectors += hibernate_el2_vectors - __hibernate_exit_text_start; /* offset */ diff --git a/arch/arm64/kernel/sdei.c b/arch/arm64/kernel/sdei.c index 2c7ca449dd51..af0ac2f920cf 100644 --- a/arch/arm64/kernel/sdei.c +++ b/arch/arm64/kernel/sdei.c @@ -200,7 +200,7 @@ unsigned long sdei_arch_get_entry_point(int conduit) * dropped to EL1 because we don't support VHE, then we can't support * SDEI. */ - if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + if (is_hyp_callable()) { pr_err("Not supported on this hardware/boot configuration\n"); goto out_err; } From patchwork Thu May 27 15:05:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C95D6C47089 for ; Thu, 27 May 2021 15:54:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 862D5613BF for ; Thu, 27 May 2021 15:54:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 862D5613BF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Z5P2vBPyqwsUFuvLrRMg53ubUWKyHW6IoDHjYwlhRYw=; b=KyZKPiMDnBSv5L gyeCGh5/RKiF9niW6l0bQf++2yCaCxq/RNwTpvJxhBDQVYwZnBrILfooUPBJgGaNX+tKiVKHM7AGh nvwkpUOQ/Xm7GreyOBW3OqKvVaAF6u+svzX96cldSJWOga0MjawUzr46QM/VtT+js535DwmbXJigx Kdvf6pNg0JPzMAE1CYYY/ck7SeMZeLWhlm3eIi2jM13CPqYSf74kKiIi1+Mf4/KmhfJth8XXX/xkp MYcVgzKW3PzTfk3bX1A6LgyLTF8sxUn1Hz1VQGBQWvAT2DR3iQYMSjUgf22+lOpTlNNOEh8omRT1m Wi6R0d6J1pS+lRCQDoeA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIIW-007Msg-TW; Thu, 27 May 2021 15:51:53 +0000 Received: from mail-qv1-xf32.google.com ([2607:f8b0:4864:20::f32]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZp-0071iZ-UE for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:41 +0000 Received: by mail-qv1-xf32.google.com with SMTP id ee9so238714qvb.8 for ; Thu, 27 May 2021 08:05:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=OoTpQqa+5r8BlTD7ijih47CrLq8A6B9RbRxoHM1uCuW1Z7lER6etpTCJW6b36pKuoO /Ib+oumN81PGrjvwhUucnW8PIIX/xfixSd/82C9Ngo0OghsxdhoPMN9YB8akCg4px1aZ XzXeqso3LANa1DVqRQz9crazOLSsLdTmD5xQcuItCBxnRmMyC6R44//8WloWI52+WiPZ UYEUxuuFElnKBjvp82oC1mMOpZWb2Z7QJCVc/Q8PHNr52Va/jYqVOiEiP6SEjdDDqhBD 2jLNo0uWAKW+fjYBYBBF+rnqF8xkm+S4zZh0UrJJEjyMpmAzJzUiWOzMjiANp5dxzvof YSNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+EiR776LGRIVUqs8E6nTUaQXNk2yNVD65j0QM2uVzow=; b=mtzSfG9gO2YGptZFdik5W/+/qGpoEPZIFeBi7GBjQedYusDVZO1rf2iHSz/tLWm1Le bGyqFIst2viCi/y9dPW7DYaW2dqv/nAVSJhUrjjBocEpMDkTtDrsuVES+gs/4RuI+pVS CLdRUBXjO1mDksgqMudRGxtauxJggCGLm6F8unZCWK2H/eWWIJ1corl993Y3QAgDyDML 3WwrYJ6b4XpBC7vvsUj9SpgVx7B70gQ/T39eKIJA5hZzB4vtqSvFbKybRsCs3I55k6TE grOGGxPAVwFl0CX3bi2HePS40MK8a9vADK7vkBt0tUODEN6XZlNI3lqjkBfBAwThCSNj l1pg== X-Gm-Message-State: AOAM532zLQNb27ZhUNMrxetXxlydR1pQcKjPxyyGCSiVkAdihPE7Q72m lJOLEK7TDT2NwNE8UGeW54fDGg== X-Google-Smtp-Source: ABdhPJzTP3oV1R+n8kGbXURQLbsLJZ4X6S2bcFpIfYYBkRufEHUPnqQYpI7uOymwC/kI/WMb3ZRyIw== X-Received: by 2002:ad4:4773:: with SMTP id d19mr4316997qvx.0.1622127936544; Thu, 27 May 2021 08:05:36 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:36 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 05/18] arm64: trans_pgd: hibernate: Add trans_pgd_copy_el2_vectors Date: Thu, 27 May 2021 11:05:13 -0400 Message-Id: <20210527150526.271941-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080538_031572_016E11F6 X-CRM114-Status: GOOD ( 17.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Users of trans_pgd may also need a copy of vector table because it is also may be overwritten if a linear map can be overwritten. Move setup of EL2 vectors from hibernate to trans_pgd, so it can be later shared with kexec as well. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 3 +++ arch/arm64/include/asm/virt.h | 3 +++ arch/arm64/kernel/hibernate.c | 28 ++++++++++------------------ arch/arm64/mm/trans_pgd.c | 20 ++++++++++++++++++++ 4 files changed, 36 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 5d08e5adf3d5..e0760e52d36d 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -36,4 +36,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors); + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h index 4216c8623538..bfbb66018114 100644 --- a/arch/arm64/include/asm/virt.h +++ b/arch/arm64/include/asm/virt.h @@ -67,6 +67,9 @@ */ extern u32 __boot_cpu_mode[2]; +extern char __hyp_stub_vectors[]; +#define ARM64_VECTOR_TABLE_LEN SZ_2K + void __hyp_set_vectors(phys_addr_t phys_vector_base); void __hyp_reset_vectors(void); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index c764574a1acb..0b8bad8bb6eb 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -48,12 +48,6 @@ */ extern int in_suspend; -/* temporary el2 vectors in the __hibernate_exit_text section. */ -extern char hibernate_el2_vectors[]; - -/* hyp-stub vectors, used to restore el2 during resume from hibernate. */ -extern char __hyp_stub_vectors[]; - /* * The logical cpu number we should resume on, initialised to a non-cpu * number. @@ -428,6 +422,7 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; + phys_addr_t el2_vectors; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -455,6 +450,14 @@ int swsusp_arch_resume(void) return -ENOMEM; } + if (is_hyp_callable()) { + rc = trans_pgd_copy_el2_vectors(&trans_info, &el2_vectors); + if (rc) { + pr_err("Failed to setup el2 vectors\n"); + return rc; + } + } + exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate @@ -467,25 +470,14 @@ int swsusp_arch_resume(void) return rc; } - /* - * The hibernate exit text contains a set of el2 vectors, that will - * be executed at el2 with the mmu off in order to reload hyp-stub. - */ - __flush_dcache_area(hibernate_exit, exit_size); - /* * KASLR will cause the el2 vectors to be in a different location in * the resumed kernel. Load hibernate's temporary copy into el2. * * We can skip this step if we booted at EL1, or are running with VHE. */ - if (is_hyp_callable()) { - phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; - el2_vectors += hibernate_el2_vectors - - __hibernate_exit_text_start; /* offset */ - + if (is_hyp_callable()) __hyp_set_vectors(el2_vectors); - } hibernate_exit(virt_to_phys(tmp_pg_dir), resume_hdr.ttbr1_el1, resume_hdr.reenter_kernel, restore_pblist, diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 527f0a39c3da..61549451ed3a 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -322,3 +322,23 @@ int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, return 0; } + +/* + * Create a copy of the vector table so we can call HVC_SET_VECTORS or + * HVC_SOFT_RESTART from contexts where the table may be overwritten. + */ +int trans_pgd_copy_el2_vectors(struct trans_pgd_info *info, + phys_addr_t *el2_vectors) +{ + void *hyp_stub = trans_alloc(info); + + if (!hyp_stub) + return -ENOMEM; + *el2_vectors = virt_to_phys(hyp_stub); + memcpy(hyp_stub, &__hyp_stub_vectors, ARM64_VECTOR_TABLE_LEN); + __flush_icache_range((unsigned long)hyp_stub, + (unsigned long)hyp_stub + ARM64_VECTOR_TABLE_LEN); + __flush_dcache_area(hyp_stub, ARM64_VECTOR_TABLE_LEN); + + return 0; +} From patchwork Thu May 27 15:05:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09B8FC4707F for ; Thu, 27 May 2021 15:59:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C4179613BF for ; Thu, 27 May 2021 15:59:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4179613BF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=f1ErHDr7klfN5loKRYtmc0uASOmqV2mlJktD0KX0ecQ=; b=nXOHybcgQ6Tu4O u/WrwbIAmOPOqhGCFnnHaT1IhnBkDlyrM8XWdGO2gDa2VXDvuel8BonuLFdsIICrk9eOHExds273b fS2MY8a753bxviICaJRAA4QXbGAb3m6pBiFwLaF3irnUQM6bweKr+0paOQ49spkzJ4/wXXxZ4/YD0 Ywd9930ZRaBWvxX/j11BUuakZLNLWL32pjFginGac+q3+lLdynXEsmAanNZJbSfwXbnAp/2wPM4kD eWXKq6wr9WYR1VJr/X6QoRxpX2a1gzo3MJQdW9sfzLC9qSsktvD98tqu7PmwSq7+sUuMVj5rIrSh2 BHVRMpjssvrqEgTVkflg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIMd-007OmU-Vy; Thu, 27 May 2021 15:56:05 +0000 Received: from mail-qk1-x730.google.com ([2607:f8b0:4864:20::730]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZs-0071jc-4y for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:43 +0000 Received: by mail-qk1-x730.google.com with SMTP id o27so912377qkj.9 for ; Thu, 27 May 2021 08:05:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uRcNIBXOyaSrsQwthVpy1hwauBBMN7Vivxx0utjD54U=; b=baw560aO1zilTRnsPH5ODSQM+umpDLJom0wWm7jrdVnxroTvnRw1ujP0UhZKxl4h16 N38nZKepAcYoE0UAEpaMfhWdP2zvyq4n22H9pFgq0X6dglCZZhNLhxbZjmXHThU2ZZJd FBIZs0K2hc/GsDRn7wCKSuSUDkhiWtOHZ6MQT3KjWhvmW/VAXRRm2CmpHf13D+JUuP5U jngl4FpHRuskS3sFTcK/T1B7mB3OYSga5Z7occBeWpVtPNsXbCtuGJExnBIOgMY8UZJJ adKCXSP8eR+kApqLmL1Xc5egVdLNQ8x4y3pH9ptqd38tuCXkVllnx9kYOl5J496pyK3b lgvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uRcNIBXOyaSrsQwthVpy1hwauBBMN7Vivxx0utjD54U=; b=GlYr7qBij9fAWFSZImwIgM6P0BlYFL4rjQNRR45gzEcJRHmyfoeFCWnKL4TJQTqjnf fD+jZDnnQVsUWFTKbsLqVN+49drLh6zEc4M/EuCUq11VtnsTw9FjdKMVgBIXgbxjG79g 035Jo2gSvMpHaX+78Swa+nrYrizdsZF7ocjQm0g/djVP4gv/AfixLYtJdOZEhQKLJA+q b99So3GJk5IZHmciMyoUfVzQRGltgCwgBscTIHWYNMweqwLqca3Y6QyBXavishLizxCB dmq1tcCqfGZe8mo4ToATfr40h4DDkjV0nhgoa6XTmHlX8WaM4wrXj9n4x7WIBYI5A4W0 fahg== X-Gm-Message-State: AOAM532FZV/RS4qPNSlB81LDJDe7NQOLrs/adDJ3LyiSM4CNrO5/E2wH E4UrqJFscS4EA2C0z2gmPbQ1WA== X-Google-Smtp-Source: ABdhPJxMXmwEgisa5gA4j+w8k9Kb9fYN4u/mCq3bmdSU3qA/vvzD6gnPAIKYpynHFW3vyiVzDRiyAQ== X-Received: by 2002:a05:620a:1126:: with SMTP id p6mr4086588qkk.120.1622127938229; Thu, 27 May 2021 08:05:38 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:37 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 06/18] arm64: hibernate: abstract ttrb0 setup function Date: Thu, 27 May 2021 11:05:14 -0400 Message-Id: <20210527150526.271941-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080540_372510_1750179F X-CRM114-Status: GOOD ( 15.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, only hibernate sets custom ttbr0 with safe idmaped function. Kexec, is also going to be using this functinality when relocation code is going to be idmapped. Move the setup seqeuence to a dedicated cpu_install_ttbr0() for custom ttbr0. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 24 ++++++++++++++++++++++++ arch/arm64/kernel/hibernate.c | 21 +-------------------- 2 files changed, 25 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index d3cef9133539..85eb92a2ffcb 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -115,6 +115,30 @@ static inline void cpu_install_idmap(void) cpu_switch_mm(lm_alias(idmap_pg_dir), &init_mm); } +/* + * Load our new page tables. A strict BBM approach requires that we ensure that + * TLBs are free of any entries that may overlap with the global mappings we are + * about to install. + * + * For a real hibernate/resume/kexec cycle TTBR0 currently points to a zero + * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI runtime + * services), while for a userspace-driven test_resume cycle it points to + * userspace page tables (and we must point it at a zero page ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). + */ +static inline void cpu_install_ttbr0(phys_addr_t ttbr0, unsigned long t0sz) +{ + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + __cpu_set_tcr_t0sz(t0sz); + + /* avoid cpu_switch_mm() and its SW-PAN and CNP interactions */ + write_sysreg(ttbr0, ttbr0_el1); + isb(); +} + /* * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 0b8bad8bb6eb..ded5115bcb63 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -206,26 +206,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (rc) return rc; - /* - * Load our new page tables. A strict BBM approach requires that we - * ensure that TLBs are free of any entries that may overlap with the - * global mappings we are about to install. - * - * For a real hibernate/resume cycle TTBR0 currently points to a zero - * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI - * runtime services), while for a userspace-driven test_resume cycle it - * points to userspace page tables (and we must point it at a zero page - * ourselves). - * - * We change T0SZ as part of installing the idmap. This is undone by - * cpu_uninstall_idmap() in __cpu_suspend_exit(). - */ - cpu_set_reserved_ttbr0(); - local_flush_tlb_all(); - __cpu_set_tcr_t0sz(t0sz); - write_sysreg(trans_ttbr0, ttbr0_el1); - isb(); - + cpu_install_ttbr0(trans_ttbr0, t0sz); *phys_dst_addr = virt_to_phys(page); return 0; From patchwork Thu May 27 15:05:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DA7BC47089 for ; Thu, 27 May 2021 15:57:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 39E16610CC for ; Thu, 27 May 2021 15:57:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39E16610CC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L9vS0tAUgnK695+xMNOhNTPthc35T9698qK9uRNN4rs=; b=1j/tk/IXQK1KFe bVAig27sUMSHcDwfTg00VtUQx9REvFIoCOIrOx+Qf18F6kuF/sCh0rHmiibl03nM41G014SaNNIbI Ag/LjkN2Tw++9ZV46j6NOMLt3TBEhxmj7GYqzuAY7VI5nrJ66cxiWvb5J65wctlERqChZMpY1FEXw hdv5rTJ1A6mRu+zQ1edw2xQLtVEX5Clz7FAuNpROlvM0YxApb3BHQw2YRRvY8hqsYFR1iV/Sj19fG i9VHxr+cezhcXv1MBOlHIqFT3rcbtSzlKV3CZQOr+kCRUI3VIoY/oIHAHOeQ/9cGSSYHQa27QpxVm WMl+pg3Cxmc+srkd/ahQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIKp-007Nuq-Hf; Thu, 27 May 2021 15:54:14 +0000 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZu-0071kd-0B for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:44 +0000 Received: by mail-qk1-x729.google.com with SMTP id j189so936487qkf.2 for ; Thu, 27 May 2021 08:05:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=FoXurcNpRVh8PW9Lk8j7yxcfb9WhsD1odEsRlBENU4YBfTWy5FE7Pci3MirsOpLizl OAwkUiXNxiixolnp818py2J+B/N0h14eTcWXM9L2T3NeUf5C9R5OGcYfwKJsn7sjcW0y RlZ2lY5o9vDQkxgcwfg0mNbVjK/w5ueBm5N8jUlDq5eFUT44aspwJXmgwff2E9EiOGGc 52TL/Kuuj7KVAqFRjlqujmS49xER8gmVrA135D9td7VASDqpb6Ou3adZ8eXy/q+KMWXP ACHn61DU7z5lzrxfnWLT9k5wcrTSFC0FGjeIZyFeGvHPa0mwpY/z2P37hkae3AzL+Xfg PhTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FcIEg/sDUKkdURMCkTF1KW5fxOPpkw674yFxDZdydcg=; b=mP7H7RgralSxl8RdIdKoVEJ9PT+x/2GgCc2PYuGXMOKus5d/zUGfm98fohph1wjxJC ic61fJIsgcAj8q8ZRtc2ZcDLnS02qZiUx59GmYwnqFU77hZzYUe7Va82D8lCuj7q9zfc z/ITyPfw15Kzqds84PaZXccijg1yzzMlKTjtpztwktZCBpj48f6f2V1h5ydRhqvWkc3x KRGI66EWo5gwx9kj+ewV7Nte7+TldnMDQx577SMV3VIkZLRbl3Ju7CKzep8/I9stOPYD 7W6BoP3VrcPLgRy3wTRNJy/oUUzvg9MOfGM3E6j5P/l0yHYauLgSoJzSdZfyfcojM0+2 W5Lg== X-Gm-Message-State: AOAM532rbCcxjF5YBhWcughwjd+m8Ir4MCAGUpe+zY2Mq0S48e4R87l1 AqiqBPv1s+CrbftBwq2LfyxTdg== X-Google-Smtp-Source: ABdhPJzfd4jTVGoM6zB9+XxkahnEtHa/XkFTFTWTzzz5x2x/RbHjlysPqBPq0fd50bFxJPKMZb48wA== X-Received: by 2002:a37:30c:: with SMTP id 12mr4018868qkd.355.1622127939672; Thu, 27 May 2021 08:05:39 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:39 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 07/18] arm64: kexec: flush image and lists during kexec load time Date: Thu, 27 May 2021 11:05:15 -0400 Message-Id: <20210527150526.271941-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080542_235304_F3A325C7 X-CRM114-Status: GOOD ( 15.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, during kexec load we are copying relocation function and flushing it. However, we can also flush kexec relocation buffers and if new kernel image is already in place (i.e. crash kernel), we can also flush the new kernel image itself. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++++++---------------- 1 file changed, 23 insertions(+), 26 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..3a034bc25709 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -59,23 +59,6 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } -int machine_kexec_post_load(struct kimage *kimage) -{ - void *reloc_code = page_to_virt(kimage->control_code_page); - - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); - - /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); - - return 0; -} - /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -152,6 +135,29 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + /* If in place flush new kernel image, else flush lists and buffers */ + if (kimage->head & IND_DONE) + kexec_segment_flush(kimage); + else + kexec_list_flush(kimage); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec - Do the kexec reboot. * @@ -169,13 +175,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - /* Flush the kimage list and its buffers. */ - kexec_list_flush(kimage); - - /* Flush the new image if already in place. */ - if ((kimage != kexec_crash_image) && (kimage->head & IND_DONE)) - kexec_segment_flush(kimage); - pr_info("Bye!\n"); local_daif_mask(); @@ -250,8 +249,6 @@ void arch_kexec_protect_crashkres(void) { int i; - kexec_segment_flush(kexec_crash_image); - for (i = 0; i < kexec_crash_image->nr_segments; i++) set_memory_valid( __phys_to_virt(kexec_crash_image->segment[i].mem), From patchwork Thu May 27 15:05:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D894C47089 for ; Thu, 27 May 2021 16:02:31 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 24FC2610CC for ; Thu, 27 May 2021 16:02:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 24FC2610CC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xjdMMiTBaisYLvNI6QuNjDZxBeqd3q/2lU5FB05UPOw=; b=aW54t0cpMOJMUN BEXpvjA+ODJAOfSmmQ/h0cr3ScFCfCK0fRmH+pUJ1Rdq5prn7fwuGd1LWECZRIGp2cDsUroRULP4t p8uKM7qCRuV3MZX1aO6ToyIU7r3Iq1N02AjHzHXcG+V0QBrszHpo7Rr4hmYubf7MXwOTnrt2ND14f 653/aS3CEDt0xgbyHhzZGOaBedB7ly7EY7YbNcGPpB/0G4255klIkuqrJoIn12ESadPpDm4gXqYsy IWC1bnz2WXaFIWgZENci0TrKAWdiY+EdpynCicfhM68/C2tCce2jIpRSw5sdbDrYz3p6OdgTfI2pk zDlrsWKbwksrVbjP6tlQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIQY-007QRT-Ib; Thu, 27 May 2021 16:00:08 +0000 Received: from mail-qv1-xf34.google.com ([2607:f8b0:4864:20::f34]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZv-0071lC-FK for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:47 +0000 Received: by mail-qv1-xf34.google.com with SMTP id v18so232079qvx.10 for ; Thu, 27 May 2021 08:05:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=WOdTNCc8/+vKdXFWJvALf/5jcbGhRyKjqHuE0XK9ac4=; b=d8BkTIMDubNMKo8lhGcNu7AKdRus5J2M4TufcGSW4HtAeu3NbJ3XThPB66dQ+1m9p4 uQSN6tv4/0GLeQQUYYLvH9KTMD/3Y0ymqQQJUncGSGi7O+XI9dAh+bjI/F2JTBUYDI/b hG56Miopzkxf4OQah6VFcnupnjMDqnyYzxqclfNuueFAlJIXCIk/mUQkSy0l9hy7LZ63 8t5Zl2QIKW2FphA5noBR5hutyqZWAAE7ORfjV0j8Vn4IoOBLjWbakCjpSu+mbfcwra9P 9PA5A7XXtZhKASA48HUe+DjXCCyoKtyblRuv7GcVXybzYhpFeowUdCednMkzMNvzqc3m sUNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WOdTNCc8/+vKdXFWJvALf/5jcbGhRyKjqHuE0XK9ac4=; b=q5WO8/QTHpo8wAHUDTyD/ESENaZ9NkbgyRqhEVmxKl5p8aQSdWg9GR+ihsFwCnwYuq LRRL4RKtRhSvdRrhDFgt6vlzgcJ4CEHQZW/RpseV1LijIN/6sUaTcpxQL0fd+xomGCMs KbU/CyiwMslQltVAibFvT4/ryqE14+PGtJ9R/vFcYVqnihkswz6H6L9gxURmc3SmuGqV nKA3/2VJhPKmdSwpaVO6UqSKCEeqMy2Z82LsGXDaprZFBYTExC2G8/ElavJBIhc4KtMX wJ3M6oDt9UqbxRyLRdXHF8sQvnJAlDWQdh1ny97keLA3j1/bJ900d01o96lOs9Ix+Ph7 4tnA== X-Gm-Message-State: AOAM531Qau5cjVG2uqnBAHPBeBqUlmfWOIh5EigQNAWKh+WMTpVzgQT0 K/t7AfO3VESgEk80AQOREB1pAw== X-Google-Smtp-Source: ABdhPJxuuInT5boY5wx2h9r0mi/iNoeH4rIqWoxJwXXRJPR79arNDGrkhRpZr8NydkZli6xNGukztQ== X-Received: by 2002:ad4:4810:: with SMTP id g16mr4197984qvy.21.1622127941083; Thu, 27 May 2021 08:05:41 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:40 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 08/18] arm64: kexec: skip relocation code for inplace kexec Date: Thu, 27 May 2021 11:05:16 -0400 Message-Id: <20210527150526.271941-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080543_659726_E5051555 X-CRM114-Status: GOOD ( 18.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In case of kdump or when segments are already in place the relocation is not needed, therefore the setup of relocation function and call to it can be skipped. Signed-off-by: Pavel Tatashin Suggested-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 34 ++++++++++++++++++----------- arch/arm64/kernel/relocate_kernel.S | 3 --- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3a034bc25709..bde623805866 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -139,21 +139,23 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); - /* If in place flush new kernel image, else flush lists and buffers */ - if (kimage->head & IND_DONE) + /* If in place, relocation is not used, only flush next kernel */ + if (kimage->head & IND_DONE) { kexec_segment_flush(kimage); - else - kexec_list_flush(kimage); + kexec_image_info(kimage); + return 0; + } memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); - kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + kexec_list_flush(kimage); + kexec_image_info(kimage); return 0; } @@ -180,19 +182,25 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the kern_reloc which contains a copy of - * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel - * uses physical addressing to relocate the new image to its final - * position and transfers control to the image entry point when the - * relocation is complete. + * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * caches. However, restart will start new kernel or purgatory directly, + * cpu_soft_restart will transfer control to arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + if (kimage->head & IND_DONE) { + typeof(__cpu_soft_restart) *restart; + + cpu_install_idmap(); + restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, + 0, 0); + } else { + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, + kimage->start, kimage->arch.dtb_mem); + } BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..8058fabe0a76 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -32,8 +32,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x16, x0 /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ - tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -65,7 +63,6 @@ SYM_CODE_START(arm64_relocate_new_kernel) .Lnext: ldr x16, [x14], #8 /* entry = *ptr++ */ tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: /* wait for writes from copy_page to finish */ dsb nsh ic iallu From patchwork Thu May 27 15:05:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6443C47089 for ; Thu, 27 May 2021 16:04:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 768B1610CC for ; Thu, 27 May 2021 16:04:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 768B1610CC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lfaG6ID0OTf0b+zZJvE8r0xv9fil3WW4pwLBA7WfwxQ=; b=GKYI6+VoErGLOl 8dHYoZyK9+Pe38TCAsEsl2woWC7dtNPNBYpLoZ7cwhbvHN7VEa3rVAYhOxSbUdckB3NmDg36f3jJ1 hYpMnz5iDhGxDL7kqMTveMp4eVWpZilCJIlsaZrdAsuok7OX8AXoGkLwy5Ylt76wupokMSnZLI2XY ab2mEMBDOmPfY9Uq3oA+p9Ryfy4XrmcbBNC60FFexC1bskhNixu1xrwZSm3PMFu1UsjY6jx80q3J4 ul9OpFEhKhxXxcAd0wBHfVadvXsTd/34vLcxGcqvTzuGN4bXImsw+/4mS2jhVZ/VbrhJxB4sEroRw jo0EP6+9D5FSHG4IjbiQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIS5-007R8k-1o; Thu, 27 May 2021 16:01:42 +0000 Received: from mail-qt1-x836.google.com ([2607:f8b0:4864:20::836]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZw-0071mE-MA for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:47 +0000 Received: by mail-qt1-x836.google.com with SMTP id t17so402639qta.11 for ; Thu, 27 May 2021 08:05:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=I6lkOzPEa6syzQf8r+RZ8XuPpKpclGEbIUwShbrK82U=; b=TnnFfUUvHvEm/etkMIl6FavDDCjxG9hxxxgfKcLeF7n8pbKBfizh0PVGR9SMdhavpQ /qaXI18/FRxFUfElEb2Mw/gUsiyEEVcvLvMB17wq4wMBqThZ8ZW1wxboXPji+jtFXAzA dmFznHFw0mFTEWzs07I5Or3POBDkvY2Li5BAWGSV9TlzSh3tpEA7QozmVmOde0tdRNWJ Ho9DhH92QuT2QsQkhMfXBenFVxdWr5De+0nSVKKvmbALAiuom+VmBP2Ocl+xIXaospwh fhvqjPCz2kafZH+AeYAvRnXpZ979KsJEGkCMNPqV9/MZKUUhRoCgzAp927vHfgXWTBlK QONw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I6lkOzPEa6syzQf8r+RZ8XuPpKpclGEbIUwShbrK82U=; b=eRDvaulw+x8XIMQu0gJIKfEzPql01lyN1erjBR2BIyr42pZXFqFP5yrmF1dTvkXU4r jXZQdEF9XbNC1xxyP4xRFbSALCCP9t0IYBu0D7McVXi3ytoyNZcferkRsVDIA9r6Ws0w t4VJ8VslkR84rGk5x1pZ0BEqAs0VMslzjI7/xQZ9S1RNCCrW0ZVfxYwv52RvDIxbr5G6 MjR8nxybkh11H3A+z6fYphkSRjjpPIM/r/sDw45hAlKiLi/YF7R0oxyWKeejFRrlmfMa 3eu+d8BZ4Bo+bLCuJYZcrAHOkfpD/+bjydItP0mffdQgdG1Fcy1iuZxNIMctqpEt8asf vyFA== X-Gm-Message-State: AOAM532pLraEPWuU3/q7t0aGTYsV8ysOXC6Kfb3l0x7mpYlAL4675jlb RgJAK8NaOrBq0C4BTzrg9obwPg== X-Google-Smtp-Source: ABdhPJyswzPAqJ6yfkg4N9Ue8mJrxUt1+TgPxwsNMw1UD8Ns1Nxidx0p88Kxuz6WsVPrCUipNJJJbg== X-Received: by 2002:ac8:5e98:: with SMTP id r24mr3491247qtx.25.1622127942481; Thu, 27 May 2021 08:05:42 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:42 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 09/18] arm64: kexec: Use dcache ops macros instead of open-coding Date: Thu, 27 May 2021 11:05:17 -0400 Message-Id: <20210527150526.271941-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080544_844180_A7297690 X-CRM114-Status: GOOD ( 12.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse kexec does dcache maintenance when it re-writes all memory. Our dcache_by_line_op macro depends on reading the sanitised DminLine from memory. Kexec may have overwritten this, so open-codes the sequence. dcache_by_line_op is a whole set of macros, it uses dcache_line_size which uses read_ctr for the sanitsed DminLine. Reading the DminLine is the first thing the dcache_by_line_op does. Rename dcache_by_line_op dcache_by_myline_op and take DminLine as an argument. Kexec can now use the slightly smaller macro. This makes up-coming changes to the dcache maintenance easier on the eye. Code generated by the existing callers is unchanged. Signed-off-by: James Morse [Fixed merging issues] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 12 ++++++++---- arch/arm64/kernel/relocate_kernel.S | 13 +++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8418c1bd8f04..f1367863d995 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -393,10 +393,9 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 - dcache_line_size \tmp1, \tmp2 + .macro dcache_by_myline_op op, domain, kaddr, size, linesz, tmp2 add \size, \kaddr, \size - sub \tmp2, \tmp1, #1 + sub \tmp2, \linesz, #1 bic \kaddr, \kaddr, \tmp2 9998: .ifc \op, cvau @@ -416,12 +415,17 @@ alternative_endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 + add \kaddr, \kaddr, \linesz cmp \kaddr, \size b.lo 9998b dsb \domain .endm + .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + dcache_line_size \tmp1, \tmp2 + dcache_by_myline_op \op, \domain, \kaddr, \size, \tmp1, \tmp2 + .endm + /* * Macro to perform an instruction cache maintenance for the interval * [start, end) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 8058fabe0a76..718037bef560 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -41,16 +41,9 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - + mov x2, x13 + mov x1, #PAGE_SIZE + dcache_by_myline_op ivac, sy, x2, x1, x15, x20 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: From patchwork Thu May 27 15:05:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E7BCC47089 for ; Thu, 27 May 2021 16:06:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E9C82610CC for ; Thu, 27 May 2021 16:06:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9C82610CC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=b8mUbE2Xn4NEcZnGX72bfeqIG7OwwTRlbTZAAwxcUzw=; b=Hqht+wpG+3xlnD D9cZGkfyKmQFfPnSDFWC6RFzT5eE8GQ0ifYCuYliCzGB1tapjORbJiGPOnMpgA1a24m9TxDIyzA5Z HZNjx5rznxGoqRfLa9FPfn49Qp89MzWlZNaKYFIE7VVKXT27/x1eq9pqXS1XJmIX62Gr3bj0n8YMK fVXFn2gy2rJDfmW79QYPqaGRS8VtmoCsr2MK5ag3tn4LZowlaNNsfB01T10wbHwooWUmnLKGzGXcS YGaxs+GbKsnvsZo3EdyV3RUOJJbHBvGyf3hyw1nVcI5VCq8UePfj8Bav0NcGtYlRJ/i39Gx+MF92e E/ebRozoCndmTfDk3/Nw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIUI-007S3X-RT; Thu, 27 May 2021 16:04:00 +0000 Received: from mail-qv1-xf32.google.com ([2607:f8b0:4864:20::f32]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZy-0071nK-Ig for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:49 +0000 Received: by mail-qv1-xf32.google.com with SMTP id o59so257936qva.1 for ; Thu, 27 May 2021 08:05:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=wPqWbOC642+FQuLJdmHZY8xtJm1ZVhhgjTRiZDTZY7Q=; b=OYcSb6MwLOEpinFGHYb3FQsN6yhufIjwNbxeyeRaz5wrZtQ/5vpDYqcsw6mj8mqgFS 5UE2pAQSBYMHHc5pzxtMNiWczCnfbE1ewNR25yNAc51ClquEAU42eWpD0wf7yKRwaa8u EJ4J1rzwEIAcPfcFUk17zjO/4bk1E+3laPOMgMNvpcUPtx9wdkjNK16ungcGbtUZZggp R0htuLwV94wC/5xA6en5rnhOodoYgPQYgWwBNINEbi9IsatqBRJokknUJ9QHGsoZbH40 o1WrbfLH0KcBdE54RrkTFHAtvePLYHlUx7qY91Nve36OHBX/DaA3M7dEWr+EOsekJxO5 HojQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wPqWbOC642+FQuLJdmHZY8xtJm1ZVhhgjTRiZDTZY7Q=; b=WYcO5ogQv3uN/MZpNdOYXaErLQluZBXatfRmROWrsuKsDryvk0Ac8pRbsCPMCGPFk1 lV3bm2KMg0KKAY4rV0fNYndlf2SNB8de3viniyMeg+s9XVtWbm6YaptkS5lhi0W05YPJ u8Y4IWSyzUyy3b/A9gaP2La0Vqk1p6AGImBbQEhSKntRRpx1zlAYK6a9XW/LoeKdMAhp TV2GC91Nl4hjDR6im63JVWwor51gFvkf7ONOsIzbsBAcmdtAqh9YXUNqPHWxInadieoq 1U4Xft8vo6OMf6043wQ/WW2qh6C/il0Y1UNAZ0zfQYx8wfvbheVtyBNbJTYF8wsLPxNH 1bYA== X-Gm-Message-State: AOAM533v6/5i2r1dIjj5i4TL0Ti7isOdtbvV5CyfOm7JCSEoHEOnrLHt 1+eQw+1ATo6JqNn/qRAxifrQOA== X-Google-Smtp-Source: ABdhPJyMEicr+0neWvvsEC3Ec9nXUiLYCTi0Iea32NIOel70L8g//7eWi+fs+fDfMn3EnbeZcrliew== X-Received: by 2002:ad4:536a:: with SMTP id e10mr3765205qvv.9.1622127943895; Thu, 27 May 2021 08:05:43 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:43 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 10/18] arm64: kexec: pass kimage as the only argument to relocation function Date: Thu, 27 May 2021 11:05:18 -0400 Message-Id: <20210527150526.271941-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080546_756096_32F20D0A X-CRM114-Status: GOOD ( 15.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Pass kimage to arm64_relocate_new_kernel, and teach it to get the required fields from kimage. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/asm-offsets.c | 7 +++++++ arch/arm64/kernel/machine_kexec.c | 6 ++++-- arch/arm64/kernel/relocate_kernel.S | 10 ++++------ 3 files changed, 15 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 0cb34ccb6e73..92c2ed846730 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include @@ -155,6 +156,12 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APIA, offsetof(struct ptrauth_keys_user, apia)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); + DEFINE(KIMAGE_START, offsetof(struct kimage, start)); + BLANK(); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index bde623805866..76a757954a70 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -83,6 +83,8 @@ static void kexec_list_flush(struct kimage *kimage) { kimage_entry_t *entry; + __flush_dcache_area(kimage, sizeof(*kimage)); + for (entry = &kimage->head; ; entry++) { unsigned int flag; void *addr; @@ -198,8 +200,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, - kimage->start, kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), + 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 718037bef560..36b4496524c3 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -27,9 +27,7 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ + ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ @@ -63,12 +61,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x1, xzr mov x2, xzr mov x3, xzr - br x17 - + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu May 27 15:05:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A81DC47089 for ; Thu, 27 May 2021 16:08:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 44DEA6100A for ; Thu, 27 May 2021 16:08:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 44DEA6100A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ghIxPeqq7DvtD6+opwOh+p/WS9ffDGkdlrPhRUFGKAQ=; b=EtrPWinIjwsdG9 pIKyvL7FgwN1aw1RQOQuCJgOkxlS9Ki2qzNui3lCTaM9Udn74irTfhRddEIle4SsXW3gjr3+ylDli frzPxykIvVsRheNLCeZvtV7i+zylD+tp23bU7LSMtdRWdbEqQVkwJMpypm5BQz7Wj66hXnZ0T42YX WXV9xfIydRf8+e2IN08ntu9FXPpfymV7LyqGlB/wRrDvxuB47ataMhKlibNw8zznMSa1VYnroAjRR 6ZK1t/piLCy+mJQbqVoPldV2rlskwUBONOPzU2mGZb708+LJhQV7BN0t1YOdpG38dJtxmzEkbkzJe gR/qB8K6AOnZ/AZCRpMQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIWL-007SrV-6x; Thu, 27 May 2021 16:06:06 +0000 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHZz-0071oB-81 for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:50 +0000 Received: by mail-qt1-x82a.google.com with SMTP id c10so408254qtx.10 for ; Thu, 27 May 2021 08:05:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=IRY2JW0Z98EgQbSxHxMAz5C8JAIM2WaC06Y7zNJ3ts0=; b=boiyFkgbfwJ557d627q5so2dUiE6lq9T3OIPmL9hmxupf/IKh//+zJ11bd9/71UrxU 4/yFmZxfMnQEIS+a9xTcY8K45N6bfu78nowipvRRn3DekJXFqP1bNQhrlM61+qnG+6U2 ij8NwJJJetIorTETOnOtqLp31yYSpvkF86AW4MLlWyUbL695KiY8+ZwAQVj/imAl3lpi jPy1zngJ8iwtzSEbmLyKysfKzGJBbciUNcKPi6YPKb1SVC28YKAyeFtS3/mNd1WPrsxg iKZRAMuk4JrqoixsI49a7oYfLibiVjfBVjiN46k/FwAz4CfK7KMbflX0j9QOdzHCqkkt p4JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IRY2JW0Z98EgQbSxHxMAz5C8JAIM2WaC06Y7zNJ3ts0=; b=JMXkyEw3IeD6YMr6GoRliWEmWEtiP5rysdmiCCdZdouvWOOdXSH4QrRcvH63NCUUkP MR/hVlM6D/+eJY/sEYtQhnM/oEXGnTCqoa75DMmquFq7JFWq2h+3rU+/lcSshECiqmLa YTSTDQS1DsFOe7IdDHD+lTOJKsDpx8tVyltsrVYzpRquw58q0gcfxyjJwLtSWkMlQno9 8k0Pvj9Gfv+mGSkWvFxbM2Acycm6EDFRB7p12HAcGT5YFP9w5A/sJRNZimBXzyA1FH0y ebnXUEeYgNARgUjPk/Ou/jGI5YWGrRA96V61taLXrL3D0zafNW36SJt0vVgjds7pB5kD /F5Q== X-Gm-Message-State: AOAM531CumUb7UVbvF4A1WFFtFidPS5tmo15ItD7tTuErprf+MO4kmQK c7uEgSEvqpD6Ub9IIzvZILGpow== X-Google-Smtp-Source: ABdhPJxvoayShnrBYhLGN1MUkwd1W/Ry1DebaPAbHABcNwTi2nuw66ZfhC1rl0DYvGfXedawKnn4vw== X-Received: by 2002:a05:622a:28f:: with SMTP id z15mr3591511qtw.260.1622127945305; Thu, 27 May 2021 08:05:45 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:44 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 11/18] arm64: kexec: kexec may require EL2 vectors Date: Thu, 27 May 2021 11:05:19 -0400 Message-Id: <20210527150526.271941-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080547_393806_B985AB32 X-CRM114-Status: GOOD ( 18.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. In preporation to MMU enabled relocation, configure our EL2 table now. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 2 +- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 31 +++++++++++++++++++++++++++++++ 4 files changed, 34 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 9f1d8566bbf9..e775e6fb03c5 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1148,7 +1148,7 @@ config CRASH_DUMP config TRANS_TABLE def_bool y - depends on HIBERNATION + depends on HIBERNATION || KEXEC_CORE config XEN_DOM0 def_bool y diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 00dbcc71aeb2..753a1c398898 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -96,6 +96,7 @@ struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t el2_vectors; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 92c2ed846730..ee37d0e0fb6d 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) #endif #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); + DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 76a757954a70..9f74791cb1d0 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -42,7 +43,9 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" dtb_mem: %pa\n", &kimage->arch.dtb_mem); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" el2_vectors: %pa\n", &kimage->arch.el2_vectors); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -137,9 +140,27 @@ static void kexec_segment_flush(const struct kimage *kimage) } } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; /* If in place, relocation is not used, only flush next kernel */ if (kimage->head & IND_DONE) { @@ -148,6 +169,14 @@ int machine_kexec_post_load(struct kimage *kimage) return 0; } + kimage->arch.el2_vectors = 0; + if (is_hyp_callable()) { + int rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); + if (rc) + return rc; + } + memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); @@ -200,6 +229,8 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + if (is_hyp_callable()) + __hyp_set_vectors(kimage->arch.el2_vectors); cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), 0, 0); } From patchwork Thu May 27 15:05:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A31CBC47089 for ; Thu, 27 May 2021 16:00:23 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6EE13613C5 for ; Thu, 27 May 2021 16:00:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6EE13613C5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Gx3J4pdV3ytEUYwO4iZYSuvzzVWVaQcM5rVaKF+hjrE=; b=lrfnUqx5MadNrN 87vmitk/OAoVCtas3UebCR16ewMfQa4p9jlGYpNig420brcX8QkMpcKTb4uRM4LnJLRHdFbRx0JC9 4GnIswOc38KwPnNYPYwK+VufzA+LU2dfGEzPlYxHermTCI4Rm8MnDLo72XmR8GJEXdyZ8ujICH/JL Cof4+YIHJ761WPNLuA12QrXLr1qw5NDAscw6WIkxgqRVkAJXpqUtB7GG4RJIJnvVb9SjOMM4tJAAM ACe0C0wMqjLCBFRWSrZxSAV+VIpPHz9/wSMV8Lz0HWgz+8eLMeCO10tFwxoAwzH4aHaVXtlfLbZ9E rEIx6jM4OWIh6TXwzAzw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIOM-007PXL-On; Thu, 27 May 2021 15:57:52 +0000 Received: from mail-qk1-x72a.google.com ([2607:f8b0:4864:20::72a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHa1-0071pO-MR for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:52 +0000 Received: by mail-qk1-x72a.google.com with SMTP id v8so945068qkv.1 for ; Thu, 27 May 2021 08:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=spq5ynnfnR8TM5XEPzllfdybPKsLwK1jYCVhSyBxi9M=; b=kEzFCK/OZq65q45023j25DBKIv6t/KPooiYfjWWPP3IJOjjHeRmCpQtuVLJkoQfk0m LLZVUvatSqsTefwkG844IqhjmqYz+eKjmIWMguCbE//G/PZilV1ovqWDnr3/kiQThoLe 8J5fE0vuDmRoHMifm12gmszoCPbgL/6vKPWVbsXnCVp6POQBaG8mMnHIcVImT2bPCI1y BV/yFm5eIJfxxYVdS1bcl46gMJrIytDeVhMlnOV5TIYrxeOwAZcaQxROJ+GyKDP9GrtS PtTEcv1/j/uuZXYBfqEmqGTXfvUSywNUrIEs2IE6VveTwxDnHBc/1L6sfb2iuapuRDjs +mIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=spq5ynnfnR8TM5XEPzllfdybPKsLwK1jYCVhSyBxi9M=; b=fFm/T0LCrMlI+hLggcsXrrwBECRF4fMJJ0/fTX1vhri+j3FhxtCV/ZgVHrpLUnBsNZ S/A+frpZ0IuWAswO/6xBlte0SB2Qq3U5hFbFakOlblpRwXvVGVrw2tiuzeBU/64C2VOC 85hdWF/Hf9YTpg7Cl6PZsKnVOWGxxJEoJm0G0u0w5FtpmADnXPwGpizpxuLcGT7sZp2o fFzN5+TvsJ8E1W2KX5iVkH1mf1OKcaWWbkVbCsncSdiMJXRaY7VFeD8dLBcC1ro8g3Q9 8tSfsyxbDR4ZhkpmuLDbas33zDBTX9fFH2Zm/HVYIg9JcqplAlaw1kYDWWtjNVtBKc1t 3zwA== X-Gm-Message-State: AOAM532JsmhvK4/qnCW0yJJpV23iZH/hHHusUII5EhRtJXAgQtgwDGEM 2yb1dqKhGF2+kU6KYo5SiwX2sQ== X-Google-Smtp-Source: ABdhPJwCPp5AbyvNOecFI+FqtUWaR34fdNcgOcCR3qCe2Y2zX902CBP0VehAshlb1076RC/1us6hfg== X-Received: by 2002:a05:620a:2282:: with SMTP id o2mr3866648qkh.479.1622127946786; Thu, 27 May 2021 08:05:46 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:46 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 12/18] arm64: kexec: relocate in EL1 mode Date: Thu, 27 May 2021 11:05:20 -0400 Message-Id: <20210527150526.271941-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080549_978155_45DC6905 X-CRM114-Status: GOOD ( 13.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since we are going to keep MMU enabled during relocation, we need to keep EL1 mode throughout the relocation. Keep EL1 enabled, and switch EL2 only before enterying the new world. Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.h | 3 +-- arch/arm64/kernel/machine_kexec.c | 4 ++-- arch/arm64/kernel/relocate_kernel.S | 13 +++++++++++-- 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index 48d0ed48c147..296abbac7192 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -20,11 +20,10 @@ static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, { typeof(__cpu_soft_restart) *restart; - unsigned long el2_switch = is_hyp_callable(); restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(0, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 9f74791cb1d0..c9993f029262 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -231,8 +231,8 @@ void machine_kexec(struct kimage *kimage) } else { if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, virt_to_phys(kimage), - 0, 0); + cpu_soft_restart(kimage->arch.kern_reloc, + virt_to_phys(kimage), 0, 0); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 36b4496524c3..df023b82544b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -13,6 +13,7 @@ #include #include #include +#include /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -61,12 +62,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + cbz x1, .Lel1 + ldr x1, [x0, #KIMAGE_START] /* relocation start */ + ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x3, xzr + mov x4, xzr + mov x0, #HVC_SOFT_RESTART + hvc #0 /* Jumps from el2 */ +.Lel1: ldr x4, [x0, #KIMAGE_START] /* relocation start */ ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ - mov x1, xzr mov x2, xzr mov x3, xzr - br x4 + br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Thu May 27 15:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F19FC47089 for ; Thu, 27 May 2021 16:10:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3267B6100A for ; Thu, 27 May 2021 16:10:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3267B6100A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jFsDBBrBuVKScNkQ19KT0M4qGYf71LCUok5iXcv3kro=; b=LWZnnf1jrg4/Jz ERcv37ZvCAh9VB6fRiLfcJIAgngbjUM489SScMDbWccotodBxMBIkcVEquwEbdcpX9Ot/DVPT+R4t xN+RcdQd1PM3pjWSVOx5qeotaYRGUmdLfJECSX2XOR9T2Fo3nGnV2v9snuAfcpPQkWqku8nTn1L6R Hp6zOXvH5Ei9QH/kZC8Xmjkhcsv8vre0Sv1LaKFbIRLmr1a/iCmv61APzHD3y2KzSLqM+vtBC9Ejs ml5AE4yY1/KecgwIOwfTjnSFkB2/FFIRuLGFCn53MqIOujO2fzgFktpl10/Nj878JxIguKjbGM6Vu jZUENy1OTCfEcNJoFAsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIYb-007TlV-V9; Thu, 27 May 2021 16:08:26 +0000 Received: from mail-qt1-x831.google.com ([2607:f8b0:4864:20::831]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHa1-0071qM-6U for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:52 +0000 Received: by mail-qt1-x831.google.com with SMTP id m13so395488qtk.13 for ; Thu, 27 May 2021 08:05:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=M+UnYXxw8nkrHqAjav5617WACApgFL7LNjySWDjNsqw=; b=OM8jv6dIvYQ/oU4lPaj545Lb6wcf2ECof0M+ej1+Y+Zo0H/YsWGYtnXhh2nlXF8qzz Ae7FdACu6ubQkw8iOcaZWTKXb3SvgrwUOl1OdUNFT0Wv76oAZtQ5DBjnYrbry7/jXH0/ 5ThzZ5f/USoigM/PsymqzJFSNqfgHrhULDkokiDhB8JL6eVaxHY/QGuuRZcY7M5QPpVR zMh4PH4RvS17sRS25iqSOVMJYGoSlE+H9TeViZoU4y8wqVULNRlAywaMpnrsLA+R7ULL qVC/ljBDadvYtAPXy4npxJ0yHe1c1a21Xt4Tf9L0tQ14+Wfa2HBtlw3qWqNLUAAACY3P o2Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M+UnYXxw8nkrHqAjav5617WACApgFL7LNjySWDjNsqw=; b=Fi6bB2yIiyBcQcxVtBmi0GgcST4B2bWg4nBgohc0PisYXq85WG97eOnMVSz+Gwsux/ M1PahMS2T0MeIoCEIzQJ0+kL1j5fm+lJwl5OVVt8K6kc3xliCwKNbuxNLKb2RepU/ibZ VvnriA6VBvdNsX7mUsx6PQ1o9Nfd8DvZPy+T/hy7xNYGhgidaOKL2Y4TTrvgt5XKXNHE PubnzxuH2YXJtRPI8ZDoPh6X7bqMI9GjEpVxj4eZihJqhC04CQ9W4Crqf6qFEzVIixCM a8OKPult79dkfkT1w76fkZ1PLk/4FP5vMQ9f3lnEopDdC8MF7hWjcyI4fN9j15z9YHfP KTrA== X-Gm-Message-State: AOAM533MQNsPZjJUnm5dN+JvFnQ/gg/zUxZ4MwyulsoROxIDt0zhyLaW IeIl0oRuYlxlR1JY+mWSr0wVkA== X-Google-Smtp-Source: ABdhPJykKRJouKWov8JqAp4TT6mHaHetjX4h/gTdtSRZ4IF/EPEd/iut0BXLaxTvC2S2l28hvd4vGg== X-Received: by 2002:ac8:13c3:: with SMTP id i3mr3613497qtj.322.1622127948200; Thu, 27 May 2021 08:05:48 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:47 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 13/18] arm64: kexec: use ld script for relocation function Date: Thu, 27 May 2021 11:05:21 -0400 Message-Id: <20210527150526.271941-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080549_641251_62891D89 X-CRM114-Status: GOOD ( 14.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute its size. The better way to do this is to use ld script incited, and put relocation function in its own section. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 14 ++++++-------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 4 files changed, 28 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index e4ad9db53af1..152cb35bf9df 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -21,5 +21,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index c9993f029262..675d27d7aea4 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,14 +20,11 @@ #include #include #include +#include #include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -157,6 +154,7 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + long reloc_size; struct trans_pgd_info info = { .trans_alloc_page = kexec_page_alloc, .trans_alloc_arg = kimage, @@ -177,14 +175,14 @@ int machine_kexec_post_load(struct kimage *kimage) return rc; } - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); kexec_list_flush(kimage); kexec_image_info(kimage); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index df023b82544b..7a600ba33ae1 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -15,6 +15,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -77,16 +78,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x3, xzr br x4 /* Jumps from el1 */ SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 709d2c433c5e..9479d66f03bf 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -63,6 +63,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -160,6 +171,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -348,3 +360,10 @@ ASSERT(swapper_pg_dir - reserved_pg_dir == RESERVED_SWAPPER_OFFSET, ASSERT(swapper_pg_dir - tramp_pg_dir == TRAMP_SWAPPER_OFFSET, "TRAMP_SWAPPER_OFFSET is wrong!") #endif + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Thu May 27 15:05:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D7DBC4707F for ; Thu, 27 May 2021 16:18:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EACA260FF3 for ; Thu, 27 May 2021 16:18:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EACA260FF3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hI5bwQCYTyOKK/UEZLA9gXq1X8JNEk8X9lqug2RJ00w=; b=ydWNrsoOTdktIs VYNZKhZa382VCPzB19mjh38O+LE6fz7SA/aoSdc+axv6l6yQU18DhZ45oH64XmguLz8c02EZff1i2 T61BZWOhU55dEeYm5/DfYpRsfEzi80v1rslZyJRN990XD7uHUmC8rpD1p5jvVgyw/K9qga7I+3R5f 3MN2UGegR7FQmAe6QJzVMWEXILHkFCyZvcSl4qDut5TnXISTKY9VxJFVxaACy7M4fzTS/5DaZd4hA IvPD66KoEhrfR97znS+XXut4xFvBn82Th4gghimguwKFrjSXwEuJVG3a2feM4XyMfkfvVMU2Oa5Gm SNxCSXyF6NZ5S3TtqjDg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIf9-007WfN-FM; Thu, 27 May 2021 16:15:13 +0000 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHa4-0071rv-8g for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:55 +0000 Received: by mail-qk1-x729.google.com with SMTP id i67so932687qkc.4 for ; Thu, 27 May 2021 08:05:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZG1hbdIv3FvUkXzGyiyJBV6DrXE2wtuUaoQN6LGlj1M=; b=PxjTnw1/0d/OHXwz8yz4wn5Hg8O1MgJq99+RWaGhWnpC2LB6uhK4c/zxkocCCd5/oC e/6xsWOBUGOE90CubgjwvpYuA2I2tZv3Sy5Zrz+zCduX7j0BuclReU68VPVaxOsPdOuh +R5wk3dSn+5+xAy+3iFGLGkGhfTBw0ZoQVJPPIdAi60oUBKmlWltJrC62fx9CgwHPz6M D59VDhjBcie7Ut7+WdA57iZ1SsSDrONvHXGGYTPo9qBTh09kD6AI68kbSEN5uYj5GaNM HXsNg85brLB3CkaI9gBcGyco1YdJubhyzY3XjTbMQBcS6vi6Xnh/IA7FTvLLdlaJenUx jr/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZG1hbdIv3FvUkXzGyiyJBV6DrXE2wtuUaoQN6LGlj1M=; b=pcZjhxDbqpQrBPGvjhlSVUF9atkbrKhczOdWLozgxdVls2jFfS1B3ChvB6uZOIJoLG 7DR4x2ntqmtxpokz4S08+idK2V8Dvi7YiAqd5qiXITof+t5LlHbPkjyGelkZD6F8+KKf 2gG5hmCUlyMql47WQlpXCT2+NzunQDlmcIqxUH7XTO/lai/rCd2iy46nNzyCNp3pDbTV zYiNhFPKeO+S3SZayPEtrxNNKnbv9r7MDPmEtxxKq63zT+HRb3BJShQqAnl2oCKVclaD 4wLM6HiEeMarlG8YfKcdi4ZYM0/uEeEQ+N9W7gPhmQvDmZk86sbPQ1xPPEvlS65k6H5l GeFA== X-Gm-Message-State: AOAM530nGY/4knFpqnykBIW3FgpS60EigAv/qAX73e2Q72pTMHzwBrT9 JVisn1KjOuHh8Zd47/mBCm/tnQ== X-Google-Smtp-Source: ABdhPJzJ1Sj2T9cni69jm3ehIaxy4vedgsKsmt7wK9VAMhgxQOjax53s6snUh5lU5bh4C1ZyuNA3UA== X-Received: by 2002:a05:620a:1126:: with SMTP id p6mr4087741qkk.120.1622127949626; Thu, 27 May 2021 08:05:49 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:49 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 14/18] arm64: kexec: install a copy of the linear-map Date: Thu, 27 May 2021 11:05:22 -0400 Message-Id: <20210527150526.271941-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080552_438238_9590CE37 X-CRM114-Status: GOOD ( 20.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To perform the kexec relocations with the MMU enabled, we need a copy of the linear map. Create one, and install it from the relocation code. This has to be done from the assembly code as it will be idmapped with TTBR0. The kernel runs in TTRB1, so can't use the break-before-make sequence on the mapping it is executing from. The makes no difference yet as the relocation code runs with the MMU disabled. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/assembler.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kexec.h | 2 ++ arch/arm64/kernel/asm-offsets.c | 2 ++ arch/arm64/kernel/hibernate-asm.S | 20 -------------------- arch/arm64/kernel/machine_kexec.c | 16 ++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 3 +++ 6 files changed, 40 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index f1367863d995..7f268bfa98c6 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -447,6 +447,25 @@ USER(\label, ic ivau, \tmp2) // invalidate I line PoU isb .endm +/* + * To prevent the possibility of old and new partial table walks being visible + * in the tlb, switch the ttbr to a zero page when we invalidate the old + * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i + * Even switching to our copied tables will cause a changed output address at + * each stage of the walk. + */ + .macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 + phys_to_ttbr \tmp, \zero_page + msr ttbr1_el1, \tmp + isb + tlbi vmalle1 + dsb nsh + phys_to_ttbr \tmp, \page_table + offset_ttbr1 \tmp, \tmp2 + msr ttbr1_el1, \tmp + isb + .endm + /* * reset_pmuserenr_el0 - reset PMUSERENR_EL0 if PMUv3 present */ diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 753a1c398898..d678f0ceb7ee 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,6 +97,8 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr1; + phys_addr_t zero_page; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index ee37d0e0fb6d..f9794cd95eef 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -160,6 +160,8 @@ int main(void) #ifdef CONFIG_KEXEC_CORE DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); + DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); BLANK(); diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..a31e621ba867 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -15,26 +15,6 @@ #include #include -/* - * To prevent the possibility of old and new partial table walks being visible - * in the tlb, switch the ttbr to a zero page when we invalidate the old - * records. D4.7.1 'General TLB maintenance requirements' in ARM DDI 0487A.i - * Even switching to our copied tables will cause a changed output address at - * each stage of the walk. - */ -.macro break_before_make_ttbr_switch zero_page, page_table, tmp, tmp2 - phys_to_ttbr \tmp, \zero_page - msr ttbr1_el1, \tmp - isb - tlbi vmalle1 - dsb nsh - phys_to_ttbr \tmp, \page_table - offset_ttbr1 \tmp, \tmp2 - msr ttbr1_el1, \tmp - isb -.endm - - /* * Resume from hibernate * diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 675d27d7aea4..fe0f9c383c17 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -153,6 +153,8 @@ static void *kexec_page_alloc(void *arg) int machine_kexec_post_load(struct kimage *kimage) { + int rc; + pgd_t *trans_pgd; void *reloc_code = page_to_virt(kimage->control_code_page); long reloc_size; struct trans_pgd_info info = { @@ -169,12 +171,22 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.el2_vectors = 0; if (is_hyp_callable()) { - int rc = trans_pgd_copy_el2_vectors(&info, - &kimage->arch.el2_vectors); + rc = trans_pgd_copy_el2_vectors(&info, + &kimage->arch.el2_vectors); if (rc) return rc; } + /* Create a copy of the linear map */ + trans_pgd = kexec_page_alloc(kimage); + if (!trans_pgd) + return -ENOMEM; + rc = trans_pgd_create_copy(&info, &trans_pgd, PAGE_OFFSET, PAGE_END); + if (rc) + return rc; + kimage->arch.ttbr1 = __pa(trans_pgd); + kimage->arch.zero_page = __pa(empty_zero_page); + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 7a600ba33ae1..e83b6380907d 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -29,10 +29,13 @@ */ SYM_CODE_START(arm64_relocate_new_kernel) /* Setup the list loop variables. */ + ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ + ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ + break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ From patchwork Thu May 27 15:05:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44A1FC4708A for ; Thu, 27 May 2021 16:15:48 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0F71B613C3 for ; Thu, 27 May 2021 16:15:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F71B613C3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=efxVQ15YZzqv3siRdTe8D+SeTzapJKbS0Ty/obIoI4c=; b=wk36qk5ZAyC0/U Q8Z79f46hvKt3TwE+f90qyqh8MoKFXGoUO9jS1zDAg5iapb0QZVzD/a1lN7pj/LXnxiW+sObZj8kE z234DPKAZ5ZJjKTHcmbburLvtA3olXsF/XnKmTzr07P5IpeBdIcvTlvH90BF6XMKoHJ1KbuJrXa52 2i1uM9PRxl/fI85+WN635CGe4XsJA7qmOUfantxkKkfmNfprg8VYy/VMdAGaOk8wkIEjU1t81i/3k bmnWhE2Lp5SUM/06aJ6aTHCTfMEEPJn6Dvcu2AmtKT67MfsnCRSPTMXGGkYoXJ9BIXKU7qtFNLUae 601pxNR3YkROe3o7/PMg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIdH-007Vpl-8U; Thu, 27 May 2021 16:13:18 +0000 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHa4-0071se-EE for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:56 +0000 Received: by mail-qt1-x82a.google.com with SMTP id g8so427355qtp.4 for ; Thu, 27 May 2021 08:05:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=AR8EfhndcJhGCfPXw14djNNwxlc4BjhxbHGlWwV0KmE=; b=cuvs0nlIyCBteOSlUbqnu7MxHnc0HMUzOaoXHVjSn2UHqFo4Hy5ar0ZeGkFqsGIbHQ xzJZpkkKF+m2C8zs8onCRxd2q4U46s+Gemio+gH+DUn+AlPyIUKZJoucrJPBQr6E7DcG N99YmAOOFqWCguZ3CDzKq5SQ30+piiw8egDDjWac9+fEafPOpRSvi5Tfwuc+6inQfQS1 SoAg8A3QZOK5Z5In4lBz/74YYIBr8KYpwrtdca9iwVy4i20iE3b8154r5qP7NoPibzoO RdPHEt00M2lkgly6+lSsC5EKnhQt1OFj9ElsVZFY/pe/I6eHNh3/VAQxne50M6M/HfL3 ab9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AR8EfhndcJhGCfPXw14djNNwxlc4BjhxbHGlWwV0KmE=; b=ujETk7XX8hB06ZUH9+wh4lVIyBsPwAE30xEhyauvdwdEdA+7InWK0EFSH9aDXUSgVI ZYywxfsLw2UG+c2DmGzsq//oo5J8LTubSCMaPoH8wqBkvjS7B6NXFG+YoJZs0V45UzSG Q6xhSDO3EwyvUVs4dSNdJCB0H7RD8Q+PV542SojR+ZX3IdBDc1hZ9RpG4G4H23MQ4I7M gDIfg7Jll/bvzvPFA56xZr48pLEUr9b1Bda1MBohLbZeo58xY43ivPCOvlHj2oeIRClL 7vDaaQnFJOqT+PaxR3hJFnwIywLkd0TzyW5phQ5tGyeJrnrHeKFzDucTTkcOQm4DpGcc ii9w== X-Gm-Message-State: AOAM530Qq1gpbX8NjGjR3ZwtI4kFYTe1GTgT2/+OPXdo7rkBDYu6fg0B b5/u+M2DJFTYS3wxobnF2WBytg== X-Google-Smtp-Source: ABdhPJyXTJpi0i100Itoz1x1obakqkcvUNpa+7nZ/5NCoxyzGOYGAHt0BiM904+IpFQ0TYxSjtJbhg== X-Received: by 2002:ac8:6e8f:: with SMTP id c15mr3508556qtv.122.1622127951311; Thu, 27 May 2021 08:05:51 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:50 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 15/18] arm64: kexec: keep MMU enabled during kexec relocation Date: Thu, 27 May 2021 11:05:23 -0400 Message-Id: <20210527150526.271941-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080552_605368_5BEB3A35 X-CRM114-Status: GOOD ( 18.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have linear map page tables configured, keep MMU enabled to allow faster relocation of segments to final destination. Cavium ThunderX2: Kernel Image size: 38M Iniramfs size: 46M Total relocation size: 84M MMU-disabled: relocation 7.489539915s MMU-enabled: relocation 0.03946095s Broadcom Stingray: The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 16 +++++++++++---- arch/arm64/kernel/relocate_kernel.S | 31 +++++++++++++++++++---------- 4 files changed, 36 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d678f0ceb7ee..dca6dedc3b25 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,8 +97,11 @@ struct kimage_arch { phys_addr_t dtb_mem; phys_addr_t kern_reloc; phys_addr_t el2_vectors; + phys_addr_t ttbr0; phys_addr_t ttbr1; phys_addr_t zero_page; + unsigned long phys_offset; + unsigned long t0sz; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index f9794cd95eef..bbdf20930491 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -161,6 +161,7 @@ int main(void) DEFINE(KIMAGE_ARCH_DTB_MEM, offsetof(struct kimage, arch.dtb_mem)); DEFINE(KIMAGE_ARCH_EL2_VECTORS, offsetof(struct kimage, arch.el2_vectors)); DEFINE(KIMAGE_ARCH_ZERO_PAGE, offsetof(struct kimage, arch.zero_page)); + DEFINE(KIMAGE_ARCH_PHYS_OFFSET, offsetof(struct kimage, arch.phys_offset)); DEFINE(KIMAGE_ARCH_TTBR1, offsetof(struct kimage, arch.ttbr1)); DEFINE(KIMAGE_HEAD, offsetof(struct kimage, head)); DEFINE(KIMAGE_START, offsetof(struct kimage, start)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fe0f9c383c17..96a478d022b6 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -190,6 +190,11 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code); + rc = trans_pgd_idmap_page(&info, &kimage->arch.ttbr0, + &kimage->arch.t0sz, reloc_code); + if (rc) + return rc; + kimage->arch.phys_offset = virt_to_phys(kimage) - (long)kimage; /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, reloc_size); @@ -223,9 +228,9 @@ void machine_kexec(struct kimage *kimage) local_daif_mask(); /* - * Both restart and cpu_soft_restart will shutdown the MMU, disable data + * Both restart and kernel_reloc will shutdown the MMU, disable data * caches. However, restart will start new kernel or purgatory directly, - * cpu_soft_restart will transfer control to arm64_relocate_new_kernel + * kernel_reloc contains the body of arm64_relocate_new_kernel * In kexec case, kimage->start points to purgatory assuming that * kernel entry and dtb address are embedded in purgatory by * userspace (kexec-tools). @@ -239,10 +244,13 @@ void machine_kexec(struct kimage *kimage) restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { + void (*kernel_reloc)(struct kimage *kimage); + if (is_hyp_callable()) __hyp_set_vectors(kimage->arch.el2_vectors); - cpu_soft_restart(kimage->arch.kern_reloc, - virt_to_phys(kimage), 0, 0); + cpu_install_ttbr0(kimage->arch.ttbr0, kimage->arch.t0sz); + kernel_reloc = (void *)kimage->arch.kern_reloc; + kernel_reloc(kimage); } BUG(); /* Should never get here. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index e83b6380907d..fc2348584a54 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -15,6 +17,13 @@ #include #include +.macro turn_off_mmu tmp1, tmp2 + mov_q \tmp1, INIT_SCTLR_EL1_MMU_OFF + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -32,22 +41,21 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x18, [x0, #KIMAGE_ARCH_ZERO_PAGE] /* x18 = zero page for BBM */ ldr x17, [x0, #KIMAGE_ARCH_TTBR1] /* x17 = linear map copy */ ldr x16, [x0, #KIMAGE_HEAD] /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ + ldr x22, [x0, #KIMAGE_ARCH_PHYS_OFFSET] /* x22 phys_offset */ raw_dcache_line_size x15, x1 /* x15 = dcache line size */ break_before_make_ttbr_switch x18, x17, x1, x2 /* set linear map */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - + sub x12, x12, x22 /* Convert x12 to virt */ /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x2, x13 - mov x1, #PAGE_SIZE - dcache_by_myline_op ivac, sy, x2, x1, x15, x20 + mov x19, x13 copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 + mov x1, #PAGE_SIZE + dcache_by_myline_op civac, sy, x19, x1, x15, x20 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination @@ -64,19 +72,20 @@ SYM_CODE_START(arm64_relocate_new_kernel) ic iallu dsb nsh isb + ldr x4, [x0, #KIMAGE_START] /* relocation start */ + ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ + ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + turn_off_mmu x12, x13 /* Start new image. */ - ldr x1, [x0, #KIMAGE_ARCH_EL2_VECTORS] /* relocation start */ cbz x1, .Lel1 - ldr x1, [x0, #KIMAGE_START] /* relocation start */ - ldr x2, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ + mov x1, x4 /* relocation start */ + mov x2, x0 /* dtb address */ mov x3, xzr mov x4, xzr mov x0, #HVC_SOFT_RESTART hvc #0 /* Jumps from el2 */ .Lel1: - ldr x4, [x0, #KIMAGE_START] /* relocation start */ - ldr x0, [x0, #KIMAGE_ARCH_DTB_MEM] /* dtb address */ mov x2, xzr mov x3, xzr br x4 /* Jumps from el1 */ From patchwork Thu May 27 15:05:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00FF1C47089 for ; Thu, 27 May 2021 16:20:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C0358613C0 for ; Thu, 27 May 2021 16:20:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0358613C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Jn71T7TqLBRDSshpR9AJnNQaZAin3eUT562GlHnz3IM=; b=m1HQ/FQlrwxswd wQCm5vZflVmotEGG5//78uega8HMGx8nqClNDJ+UGGU9eceXrr972z9pNTco2a6MWP6TKwroEnX8z 8AnW31g4nppVJ9knNStPpHOwH1BxHf1P/KzkiIpxYF3bzZ7n9b0NSYSvafnXKf2xWwux6q2PkBjun 9UGq/6SjVwcP9EHNHTkgby1u/XMgDvXpPQGaN89AMpDdxrm/OcPQ57uyhy6AVX9whHT0mM5C+Ddpy 7Rx6fOc6hC7IugWPe3aKa6Zy4qwLfLNvCnS0BxGjC6vu1q53QhlegQ+I5tuQzqpq51Ga3Ig6InP8N HYs/8iP/ZEDj0DSFcZGA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIhM-007XYn-U4; Thu, 27 May 2021 16:17:30 +0000 Received: from mail-qt1-x832.google.com ([2607:f8b0:4864:20::832]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHa7-0071u7-4D for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:05:57 +0000 Received: by mail-qt1-x832.google.com with SMTP id h24so398983qtm.12 for ; Thu, 27 May 2021 08:05:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HZHOJzSFFbUuzx7eGvKvsp2IVzsfaWJrcIfDcXhfmjQ=; b=OE2Nv9rI1yl+eDr9xb/59iX9oopmd3JLdS6dWVrola8Od1OZixPXWsBxDTcSsG/BNj xqiP3/MtToHN8To0OOVfsTQ3FBtribEObmkQCfcGTb2XGrrt2TWIa9A+RUVytD4Z/F1Z kKpuTAqZN0nSiaH9NmXyCZVROaX3De/0QTwv6XiQhtVj2j7cm6+ofwNN8zS5yUCs8bE9 5f7KTwgzvzQxt+xXns0fOIqMOfXSXcBgLh+YZVajTxS7x88UNgl5ov6gRjoqo7qNpKeE aJtrlwWbPDLsQi9BZulXO6txvlllHE94kB0RXJGsAb9bZtu3H53chND0bq2jH+PtgScr gSdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HZHOJzSFFbUuzx7eGvKvsp2IVzsfaWJrcIfDcXhfmjQ=; b=LszZa1KX39HqiSicXR9v9Z19ZvlDcxonyHlggfNujj4qea3a4kl8InhHZCd7HBAmT6 yU1JKe8lSRpY0MIIzbGuUIUI9rK0AaqUMIMiGdWE6nGR+cj6mRUPpyQN0bSzkT4m9Peo xGfA1TxbboamVz2ftEKqfhYM6jUlABRiH8/Axs17Bm2SUP2cLFoqdbUx+G8uJn9rT0Sb L05GxPnQ6im5Yzu2zHgbg/DoMkVyuV+yrl62O6SvT2ayRRhJjHYNCokRSB0kXV+JJR+c 1AJuDVRUEocVktKY2U1HISTbqAMJfPCC8cHI24/JrsQ/EYyrNSS57UkaYZORX+77qWga t2GQ== X-Gm-Message-State: AOAM531iLQ3W00mKkJB64/NJ4vMB4V+hC8f8feWezxHyfE5om0oYP3ZC 5QVy+OvyMJZroDiwkNXoUdkbcg== X-Google-Smtp-Source: ABdhPJwQ7PTEJ4bfTYMlyGqHPr1GFZ98OMMmMZTDk2VsdokJOvRo2GHvow0yZur+w4JE/untrcRTcA== X-Received: by 2002:ac8:66da:: with SMTP id m26mr3565077qtp.102.1622127952711; Thu, 27 May 2021 08:05:52 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:52 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 16/18] arm64: kexec: remove the pre-kexec PoC maintenance Date: Thu, 27 May 2021 11:05:24 -0400 Message-Id: <20210527150526.271941-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080555_190533_6BF5C900 X-CRM114-Status: GOOD ( 11.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now that kexec does its relocations with the MMU enabled, we no longer need to clean the relocation data to the PoC. Co-developed-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 40 ------------------------------- 1 file changed, 40 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 96a478d022b6..afbde4bb1829 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -76,45 +76,6 @@ int machine_kexec_prepare(struct kimage *kimage) return 0; } -/** - * kexec_list_flush - Helper to flush the kimage list and source pages to PoC. - */ -static void kexec_list_flush(struct kimage *kimage) -{ - kimage_entry_t *entry; - - __flush_dcache_area(kimage, sizeof(*kimage)); - - for (entry = &kimage->head; ; entry++) { - unsigned int flag; - void *addr; - - /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); - - flag = *entry & IND_FLAGS; - if (flag == IND_DONE) - break; - - addr = phys_to_virt(*entry & PAGE_MASK); - - switch (flag) { - case IND_INDIRECTION: - /* Set entry point just before the new list page. */ - entry = (kimage_entry_t *)addr - 1; - break; - case IND_SOURCE: - /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); - break; - case IND_DESTINATION: - break; - default: - BUG(); - } - } -} - /** * kexec_segment_flush - Helper to flush the kimage segments to PoC. */ @@ -200,7 +161,6 @@ int machine_kexec_post_load(struct kimage *kimage) __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + reloc_size); - kexec_list_flush(kimage); kexec_image_info(kimage); return 0; From patchwork Thu May 27 15:05:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F205C4707F for ; Thu, 27 May 2021 16:13:14 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A255613C0 for ; Thu, 27 May 2021 16:13:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A255613C0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TRWb4UI09AtVhGu8ImDP29UFLQpdQM5jWkBF9BqBVXg=; b=rjqpzzBwgtlPfy R675uoXaIhoJgpdUeWEwaloXvmB+Q68hGisf9XmK/vaPM0lo16j7Rm1v2a6atYzJOuSBj28ymy5h4 dZSvYIO5mnGj8JIObwK1X340yXa8AfaQuXzDIiasJP3w1+aR02DPUG709M2lWQrA+vgG/XXmeHv2n Rhi09MYLT6/wCNLNz8c5Ezc3GRAxU1IQt5uc4W8tdkZ3cS/YttVS0AxgiPypBMob4ErIHkdg8wbUb u9+lKlGdEj3J3cizg+NKjAIp/WYCFYdUUrd7/OOXjDRsAqQF40yN8LDwWZ6jYHWGDR5fBFIW1jErA vjDrIQ6yu2oEU1DtxcWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIaj-007Ucu-7V; Thu, 27 May 2021 16:10:38 +0000 Received: from mail-qt1-x82d.google.com ([2607:f8b0:4864:20::82d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHa9-0071vg-Ob for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:06:00 +0000 Received: by mail-qt1-x82d.google.com with SMTP id g17so406868qtk.9 for ; Thu, 27 May 2021 08:05:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=2lq6+MakGPQuirNtQQTDFwD5XrRoYT+d3t/Oh58mP4Q=; b=U1ehG/zeOWYwlR4hoxP5YnaQEmhc2GgjVwIbKmIULFiONY3PaoltZVzgCzAjQs6Cse rHgAqKTLHeMqW/YgTshpwJSBqtCRvtJiKN0ClUGwPUGhbkCYVJeGepTiTy/8wkMlvlb5 O6FMOwriPuaVqJqkKKAdoH5VSffTa2u3TOMeLapsQiGhEsbWdbrLh+ouQplarW+mf/5B gwKH5PslhIVaCy6gjfF3xuACoxFc3zACPGM4DjE44jMUaun/550UV6jajDZvbOkwpxWC Bn+T9v45T5vbIL39tsfl//beVDi5b2Jx/5wSUfXuved3P8Fn99uz79nTcDWD/wj8e3d6 WAxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2lq6+MakGPQuirNtQQTDFwD5XrRoYT+d3t/Oh58mP4Q=; b=B7o4fHUufDb2NMZKNGXhgSyijc1UlhtuBsA1PVQZgU3D8p4QszO89GUtFm6varG4Mx //nxdPrqJ5oCiZVNV/CfcyvKy2isQxQlzmHMUZwh2yNH7bX0PFxTo7J2tfJwsd5BZUdY BTm6pqfZG0xEbQi6F4Kpyd17d1QZPfw7icrA5nwnIjRDD6o5STmSo6D4b6OfZL1ufgK9 Q7/9fDFC7YOM0Ea2Sd186vVnDiW9rBwGdiYyyVJd9dRh6WsVWlJrpj77XMwTpSuKsCta ZOx1O6Yr+Mjba4tzeXVfgKyuEdpfaGOSwnK/RQz59rv7BaElHj9e3v92Wc7wDb3VyQvX 0rGA== X-Gm-Message-State: AOAM532jMNc92JHBuw8e17l/UrWMycmnH617/mikG2wIT1vRLKyzzSTE U8a3bCj4hDZgZBNNwq3kH9LV8g== X-Google-Smtp-Source: ABdhPJw0pPua175hZLYTT6CqVpGguaKlKVb56ZVRDoJK9z9ajvuvB1dR7EF+lT603pTxYBhMr1txXQ== X-Received: by 2002:ac8:5e0e:: with SMTP id h14mr3514930qtx.117.1622127954155; Thu, 27 May 2021 08:05:54 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:53 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 17/18] arm64: kexec: Remove cpu-reset.h Date: Thu, 27 May 2021 11:05:25 -0400 Message-Id: <20210527150526.271941-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080558_025425_67F458AA X-CRM114-Status: GOOD ( 15.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This header contains only cpu_soft_restart() which is never used directly anymore. So, remove this header, and rename the helper to be cpu_soft_restart(). Suggested-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 6 ++++++ arch/arm64/kernel/cpu-reset.S | 7 +++---- arch/arm64/kernel/cpu-reset.h | 30 ------------------------------ arch/arm64/kernel/machine_kexec.c | 6 ++---- 4 files changed, 11 insertions(+), 38 deletions(-) delete mode 100644 arch/arm64/kernel/cpu-reset.h diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index dca6dedc3b25..9839bfc163d7 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,12 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +void cpu_soft_restart(unsigned long el2_switch, unsigned long entry, + unsigned long arg0, unsigned long arg1, + unsigned long arg2); +#endif + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index d47ff63a5b66..48a8af97faa9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,8 +16,7 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. @@ -29,7 +28,7 @@ * branch to what would be the reset vector. It must be executed with the * flat identity mapping. */ -SYM_CODE_START(__cpu_soft_restart) +SYM_CODE_START(cpu_soft_restart) mov_q x12, INIT_SCTLR_EL1_MMU_OFF pre_disable_mmu_workaround /* @@ -48,6 +47,6 @@ SYM_CODE_START(__cpu_soft_restart) mov x1, x3 // arg1 mov x2, x4 // arg2 br x8 -SYM_CODE_END(__cpu_soft_restart) +SYM_CODE_END(cpu_soft_restart) .popsection diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h deleted file mode 100644 index 296abbac7192..000000000000 --- a/arch/arm64/kernel/cpu-reset.h +++ /dev/null @@ -1,30 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * CPU reset routines - * - * Copyright (C) 2015 Huawei Futurewei Technologies. - */ - -#ifndef _ARM64_CPU_RESET_H -#define _ARM64_CPU_RESET_H - -#include - -void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); - -static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) -{ - typeof(__cpu_soft_restart) *restart; - - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); - - cpu_install_idmap(); - restart(0, entry, arg0, arg1, arg2); - unreachable(); -} - -#endif diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index afbde4bb1829..16996b568fd3 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -23,8 +23,6 @@ #include #include -#include "cpu-reset.h" - /** * kexec_image_info - For debugging output. */ @@ -197,10 +195,10 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ if (kimage->head & IND_DONE) { - typeof(__cpu_soft_restart) *restart; + typeof(cpu_soft_restart) *restart; cpu_install_idmap(); - restart = (void *)__pa_symbol(function_nocfi(__cpu_soft_restart)); + restart = (void *)__pa_symbol(function_nocfi(cpu_soft_restart)); restart(is_hyp_callable(), kimage->start, kimage->arch.dtb_mem, 0, 0); } else { From patchwork Thu May 27 15:05:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12284749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4621BC47089 for ; Thu, 27 May 2021 16:22:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0568D60FF3 for ; Thu, 27 May 2021 16:22:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0568D60FF3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gnMygclAlzpdngitI37sF+QFRfuXS5DhT8K63tAqGZ0=; b=RscQCRasWE76F4 NhMcrYFtBb4QrWVNXCPhRBnEXJOLKQYFtg0tHWaGmDT7ezWwSav/5/92mRVug+Mkl2Mr/+o6QsvoJ tOHWKCZSl9DYVxTlB1pyFWR6T87I/df6a5BgOhpu/jxvzN0HByH7SsC7t4S3ZKCdtdO0HnxsVOp+q 4aXZEnE1ObzPZheEQPEpPDoq5eWHa+RUTsS0DBYgYTFqJ3xmfnAb4OkRbzZwhlbZykDN5Hd6hbOk0 vWbFAu238fzKWlzHQhMpJ1S+iPA8YvaT7CKTHhhUICg7YaFea4lfKJlMatQdMdzG+b4tPNcxNVjXl Jo63t1jt9vjkFifaWV4A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmIjZ-007YVd-K3; Thu, 27 May 2021 16:19:47 +0000 Received: from mail-qv1-xf31.google.com ([2607:f8b0:4864:20::f31]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lmHaA-0071w2-Pr for linux-arm-kernel@lists.infradead.org; Thu, 27 May 2021 15:06:02 +0000 Received: by mail-qv1-xf31.google.com with SMTP id ez19so254982qvb.3 for ; Thu, 27 May 2021 08:05:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=Iq2Q5NY3yposm0EzkEbNN3/+GTxojDq2PR6h8IMwJ6xZAn+A1cRGovJwR2aPkV8hn7 w+eFIRjxUsghtCzfIod0bzTDP+pcAgOSje5vB8U3q+aXKe+waXT2d6Ggk1uAUzfsA4gf +QgHKhRNtnebjuIuJILNouevRpewJ1PPo8mfAdzpLMPyW9f25kw4YpwOoRwxkKmbdVa8 OfRtGvLr0zlSKmJy2Vzk7F0qHuaZXgJecrZvHqqRzNFweTj/J9gd2alfHSMSVk+7BIop 627eTfXCJVf+/mfbfdy20rrkK0z6CFDL4pu/MrBdiV/h6BwYsj88TvOrAtjk36oXx7Dh 13cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KXQ+yohNMyyqRPEeNONrEzPBCefWBOBuVaWouKl2Q04=; b=a2kBTEHhTdpWutQHbnhorPfkurJ9R5xuFTEMBreFXp9SjQdogSkBouDK/XYjQ/C3Q9 mNclg6HnjQHysXM5Ng2dyKrqObl5exFHiAAcs02E+Tx2lzHN0aVGr2fxbhQkRmql2llF LFB4xTyzCjxgwsouQFdjjYSuy+JS9CGCCTEDBfyDN2xZuOpVdzGQ0GNcrZC5YWsag4xd 0ZJNjbgiUfomCNb/uNSG7VD5UBLgHAwaiLGaORiQoZ5goQHalrt7GafmlW1Tt8zWhVrv fcV+D8Z7tZY+oyhG8kRn9KLXq/evZGcp+UXAuBzD+dhJEkBuLf52MdxTpMoaotrtoCpw EA6w== X-Gm-Message-State: AOAM5330cX0S06wyrc5umDOs6CQXxXZyrzYNw3TwT4LOY1SOabXmRgXc NGRJNBuzrLNNjd4xmhY/9K07OKRepUb6nc6H X-Google-Smtp-Source: ABdhPJzsC2TgTF9Gmqd9ulqOby16oaHRzRYnnCr7Rt0hFZaf2VvH052e/3SGjlSguYtwryEqV/XBTg== X-Received: by 2002:a0c:9e24:: with SMTP id p36mr4019323qve.60.1622127956289; Thu, 27 May 2021 08:05:56 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id x10sm1447468qkh.124.2021.05.27.08.05.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 May 2021 08:05:55 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com, kernelfans@gmail.com, akpm@linux-foundation.org, madvenka@linux.microsoft.com Subject: [PATCH 18/18] arm64/mm: remove useless trans_pgd_map_page() Date: Thu, 27 May 2021 11:05:26 -0400 Message-Id: <20210527150526.271941-19-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210527150526.271941-1-pasha.tatashin@soleen.com> References: <20210527150526.271941-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210527_080559_227091_1671A7DB X-CRM114-Status: GOOD ( 14.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Pingfan Liu The intend of trans_pgd_map_page() was to map contigous range of VA memory to the memory that is getting relocated during kexec. However, since we are now using linear map instead of contigous range this function is not needed Signed-off-by: Pingfan Liu [Changed commit message] Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 5 +-- arch/arm64/mm/trans_pgd.c | 57 ------------------------------ 2 files changed, 1 insertion(+), 61 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index e0760e52d36d..234353df2f13 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -15,7 +15,7 @@ /* * trans_alloc_page * - Allocator that should return exactly one zeroed page, if this - * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * allocator fails, trans_pgd_create_copy() and trans_pgd_idmap_page() * return -ENOMEM error. * * trans_alloc_arg @@ -30,9 +30,6 @@ struct trans_pgd_info { int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, unsigned long start, unsigned long end); -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot); - int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, unsigned long *t0sz, void *page); diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 61549451ed3a..e24a749013c1 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -217,63 +217,6 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, return rc; } -/* - * Add map entry to trans_pgd for a base-size page at PTE level. - * info: contains allocator and its argument - * trans_pgd: page table in which new map is added. - * page: page to be mapped. - * dst_addr: new VA address for the page - * pgprot: protection for the page. - * - * Returns 0 on success, and -ENOMEM on failure. - */ -int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, - void *page, unsigned long dst_addr, pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = trans_alloc(info); - if (!pgdp) - return -ENOMEM; - pgd_populate(NULL, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = trans_alloc(info); - if (!pudp) - return -ENOMEM; - p4d_populate(NULL, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = trans_alloc(info); - if (!pmdp) - return -ENOMEM; - pud_populate(NULL, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = trans_alloc(info); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(NULL, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); - - return 0; -} - /* * The page we want to idmap may be outside the range covered by VA_BITS that * can be built using the kernel's p?d_populate() helpers. As a one off, for a