From patchwork Mon Jan 25 19:19:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79F92C433E0 for ; Mon, 25 Jan 2021 19:22:00 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 226EB21D94 for ; Mon, 25 Jan 2021 19:22:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 226EB21D94 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=TeubOPPDJNGcQUvm6HTY7PAVrMG6VwgxX1YXdNf/fLg=; b=UpDna7lniGV12Djzn3ia2Wj/yv 0PlRXZ97wA5lVixATrzMRT8ZwWy37V5a6UhPONzx+jYjXmN4TuGbRTuefU2sEO69Pm0zu1+XD6WPA KnvKHDWxBhjQqgleWHS8QI8kR68PQTXiDHmNMrIGsO16pJELimROJc150f6sa5msvKxZqFegY9bqR cPz6WLQHI01ehXVpr0uix8U6319LMDAG+DHozqxFobL4XR70Lpxxp+V/+bsCl8xUuPjjAZBWpvB+U hnxHmP8/3KtoKJOQb2p4lnSk697yN7Sx4hBwdhfkhrRVf9tmkZL1wZq2wFtWLLyV6R0LaW6gAeutG 5MIMt0aA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Op-0007BA-QR; Mon, 25 Jan 2021 19:19:43 +0000 Received: from mail-qv1-xf30.google.com ([2607:f8b0:4864:20::f30]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Oa-00073E-OW for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:31 +0000 Received: by mail-qv1-xf30.google.com with SMTP id j13so1381361qvu.10 for ; Mon, 25 Jan 2021 11:19:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=etL32FYt+98u1sEnSN2D9m2zEWLuZNtfowf09CYacIU=; b=fd0Lk6UOzO0TicMR5oWXHg+JuTghXP3CauOmv+oga2qmCQO/ZG/IiT8HXfv2U++qqc KbyfcrG5gvFX62FWqf92w2nWIOCis9V5mQX1tQd+Ywe6zBeMa5xAdRflPHcI/Vhb+CB5 ArtjxyEr8hBh7PGYANxNGz7SyV5FeTRqEFDuquVkiK6f2rPEjTV8PU5aIhSSDteWN8nQ cZbAyCVs6sQpDayWlXkRJUoDMIX5pQAeFQZ5a4K8F5nO0isXUUoeNe3znwWYTxHKC6pL EB42Cxu/hu0LdmQdPRypPBrnMU+fYdv0cKqxTUvu29es2XrwE/96x6W4ShlArBm+SW68 /Sog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=etL32FYt+98u1sEnSN2D9m2zEWLuZNtfowf09CYacIU=; b=SOhsvzIrtLwK0K8JSVHhpTZN4Lku7d9Gii3v9fa7QCQjMCDRsQLqpdzxWOzYzzRnhG WcKLlNk2+4THJyIg3ZgnMmksSJsazb+Fpf3n7FQYw2I/iQZmyKE4rmg7nlYb2i1j/3X3 N/jsSxs+6gzPSMQGm+z97ZWtssMzyriUplkWesbqD3SKgoM94Yi7lVjyHrNXRzJ9mHRY ndNOV1kMypUTmodzF9HWeklAOXzvSZDeiWrDm9Fya5Ne2fCoEOU+jKl5mrkmE6zbt26H 8B4fmblqQIXZn17r5+hdKnZpdk5DYxAK41r8I25Qy9/0E42zcvcgBlLALi+Celtasla5 z5DQ== X-Gm-Message-State: AOAM533tY/n5XhWd4ZPd1q8uG2kw2rblvNB+NJ8T3OB8NZmasR+TzFmr EsMkgG5Wbm3RjkTGk69tJoaThQ== X-Google-Smtp-Source: ABdhPJxsUsLk3uypyODw2uSFbzFEEVUFtn5pCaoqujizEArj1agTAVjfzSE4qT7+XYJRN9kW/PJkwQ== X-Received: by 2002:a05:6214:36b:: with SMTP id t11mr2228378qvu.33.1611602367042; Mon, 25 Jan 2021 11:19:27 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:26 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 01/18] arm64: kexec: make dtb_mem always enabled Date: Mon, 25 Jan 2021 14:19:06 -0500 Message-Id: <20210125191923.1060122-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141928_823050_6634C2D2 X-CRM114-Status: GOOD ( 12.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, dtb_mem is enabled only when CONFIG_KEXEC_FILE is enabled. This adds ugly ifdefs to c files. Always enabled dtb_mem, when it is not used, it is NULL. Change the dtb_mem to phys_addr_t, as it is a physical address. Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/include/asm/kexec.h | 4 ++-- arch/arm64/kernel/machine_kexec.c | 6 +----- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d24b527e8c00..61530ec3a9b1 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,18 +90,18 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif -#ifdef CONFIG_KEXEC_FILE #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { void *dtb; - unsigned long dtb_mem; + phys_addr_t dtb_mem; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; unsigned long elf_headers_sz; }; +#ifdef CONFIG_KEXEC_FILE extern const struct kexec_file_ops kexec_image_ops; struct kimage; diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a0b144cfaea7..8096a6aa1d49 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -204,11 +204,7 @@ void machine_kexec(struct kimage *kimage) * In kexec_file case, the kernel starts directly without purgatory. */ cpu_soft_restart(reboot_code_buffer_phys, kimage->head, kimage->start, -#ifdef CONFIG_KEXEC_FILE - kimage->arch.dtb_mem); -#else - 0); -#endif + kimage->arch.dtb_mem); BUG(); /* Should never get here. */ } From patchwork Mon Jan 25 19:19:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1ECDC433E0 for ; Mon, 25 Jan 2021 19:21:32 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4190421744 for ; Mon, 25 Jan 2021 19:21:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4190421744 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=p0AAK75IWYYSRYkDuY22SVbXiFuufudhf8fmT/3Aphc=; b=oHiE4cT8MVFTK89s5k2ZKMbfVi KdtvaS9/OvGPkLa7mFN2Sp3iKyvOuhlYGFs+KiWMHYJEXo3VLb9lsaomFJToeSk0+qj1u+iK1i3Gr 29gRXdLCpq8M9xjX644oRKP9uLSl0ucLmfXQAzSrM+s/meRx8511/rV61ZQIxqDUXpcQeQzr+XZyc 2UonXhh8DPnw09RrUDa/A1R+JmxrycNrSxMWYhazarMt7YfuvCvMqOea/YLfymy/eNnOmsW6rQdqw 6SU61oDIGSSDrM29Ni97ki2krFs6LTSlpMPR+SGEwQBTc3XwVNN+1Q4sKvj5spfmpBglFUzPOgSLp 9FtEhbPg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47P0-0007EH-RV; Mon, 25 Jan 2021 19:19:54 +0000 Received: from mail-qv1-xf2c.google.com ([2607:f8b0:4864:20::f2c]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Oc-00074C-Md for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:37 +0000 Received: by mail-qv1-xf2c.google.com with SMTP id p5so6711897qvs.7 for ; Mon, 25 Jan 2021 11:19:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=rWpWL/DRiZPfCy8fAJD8ErDuDMoAsKZLcMwbx3gozjY=; b=NZ5XxFttV3baXG5fjW/ZpaC7LBvDpicX5uySrcQVbyy1c18h0xyqlMmWuyCRowKzIL 3e33S4LCQP2unKI4CIWKN0KSN5UOTN71ef3uB9naUZwGHi6huRuILd95UDOOciWb/aN8 ULr8BmGlF99/Ts9F6CX+/Hl3EWpBrY4QuAZIvNHgadEQ7LrCErmysGI0hmBvGVKLOvww uaYh2RRy2tdHXfkNRSNPuja7VlZtWq13bIgkwLZe/uCR9EKKqC52TRzHPnJeGklcyVz4 aNd3+2sBb1fkTBenfwdJob3WSuHRrYk21+XJDoA0CWQJZBnq2vPRu1wy+jEOG6/FSzt0 YQNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=rWpWL/DRiZPfCy8fAJD8ErDuDMoAsKZLcMwbx3gozjY=; b=GBw9lNt9GK/0qTDrbQtLTObulnLys002b9QhimKOLE3sW/L2HP9tCUCZXJWkDZnJql 35sKgdm9rZ1wKZGseOGnciW/r+yDBzBecix4eaKS7cpJ8FGgouKi2nnbj6bBR817fIXy Z8YBm0f70rkNzuQAtHcrfbLww2jD5x3GCN/HJPIGzt0SMyCxnWEzxB8OoenyplKer+ad NsgNiyhkJ52ICwSgSpADUR7lUoVLkREvCUdndDDQsYJg8MyNwqRXwjdga6hnlwBTIDJG 4choHtKpG44xDA/XLTpLeeB1s8T/POYjGlABGSc650ZZUZJpMHT+U3B9DYrlN55ylopw hfPA== X-Gm-Message-State: AOAM530CXbbB1nhI4JDCJcOVOd1UnzwPszJwgDLm7dCPlcl0HxWkxM3P KYpSD9SPL+Jl1FsUenCofzdQag== X-Google-Smtp-Source: ABdhPJwtUDo/AEU+wN44OEcc7AdxoOnqTZ02B3zhpDKqDqRd5gONz17nKuvw43h6SxCW1YGZ7BgVUQ== X-Received: by 2002:ad4:4e86:: with SMTP id dy6mr2312415qvb.4.1611602368593; Mon, 25 Jan 2021 11:19:28 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:28 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 02/18] arm64: hibernate: variable pudp is used instead of pd4dp Date: Mon, 25 Jan 2021 14:19:07 -0500 Message-Id: <20210125191923.1060122-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141930_872289_6D93FAAD X-CRM114-Status: GOOD ( 13.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org There should be p4dp used when p4d page is allocated. This is not a functional issue, but for the logical correctness this should be fixed. Fixes: e9f6376858b9 ("arm64: add support for folded p4d page tables") Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hibernate.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 9c9f47e9f7f4..0a54d81c90f9 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -190,10 +190,10 @@ static int trans_pgd_map_page(pgd_t *trans_pgd, void *page, pgdp = pgd_offset_pgd(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { - pudp = (void *)get_safe_page(GFP_ATOMIC); - if (!pudp) + p4dp = (void *)get_safe_page(GFP_ATOMIC); + if (!pgdp) return -ENOMEM; - pgd_populate(&init_mm, pgdp, pudp); + pgd_populate(&init_mm, pgdp, p4dp); } p4dp = p4d_offset(pgdp, dst_addr); From patchwork Mon Jan 25 19:19:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 487F9C433DB for ; Mon, 25 Jan 2021 19:22:24 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B801420E65 for ; Mon, 25 Jan 2021 19:22:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B801420E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+rIDPzkvA/1j4DdfjZ90DJUi5upNJJ/S9yunyI1Jvv4=; b=BvTMyfuLjneNQff90Sv27U1BBU ZiF39eWFA99rZdElihUjCTxpDAFFkuulLnSHGOlj4lJu0xHvp7QBilWLhPqFxaMGB3DW0Zpc6IHUl VuVl3neDtj33iSnnzbE/MSXAmUWNcB0jzFfeO345VGAGiqRJ0h8TDC3nvB3JTV4He2MPZN4n8+IUh 2XwQ0ffUfSWRnMsVnIeNDQ7Q80CQhirAOJ8IBYLFaE4Ge0+xo0wd3kCQu7Gu3LY7gbXwLGBidlAnJ Rydub3jmFbDGGyRxSZx8y99Ngel33+zAPTKkW+xz0AZp+mTIyuklYd453gGE8csiiiqv3KFwt13QM ZIFe21Tw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Po-0007Wb-Sz; Mon, 25 Jan 2021 19:20:44 +0000 Received: from mail-qk1-x733.google.com ([2607:f8b0:4864:20::733]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Oe-00074t-D8 for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:41 +0000 Received: by mail-qk1-x733.google.com with SMTP id a7so6759985qkb.13 for ; Mon, 25 Jan 2021 11:19:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=gWo+XOLlvSFu2Q0rUJUVRXtmMkgZkyDfo2dHFScNBUo=; b=LqDMe3eOPRCh0LmJCvwRCsUvTSYvc9yktr4rl92IrXlPlKxe8yf2ehoJ93OT5QGF8X ka8m82UHigU51KhTc5eitfsRDuGL4atkVJeiuPTnOP3xSVFBrIWJ4TS7tJPHhNvlIV7E OnsfQhqph1ARfW7bnKbJs1NevlHz3IClvYTWA7bIRlOJW9VoOhm18BDlB5YPIoEFLYJv SWoQnbL1gec/EP/A7EVGgMGihzNLYr+bLaWsZUCZyXLGpdfwomncaAETHwm4pTWMauzh rSgVPlbE4YLwJTG5QzjeXpTxo81QtrEq6NBne8uI3VFzz3q+Ovgih6LiFF1ApJrbZyps w44Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=gWo+XOLlvSFu2Q0rUJUVRXtmMkgZkyDfo2dHFScNBUo=; b=BBqlKa78cVoH9kM6M7Ffxs+/HFAEUPfdbT06IALZRft4m8WiDRbbuJmgOrNYndzU34 eFGJbOJd2lxlBYDC5arURaxpmv/1ZPZ9KBnsgDsRKpliHs/DkyHIuXRFtq91ywU5GkRF C8U/iPsfRWRtww4MYZqycKwmsftUVtdZnMux2+HvxhCseOHgrlu/AouMx3ppAtTFtu5A 8D94EqXejiJjxlgoantz1I9Uezn7RcjadMo0hHYdonpuL0D4NbAHILfFvtogTdquaFct PU3tPHFN0wpKkD1l1uJu4csA4mzezel0LYBxc97O1pf6EpuAmJc4SK+NtmNECDJw1KTA DIcg== X-Gm-Message-State: AOAM532mte2HOnKHGx7W+zh96vlng7GBWyyCQ7SpDnXIi3EOes6KcmYP qBhYfHDYYnu6B3tx7dQ5gMOymw== X-Google-Smtp-Source: ABdhPJwRsutfqZPg7CQkhtjt0vPU0WCJTPCcRsyd2TE8K/vxLVCDWuFPNz3zCWV4YhPp49kMjVURcA== X-Received: by 2002:a37:a50e:: with SMTP id o14mr2319608qke.250.1611602370201; Mon, 25 Jan 2021 11:19:30 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:29 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 03/18] arm64: hibernate: move page handling function to new trans_pgd.c Date: Mon, 25 Jan 2021 14:19:08 -0500 Message-Id: <20210125191923.1060122-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141932_544130_8FDD1570 X-CRM114-Status: GOOD ( 26.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we abstracted the required functions move them to a new home. Later, we will generalize these function in order to be useful outside of hibernation. Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/Kconfig | 4 + arch/arm64/include/asm/trans_pgd.h | 21 +++ arch/arm64/kernel/hibernate.c | 228 +------------------------- arch/arm64/mm/Makefile | 1 + arch/arm64/mm/trans_pgd.c | 250 +++++++++++++++++++++++++++++ 5 files changed, 277 insertions(+), 227 deletions(-) create mode 100644 arch/arm64/include/asm/trans_pgd.h create mode 100644 arch/arm64/mm/trans_pgd.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index f39568b28ec1..fc0ed9d6e011 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1132,6 +1132,10 @@ config CRASH_DUMP For more details see Documentation/admin-guide/kdump/kdump.rst +config TRANS_TABLE + def_bool y + depends on HIBERNATION + config XEN_DOM0 def_bool y depends on XEN diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h new file mode 100644 index 000000000000..23153c13d1ce --- /dev/null +++ b/arch/arm64/include/asm/trans_pgd.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2020, Microsoft Corporation. + * Pavel Tatashin + */ + +#ifndef _ASM_TRANS_TABLE_H +#define _ASM_TRANS_TABLE_H + +#include +#include +#include + +int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, + unsigned long end); + +int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, + pgprot_t pgprot); + +#endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 0a54d81c90f9..4a38662f0d90 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -16,7 +16,6 @@ #define pr_fmt(x) "hibernate: " x #include #include -#include #include #include #include @@ -31,13 +30,12 @@ #include #include #include -#include -#include #include #include #include #include #include +#include #include /* @@ -178,54 +176,6 @@ int arch_hibernation_header_restore(void *addr) } EXPORT_SYMBOL(arch_hibernation_header_restore); -static int trans_pgd_map_page(pgd_t *trans_pgd, void *page, - unsigned long dst_addr, - pgprot_t pgprot) -{ - pgd_t *pgdp; - p4d_t *p4dp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_pgd(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = (void *)get_safe_page(GFP_ATOMIC); - if (!pgdp) - return -ENOMEM; - pgd_populate(&init_mm, pgdp, p4dp); - } - - p4dp = p4d_offset(pgdp, dst_addr); - if (p4d_none(READ_ONCE(*p4dp))) { - pudp = (void *)get_safe_page(GFP_ATOMIC); - if (!pudp) - return -ENOMEM; - p4d_populate(&init_mm, p4dp, pudp); - } - - pudp = pud_offset(p4dp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = (void *)get_safe_page(GFP_ATOMIC); - if (!pmdp) - return -ENOMEM; - pud_populate(&init_mm, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = (void *)get_safe_page(GFP_ATOMIC); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(&init_mm, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); - - return 0; -} - /* * Copies length bytes, starting at src_start into an new page, * perform cache maintenance, then maps it at the specified address low @@ -462,182 +412,6 @@ int swsusp_arch_suspend(void) return ret; } -static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) -{ - pte_t pte = READ_ONCE(*src_ptep); - - if (pte_valid(pte)) { - /* - * Resume will overwrite areas that may be marked - * read only (code, rodata). Clear the RDONLY bit from - * the temporary mappings we use during restore. - */ - set_pte(dst_ptep, pte_mkwrite(pte)); - } else if (debug_pagealloc_enabled() && !pte_none(pte)) { - /* - * debug_pagealloc will removed the PTE_VALID bit if - * the page isn't in use by the resume kernel. It may have - * been in use by the original kernel, in which case we need - * to put it back in our copy to do the restore. - * - * Before marking this entry valid, check the pfn should - * be mapped. - */ - BUG_ON(!pfn_valid(pte_pfn(pte))); - - set_pte(dst_ptep, pte_mkpresent(pte_mkwrite(pte))); - } -} - -static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, - unsigned long end) -{ - pte_t *src_ptep; - pte_t *dst_ptep; - unsigned long addr = start; - - dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC); - if (!dst_ptep) - return -ENOMEM; - pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); - dst_ptep = pte_offset_kernel(dst_pmdp, start); - - src_ptep = pte_offset_kernel(src_pmdp, start); - do { - _copy_pte(dst_ptep, src_ptep, addr); - } while (dst_ptep++, src_ptep++, addr += PAGE_SIZE, addr != end); - - return 0; -} - -static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, - unsigned long end) -{ - pmd_t *src_pmdp; - pmd_t *dst_pmdp; - unsigned long next; - unsigned long addr = start; - - if (pud_none(READ_ONCE(*dst_pudp))) { - dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); - if (!dst_pmdp) - return -ENOMEM; - pud_populate(&init_mm, dst_pudp, dst_pmdp); - } - dst_pmdp = pmd_offset(dst_pudp, start); - - src_pmdp = pmd_offset(src_pudp, start); - do { - pmd_t pmd = READ_ONCE(*src_pmdp); - - next = pmd_addr_end(addr, end); - if (pmd_none(pmd)) - continue; - if (pmd_table(pmd)) { - if (copy_pte(dst_pmdp, src_pmdp, addr, next)) - return -ENOMEM; - } else { - set_pmd(dst_pmdp, - __pmd(pmd_val(pmd) & ~PMD_SECT_RDONLY)); - } - } while (dst_pmdp++, src_pmdp++, addr = next, addr != end); - - return 0; -} - -static int copy_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start, - unsigned long end) -{ - pud_t *dst_pudp; - pud_t *src_pudp; - unsigned long next; - unsigned long addr = start; - - if (p4d_none(READ_ONCE(*dst_p4dp))) { - dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC); - if (!dst_pudp) - return -ENOMEM; - p4d_populate(&init_mm, dst_p4dp, dst_pudp); - } - dst_pudp = pud_offset(dst_p4dp, start); - - src_pudp = pud_offset(src_p4dp, start); - do { - pud_t pud = READ_ONCE(*src_pudp); - - next = pud_addr_end(addr, end); - if (pud_none(pud)) - continue; - if (pud_table(pud)) { - if (copy_pmd(dst_pudp, src_pudp, addr, next)) - return -ENOMEM; - } else { - set_pud(dst_pudp, - __pud(pud_val(pud) & ~PUD_SECT_RDONLY)); - } - } while (dst_pudp++, src_pudp++, addr = next, addr != end); - - return 0; -} - -static int copy_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, - unsigned long end) -{ - p4d_t *dst_p4dp; - p4d_t *src_p4dp; - unsigned long next; - unsigned long addr = start; - - dst_p4dp = p4d_offset(dst_pgdp, start); - src_p4dp = p4d_offset(src_pgdp, start); - do { - next = p4d_addr_end(addr, end); - if (p4d_none(READ_ONCE(*src_p4dp))) - continue; - if (copy_pud(dst_p4dp, src_p4dp, addr, next)) - return -ENOMEM; - } while (dst_p4dp++, src_p4dp++, addr = next, addr != end); - - return 0; -} - -static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, - unsigned long end) -{ - unsigned long next; - unsigned long addr = start; - pgd_t *src_pgdp = pgd_offset_k(start); - - dst_pgdp = pgd_offset_pgd(dst_pgdp, start); - do { - next = pgd_addr_end(addr, end); - if (pgd_none(READ_ONCE(*src_pgdp))) - continue; - if (copy_p4d(dst_pgdp, src_pgdp, addr, next)) - return -ENOMEM; - } while (dst_pgdp++, src_pgdp++, addr = next, addr != end); - - return 0; -} - -static int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, - unsigned long end) -{ - int rc; - pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); - - if (!trans_pgd) { - pr_err("Failed to allocate memory for temporary page tables.\n"); - return -ENOMEM; - } - - rc = copy_page_tables(trans_pgd, start, end); - if (!rc) - *dst_pgdp = trans_pgd; - - return rc; -} - /* * Setup then Resume from the hibernate image using swsusp_arch_suspend_exit(). * diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 5ead3c3de3b6..77222d92667a 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -6,6 +6,7 @@ obj-y := dma-mapping.o extable.o fault.o init.o \ obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PTDUMP_DEBUGFS) += ptdump_debugfs.o +obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o obj-$(CONFIG_ARM64_MTE) += mteswap.o diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c new file mode 100644 index 000000000000..e048d1f5c912 --- /dev/null +++ b/arch/arm64/mm/trans_pgd.c @@ -0,0 +1,250 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Transitional page tables for kexec and hibernate + * + * This file derived from: arch/arm64/kernel/hibernate.c + * + * Copyright (c) 2020, Microsoft Corporation. + * Pavel Tatashin + * + */ + +/* + * Transitional tables are used during system transferring from one world to + * another: such as during hibernate restore, and kexec reboots. During these + * phases one cannot rely on page table not being overwritten. This is because + * hibernate and kexec can overwrite the current page tables during transition. + */ + +#include +#include +#include +#include +#include +#include +#include + +static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) +{ + pte_t pte = READ_ONCE(*src_ptep); + + if (pte_valid(pte)) { + /* + * Resume will overwrite areas that may be marked + * read only (code, rodata). Clear the RDONLY bit from + * the temporary mappings we use during restore. + */ + set_pte(dst_ptep, pte_mkwrite(pte)); + } else if (debug_pagealloc_enabled() && !pte_none(pte)) { + /* + * debug_pagealloc will removed the PTE_VALID bit if + * the page isn't in use by the resume kernel. It may have + * been in use by the original kernel, in which case we need + * to put it back in our copy to do the restore. + * + * Before marking this entry valid, check the pfn should + * be mapped. + */ + BUG_ON(!pfn_valid(pte_pfn(pte))); + + set_pte(dst_ptep, pte_mkpresent(pte_mkwrite(pte))); + } +} + +static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, + unsigned long end) +{ + pte_t *src_ptep; + pte_t *dst_ptep; + unsigned long addr = start; + + dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC); + if (!dst_ptep) + return -ENOMEM; + pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); + dst_ptep = pte_offset_kernel(dst_pmdp, start); + + src_ptep = pte_offset_kernel(src_pmdp, start); + do { + _copy_pte(dst_ptep, src_ptep, addr); + } while (dst_ptep++, src_ptep++, addr += PAGE_SIZE, addr != end); + + return 0; +} + +static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, + unsigned long end) +{ + pmd_t *src_pmdp; + pmd_t *dst_pmdp; + unsigned long next; + unsigned long addr = start; + + if (pud_none(READ_ONCE(*dst_pudp))) { + dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); + if (!dst_pmdp) + return -ENOMEM; + pud_populate(&init_mm, dst_pudp, dst_pmdp); + } + dst_pmdp = pmd_offset(dst_pudp, start); + + src_pmdp = pmd_offset(src_pudp, start); + do { + pmd_t pmd = READ_ONCE(*src_pmdp); + + next = pmd_addr_end(addr, end); + if (pmd_none(pmd)) + continue; + if (pmd_table(pmd)) { + if (copy_pte(dst_pmdp, src_pmdp, addr, next)) + return -ENOMEM; + } else { + set_pmd(dst_pmdp, + __pmd(pmd_val(pmd) & ~PMD_SECT_RDONLY)); + } + } while (dst_pmdp++, src_pmdp++, addr = next, addr != end); + + return 0; +} + +static int copy_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start, + unsigned long end) +{ + pud_t *dst_pudp; + pud_t *src_pudp; + unsigned long next; + unsigned long addr = start; + + if (p4d_none(READ_ONCE(*dst_p4dp))) { + dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC); + if (!dst_pudp) + return -ENOMEM; + p4d_populate(&init_mm, dst_p4dp, dst_pudp); + } + dst_pudp = pud_offset(dst_p4dp, start); + + src_pudp = pud_offset(src_p4dp, start); + do { + pud_t pud = READ_ONCE(*src_pudp); + + next = pud_addr_end(addr, end); + if (pud_none(pud)) + continue; + if (pud_table(pud)) { + if (copy_pmd(dst_pudp, src_pudp, addr, next)) + return -ENOMEM; + } else { + set_pud(dst_pudp, + __pud(pud_val(pud) & ~PUD_SECT_RDONLY)); + } + } while (dst_pudp++, src_pudp++, addr = next, addr != end); + + return 0; +} + +static int copy_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, + unsigned long end) +{ + p4d_t *dst_p4dp; + p4d_t *src_p4dp; + unsigned long next; + unsigned long addr = start; + + dst_p4dp = p4d_offset(dst_pgdp, start); + src_p4dp = p4d_offset(src_pgdp, start); + do { + next = p4d_addr_end(addr, end); + if (p4d_none(READ_ONCE(*src_p4dp))) + continue; + if (copy_pud(dst_p4dp, src_p4dp, addr, next)) + return -ENOMEM; + } while (dst_p4dp++, src_p4dp++, addr = next, addr != end); + + return 0; +} + +static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, + unsigned long end) +{ + unsigned long next; + unsigned long addr = start; + pgd_t *src_pgdp = pgd_offset_k(start); + + dst_pgdp = pgd_offset_pgd(dst_pgdp, start); + do { + next = pgd_addr_end(addr, end); + if (pgd_none(READ_ONCE(*src_pgdp))) + continue; + if (copy_p4d(dst_pgdp, src_pgdp, addr, next)) + return -ENOMEM; + } while (dst_pgdp++, src_pgdp++, addr = next, addr != end); + + return 0; +} + +int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, + unsigned long end) +{ + int rc; + pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); + + if (!trans_pgd) { + pr_err("Failed to allocate memory for temporary page tables.\n"); + return -ENOMEM; + } + + rc = copy_page_tables(trans_pgd, start, end); + if (!rc) + *dst_pgdp = trans_pgd; + + return rc; +} + +int trans_pgd_map_page(pgd_t *trans_pgd, void *page, + unsigned long dst_addr, + pgprot_t pgprot) +{ + pgd_t *pgdp; + p4d_t *p4dp; + pud_t *pudp; + pmd_t *pmdp; + pte_t *ptep; + + pgdp = pgd_offset_pgd(trans_pgd, dst_addr); + if (pgd_none(READ_ONCE(*pgdp))) { + p4dp = (void *)get_safe_page(GFP_ATOMIC); + if (!pgdp) + return -ENOMEM; + pgd_populate(&init_mm, pgdp, p4dp); + } + + p4dp = p4d_offset(pgdp, dst_addr); + if (p4d_none(READ_ONCE(*p4dp))) { + pudp = (void *)get_safe_page(GFP_ATOMIC); + if (!pudp) + return -ENOMEM; + p4d_populate(&init_mm, p4dp, pudp); + } + + pudp = pud_offset(p4dp, dst_addr); + if (pud_none(READ_ONCE(*pudp))) { + pmdp = (void *)get_safe_page(GFP_ATOMIC); + if (!pmdp) + return -ENOMEM; + pud_populate(&init_mm, pudp, pmdp); + } + + pmdp = pmd_offset(pudp, dst_addr); + if (pmd_none(READ_ONCE(*pmdp))) { + ptep = (void *)get_safe_page(GFP_ATOMIC); + if (!ptep) + return -ENOMEM; + pmd_populate_kernel(&init_mm, pmdp, ptep); + } + + ptep = pte_offset_kernel(pmdp, dst_addr); + set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); + + return 0; +} From patchwork Mon Jan 25 19:19:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00CCFC433DB for ; Mon, 25 Jan 2021 19:22:05 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C7CB21744 for ; Mon, 25 Jan 2021 19:22:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C7CB21744 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ClFiAlw3xeSzwBIbIL2yrkVdyfW39eC6GYJ9DxKPX98=; b=aAeKjg85selfCjxNt0l8GgQRCA rZKlwW83Ys7VOLIujYlIeEnDVjqjwhkgYwVbPrVTRFnd3NpNGAtAaa/XzV2w9kRPy+PN7zlMXctjn mukmFWewG1hLtzUdXmrMW8phUtvUBfcpFjAjj46KwOHIGRnFtAH8ZO/ktXknkG2rP9jdqVyG/JK57 2Ilti13eokvLmQ+oHV8L7lHrgIv+AzwusWsSZPXO9so2XsD/9CbARuOTkwAEYvKaG4YECLmhCcyHH 1PGJABiF0o4KwymX5HNH/NdLB3karIJwq6rhaWXCDg6qMzpMSUVc29HAn1JC0sjeQ8bxd9W7r+ash zIRTp8Lw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47PN-0007ME-LT; Mon, 25 Jan 2021 19:20:17 +0000 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Og-00075t-Kp for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:41 +0000 Received: by mail-qt1-x82a.google.com with SMTP id o18so10495366qtp.10 for ; Mon, 25 Jan 2021 11:19:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uo4pw5FNnmKWVvyw7b3Lb3eRlMWvIqi1VadyjI7Jm9U=; b=IE28xogklAT4sOez1Dy/Fu+fa8eNdiI9L10ZSXU1+5/yWipPF2Sx81TPKmgb+uw1dh jGYoNPksCaDXyypSXPaeWhb5sxug0tr/JO/83cvLPQ3M6Sp5JxgFVfXeuiQK0TBWG+0I Efgtb7wfBRSoCphyFE0zXigGVWsMBfWzO0k3CmG0zwG7rP+CUEgCjweGjk4Owkako7sg kGRSDnlmgy8f/demaZoW6ffBD7eiyP6ABCLBJuVkvjc3wuLlN02lRkjcwMkp/bRVrkrH l8gA4TsGufT1dkwtXchxNDTuRNc+TAcrp7WwV6d68TZzzukHJtRV0qHNiDJy4CL7/tD5 95Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uo4pw5FNnmKWVvyw7b3Lb3eRlMWvIqi1VadyjI7Jm9U=; b=W37yqoApeOisJOucFQdesXBZa8JCjCxF/stES+SHKM8W8eE4E1G19x4ZZ2Zmqe5Lhm Oqu1npJkW/zwVBcHIPGEAAAx2wX5BoH8IEOsR+RVv/KO5Haa8Gwle8rL4HJXZTs4EnUT /cPw2L0UIj9B8gYU4BUYkJWKBgRraOqC0oVqyl/gHPyWYPsMjf7UkxPlk70F0OUFIIX7 75SmUIv0qzlWLQw/gxL50MNM9SiC7sH3cWICdxCbab7F3VV9nBewTY31jVuwHabT82fQ vZLg+GCk/8jggS3m8JjCpXkI2/5YwExIAMcIQ0S9b6K9OaAM1chW/sKCX7W3lIm3vM05 lerg== X-Gm-Message-State: AOAM531sais25Qg9KGZa3Ty1mFZXE8RyQCxuDJUUx5ubgHSMoX/987ab JWKoFLusbgJQeKfA6kIzXH9W8g== X-Google-Smtp-Source: ABdhPJzZPCVq7UEdZey87cJYsvOc8buPPhi2/MEfXEKJx3amijR4SVfdPvhZKax11XK2NKqWOtCGUA== X-Received: by 2002:ac8:604b:: with SMTP id k11mr1970803qtm.321.1611602371717; Mon, 25 Jan 2021 11:19:31 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:31 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 04/18] arm64: trans_pgd: make trans_pgd_map_page generic Date: Mon, 25 Jan 2021 14:19:09 -0500 Message-Id: <20210125191923.1060122-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141934_757152_93F4B1AC X-CRM114-Status: GOOD ( 16.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org kexec is going to use a different allocator, so make trans_pgd_map_page to accept allocator as an argument, and also kexec is going to use a different map protection, so also pass it via argument. Signed-off-by: Pavel Tatashin Reviewed-by: Matthias Brugger --- arch/arm64/include/asm/trans_pgd.h | 19 +++++++++++++++++-- arch/arm64/kernel/hibernate.c | 12 +++++++++++- arch/arm64/mm/trans_pgd.c | 30 ++++++++++++++++++++++-------- 3 files changed, 50 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 23153c13d1ce..b46409b25234 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -12,10 +12,25 @@ #include #include +/* + * trans_alloc_page + * - Allocator that should return exactly one zeroed page, if this + * allocator fails, trans_pgd_create_copy() and trans_pgd_map_page() + * return -ENOMEM error. + * + * trans_alloc_arg + * - Passed to trans_alloc_page as an argument + */ + +struct trans_pgd_info { + void * (*trans_alloc_page)(void *arg); + void *trans_alloc_arg; +}; + int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, unsigned long end); -int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, - pgprot_t pgprot); +int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, + void *page, unsigned long dst_addr, pgprot_t pgprot); #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 4a38662f0d90..c173f280bfea 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -176,6 +176,11 @@ int arch_hibernation_header_restore(void *addr) } EXPORT_SYMBOL(arch_hibernation_header_restore); +static void *hibernate_page_alloc(void *arg) +{ + return (void *)get_safe_page((gfp_t)(unsigned long)arg); +} + /* * Copies length bytes, starting at src_start into an new page, * perform cache maintenance, then maps it at the specified address low @@ -192,6 +197,11 @@ static int create_safe_exec_page(void *src_start, size_t length, unsigned long dst_addr, phys_addr_t *phys_dst_addr) { + struct trans_pgd_info trans_info = { + .trans_alloc_page = hibernate_page_alloc, + .trans_alloc_arg = (void *)GFP_ATOMIC, + }; + void *page = (void *)get_safe_page(GFP_ATOMIC); pgd_t *trans_pgd; int rc; @@ -206,7 +216,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (!trans_pgd) return -ENOMEM; - rc = trans_pgd_map_page(trans_pgd, page, dst_addr, + rc = trans_pgd_map_page(&trans_info, trans_pgd, page, dst_addr, PAGE_KERNEL_EXEC); if (rc) return rc; diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index e048d1f5c912..f28eceba2242 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -25,6 +25,11 @@ #include #include +static void *trans_alloc(struct trans_pgd_info *info) +{ + return info->trans_alloc_page(info->trans_alloc_arg); +} + static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) { pte_t pte = READ_ONCE(*src_ptep); @@ -201,9 +206,18 @@ int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, return rc; } -int trans_pgd_map_page(pgd_t *trans_pgd, void *page, - unsigned long dst_addr, - pgprot_t pgprot) +/* + * Add map entry to trans_pgd for a base-size page at PTE level. + * info: contains allocator and its argument + * trans_pgd: page table in which new map is added. + * page: page to be mapped. + * dst_addr: new VA address for the page + * pgprot: protection for the page. + * + * Returns 0 on success, and -ENOMEM on failure. + */ +int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, + void *page, unsigned long dst_addr, pgprot_t pgprot) { pgd_t *pgdp; p4d_t *p4dp; @@ -213,7 +227,7 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, pgdp = pgd_offset_pgd(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { - p4dp = (void *)get_safe_page(GFP_ATOMIC); + p4dp = trans_alloc(info); if (!pgdp) return -ENOMEM; pgd_populate(&init_mm, pgdp, p4dp); @@ -221,7 +235,7 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, p4dp = p4d_offset(pgdp, dst_addr); if (p4d_none(READ_ONCE(*p4dp))) { - pudp = (void *)get_safe_page(GFP_ATOMIC); + pudp = trans_alloc(info); if (!pudp) return -ENOMEM; p4d_populate(&init_mm, p4dp, pudp); @@ -229,7 +243,7 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, pudp = pud_offset(p4dp, dst_addr); if (pud_none(READ_ONCE(*pudp))) { - pmdp = (void *)get_safe_page(GFP_ATOMIC); + pmdp = trans_alloc(info); if (!pmdp) return -ENOMEM; pud_populate(&init_mm, pudp, pmdp); @@ -237,14 +251,14 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, pmdp = pmd_offset(pudp, dst_addr); if (pmd_none(READ_ONCE(*pmdp))) { - ptep = (void *)get_safe_page(GFP_ATOMIC); + ptep = trans_alloc(info); if (!ptep) return -ENOMEM; pmd_populate_kernel(&init_mm, pmdp, ptep); } ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); + set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); return 0; } From patchwork Mon Jan 25 19:19:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCA72C433DB for ; Mon, 25 Jan 2021 19:23:57 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6838221D79 for ; Mon, 25 Jan 2021 19:23:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6838221D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+Iswnc0U/TybhMQfX7y2xD1hHz4GKIGwLaU3EIcNrXo=; b=BV7YSURRpnR9SFZwRLs5ozQHb1 p3ajCTV2uwXy0JztCqA8YTEmci1clUWiRMfL3BYPsncrHn/2oGvDqWar1+Kw5RHa1xvY/UANbU/6Z ZxDzKwULJcUofJsgdnsid0P/BRtBZNKnpDzdtyj6UJavImPHgAxwg0O04h0kgsd7KnBjl3wkJvXDF ZPTOY9emp1XYhECIrrkEUOHx1vMu85zJSRRgeIKlQm+i6/csP2AIm2bTBBeQrkTVPLT4MwTOBuS1F w9J/9PO9BPCA24xWR0t4RX6V9RTS1EtCm4M8nMgb0PGsKiYuS7pkzkyFOe39S1LaDdykeHl+lIVnj GoSoc5dg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47QL-0007j4-ED; Mon, 25 Jan 2021 19:21:17 +0000 Received: from mail-qv1-xf2d.google.com ([2607:f8b0:4864:20::f2d]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Og-00076s-OO for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:46 +0000 Received: by mail-qv1-xf2d.google.com with SMTP id es14so2919524qvb.3 for ; Mon, 25 Jan 2021 11:19:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FGDp7jvUOvQaUtmYsOORS7JatJv8YjyLST8iVKp3uek=; b=MCIWlM9LqtrBZhnKkAMxaF3ZB6y612DYB/9YdT3+aJCN8N2Ped81zgD28h2d7Y2eui M1kCjXBd2IUYY5Wyk+2REfe21/3BZvjMKmy50MwmvYJV75E7R+PJ815FzSmDwolsQ/+t jlzH6h1VWG0WxXPPh/t5QYrP0sv8OosPme/9H8Iwiou9nQxfDuKMIQgpe2JtLd2pX+eB BFJlTpWqsf8V1rCwfzGPAjoaSrlSoDrH3R8qJn6tiYSEnQyOr+NSmm1VaS2m2J1QzlKM RDecp+ibsQWFrMRfWAQ8r7dgwQY5oFFokAh78jU24QV6JnsCbEPYzVXC4FKym3jn74BI 5+Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FGDp7jvUOvQaUtmYsOORS7JatJv8YjyLST8iVKp3uek=; b=LqAthoh62xRWcC3ZtwyiPJWGdRnHsBMIWbdqQ2MbVunZHDQqApNdk7q+i4WrP3Xn9d lozRHOYhpYF0RaF9N0sxnZYEJwAac5v1KjP8S+zQXyzY2tV0ivTgqy31NOFs5KWNUI8v VuSQmi4kbuXudTmixnR75K2LuhsIQmIWSDVqsjjEVlbmisvY0oEtHojQQuiI/84rWVY0 6v3IShRkillGsI28C9RJwH3s+7tKkKBgkBfuEUYJ0cjfEZiw6klbm3/Tospp78GdD2zl gTBkSfmKMzDkt801t6ijaeTAEb5cH/z0dQ6ojl01DaPyICV7i7MJbzGb/+IZICkOTK7w DTPQ== X-Gm-Message-State: AOAM533o5nTqXIMtGkR8Moe7HqNaMS1+JUGVPZ/VnYQkAe+ZWXY9jMtW rOYhBBFfhcK0/fRR0Neja85QBQ== X-Google-Smtp-Source: ABdhPJxxx+xP/tQ5CLVtjlwvppB0m/QNU5v1txKp9ktgaWccMjRmTAmf67LKfYWEvAAHjJJ9HcNAlg== X-Received: by 2002:ad4:4e8a:: with SMTP id dy10mr2245973qvb.36.1611602373235; Mon, 25 Jan 2021 11:19:33 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:32 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 05/18] arm64: trans_pgd: pass allocator trans_pgd_create_copy Date: Mon, 25 Jan 2021 14:19:10 -0500 Message-Id: <20210125191923.1060122-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141934_913812_7D44E38B X-CRM114-Status: GOOD ( 16.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make trans_pgd_create_copy and its subroutines to use allocator that is passed as an argument Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/include/asm/trans_pgd.h | 4 +-- arch/arm64/kernel/hibernate.c | 7 ++++- arch/arm64/mm/trans_pgd.c | 49 ++++++++++++++++++------------ 3 files changed, 38 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index b46409b25234..7fbf6a3ccff7 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -27,8 +27,8 @@ struct trans_pgd_info { void *trans_alloc_arg; }; -int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, - unsigned long end); +int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, + unsigned long start, unsigned long end); int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, void *page, unsigned long dst_addr, pgprot_t pgprot); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index c173f280bfea..94fc275cdd21 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -437,13 +437,18 @@ int swsusp_arch_resume(void) phys_addr_t phys_hibernate_exit; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); + struct trans_pgd_info trans_info = { + .trans_alloc_page = hibernate_page_alloc, + .trans_alloc_arg = (void *)GFP_ATOMIC, + }; /* * Restoring the memory image will overwrite the ttbr1 page tables. * Create a second copy of just the linear map, and use this when * restoring. */ - rc = trans_pgd_create_copy(&tmp_pg_dir, PAGE_OFFSET, PAGE_END); + rc = trans_pgd_create_copy(&trans_info, &tmp_pg_dir, PAGE_OFFSET, + PAGE_END); if (rc) return rc; diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index f28eceba2242..47b6b7029907 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -57,14 +57,14 @@ static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) } } -static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, - unsigned long end) +static int copy_pte(struct trans_pgd_info *info, pmd_t *dst_pmdp, + pmd_t *src_pmdp, unsigned long start, unsigned long end) { pte_t *src_ptep; pte_t *dst_ptep; unsigned long addr = start; - dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC); + dst_ptep = trans_alloc(info); if (!dst_ptep) return -ENOMEM; pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); @@ -78,8 +78,8 @@ static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, return 0; } -static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, - unsigned long end) +static int copy_pmd(struct trans_pgd_info *info, pud_t *dst_pudp, + pud_t *src_pudp, unsigned long start, unsigned long end) { pmd_t *src_pmdp; pmd_t *dst_pmdp; @@ -87,7 +87,7 @@ static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, unsigned long addr = start; if (pud_none(READ_ONCE(*dst_pudp))) { - dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); + dst_pmdp = trans_alloc(info); if (!dst_pmdp) return -ENOMEM; pud_populate(&init_mm, dst_pudp, dst_pmdp); @@ -102,7 +102,7 @@ static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, if (pmd_none(pmd)) continue; if (pmd_table(pmd)) { - if (copy_pte(dst_pmdp, src_pmdp, addr, next)) + if (copy_pte(info, dst_pmdp, src_pmdp, addr, next)) return -ENOMEM; } else { set_pmd(dst_pmdp, @@ -113,7 +113,8 @@ static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, return 0; } -static int copy_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start, +static int copy_pud(struct trans_pgd_info *info, p4d_t *dst_p4dp, + p4d_t *src_p4dp, unsigned long start, unsigned long end) { pud_t *dst_pudp; @@ -122,7 +123,7 @@ static int copy_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start, unsigned long addr = start; if (p4d_none(READ_ONCE(*dst_p4dp))) { - dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC); + dst_pudp = trans_alloc(info); if (!dst_pudp) return -ENOMEM; p4d_populate(&init_mm, dst_p4dp, dst_pudp); @@ -137,7 +138,7 @@ static int copy_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start, if (pud_none(pud)) continue; if (pud_table(pud)) { - if (copy_pmd(dst_pudp, src_pudp, addr, next)) + if (copy_pmd(info, dst_pudp, src_pudp, addr, next)) return -ENOMEM; } else { set_pud(dst_pudp, @@ -148,7 +149,8 @@ static int copy_pud(p4d_t *dst_p4dp, p4d_t *src_p4dp, unsigned long start, return 0; } -static int copy_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, +static int copy_p4d(struct trans_pgd_info *info, pgd_t *dst_pgdp, + pgd_t *src_pgdp, unsigned long start, unsigned long end) { p4d_t *dst_p4dp; @@ -162,15 +164,15 @@ static int copy_p4d(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, next = p4d_addr_end(addr, end); if (p4d_none(READ_ONCE(*src_p4dp))) continue; - if (copy_pud(dst_p4dp, src_p4dp, addr, next)) + if (copy_pud(info, dst_p4dp, src_p4dp, addr, next)) return -ENOMEM; } while (dst_p4dp++, src_p4dp++, addr = next, addr != end); return 0; } -static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, - unsigned long end) +static int copy_page_tables(struct trans_pgd_info *info, pgd_t *dst_pgdp, + unsigned long start, unsigned long end) { unsigned long next; unsigned long addr = start; @@ -181,25 +183,34 @@ static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, next = pgd_addr_end(addr, end); if (pgd_none(READ_ONCE(*src_pgdp))) continue; - if (copy_p4d(dst_pgdp, src_pgdp, addr, next)) + if (copy_p4d(info, dst_pgdp, src_pgdp, addr, next)) return -ENOMEM; } while (dst_pgdp++, src_pgdp++, addr = next, addr != end); return 0; } -int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, - unsigned long end) +/* + * Create trans_pgd and copy linear map. + * info: contains allocator and its argument + * dst_pgdp: new page table that is created, and to which map is copied. + * start: Start of the interval (inclusive). + * end: End of the interval (exclusive). + * + * Returns 0 on success, and -ENOMEM on failure. + */ +int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, + unsigned long start, unsigned long end) { int rc; - pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); + pgd_t *trans_pgd = trans_alloc(info); if (!trans_pgd) { pr_err("Failed to allocate memory for temporary page tables.\n"); return -ENOMEM; } - rc = copy_page_tables(trans_pgd, start, end); + rc = copy_page_tables(info, trans_pgd, start, end); if (!rc) *dst_pgdp = trans_pgd; From patchwork Mon Jan 25 19:19:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0297C433DB for ; Mon, 25 Jan 2021 20:31:55 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58F92216FD for ; Mon, 25 Jan 2021 20:31:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58F92216FD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=4SQ9oG17TDLioUtBxbC4MIm0ohmABfP3NDkpMCkutlU=; b=AGpBtq+cARE5pfHR2Pk8tnFxKo pCE7HVbb7TDz79Lhls2z5q6F5yb7DNSMEYSSGgeOGTddNBXCljV2WqNhIygW3Q8L6xycmHZZPvjNI NJGdwy1YKj2A+Hlv5K6vuuXWW+v8sD18DM3UtQkdiUKkNAABpQayWXkBBnC6v2oGMEXDYnVjANB4h w/z/3/rZ/GMjmoSPaJ05E/2CxZxrCd6hkgiVkcuUsKbSnf0AXf3kZdvZFUZ2hfX18DfFg6T/HFvAJ d2s3HatVm1LPf4BLmnHkGU3nFPsO+D8gwFWoOd1j0KUbVEYgZHyhZ/O1aarefxl7w3luQbAcqTOm3 TdPlClLw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l48Vj-0003RK-R2; Mon, 25 Jan 2021 20:30:55 +0000 Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Oi-00077i-46 for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:46 +0000 Received: by mail-qt1-x82a.google.com with SMTP id e17so10529122qto.3 for ; Mon, 25 Jan 2021 11:19:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3FwrG1DRKa930mcqv1CQX/Ow19T+ppBoJGHaLYA3Zn4=; b=OMM+n/NgLxxolTPSpQfFupxi8pL7prIM8KcL0yuUt6nWfKRTlR6APYpr/V8F9WNvA1 ikIhnojbMioMSri7RLzroDZGFYn/E6gh220XP8IJyVS3exiStf/6gCaet1KfKtTGtzSB hUwZY6sdDa4bLX25PetOw90C87Q6MClcrvHBbWn37ai8bBgV5A9XsgIe+309KgWBm0o4 PM5Hgbdxd/Z4Wf3Z9Uy9rmMFm46l1p1ggWGl3Glxu55yQPOxsT+XaTwTkEaSpvjMbD6M 6rT7T58457HNNABrDb3FqPJjaoGnar+ESiEAa6nYAAfBmFI+ipXzsBU1dI23SbYeBsye z4Gw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3FwrG1DRKa930mcqv1CQX/Ow19T+ppBoJGHaLYA3Zn4=; b=Ey7gTsuszfd6FG+zff+qMt9nmjC5t/2vr8QVOspyB0NCptScIwQITfuxYibD1AJeGs FC0x6JG77aXIfs09A9g+2jN7GXEuk7ynrM9S2WIG7C+ESWqogU+zuHErVjT4hHW43EF0 OQLpKCZN5GUsUvWSw1WCY8veGLwC4e/ayBTvNTWVsnp922+Ngixb2cFpFkr1fkZRXF97 0EXg4T+lpfLNh9n1nVj/bCx1EsIMmLSdbFq68uu9rhlR7J0RN8+RfyG/WB2spwgROppM RpxaUsw663+e7tjEWZxf0GtClaxgVgQIzgtnwBdx/gRBWTNY8k++Nyj9T28N2P0n0r0o bIsw== X-Gm-Message-State: AOAM53255ESd3yh+EUK3b1l6DZVuOXocGbQ/k54Oju2HtPmkBclxkkQl n2fCvhyNC4ERnJJaACwAU1/OVw== X-Google-Smtp-Source: ABdhPJzYpijMw9x6eiYnJkfIhvZXHqaaGkTUev606A5PQKAYeMNfq2YM+YhFDIUaCji9oVYPnb8lLA== X-Received: by 2002:ac8:3571:: with SMTP id z46mr2020689qtb.83.1611602374721; Mon, 25 Jan 2021 11:19:34 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:34 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 06/18] arm64: trans_pgd: pass NULL instead of init_mm to *_populate functions Date: Mon, 25 Jan 2021 14:19:11 -0500 Message-Id: <20210125191923.1060122-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141936_912554_F875D46A X-CRM114-Status: GOOD ( 11.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org trans_pgd_* should be independent from mm context because the tables that are created by this code are used when there are no mm context around, as it is between kernels. Simply replace mm_init's with NULL. Signed-off-by: Pavel Tatashin Acked-by: James Morse --- arch/arm64/mm/trans_pgd.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 47b6b7029907..ded8e2ba0308 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -67,7 +67,7 @@ static int copy_pte(struct trans_pgd_info *info, pmd_t *dst_pmdp, dst_ptep = trans_alloc(info); if (!dst_ptep) return -ENOMEM; - pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); + pmd_populate_kernel(NULL, dst_pmdp, dst_ptep); dst_ptep = pte_offset_kernel(dst_pmdp, start); src_ptep = pte_offset_kernel(src_pmdp, start); @@ -90,7 +90,7 @@ static int copy_pmd(struct trans_pgd_info *info, pud_t *dst_pudp, dst_pmdp = trans_alloc(info); if (!dst_pmdp) return -ENOMEM; - pud_populate(&init_mm, dst_pudp, dst_pmdp); + pud_populate(NULL, dst_pudp, dst_pmdp); } dst_pmdp = pmd_offset(dst_pudp, start); @@ -126,7 +126,7 @@ static int copy_pud(struct trans_pgd_info *info, p4d_t *dst_p4dp, dst_pudp = trans_alloc(info); if (!dst_pudp) return -ENOMEM; - p4d_populate(&init_mm, dst_p4dp, dst_pudp); + p4d_populate(NULL, dst_p4dp, dst_pudp); } dst_pudp = pud_offset(dst_p4dp, start); @@ -241,7 +241,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, p4dp = trans_alloc(info); if (!pgdp) return -ENOMEM; - pgd_populate(&init_mm, pgdp, p4dp); + pgd_populate(NULL, pgdp, p4dp); } p4dp = p4d_offset(pgdp, dst_addr); @@ -249,7 +249,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, pudp = trans_alloc(info); if (!pudp) return -ENOMEM; - p4d_populate(&init_mm, p4dp, pudp); + p4d_populate(NULL, p4dp, pudp); } pudp = pud_offset(p4dp, dst_addr); @@ -257,7 +257,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, pmdp = trans_alloc(info); if (!pmdp) return -ENOMEM; - pud_populate(&init_mm, pudp, pmdp); + pud_populate(NULL, pudp, pmdp); } pmdp = pmd_offset(pudp, dst_addr); @@ -265,7 +265,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, ptep = trans_alloc(info); if (!ptep) return -ENOMEM; - pmd_populate_kernel(&init_mm, pmdp, ptep); + pmd_populate_kernel(NULL, pmdp, ptep); } ptep = pte_offset_kernel(pmdp, dst_addr); From patchwork Mon Jan 25 19:19:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E418C433E0 for ; Mon, 25 Jan 2021 19:23:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0DA5C21D79 for ; Mon, 25 Jan 2021 19:23:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0DA5C21D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Cb7lUtrdJEe0du49V3IcGM9L0UUL/aUMMHdq8ijt4t8=; b=Xle+lpI2RAxr3uRHyxih7akj3s o+eaUnLDLTjNktg4FQfGGSN20xilp7C74pujwU4HMgH30m08yYQ89tmJzFeI6VE3OViqVEyxChXsR zLXdiKZjnIt38TIh39dlLnUAd/c5WJpkvJoeWb7GY11m+mNzI5vfsCdJkbTloveyuNoE8pPk/65px bmZ4Y4PaYUk4iCzYnOckEtXTHuzM8xOwsfzZz8NUyV6iJzOLqv3R/x0Z4gt1ChlIkmStCUd4u6Vgv YmekzJX/i1DIZJz6sqXC6oiaBcAJEsjy9yuRB1LXTjtKWiDvuJtn4fHzTkWZP6wFc4SNDM2R2VPKt Xzasepeg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47RC-00084Y-8c; Mon, 25 Jan 2021 19:22:10 +0000 Received: from mail-qv1-xf2a.google.com ([2607:f8b0:4864:20::f2a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Ok-00078E-D4 for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:56 +0000 Received: by mail-qv1-xf2a.google.com with SMTP id n14so1953460qvg.5 for ; Mon, 25 Jan 2021 11:19:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=I2bLiPX3VTIFYdlWvHrlv9bi90SZn0tAgsUExdtL3vw=; b=e7EeqPlQvWqFBbDxIS+wdmiqyKoOqI9OZZCgYolvZoI6Xiol0eFKvHPC1pWgm6Fiay k8gT8zBoIce+Xnc0+wfreyByeComQF0lOYUuI2FV338Zysdkz2m76fy7N3c+/gmDJcw9 Su+R42Mjqxnq0Z04YmcTRsTv2Dkqh0+vHQYSAzntfUfSzhuEiRsIZKtRGZNohQuoiMP1 37lX2zUf84XJ8SC5th4NIOoejS1ep2O1KTF5HUrMordiA0b/YGqqyz/FlLrRWNDMSTQ+ KGVrdwd2jN+Ly8WZH1xArA9m5WyRxE/jCVrqctcOR1sw8aXhiCqDGpRsGzUEBQZL0sci 0F2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I2bLiPX3VTIFYdlWvHrlv9bi90SZn0tAgsUExdtL3vw=; b=iEc3YwQ2h+DmslPheE3zYa8UIkZWq4iGT1p0BevkKTuF2bainAgDnojwt/FSGhlYdf fqsfMM5lWNQ49RcRRMMRvasEVSnPnS5onHWI1TyXHZ8jub10NFBynio/6TkET5zzP9zT kdI1MNLW8aHLooC3DYWFb/newvuEggadExb3HrnvA3+/o7DAPYhhqDiI+PeSBnm18eTY Zo35pBXUyXfwXdtXURnYhPQENO9ZLwOmw0Qq+ZpXR4W+mY7iU9aP0QRfIfBucwtoIWu5 g/R7lKnOvDvA07gu3BefHB91QCwiQw62/x+nxGmfCei1FDAFMr86V3dLe8Ewq/1MQw4K brwg== X-Gm-Message-State: AOAM5305hUORTNoEI1kGU0nqEsnchFg0m4n52BXlxhwWl5b45SfkHvrf cf94Q70sQglpNo6bD6U2TYrLaA== X-Google-Smtp-Source: ABdhPJzA63TTz10VNLj7mmheU/keqIHP7LOeLAzUTHMXQrqU9ZPOb8WN2VV8RoUDjNxuh0fZX6dq9g== X-Received: by 2002:a0c:d403:: with SMTP id t3mr2275554qvh.4.1611602376226; Mon, 25 Jan 2021 11:19:36 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:35 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 07/18] arm64: mm: Always update TCR_EL1 from __cpu_set_tcr_t0sz() Date: Mon, 25 Jan 2021 14:19:12 -0500 Message-Id: <20210125191923.1060122-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141938_528786_A4A62EAD X-CRM114-Status: GOOD ( 13.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse Because only the idmap sets a non-standard T0SZ, __cpu_set_tcr_t0sz() can check for platforms that need to do this using __cpu_uses_extended_idmap() before doing its work. The idmap is only built with enough levels, (and T0SZ bits) to map its single page. To allow hibernate, and then kexec to idmap their single page copy routines, __cpu_set_tcr_t0sz() needs to consider additional users, who may need a different number of levels/T0SZ-bits to the idmap. (i.e. VA_BITS may be enough for the idmap, but not hibernate/kexec) Always read TCR_EL1, and check whether any work needs doing for this request. __cpu_uses_extended_idmap() remains as it is used by KVM, whose idmap is also part of the kernel image. This mostly affects the cpuidle path, where we now get an extra system register read . CC: Lorenzo Pieralisi CC: Sudeep Holla Signed-off-by: James Morse Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/mmu_context.h | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 0b3079fd28eb..70ce8c1d2b07 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -81,16 +81,15 @@ static inline bool __cpu_uses_extended_idmap_level(void) } /* - * Set TCR.T0SZ to its default value (based on VA_BITS) + * Ensure TCR.T0SZ is set to the provided value. */ static inline void __cpu_set_tcr_t0sz(unsigned long t0sz) { - unsigned long tcr; + unsigned long tcr = read_sysreg(tcr_el1); - if (!__cpu_uses_extended_idmap()) + if ((tcr & TCR_T0SZ_MASK) >> TCR_T0SZ_OFFSET == t0sz) return; - tcr = read_sysreg(tcr_el1); tcr &= ~TCR_T0SZ_MASK; tcr |= t0sz << TCR_T0SZ_OFFSET; write_sysreg(tcr, tcr_el1); From patchwork Mon Jan 25 19:19:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D1EFC433DB for ; Mon, 25 Jan 2021 19:24:12 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BBCFC21D79 for ; Mon, 25 Jan 2021 19:24:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBCFC21D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JAOGmUoDXOSr5LNhe67ik0FMhyAXz7kEw5wR8aR0cKI=; b=je1p8T2dTHOeTvUNhTftcafoeD sma9j+jJC6cr5GvXCcOEXUgpzFNEe3luOtFWv9i/qkFy2ZSGd+j+VxJzzBYtY7pC4/pkQzNKNf6aG kYYUQgNhvMmHg7SVdFYiTP1qlKHjVYFd9Fj8FuFqf9Ip3oob+6atNTgP26I8GY1zrbXeiwU+vjg4E di31X5iXgvKMbvVKxJ+PcpGK/0n17oLABb5gHcAFEiiwXMQojwXmUr9vWuqoUHhimmcGDQ3YzyEyn ogSjYR0tFtFl/LZVJN8JPPnAdLU8A2O249suuhzubtUHlOhGqaQak2oTv21JU8x4WLwtVJtt7aRSJ xM+jPASA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47RH-000878-GH; Mon, 25 Jan 2021 19:22:15 +0000 Received: from mail-qk1-x735.google.com ([2607:f8b0:4864:20::735]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Ol-00079R-KA for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:19:59 +0000 Received: by mail-qk1-x735.google.com with SMTP id n15so8954554qkh.8 for ; Mon, 25 Jan 2021 11:19:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=yxqxdjcYeXzHRYJR2uL7fg5YfMG49Nqr7zKyoy7ZCso=; b=B+qQeHZmm1nCpaer5jGR+2XL6vSG8oc6ubxBiW55/ZEpjT2ip14JUJefspWFfBQ6rt iBqlLQCI3KgXOUvLw0zhTVAQK5uxX/UjiKR/MkioK7T2r12lNsCoQBZbQn+9NVQ2oNuT YK+uko972boltBWppWe8whrFOtZ2Mhtqv/iP9xoIMVV3NqgpyuNg2c2fwqeyz4qeD/o3 ktZpoUs8f9jp3HUc1a5pQ/Utk2SPkmBTyIeWR5osMkCLaaoOXqGVKy3KDgzs4me7BHgM zboESC1tS7LywP8+X1Y0VNi5CfUjDoofhhyY29oDe1fIrxlNrkHZWRfJADOUB+qVFZGG qrAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yxqxdjcYeXzHRYJR2uL7fg5YfMG49Nqr7zKyoy7ZCso=; b=iJWeiPGPeahaka6Bw2T0TPgf0jIzeXqUL8D6lmzOsMFvYKM61vpgXi/zj8AycS21+U 52coUEx4JLW7vR9N9pN3phylXA9YOun+BbP7wfHhuxOZXkjlnEEW7XJZaCDVUOh3uH4V B0/67jG82pzC6SzSHDEaFfAbtQQHfM6Yhr3jAFmF7f1He85bjIrE9KypIFN3TLnZfqWh a3FmFRtA5kIj2lU5NpsLx46fPtrrwo7UK0YTawqimRmtP8LH2eOGGoIa/RiAxgmzvO9H jxq65yJvv+EINLOWm/QXtmB+BrZV1RctiiDkqpxEBlIThoelNzX0OJwSkmB7BFvxwOzp KNow== X-Gm-Message-State: AOAM533SjiudlzS6CKrii2dxJBRHb8ddcp5kMA8fwdWgZJP/XSqv+5UH JBbG3gt7HPMVMvidhMHYJd5nHg== X-Google-Smtp-Source: ABdhPJzSy2uQ+VCegZg1vz7vlssjTu7VocHSaS3LdXt5GoYDFqUz9exkpacYbL1y/NfjKgTESmOAjQ== X-Received: by 2002:a05:620a:1406:: with SMTP id d6mr2217884qkj.312.1611602377878; Mon, 25 Jan 2021 11:19:37 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:37 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 08/18] arm64: trans_pgd: hibernate: idmap the single page that holds the copy page routines Date: Mon, 25 Jan 2021 14:19:13 -0500 Message-Id: <20210125191923.1060122-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141939_764870_643D5F54 X-CRM114-Status: GOOD ( 24.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: James Morse To resume from hibernate, the contents of memory are restored from the swap image. This may overwrite any page, including the running kernel and its page tables. Hibernate copies the code it uses to do the restore into a single page that it knows won't be overwritten, and maps it with page tables built from pages that won't be overwritten. Today the address it uses for this mapping is arbitrary, but to allow kexec to reuse this code, it needs to be idmapped. To idmap the page we must avoid the kernel helpers that have VA_BITS baked in. Convert create_single_mapping() to take a single PA, and idmap it. The page tables are built in the reverse order to normal using pfn_pte() to stir in any bits between 52:48. T0SZ is always increased to cover 48bits, or 52 if the copy code has bits 52:48 in its PA. Signed-off-by: James Morse [Adopted the original patch from James to trans_pgd interface, so it can be commonly used by both Kexec and Hibernate. Some minor clean-ups.] Signed-off-by: Pavel Tatashin Link: https://lore.kernel.org/linux-arm-kernel/20200115143322.214247-4-james.morse@arm.com/ --- arch/arm64/include/asm/trans_pgd.h | 3 ++ arch/arm64/kernel/hibernate.c | 32 +++++++------------ arch/arm64/mm/trans_pgd.c | 49 ++++++++++++++++++++++++++++++ 3 files changed, 63 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index 7fbf6a3ccff7..5d08e5adf3d5 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -33,4 +33,7 @@ int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, void *page, unsigned long dst_addr, pgprot_t pgprot); +int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, + unsigned long *t0sz, void *page); + #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 94fc275cdd21..9df32ba0d574 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -194,7 +194,6 @@ static void *hibernate_page_alloc(void *arg) * page system. */ static int create_safe_exec_page(void *src_start, size_t length, - unsigned long dst_addr, phys_addr_t *phys_dst_addr) { struct trans_pgd_info trans_info = { @@ -203,7 +202,8 @@ static int create_safe_exec_page(void *src_start, size_t length, }; void *page = (void *)get_safe_page(GFP_ATOMIC); - pgd_t *trans_pgd; + phys_addr_t trans_ttbr0; + unsigned long t0sz; int rc; if (!page) @@ -211,13 +211,7 @@ static int create_safe_exec_page(void *src_start, size_t length, memcpy(page, src_start, length); __flush_icache_range((unsigned long)page, (unsigned long)page + length); - - trans_pgd = (void *)get_safe_page(GFP_ATOMIC); - if (!trans_pgd) - return -ENOMEM; - - rc = trans_pgd_map_page(&trans_info, trans_pgd, page, dst_addr, - PAGE_KERNEL_EXEC); + rc = trans_pgd_idmap_page(&trans_info, &trans_ttbr0, &t0sz, page); if (rc) return rc; @@ -230,12 +224,15 @@ static int create_safe_exec_page(void *src_start, size_t length, * page, but TLBs may contain stale ASID-tagged entries (e.g. for EFI * runtime services), while for a userspace-driven test_resume cycle it * points to userspace page tables (and we must point it at a zero page - * ourselves). Elsewhere we only (un)install the idmap with preemption - * disabled, so T0SZ should be as required regardless. + * ourselves). + * + * We change T0SZ as part of installing the idmap. This is undone by + * cpu_uninstall_idmap() in __cpu_suspend_exit(). */ cpu_set_reserved_ttbr0(); local_flush_tlb_all(); - write_sysreg(phys_to_ttbr(virt_to_phys(trans_pgd)), ttbr0_el1); + __cpu_set_tcr_t0sz(t0sz); + write_sysreg(trans_ttbr0, ttbr0_el1); isb(); *phys_dst_addr = virt_to_phys(page); @@ -434,7 +431,6 @@ int swsusp_arch_resume(void) void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; - phys_addr_t phys_hibernate_exit; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); struct trans_pgd_info trans_info = { @@ -462,19 +458,13 @@ int swsusp_arch_resume(void) return -ENOMEM; } - /* - * Locate the exit code in the bottom-but-one page, so that *NULL - * still has disastrous affects. - */ - hibernate_exit = (void *)PAGE_SIZE; exit_size = __hibernate_exit_text_end - __hibernate_exit_text_start; /* * Copy swsusp_arch_suspend_exit() to a safe page. This will generate * a new set of ttbr0 page tables and load them. */ rc = create_safe_exec_page(__hibernate_exit_text_start, exit_size, - (unsigned long)hibernate_exit, - &phys_hibernate_exit); + (phys_addr_t *)&hibernate_exit); if (rc) { pr_err("Failed to create safe executable page for hibernate_exit code.\n"); return rc; @@ -493,7 +483,7 @@ int swsusp_arch_resume(void) * We can skip this step if we booted at EL1, or are running with VHE. */ if (el2_reset_needed()) { - phys_addr_t el2_vectors = phys_hibernate_exit; /* base */ + phys_addr_t el2_vectors = (phys_addr_t)hibernate_exit; el2_vectors += hibernate_el2_vectors - __hibernate_exit_text_start; /* offset */ diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index ded8e2ba0308..527f0a39c3da 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -273,3 +273,52 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, return 0; } + +/* + * The page we want to idmap may be outside the range covered by VA_BITS that + * can be built using the kernel's p?d_populate() helpers. As a one off, for a + * single page, we build these page tables bottom up and just assume that will + * need the maximum T0SZ. + * + * Returns 0 on success, and -ENOMEM on failure. + * On success trans_ttbr0 contains page table with idmapped page, t0sz is set to + * maximum T0SZ for this page. + */ +int trans_pgd_idmap_page(struct trans_pgd_info *info, phys_addr_t *trans_ttbr0, + unsigned long *t0sz, void *page) +{ + phys_addr_t dst_addr = virt_to_phys(page); + unsigned long pfn = __phys_to_pfn(dst_addr); + int max_msb = (dst_addr & GENMASK(52, 48)) ? 51 : 47; + int bits_mapped = PAGE_SHIFT - 4; + unsigned long level_mask, prev_level_entry, *levels[4]; + int this_level, index, level_lsb, level_msb; + + dst_addr &= PAGE_MASK; + prev_level_entry = pte_val(pfn_pte(pfn, PAGE_KERNEL_EXEC)); + + for (this_level = 3; this_level >= 0; this_level--) { + levels[this_level] = trans_alloc(info); + if (!levels[this_level]) + return -ENOMEM; + + level_lsb = ARM64_HW_PGTABLE_LEVEL_SHIFT(this_level); + level_msb = min(level_lsb + bits_mapped, max_msb); + level_mask = GENMASK_ULL(level_msb, level_lsb); + + index = (dst_addr & level_mask) >> level_lsb; + *(levels[this_level] + index) = prev_level_entry; + + pfn = virt_to_pfn(levels[this_level]); + prev_level_entry = pte_val(pfn_pte(pfn, + __pgprot(PMD_TYPE_TABLE))); + + if (level_msb == max_msb) + break; + } + + *trans_ttbr0 = phys_to_ttbr(__pfn_to_phys(pfn)); + *t0sz = TCR_T0SZ(max_msb + 1); + + return 0; +} From patchwork Mon Jan 25 19:19:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5443AC433DB for ; Mon, 25 Jan 2021 19:24:18 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D425F21D79 for ; Mon, 25 Jan 2021 19:24:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D425F21D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pto9FCRTOBBKLggelHZCbt3Nvf+bikZla1j4sK8x068=; b=eiSc2AmK0GrxZsTxbiJaqOzIs4 Ul1NNGU6vNVobCYZeMbFPNQp0XPeUMNE9CXsh6D/WrXwkDnt+ijyRAciZYq8UunEFB8u8kPWpGMUp +3rWQLAhz76HXH96m2LZsSS9J0KxlGm+lpRFwxuEUE7gJjrHZCY0i5lvd5A9ju5bUEmBPCS8LXn7L dMtd0z/sHQ/EpCyPAzWHmKoOxSb9Nk29ffE57LDCM8eP0FSs3sNQs11KKZ6rYbLUBAa8oWzyTCczF DeGxrv1ZfyLNiYA5pgkzCZTpNzx6KwASW4Ms3BQ3MgL1Kv3OrD/RBH9urOkAfTD1gdfNYiyijXEE1 +DQMptWg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47RX-0008Fg-O9; Mon, 25 Jan 2021 19:22:31 +0000 Received: from mail-qt1-x830.google.com ([2607:f8b0:4864:20::830]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Om-0007A0-Pd for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:02 +0000 Received: by mail-qt1-x830.google.com with SMTP id c1so10551568qtc.1 for ; Mon, 25 Jan 2021 11:19:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zFOq/S9iOYCDWpf61SwLzKvAyTai8NEIz55jjvM4EVM=; b=ckXcD39ZeMZdTbgyZrKYzdjdsYWAcQx3VmO7mRxHypwrmjO4e8B0MSbG5sIlFuDpa7 W3rCPkP3Lgu5Lzz/yrTN4X+QZex8zNY8KP5B82WyOu1kph3LCamxEBXe58RAj4qI1vFl niZ2GvFNe2pz6XjLG5sXaa5Q9jjgPm4je/lUH3YbWKZmkGCn8Bh4Yn7/P+MYmvbJ4ox8 ldCHR7bGVIxMH69hJwYJjb377KZXnzdMapSbUePN+4ASS1+puTtcdF7qmAlfiIhpsqXC upgE85LngJaiKuGgdKWtSgT3qxYdiRFPh4vC1yi7clw7nnSYUT2JEVyJExz/S4Qk0CcJ 0Bug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zFOq/S9iOYCDWpf61SwLzKvAyTai8NEIz55jjvM4EVM=; b=bYoRGwytc3YOqAeCRaOB3k+m5xhCpNmVD9Ne8qgpDcdLzqwgawT6rNjpWa7glmnRtX N6EbQHD32FNxBErvMkOOHSMIwX9oTW5bueh2iPjcRX/BQkpnRec3jW+MtlS0oai9skSp Nzv4RwvqisskDtqiMTCyUfVNxaXM5HA+Wf18e254LsHVE/u4lAv1ojRR21SRVozW46rU 3R0EBgt3h3tMNOCHRScCU1/JKUEtT591qbI3Ql+aABEOCH/AOO56dxq3ZrhdyFzkvvzZ 70eZ/ChBAFDz7v9ZnujARzfR0APWstY5VGNfyZKGHdZFkuV8aiGzzZwLM8mwJhhh9ax8 EcZQ== X-Gm-Message-State: AOAM532QlHiB2G6MMn4nS7x0Ku410hgpaGVVmH9c1IalgmfvDmB/RPbo 2zc3UaBJvgIzjyiZ4rIT/BNw4Q== X-Google-Smtp-Source: ABdhPJzOtp3LRX+RA164VIS3opw37sa157PwXQZoQIDOfXAj8170m0WLiADo7/egWOhgpbrd6Sy/vg== X-Received: by 2002:ac8:57c1:: with SMTP id w1mr1954928qta.313.1611602379388; Mon, 25 Jan 2021 11:19:39 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:38 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 09/18] arm64: kexec: move relocation function setup Date: Mon, 25 Jan 2021 14:19:14 -0500 Message-Id: <20210125191923.1060122-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141941_254251_77349D0A X-CRM114-Status: GOOD ( 21.09 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kernel relocation function is configured in machine_kexec() at the time of kexec reboot by using control_code_page. This operation, however, is more logical to be done during kexec_load, and thus remove from reboot time. Move, setup of this function to newly added machine_kexec_post_load(). Because once MMU is enabled, kexec control page will contain more than relocation kernel, but also vector table, add pointer to the actual function within this page arch.kern_reloc. Currently, it equals to the beginning of page, we will add offsets later, when vector table is added. Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/include/asm/kexec.h | 1 + arch/arm64/kernel/machine_kexec.c | 46 +++++++++++++------------------ 2 files changed, 20 insertions(+), 27 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 61530ec3a9b1..9befcd87e9a8 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -95,6 +95,7 @@ static inline void crash_post_resume(void) {} struct kimage_arch { void *dtb; phys_addr_t dtb_mem; + phys_addr_t kern_reloc; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 8096a6aa1d49..a8aaa6562429 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -42,6 +42,7 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -58,6 +59,22 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } +int machine_kexec_post_load(struct kimage *kimage) +{ + void *reloc_code = page_to_virt(kimage->control_code_page); + + memcpy(reloc_code, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = __pa(reloc_code); + + /* Flush the reloc_code in preparation for its execution. */ + __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); + + return 0; +} + /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -143,8 +160,6 @@ static void kexec_segment_flush(const struct kimage *kimage) */ void machine_kexec(struct kimage *kimage) { - phys_addr_t reboot_code_buffer_phys; - void *reboot_code_buffer; bool in_kexec_crash = (kimage == kexec_crash_image); bool stuck_cpus = cpus_are_stuck_in_kernel(); @@ -155,31 +170,8 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - reboot_code_buffer_phys = page_to_phys(kimage->control_code_page); - reboot_code_buffer = phys_to_virt(reboot_code_buffer_phys); - kexec_image_info(kimage); - /* - * Copy arm64_relocate_new_kernel to the reboot_code_buffer for use - * after the kernel is shut down. - */ - memcpy(reboot_code_buffer, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - - /* Flush the reboot_code_buffer in preparation for its execution. */ - __flush_dcache_area(reboot_code_buffer, arm64_relocate_new_kernel_size); - - /* - * Although we've killed off the secondary CPUs, we don't update - * the online mask if we're handling a crash kernel and consequently - * need to avoid flush_icache_range(), which will attempt to IPI - * the offline CPUs. Therefore, we must use the __* variant here. - */ - __flush_icache_range((uintptr_t)reboot_code_buffer, - (uintptr_t)reboot_code_buffer + - arm64_relocate_new_kernel_size); - /* Flush the kimage list and its buffers. */ kexec_list_flush(kimage); @@ -193,7 +185,7 @@ void machine_kexec(struct kimage *kimage) /* * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the reboot_code_buffer which contains a copy of + * transfer control to the kern_reloc which contains a copy of * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel * uses physical addressing to relocate the new image to its final * position and transfers control to the image entry point when the @@ -203,7 +195,7 @@ void machine_kexec(struct kimage *kimage) * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(reboot_code_buffer_phys, kimage->head, kimage->start, + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, kimage->arch.dtb_mem); BUG(); /* Should never get here. */ From patchwork Mon Jan 25 19:19:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70149C433E0 for ; Mon, 25 Jan 2021 19:28:55 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1301E224DF for ; Mon, 25 Jan 2021 19:28:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1301E224DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=G401h2VXwk+/sz6hueRezidQ1PAYED3y2DCTf2Ohprs=; b=JxeLBJBJPp0d7sO+v3aFaRzpcz 8yvNfnG77OLlVrzcr7uVaoyiJbFYnlgH7sxdwf6ETulYpZvEYrjvfScQz0LgM0wIyVTvDMQvVMBHJ 7b0fY1xkxN0d/jqX/qgATMNeAdenAeqHXcLc+FKZd1E6xt3qHPQkkAPAn9U2Vl+gPfcHvh8LLFYD/ 6t/YxqlywV1GSTCtkbkBjSCTo+1xTPJEfW051Jmv5DaU70oD68vPUjmqIhbk+2kQEMIXQFq5TZ3Wy EpCXZOv/0QZD/9eXQAlo+bPzrC/tMf3HSAECMRPIAQyOYCdOJahAD1MltgySFdvBjd7C0fChR35LP 9k3MZMRw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47VN-0001hU-Qx; Mon, 25 Jan 2021 19:26:30 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Q3-0007eZ-CB for linux-arm-kernel@merlin.infradead.org; Mon, 25 Jan 2021 19:20:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=5oItsXz80VGMkIK9n/+4yNTlc3wm3VkQinupbSJHsbU=; b=ZP33txIsk3qoNdFCoVtuAbOGck C9sgiPzLCR+spCygvCl1XOX5WcvveLSYDgHOc6/G+XjKG6cKA6nmOZt3o7PskZ1jSooCvPfWdAI8E YtqlfGLPCBsFf8ciqV8vUs8wJwrDSambaWqyleNyX1ybDfLujoCZgGlb2LDQeX/soWLkBRIOcZlGD 0fQTvuou0lrlrGl6cheDrs5jdlyxjaCPYHD4BMQzMCk8r3hnjjcV/4CwxJQHDBG7vlywl9IZQIGj+ l+B0YB3J5BW6NVCbpHxVILnLZ7IlrlVnpWhZvn5ZpPXmYSDc54pKfgvzoreftF4kgvac7tFIkQbCU DJwG3lgQ==; Received: from mail-qt1-x82a.google.com ([2607:f8b0:4864:20::82a]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1l47Ow-004Z3m-AG for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:29 +0000 Received: by mail-qt1-x82a.google.com with SMTP id t14so6905975qto.8 for ; Mon, 25 Jan 2021 11:19:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5oItsXz80VGMkIK9n/+4yNTlc3wm3VkQinupbSJHsbU=; b=PZYpwfrFvSlaUCkpDD05HG1cJCDlDOf+CUaa46d/3Sf4IeCJvLQ226vo3BkaPzPeVK JZbb1ZDcCU7ye9jeVV1OG5khyvwWUIfNQO1CqaBvXmiHejT5L+rDVKo6Rm4kpcbugs9G ne9oGFKvOQsVFYgujx0RQze9i3AqNviHN45Bb0YFDIo741ZRjS1YRjfDGXn69eAi4PAV JIhvMTg3YI2g+4P9Vz7u/dSGxJd8MHa37mdnHAj1kcTor+MsBSmxNUee4ll41ONPNKp7 VCcG8A9NH9nt8ykRu91p3nz82wHjF6r3JWAGbffQLeTodD/nuGq7TYWmm9RrzYzoTo1a sSsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5oItsXz80VGMkIK9n/+4yNTlc3wm3VkQinupbSJHsbU=; b=Zte3uNaUEjdLu5nKX/s2uWKo9A8qpBDT1Ab5KrhzZhmHESByPK9goy35J0bXGQw3RB qCvfsD0lFi6Wb5PGGFyjNkzsKrzXzHHGKlYVEVEwiDgvpQmy5s8kRFoNkodXduS4Bec1 zSWBPeLHKKcMJm9Q4yqTqigXDQzj8TQxT6QsTrc/fLm9V+6TAgk+cD3CDzBgKqPzxJwR bqEu+Pd8P0fBLiUykC0O27lSrdb9I7uY7KV7r/Bdr3fpJev1Te1XtEkmgZyWpR2XNuq6 X4D8F12vwKoUiaC60f561lC64IUgV2T9UTvcEppmfLOrtYebkYGWOJGFzqblV9jnjKpS bm1g== X-Gm-Message-State: AOAM532sJorekFetRG7Y/0ydRiMpKWnJQWwIzwMfSUYtuQgE0O33YgwS BbdL4VIaepgOdNAeA+9R9FXts6kwMgYsmg== X-Google-Smtp-Source: ABdhPJydaAMyVai0d0oDbWxmrL/nu0WOLUZ8DiszpSfhSyLHotQkTYoRRSqhOnxoO+wBONtIKJSi8w== X-Received: by 2002:ac8:490e:: with SMTP id e14mr1966761qtq.99.1611602380917; Mon, 25 Jan 2021 11:19:40 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:40 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 10/18] arm64: kexec: call kexec_image_info only once Date: Mon, 25 Jan 2021 14:19:15 -0500 Message-Id: <20210125191923.1060122-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_192028_783879_FB3C07B5 X-CRM114-Status: GOOD ( 11.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec_image_info() is called during load time, and right before kernel is being kexec'ed. There is no need to do both. So, call it only once when segments are loaded and the physical location of page with copy of arm64_relocate_new_kernel is known. Signed-off-by: Pavel Tatashin Acked-by: James Morse --- arch/arm64/kernel/machine_kexec.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a8aaa6562429..90a335c74442 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -66,6 +66,7 @@ int machine_kexec_post_load(struct kimage *kimage) memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); + kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); @@ -84,8 +85,6 @@ int machine_kexec_post_load(struct kimage *kimage) */ int machine_kexec_prepare(struct kimage *kimage) { - kexec_image_info(kimage); - if (kimage->type != KEXEC_TYPE_CRASH && cpus_are_stuck_in_kernel()) { pr_err("Can't kexec: CPUs are stuck in the kernel.\n"); return -EBUSY; @@ -170,8 +169,6 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - kexec_image_info(kimage); - /* Flush the kimage list and its buffers. */ kexec_list_flush(kimage); From patchwork Mon Jan 25 19:19:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D580C433E0 for ; Mon, 25 Jan 2021 19:24:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5F58121D79 for ; Mon, 25 Jan 2021 19:24:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F58121D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7Pq5jkX25TbzueftKKl+uOz9L9QLGUOQeVZ4dgXxFnQ=; b=Tn2Bl7nyUl7to0wPxBtS5Th1r+ jTiE78KPoM+4BeNFVPvlkO5hb+ywaOLGqzWW2Tf/+iHc/zv96WszXWnbm0xkc5qV5RQjD0zBxOdE3 pOQA2CZ564Fc/jQ0EzrF3mGEzxkAlYM7FBZ9iUZENdqbSsraudKO3M4iQ/X+Wsh0MPiALD6p0ObP1 dRdFB9mKk2OijuJ7wXkl8aWxyXlX/zoetRNGMn/WoyxOubylPZsIjH2jt9HdkQFEdSWa1zP5OiT4I 8Q2UTmmZxCS9cb2wpixlnjg8zKfHLSgtKwz5KAUFGGsSDLv36xDO9EmNXWNIhLXUhbEGogf3GmMsg iUOeKeZg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Rr-0008NT-37; Mon, 25 Jan 2021 19:22:51 +0000 Received: from mail-qt1-x832.google.com ([2607:f8b0:4864:20::832]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Oq-0007BS-EI for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:07 +0000 Received: by mail-qt1-x832.google.com with SMTP id e17so10529433qto.3 for ; Mon, 25 Jan 2021 11:19:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fYtwEkSD1g4zgc3mwZlXud5vJEM5wBsBl27BfuiCCGo=; b=U2AIVAJVNsrZwMQGm3jBTprnYlMZE2r0f07KZ/zh8a2HtVegm5Vrw9+CSVp/lI7Olr b3BHv9VX9zYpOGGmLuvOmzIVhSJR0S+/r/mkSAONsXUZHdQBInsFN0VBlaumZAGrgg/F s4k2BJaAFHTTlqUf+cDOFIXSAPXmQj1K0VWS9M4Dhs8r6V23ddNJ1CPgIA5l3dp1IaPX 9gBIB7dFsc7RCkiDRTBhE9rOUkOsra1PlaC4eXs98zEiGF268+JK7j0dEP/u53Tvk+FU fIy+tzeA6q2grkIIzOJ3NToRGJ3bHoTlX+Ix/f+DR7XKJX/ZM7gvINzw0/dtPHUm1kYk PTgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fYtwEkSD1g4zgc3mwZlXud5vJEM5wBsBl27BfuiCCGo=; b=mDEgTzeX3SDh6dqORIoJi1yki22/Wv06gXhBAsKK+ZEHoN66bliKQ+xmZrs1zWwQTQ FIeXGNeNU5CCtnH3ywgSE8YvVgG2SiF29YEGVoZbUFk0r+1uZ+8mPq+TjHXVxRVvnTls N8F36tZFS9tfoIYeHKPdv2Oo6dy5BFc+jPqeGlkIHmCaneS2PqTVDGSjlkJKS/EcOUu3 3SFPlzpXKWGJwX3JuBIZoR4TOQ9L/7zbJafyain6OFuQ2vlyFQ/cXkUsZZhQmi6g6J5n G6OTa0tHS+lS324MFFblYq5mPIB6BsXss0koGUxP9ra1M8skw+CKmpp4lg0lv11oj7zC 7wIA== X-Gm-Message-State: AOAM5308L/BhtjALLmU7ZeotjOZ+2WJM+zx1aSy+mlDr1QQivq2Z7xVs UqGZ5YhJFg2wB6AoJzcSKbTnyQ== X-Google-Smtp-Source: ABdhPJyI8k+86XaV/8c2jn/BKReyE9eDvlppb7XnDOdVY2z1s16lwW29T78nGb4fSsM7KDSok6C8eQ== X-Received: by 2002:ac8:a82:: with SMTP id d2mr1908195qti.343.1611602382398; Mon, 25 Jan 2021 11:19:42 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:41 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 11/18] arm64: kexec: arm64_relocate_new_kernel clean-ups and optimizations Date: Mon, 25 Jan 2021 14:19:16 -0500 Message-Id: <20210125191923.1060122-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141944_556979_86CF5519 X-CRM114-Status: GOOD ( 14.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In preparation to bigger changes to arm64_relocate_new_kernel that would enable this function to do MMU backed memory copy, do few clean-ups and optimizations. These include: 1. Call raw_dcache_line_size() only when relocation is actually going to happen. i.e. kdump type kexec, does not need it. 2. copy_page(dest, src, tmps...) increments dest and src by PAGE_SIZE, so no need to store dest prior to calling copy_page and increment it after. Also, src is not used after a copy, not need to copy either. 3. For consistency use comment on the same line with instruction when it describes the instruction itself. 4. Some comment corrections Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/relocate_kernel.S | 36 +++++++---------------------- 1 file changed, 8 insertions(+), 28 deletions(-) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 84eec95ec06c..462ffbc37071 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -17,28 +17,24 @@ /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * - * The memory that the old kernel occupies may be overwritten when coping the + * The memory that the old kernel occupies may be overwritten when copying the * new image to its final location. To assure that the * arm64_relocate_new_kernel routine which does that copy is not overwritten, * all code and data needed by arm64_relocate_new_kernel must be between the * symbols arm64_relocate_new_kernel and arm64_relocate_new_kernel_end. The * machine_kexec() routine will copy arm64_relocate_new_kernel to the kexec - * control_code_page, a special page which has been set up to be preserved - * during the copy operation. + * safe memory that has been set up to be preserved during the copy operation. */ SYM_CODE_START(arm64_relocate_new_kernel) - /* Setup the list loop variables. */ mov x18, x2 /* x18 = dtb address */ mov x17, x1 /* x17 = kimage_start */ mov x16, x0 /* x16 = kimage_head */ - raw_dcache_line_size x15, x0 /* x15 = dcache line size */ mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Check if the new image needs relocation. */ tbnz x16, IND_DONE_BIT, .Ldone - + raw_dcache_line_size x15, x0 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -57,34 +53,18 @@ SYM_CODE_START(arm64_relocate_new_kernel) b.lo 2b dsb sy - mov x20, x13 - mov x21, x12 - copy_page x20, x21, x0, x1, x2, x3, x4, x5, x6, x7 - - /* dest += PAGE_SIZE */ - add x13, x13, PAGE_SIZE + copy_page x13, x12, x0, x1, x2, x3, x4, x5, x6, x7 b .Lnext - .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination - - /* ptr = addr */ - mov x14, x12 + mov x14, x12 /* ptr = addr */ b .Lnext - .Ltest_destination: tbz x16, IND_DESTINATION_BIT, .Lnext - - /* dest = addr */ - mov x13, x12 - + mov x13, x12 /* dest = addr */ .Lnext: - /* entry = *ptr++ */ - ldr x16, [x14], #8 - - /* while (!(entry & DONE)) */ - tbz x16, IND_DONE_BIT, .Lloop - + ldr x16, [x14], #8 /* entry = *ptr++ */ + tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ .Ldone: /* wait for writes from copy_page to finish */ dsb nsh From patchwork Mon Jan 25 19:19:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42BA2C433DB for ; Mon, 25 Jan 2021 19:45:48 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C758F2251F for ; Mon, 25 Jan 2021 19:45:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C758F2251F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=6Rb9ztuAj51wsXtQj1Z96DHR9hriY/PsrHnNV3lddaQ=; b=FwdioyOYAbHvjWCsvDc4J4s9HC zsUpB6SuPjYfHqG634NJEOLiDeVycdMaO/6mSLnZS6/IYZS78VG2F2k3WYCLumm1I6b2qzupDhiKd Olsk2PIliydNrnMES3u6cXOJq+WJEyBGeRw+YNjKuCR6im+7fSMcyxjfxs5zXCEuJFTyCZRN3Vxp+ IQGaywtk+5TKp+oUZnso7qoB9YmVuLmaGEGoe+yKmWLy7pSihwmm7qmVKFdto7eqAYEdF6GyKvCUw TnQxoHlCb5kyJcVKvL5LyTBjOt2xNUjkMPMWZuKVpUJ0bvZr+7l9nqEFjTAEPEtd4T6sib2qKIhs+ ORN8OXuw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47mb-0005zr-8d; Mon, 25 Jan 2021 19:44:17 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47mX-0005aF-Ft for linux-arm-kernel@merlin.infradead.org; Mon, 25 Jan 2021 19:44:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=tiSMngDA2ABiEtmeVcai+dAKxWkYAh1htex3hJFUjpk=; b=JuimPJqLSeGNobAii6k71IZ9Nj 6KPR+WoVfHsI7WMxroKioWxD/VqM5EUwUp4mr4recBzQNGzuMqulTHJHXotv6tEFKj2EfaX9EVGrz Q84IYPaJx6aX+65G2W8qN5vkik0bRbBmYIfNZ413dH/XAhv74gu5kl7hHwBN1KY7TaMghgoe9xZ2Q RupYGpsRr955GWgp5jITMJbYp3eB1r3qDqXjoy2xnlRv/FcO/vjQUtGitbMAoiRiLhso4vj+PPjRV kUrtNimyQEOJIHxyxy9TsWxesGJhHYFZWCLNnVh3kXF9rW1pl9uPjWflnpmffHj8guDe4Xez808F5 ZbWiBJDg==; Received: from mail-qt1-x82e.google.com ([2607:f8b0:4864:20::82e]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1l47PA-004Z4U-KX for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:40 +0000 Received: by mail-qt1-x82e.google.com with SMTP id e17so10529524qto.3 for ; Mon, 25 Jan 2021 11:19:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=tiSMngDA2ABiEtmeVcai+dAKxWkYAh1htex3hJFUjpk=; b=ViZk57Ck4HUJG00kgKhICCWv1eeMV2KNwvtWbX+xyUO32m6U7YnxxkY3kd7tyt7Ng+ XrR64IVqIQXWuSCO/zvWvsnc220koQtIQ6BeCpcN3m5qJtxE8hQd+lqOk1SSUws39mJ5 2UU0xjoEra/u8cu7+zZxLeSEhnlev+FK43G7b4wigJE0ZW7Fxmwc4f+5WKP6PGVvLDKD CevQSfsHTtUhlePppvfpl/vrZuOFENi0k/k70Vz6s/LnbGLcFohCKRaCw+y6dHHnaupf dxxTdXFgaF41aR4eCDsfpBOzusaAodDscncPCsGYESYFYQ62GgZzY2GbN4qZLZDAgGys GB3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tiSMngDA2ABiEtmeVcai+dAKxWkYAh1htex3hJFUjpk=; b=dkJQLxpvq5TbAaBGJNTO3HwjaQ2qDRSA50V1qgwzn0a0d98greDpMRpAuoKsmym2Mz KBmnfv5fGLQ8gV86qnWayxXFxX9QW+s/4jIuseakGbhDtZ+CwZzTljfdDh0ujpIVpBOW QFG/iKh+ltXSGJYmXMeF8e/Mwdc0gNQ2l2KcpOnLStbQ+CodXxsmytx1+DKXEVLFicyB Sd8IM1tNqo2R6l1vOdosSJwRNAXDI38FW9Rm2W6wLdn8Q1uQGC9/0/sEVreZTww/e9d/ lDOO0+RcLy45fH57d8Pb3LtliiUq4LD1rzgwxUeyxnFNoMHXNY34iGycvt1SPNgCtJ/B 1e+w== X-Gm-Message-State: AOAM533NHg6Zd2pPvC3n1aSLu6aCSAnJQG4MVY2lWxz+/Qitd7TIPTq7 OMPnB6DtITZxB976oojQ1iOLXg== X-Google-Smtp-Source: ABdhPJzn7ahPtOGX/KTR0Kc/e2td2Abo5VWVmrRWEDIeC/ylKNtmaaW0TcaryJ7Q6EPO8SaSfFQgTQ== X-Received: by 2002:ac8:4d93:: with SMTP id a19mr1928434qtw.356.1611602383884; Mon, 25 Jan 2021 11:19:43 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:43 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 12/18] arm64: kexec: arm64_relocate_new_kernel don't use x0 as temp Date: Mon, 25 Jan 2021 14:19:17 -0500 Message-Id: <20210125191923.1060122-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_192040_131431_F8E1195F X-CRM114-Status: GOOD ( 10.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org x0 will contain the only argument to arm64_relocate_new_kernel; don't use it as a temp. Reassigned registers to free-up x0 so we won't need to copy argument, and can use it at the beginning and at the end of the function. Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/kernel/relocate_kernel.S | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 462ffbc37071..b78ea5de97a4 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -34,7 +34,7 @@ SYM_CODE_START(arm64_relocate_new_kernel) mov x13, xzr /* x13 = copy dest */ /* Check if the new image needs relocation. */ tbnz x16, IND_DONE_BIT, .Ldone - raw_dcache_line_size x15, x0 /* x15 = dcache line size */ + raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ @@ -43,17 +43,17 @@ SYM_CODE_START(arm64_relocate_new_kernel) tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x0, x13 - add x20, x0, #PAGE_SIZE + mov x2, x13 + add x20, x2, #PAGE_SIZE sub x1, x15, #1 - bic x0, x0, x1 -2: dc ivac, x0 - add x0, x0, x15 - cmp x0, x20 + bic x2, x2, x1 +2: dc ivac, x2 + add x2, x2, x15 + cmp x2, x20 b.lo 2b dsb sy - copy_page x13, x12, x0, x1, x2, x3, x4, x5, x6, x7 + copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination From patchwork Mon Jan 25 19:19:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E0CDC433DB for ; Mon, 25 Jan 2021 19:44:59 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BA3962251F for ; Mon, 25 Jan 2021 19:44:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA3962251F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=a6ywbOxFuOV9Zz0N8FvTx/aqE46ipubT08+o0vaw9IE=; b=FndKli2asEwY1Qbfl4UnQXUxDO rj98R33BfXrB+xXBFiuV0Sbg2AmzY8KmbIdq0JlWF4FuQfBNN3Pz2TDMU4rQb2Mu+j52Z7qIfR5yM AENl0bvPJRjgrKGUo9a6EnMSerR148f+uUqto7hmBCzmW4pYihZLKmhCCalwHZG6RkMHIlioiBBJU FZZmGOMJiV6pH6jNyhNwRXJ1I+02w9dGj0TNPgKjIDoePdBVP4d6uxDFsihLUx4Q5uDxZylzoAKxd SZz5Ky5FV57TXVbQMVCindzuCTtItXC2HmzBPaa8fvk32XOyswDcDwNlpT5IRnb8bwbcqgmWLDBBI 7a30MBMg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47lQ-0005kV-N7; Mon, 25 Jan 2021 19:43:04 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47lN-0005aF-C1 for linux-arm-kernel@merlin.infradead.org; Mon, 25 Jan 2021 19:43:01 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: Content-Type:Content-ID:Content-Description; bh=ASygAqr57jUZ78nCjocRBoBMKp4WVYWpgcDuzWEf5/U=; b=TegYRzyHhgZpxq4Mfs0xTbQXrx N3IKOJXTl5C6vADY6BQTjyekvxG3CzyPoSV9DbYB0Mee2r/rco5hM510/ySUfzia6KjFkwPNtDgG8 ePiproTFqZVxEqtEHybQQs+a/D+2G9Ev+Cj0B5cqdhXxNY/nQr9DnQONIJcpYvZQD1TsMzMzqY0Eh 742YklhgswMlhg2rKwnhtw3PaCnkn3gaxTuB817skj+SEEirP+0aoGGfL3kns7Sn5EuZ6bO6xNAfE ym0zTewQu9pQF97Kw0WETu+/1sYpqSlMX7bEdEdI65QbZxXKESOXck6jLk2vbjY2VBOex5+uqZNo7 gKC/zofw==; Received: from mail-qv1-xf2b.google.com ([2607:f8b0:4864:20::f2b]) by casper.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1l47PA-004Z4c-Ka for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:40 +0000 Received: by mail-qv1-xf2b.google.com with SMTP id l14so4656450qvp.2 for ; Mon, 25 Jan 2021 11:19:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ASygAqr57jUZ78nCjocRBoBMKp4WVYWpgcDuzWEf5/U=; b=kwanBJbnq5DMQn3NNdwALjHbGNIQHRgVMzw/Xc+aSak11cDNL99h8N7xtx7c8gngNP e3KAIrRTpyBGeF9fdLyOiJ3hPzW5AZaC88I9XyuG+3a1+/V4dltXX9U7nM7+9pEIB3Me 86UUqy73HTQ2qKV07Y9Mb83Y/TbL1+Ig8KWvxl5bPCfQ5cmcvHkndt40H4FloM3GoHcQ EFYUj2luKiryZhIHw/bvSX5V6SIKGJwx6oKh8SXtCholJ29JzgmOGyKpqHGvtbcO59Cj mW9VwXzYubwJ9ABOmue84kkNoRHqo7cHaIru9BKsgyCA6UBWVqBw9hwfaqR8LI9kWodg fCwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ASygAqr57jUZ78nCjocRBoBMKp4WVYWpgcDuzWEf5/U=; b=qFgGWlmK+tGHsn7vUEkmxEHSStNAnGih3r7JOw0ZKyngqPxagDfdsyJ5ErjIrTNtPv GX1UaEjL1XPUMTJBBUbvDitVrz2jwoHHeQSv2WsLmbUKR/mWJ/f/2ngxbafP5cXD5V9/ UCxkiEiGu98/qs/1pEDwB5BD5UsH3TYtsm8VNx+eKNWds+FhnpyL4OoKCSK82GyMXgKn BhDq7kBZFkzqXuX2Z+qn29IhiKjjUNk2mYC8hMF2kAcgJJMHj3QNwYhJjh/75AzvQ2ZI 2x3P738WZ45Ni5TdaqFQqfFoxtzpcyUMabOYQSW7Oqeehkf5+VGDDrUM9V8WACmBfObp zSSQ== X-Gm-Message-State: AOAM531rSZ5WFzStWMUNSxy7He7Wy4AHzFiOF6xA+GKsRLG+jnFifsGi Jauq8gPCrloVkcrUCG8AfpbPr4hHrJH41w== X-Google-Smtp-Source: ABdhPJyrusG+lMDL+aP7VGwdEsWxy6Ayk8cCYPTOqt0xOkOzZ3Z/dofRUCKSU1J661lfcVzfB4nlDQ== X-Received: by 2002:a0c:b99c:: with SMTP id v28mr2278172qvf.12.1611602385413; Mon, 25 Jan 2021 11:19:45 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:44 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 13/18] arm64: kexec: add expandable argument to relocation function Date: Mon, 25 Jan 2021 14:19:18 -0500 Message-Id: <20210125191923.1060122-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_192040_222995_D0E1AFD0 X-CRM114-Status: GOOD ( 24.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments (hypervisor abi). And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Add a new struct: kern_reloc_arg, and place it in kexec safe page (i.e memory that is not overwritten during relocation). Thus, make arm64_relocate_new_kernel to only take one argument, that contains all the needed information. Note: Another benefit of allowing this function to accept more arguments, is that kernel can actually accept up to 4 arguments (x0-x3), however currently only one is used, but if in the future we will need for more (for example, pass information about when previous kernel exited to have a precise measurement in time spent in purgatory), we won't be easilty do that if arm64_relocate_new_kernel can't accept more arguments. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 18 ++++++++++++++++++ arch/arm64/kernel/asm-offsets.c | 9 +++++++++ arch/arm64/kernel/cpu-reset.S | 11 +++-------- arch/arm64/kernel/cpu-reset.h | 8 +++----- arch/arm64/kernel/machine_kexec.c | 27 +++++++++++++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 21 ++++++++------------- 6 files changed, 66 insertions(+), 28 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 9befcd87e9a8..990185744148 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,12 +90,30 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +/* + * kern_reloc_arg is passed to kernel relocation function as an argument. + * head kimage->head, allows to traverse through relocation segments. + * entry_addr kimage->start, where to jump from relocation function (new + * kernel, or purgatory entry address). + * kern_arg0 first argument to kernel is its dtb address. The other + * arguments are currently unused, and must be set to 0 + */ +struct kern_reloc_arg { + phys_addr_t head; + phys_addr_t entry_addr; + phys_addr_t kern_arg0; + phys_addr_t kern_arg1; + phys_addr_t kern_arg2; + phys_addr_t kern_arg3; +}; + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { void *dtb; phys_addr_t dtb_mem; phys_addr_t kern_reloc; + phys_addr_t kern_reloc_arg; /* Core ELF header buffer */ void *elf_headers; unsigned long elf_headers_mem; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 301784463587..6067a288f568 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -23,6 +23,7 @@ #include #include #include +#include int main(void) { @@ -150,6 +151,14 @@ int main(void) DEFINE(PTRAUTH_USER_KEY_APGA, offsetof(struct ptrauth_keys_user, apga)); DEFINE(PTRAUTH_KERNEL_KEY_APIA, offsetof(struct ptrauth_keys_kernel, apia)); BLANK(); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KEXEC_KRELOC_HEAD, offsetof(struct kern_reloc_arg, head)); + DEFINE(KEXEC_KRELOC_ENTRY_ADDR, offsetof(struct kern_reloc_arg, entry_addr)); + DEFINE(KEXEC_KRELOC_KERN_ARG0, offsetof(struct kern_reloc_arg, kern_arg0)); + DEFINE(KEXEC_KRELOC_KERN_ARG1, offsetof(struct kern_reloc_arg, kern_arg1)); + DEFINE(KEXEC_KRELOC_KERN_ARG2, offsetof(struct kern_reloc_arg, kern_arg2)); + DEFINE(KEXEC_KRELOC_KERN_ARG3, offsetof(struct kern_reloc_arg, kern_arg3)); #endif return 0; } diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 37721eb6f9a1..bbf70db43744 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,14 +16,11 @@ .pushsection .idmap.text, "awx" /* - * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for - * cpu_soft_restart. + * __cpu_soft_restart(el2_switch, entry, arg) - Helper for cpu_soft_restart. * * @el2_switch: Flag to indicate a switch to EL2 is needed. * @entry: Location to jump to for soft reset. - * arg0: First argument passed to @entry. (relocation list) - * arg1: Second argument passed to @entry.(physical kernel entry) - * arg2: Third argument passed to @entry. (physical dtb address) + * arg: Entry argument * * Put the CPU into the same state as it would be if it had been reset, and * branch to what would be the reset vector. It must be executed with the @@ -47,9 +44,7 @@ SYM_CODE_START(__cpu_soft_restart) hvc #0 // no return 1: mov x8, x1 // entry - mov x0, x2 // arg0 - mov x1, x3 // arg1 - mov x2, x4 // arg2 + mov x0, x2 // arg br x8 SYM_CODE_END(__cpu_soft_restart) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..7a8720ff186f 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -11,12 +11,10 @@ #include void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); + unsigned long arg); static inline void __noreturn cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) + unsigned long arg) { typeof(__cpu_soft_restart) *restart; @@ -25,7 +23,7 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(el2_switch, entry, arg); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..679db3f1e0c5 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -43,6 +43,7 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" kern_reloc_arg: %pa\n", &kimage->arch.kern_reloc_arg); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -59,19 +60,42 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); + struct kern_reloc_arg *kern_reloc_arg = kexec_page_alloc(kimage); + + if (!kern_reloc_arg) + return -ENOMEM; memcpy(reloc_code, arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); kimage->arch.kern_reloc = __pa(reloc_code); + kimage->arch.kern_reloc_arg = __pa(kern_reloc_arg); + kern_reloc_arg->head = kimage->head; + kern_reloc_arg->entry_addr = kimage->start; + kern_reloc_arg->kern_arg0 = kimage->arch.dtb_mem; kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); + __flush_dcache_area(kern_reloc_arg, sizeof(struct kern_reloc_arg)); return 0; } @@ -192,8 +216,7 @@ void machine_kexec(struct kimage *kimage) * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, kimage->arch.kern_reloc_arg); BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index b78ea5de97a4..c92228aeddca 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -8,7 +8,7 @@ #include #include - +#include #include #include #include @@ -26,13 +26,8 @@ * safe memory that has been set up to be preserved during the copy operation. */ SYM_CODE_START(arm64_relocate_new_kernel) - /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ /* Check if the new image needs relocation. */ + ldr x16, [x0, #KEXEC_KRELOC_HEAD] /* x16 = kimage_head */ tbnz x16, IND_DONE_BIT, .Ldone raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: @@ -73,12 +68,12 @@ SYM_CODE_START(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 - mov x1, xzr - mov x2, xzr - mov x3, xzr - br x17 - + ldr x4, [x0, #KEXEC_KRELOC_ENTRY_ADDR] /* x4 = kimage_start */ + ldr x3, [x0, #KEXEC_KRELOC_KERN_ARG3] + ldr x2, [x0, #KEXEC_KRELOC_KERN_ARG2] + ldr x1, [x0, #KEXEC_KRELOC_KERN_ARG1] + ldr x0, [x0, #KEXEC_KRELOC_KERN_ARG0] /* x0 = dtb address */ + br x4 SYM_CODE_END(arm64_relocate_new_kernel) .align 3 /* To keep the 64-bit values below naturally aligned. */ From patchwork Mon Jan 25 19:19:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8929AC433DB for ; Mon, 25 Jan 2021 19:26:10 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DE88921D79 for ; Mon, 25 Jan 2021 19:26:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE88921D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=5+oI0Cyas0ypQoYbjUnCAnh58tcyHC22teXAVnJeNpw=; b=IsqpERnM9YDGPxvN4GpbHDTVjC rUS4/MMXnOQbzGIFzQ8uu4TnheTA2rIEdwrTR6AtikpxbyEv6w1UZft7/RNI3y3RW4ZhwxnOd2Zun JJ2XcgicM76RzTxVhbYW6C2wBJc1e27cM8h38Mtf28kvPGvxJS5O3IaV7ZkAG6mfBX9FesciffyVl LqcsFFXa0bf21iUrYGTGeVCdajA9CzvA8yIoNEuB1PSss6bUpGGIXFuLox25gdbkCwB7zn0s3FcIx NcsoF1kDyIpMg1VzRT95xDB49OF76gnXP4Ad0Fx07VXfhpETQVxZTjKVPbEU/PXn/ScZD39lAMIUz Uh8RDD4Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Sk-0000RQ-0e; Mon, 25 Jan 2021 19:23:46 +0000 Received: from mail-qv1-xf33.google.com ([2607:f8b0:4864:20::f33]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Ou-0007Ca-Fc for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:14 +0000 Received: by mail-qv1-xf33.google.com with SMTP id l14so4656494qvp.2 for ; Mon, 25 Jan 2021 11:19:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=iM7vyMjvp8HFtXYgFN4DAarGfV3AZjiwX6w+vbwvdyg=; b=cNVMTo7wVOFzkosxcDfMWZ2s1tXW6ofVvoFzZeHrw+FNJleyDydfxbhz3EfNsvpFJ/ 2s2rNi8tlVz1731hYXPPp+grZOrvK99ScwJsmxiKRan0biFSicPEerEV/fP0VD2+s3c0 Q1CpZE03nirV+yKYJKAtAexxmsocy0Px9UBV7F6Y40euZaJWsR3luw71jBO7izoGBRQx 6tTMZCqVvUdXzUTfrLOG5sToHBzDtqGXMuKPbC+C/uKPt/qHUepYF71bZy78jfYg1Yd2 z1w0Pu0vk8UauVFL/7IgUBGL2IigtR4ZgVXHJa5k4bOOK6+3w66dkb66fuid43YHPtow s98g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iM7vyMjvp8HFtXYgFN4DAarGfV3AZjiwX6w+vbwvdyg=; b=OSwtvYmAQzu1uyDhoCVQmYiBGNKdK+pBR2Wp59IWkBm7K3OLsxN3S+qWBCAy5m0GJ2 7wesLvKdhXkJ5m4LQDgBDhCGPYfykmn0z3/kD6gzQ+WPkuZhlysfyRGN03VAZkr7oX6e TCLUpWcNdf9BZISuT1skMex4kHAF46cV1tq2lQbqhO/YJO6PHB9pEJMFJrmUPD2kGOaw 9y5p85dwEn8CBwPsDEQC4+SSRrOHEP0Pv6EJJkfBz+fABHZN18krPNAZlFQZtZQ8KaMe lCvFMwHYRc96/gHfeHu94bRnW5YFImsJEGBKks9W5s9Cvj+9YXhjfu7bLDG6xqpraPR/ qR/A== X-Gm-Message-State: AOAM530Bdw3lTtQ7wwUz3bmd5ZA4QFI/1dzLf6WGqxspvh9nxrgizXGW L6Aa0MQbiF0f7YEKcvsmHFU6kg== X-Google-Smtp-Source: ABdhPJxSLliyq8RV2Enm412pyUsXXb9GvTpxItvdOl+cT7BuQwZg2qzmybsh1JfoC5G0oaCSxQ/2hw== X-Received: by 2002:a0c:c38e:: with SMTP id o14mr2264022qvi.29.1611602386886; Mon, 25 Jan 2021 11:19:46 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:46 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 14/18] arm64: kexec: use ld script for relocation function Date: Mon, 25 Jan 2021 14:19:19 -0500 Message-Id: <20210125191923.1060122-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141948_611629_0F788520 X-CRM114-Status: GOOD ( 17.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, relocation code declares start and end variables which are used to compute it size. The better way to do this is to use ld script incited, and put relocation function in its own section. Soon, relocation function will share the same page with EL2 vectors. So, proper marking is needed. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 4 ++++ arch/arm64/include/asm/sections.h | 1 + arch/arm64/kernel/machine_kexec.c | 17 ++++++++--------- arch/arm64/kernel/relocate_kernel.S | 15 ++------------- arch/arm64/kernel/vmlinux.lds.S | 19 +++++++++++++++++++ 5 files changed, 34 insertions(+), 22 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 990185744148..7f4f9abdf049 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,10 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +extern const char arm64_relocate_new_kernel[]; +#endif + /* * kern_reloc_arg is passed to kernel relocation function as an argument. * head kimage->head, allows to traverse through relocation segments. diff --git a/arch/arm64/include/asm/sections.h b/arch/arm64/include/asm/sections.h index 8ff579361731..ae873eb22205 100644 --- a/arch/arm64/include/asm/sections.h +++ b/arch/arm64/include/asm/sections.h @@ -19,5 +19,6 @@ extern char __exittext_begin[], __exittext_end[]; extern char __irqentry_text_start[], __irqentry_text_end[]; extern char __mmuoff_data_start[], __mmuoff_data_end[]; extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; +extern char __relocate_new_kernel_start[], __relocate_new_kernel_end[]; #endif /* __ASM_SECTIONS_H */ diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 679db3f1e0c5..361a4d082093 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,13 +20,10 @@ #include #include #include +#include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -78,13 +75,15 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); struct kern_reloc_arg *kern_reloc_arg = kexec_page_alloc(kimage); + long func_offset, reloc_size; if (!kern_reloc_arg) return -ENOMEM; - memcpy(reloc_code, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - kimage->arch.kern_reloc = __pa(reloc_code); + func_offset = arm64_relocate_new_kernel - __relocate_new_kernel_start; + reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); + kimage->arch.kern_reloc = __pa(reloc_code) + func_offset; kimage->arch.kern_reloc_arg = __pa(kern_reloc_arg); kern_reloc_arg->head = kimage->head; kern_reloc_arg->entry_addr = kimage->start; @@ -92,9 +91,9 @@ int machine_kexec_post_load(struct kimage *kimage) kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area(reloc_code, reloc_size); flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + reloc_size); __flush_dcache_area(kern_reloc_arg, sizeof(struct kern_reloc_arg)); return 0; diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index c92228aeddca..d2a4a0b0d76b 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -14,6 +14,7 @@ #include #include +.pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -75,16 +76,4 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x0, [x0, #KEXEC_KRELOC_KERN_ARG0] /* x0 = dtb address */ br x4 SYM_CODE_END(arm64_relocate_new_kernel) - -.align 3 /* To keep the 64-bit values below naturally aligned. */ - -.Lcopy_end: -.org KEXEC_CONTROL_PAGE_SIZE - -/* - * arm64_relocate_new_kernel_size - Number of bytes to copy to the - * control_code_page. - */ -.globl arm64_relocate_new_kernel_size -arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel +.popsection diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 4c0b0c89ad59..33b0d3c9fd3b 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -82,6 +83,16 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_KEXEC_CORE +#define KEXEC_TEXT \ + . = ALIGN(SZ_4K); \ + __relocate_new_kernel_start = .; \ + *(.kexec_relocate.text) \ + __relocate_new_kernel_end = .; +#else +#define KEXEC_TEXT +#endif + #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 #define TRAMP_TEXT \ . = ALIGN(PAGE_SIZE); \ @@ -142,6 +153,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + KEXEC_TEXT TRAMP_TEXT *(.fixup) *(.gnu.warning) @@ -316,3 +328,10 @@ ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) == PAGE_SIZE, * If padding is applied before .head.text, virt<->phys conversions will fail. */ ASSERT(_text == KIMAGE_VADDR, "HEAD is misaligned") + +#ifdef CONFIG_KEXEC_CORE +/* kexec relocation code should fit into one KEXEC_CONTROL_PAGE_SIZE */ +ASSERT(__relocate_new_kernel_end - (__relocate_new_kernel_start & ~(SZ_4K - 1)) + <= SZ_4K, "kexec relocation code is too big or misaligned") +ASSERT(KEXEC_CONTROL_PAGE_SIZE >= SZ_4K, "KEXEC_CONTROL_PAGE_SIZE is brokern") +#endif From patchwork Mon Jan 25 19:19:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E089FC433E0 for ; Mon, 25 Jan 2021 19:26:28 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8908621D94 for ; Mon, 25 Jan 2021 19:26:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8908621D94 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=r1t657QmlXGq4RI+1wOInYwbqlb1bfJNsUr39mz3UfA=; b=LLQmOzFs4QcmKgTpQaTC4/1Xau Ua7x3jm+iS4AVxgc86A1CgYBTHJ7E5OdxRAZV6v2l+Ipd8RxIcWxrfK2tLpwCvgRailc+fCkGP6Fv QF8F/pLlSjTKcqRwxjWPCD9EvKw2JtZFtbskwwyCTeUUXvLMe71bWul+7ygBdqSV1+hVLiLc8/4uq tg17gCAAT56kvb6BHS1/QJWVILpf4OfqfY+f2NC3o1NWZK+/924Gr1Lb6NL08B9T5AxAjAWYc0/OA 1QZyLb1Fr0BwacPFGV8xTbLnfKNkScVLLbmUtiZys9VATsU8CBIixRSeYikhOMf+0xIk4aA7E+SGl ik4vr+bA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47T1-0000bS-1M; Mon, 25 Jan 2021 19:24:03 +0000 Received: from mail-qk1-x729.google.com ([2607:f8b0:4864:20::729]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Ow-0007D8-Kg for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:14 +0000 Received: by mail-qk1-x729.google.com with SMTP id n7so2317099qkc.4 for ; Mon, 25 Jan 2021 11:19:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lbvVstnavugORmz+7RoYwB1ceirdtrsJhtHKQ5onWhE=; b=IK8GwsbNp05JTiDZeLKhus1Kd2crIvvLrCxhH/bhmegAMs2Z7vCp9d9KDysrBeDIzB KfxvBNXH0aHs05LraLi8jmw16detVwNx9DuZtp/sIo3qMZqxfqpu59gbI149OG92xdpR BrFyLDIenN5JcmuFpWIbMiKc4uzG51cstbavtA7pMHPtalMM29HTadK2mLoOXJWU5BxK 45HH6pKYIKWhfQ7huEmlUwbahY1bfK2R1z7LNKLBw4FcrzBIC5JKeyhBB9/bQJ+AHQBZ MQM5GrrFPz30IMgWgwFEjypPYyxfGfRD+/2oR5QMd8KQG6sPGUEoLJ4oRkCqK0gJSZZK orNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lbvVstnavugORmz+7RoYwB1ceirdtrsJhtHKQ5onWhE=; b=MYfboLiQKvixGzryqnJ6GbqJMMWQeOVbyQzmBzBuHvuEtPgF9/9t+uBi2vA2wyBAg4 0EG9ZWac7wylWYyixSrn4cPPKfZKhcx6JMmeJpxe/KgyABGWUG7lf9g1wu00pbdewwAm B9OJ3whVJ+dwq8G0Ojic+VEPt0l67eN5PHdyq1EMI4rVR/8BASpxS0JeSU7Rtcbm8ySQ sCvEHM5MygJ2eamJVHu+2aD3Fjwj+7glo0N9pVdTDxrbyjtYt17rMwqikMh6mf2i10CG O1NgJdgXbTFWD4rTawYvMRpfD1+f6IPNo9+/W0P3oNAq2RN6pw/A+/3jd9bv5cYMnCUZ +MuQ== X-Gm-Message-State: AOAM531xoWYlKcrK/oFefNEtzlj8ID7Lo9Ta4llAKZKQs9lKxwK0GgGz T4PX5mqn20Gg3puzoOv31IGZmw== X-Google-Smtp-Source: ABdhPJw6EjzUg6gmYM29zXvOx243XtORadzlnKH8/gi9zPCw6PpLiwryRc9t/9ZWzJPnyqYyMiOLXw== X-Received: by 2002:ae9:dc87:: with SMTP id q129mr2285101qkf.297.1611602388372; Mon, 25 Jan 2021 11:19:48 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:47 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 15/18] arm64: kexec: kexec may require EL2 vectors Date: Mon, 25 Jan 2021 14:19:20 -0500 Message-Id: <20210125191923.1060122-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141950_766610_BE9A2AD1 X-CRM114-Status: GOOD ( 17.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we have a EL2 mode without VHE, the EL2 vectors are needed in order to switch to EL2 and jump to new world with hypervisor privileges. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 5 +++++ arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/machine_kexec.c | 9 +++++++- arch/arm64/kernel/relocate_kernel.S | 35 +++++++++++++++++++++++++++++ 4 files changed, 49 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 7f4f9abdf049..b96d8a6aac80 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -92,6 +92,7 @@ static inline void crash_post_resume(void) {} #if defined(CONFIG_KEXEC_CORE) extern const char arm64_relocate_new_kernel[]; +extern const char arm64_kexec_el2_vectors[]; #endif /* @@ -101,6 +102,9 @@ extern const char arm64_relocate_new_kernel[]; * kernel, or purgatory entry address). * kern_arg0 first argument to kernel is its dtb address. The other * arguments are currently unused, and must be set to 0 + * el2_vector If present means that relocation routine will go to EL1 + * from EL2 to do the copy, and then back to EL2 to do the jump + * to new world. */ struct kern_reloc_arg { phys_addr_t head; @@ -109,6 +113,7 @@ struct kern_reloc_arg { phys_addr_t kern_arg1; phys_addr_t kern_arg2; phys_addr_t kern_arg3; + phys_addr_t el2_vector; }; #define ARCH_HAS_KIMAGE_ARCH diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 6067a288f568..8a9475be1b62 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -159,6 +159,7 @@ int main(void) DEFINE(KEXEC_KRELOC_KERN_ARG1, offsetof(struct kern_reloc_arg, kern_arg1)); DEFINE(KEXEC_KRELOC_KERN_ARG2, offsetof(struct kern_reloc_arg, kern_arg2)); DEFINE(KEXEC_KRELOC_KERN_ARG3, offsetof(struct kern_reloc_arg, kern_arg3)); + DEFINE(KEXEC_KRELOC_EL2_VECTOR, offsetof(struct kern_reloc_arg, el2_vector)); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 361a4d082093..41d1e3ca13f8 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -75,19 +75,26 @@ int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); struct kern_reloc_arg *kern_reloc_arg = kexec_page_alloc(kimage); - long func_offset, reloc_size; + long func_offset, vector_offset, reloc_size; if (!kern_reloc_arg) return -ENOMEM; func_offset = arm64_relocate_new_kernel - __relocate_new_kernel_start; reloc_size = __relocate_new_kernel_end - __relocate_new_kernel_start; + vector_offset = arm64_kexec_el2_vectors - __relocate_new_kernel_start; + memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code) + func_offset; kimage->arch.kern_reloc_arg = __pa(kern_reloc_arg); kern_reloc_arg->head = kimage->head; kern_reloc_arg->entry_addr = kimage->start; kern_reloc_arg->kern_arg0 = kimage->arch.dtb_mem; + + /* Setup vector table only when EL2 is available, but no VHE */ + if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) + kern_reloc_arg->el2_vector = __pa(reloc_code) + vector_offset; + kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index d2a4a0b0d76b..c6178b1a4e60 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -14,6 +14,17 @@ #include #include +.macro el1_sync_64 + .align 7 + br x4 /* Jump to new world from el2 */ +.endm + +.macro invalid_vector label +\label: + .align 7 + b \label +.endm + .pushsection ".kexec_relocate.text", "ax" /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. @@ -76,4 +87,28 @@ SYM_CODE_START(arm64_relocate_new_kernel) ldr x0, [x0, #KEXEC_KRELOC_KERN_ARG0] /* x0 = dtb address */ br x4 SYM_CODE_END(arm64_relocate_new_kernel) + +/* el2 vectors - switch el2 here while we restore the memory image. */ + .align 11 +SYM_CODE_START(arm64_kexec_el2_vectors) + invalid_vector el2_sync_invalid_sp0 /* Synchronous EL2t */ + invalid_vector el2_irq_invalid_sp0 /* IRQ EL2t */ + invalid_vector el2_fiq_invalid_sp0 /* FIQ EL2t */ + invalid_vector el2_error_invalid_sp0 /* Error EL2t */ + + invalid_vector el2_sync_invalid_spx /* Synchronous EL2h */ + invalid_vector el2_irq_invalid_spx /* IRQ EL2h */ + invalid_vector el2_fiq_invalid_spx /* FIQ EL2h */ + invalid_vector el2_error_invalid_spx /* Error EL2h */ + + el1_sync_64 /* Synchronous 64-bit EL1 */ + invalid_vector el1_irq_invalid_64 /* IRQ 64-bit EL1 */ + invalid_vector el1_fiq_invalid_64 /* FIQ 64-bit EL1 */ + invalid_vector el1_error_invalid_64 /* Error 64-bit EL1 */ + + invalid_vector el1_sync_invalid_32 /* Synchronous 32-bit EL1 */ + invalid_vector el1_irq_invalid_32 /* IRQ 32-bit EL1 */ + invalid_vector el1_fiq_invalid_32 /* FIQ 32-bit EL1 */ + invalid_vector el1_error_invalid_32 /* Error 32-bit EL1 */ +SYM_CODE_END(arm64_kexec_el2_vectors) .popsection From patchwork Mon Jan 25 19:19:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44198C433E0 for ; Mon, 25 Jan 2021 19:27:20 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D924221D79 for ; Mon, 25 Jan 2021 19:27:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D924221D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GI/LStg2tyX4DPcdRAPZjwcb1CeAKtzw7kc1ZoEYOdY=; b=YcF/QdsOjnS3G0eKZdXCHcEva6 LdvOr3YqWRFSxNGs8ekoLyzXgeQH47AB9q8Cx6Dk5kXFy3BgabtuxMMYccjgP0fXWm9tMBX3RuMI5 GY17PB6zdwTDeglfgyVTA4DzYueJ853EM5A4/VlW5baRDGXQ0PD+HIEorCH0Up7+8bj/uNhhfS3jo sp4vZGe5C1sn75zT0SI44EBOvdgn71lOaHd0ZPZDebK8rf1/PLMaztUKflFhG7QM8vRR50laETPUF N6FLGZFnXlYsE+lPVeUgjkCr5RrSO1VRXaxfpLT87ozTuxpbPKCYTwRy5fUKyZ0JSE9kAuhDhlkRk PrxcQDbg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47TU-0000o6-7d; Mon, 25 Jan 2021 19:24:32 +0000 Received: from mail-qv1-xf2f.google.com ([2607:f8b0:4864:20::f2f]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47P1-0007Ew-LT for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:19 +0000 Received: by mail-qv1-xf2f.google.com with SMTP id u16so4301709qvo.9 for ; Mon, 25 Jan 2021 11:19:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kuMkw7AdZybvslqmQltvC+V+qU5cLv31NDZ09K7EBEg=; b=oK64oKKQQBz3eN6h0+/Lx2yuanWsQs+hJCSPz+vyh29OFbeOFLT5Xe23Rr5vqAqqia hDdfTHUDsxKiPIVXSKssSWesBy0Uj4Qrv5NzWGtsu6VF0JIydxN9h37COcwJgenkEPlX dcq43hTSCusUMFWUgStPOoOET5G5vjjE0m/ScUoGS8mk88VVfb5cpkQpKS/hAsZ/9UoD v3LhLDNenT8aU7UnkW00YTObUx9CivESou5K9WMYZNr0kwf7BTb3869wXyfYMYY5rXJK 7qtuD6Bqxy7QIAS+LN3Lh374x141W+2qVud+Q21sZ79atN4ep8n8Gb9f0uzb6Q+S6fDP Omfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kuMkw7AdZybvslqmQltvC+V+qU5cLv31NDZ09K7EBEg=; b=nsidgogyHk56JYgZ+uTvsuFrB0tB0rciooIS9J+pwI3P7noRaIppjC14sEU9vKMVKf dPFfijEoGN7XWxwxlEtShFlq9lC6V0+JB3kReElvetcBmqaSDIUuiuJC5OIRBuL4mbm7 omgaAw5EKa9XyYxxS7jZjBOX0wftPOonlzEQD19U4lYFy3L0Stlb8wmiC/yASyMoLJJ5 JkRPNb1fIk3RdOKhfNudSX8lmfJSLnjcAhhR/XjaYkag/slpQJJCaux9HYEIRnLv6TH7 S0mcQs5y6+4KolzDc9WMB4c9QNnMSZhOiyNSG3sSo3LrEARPxAKMfzTF5RL7afkO25iu 2Zjw== X-Gm-Message-State: AOAM531fxWJnZHOcRAnc7RK2K3oUCprZWj5l+Ec9oriWwqtgdtK2EmLR lzpNgCBVGXQ0yfauKzqxkreN/A== X-Google-Smtp-Source: ABdhPJy1zPtPtrFKvie4dKIh/MK1P7SzhxNpMb4yTYY+AVvRu6RGBMRFYJn03biHv9W5Chg441JPkw== X-Received: by 2002:a0c:fcca:: with SMTP id i10mr2068628qvq.38.1611602393196; Mon, 25 Jan 2021 11:19:53 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:49 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 16/18] arm64: kexec: configure trans_pgd page table for kexec Date: Mon, 25 Jan 2021 14:19:21 -0500 Message-Id: <20210125191923.1060122-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141955_775286_609076E7 X-CRM114-Status: GOOD ( 20.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Configure a page table located in kexec-safe memory that has the following mappings: 1. identity mapping for text of relocation function with executable permission. 2. va mappings for all source ranges 3. va mappings for all destination ranges. Signed-off-by: Pavel Tatashin Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 12 ++++ arch/arm64/kernel/asm-offsets.c | 6 ++ arch/arm64/kernel/machine_kexec.c | 91 ++++++++++++++++++++++++++++++- 3 files changed, 108 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index b96d8a6aac80..049cde429b1b 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -105,6 +105,12 @@ extern const char arm64_kexec_el2_vectors[]; * el2_vector If present means that relocation routine will go to EL1 * from EL2 to do the copy, and then back to EL2 to do the jump * to new world. + * trans_ttbr0 idmap for relocation function and its argument + * trans_ttbr1 map for source/destination addresses. + * trans_t0sz t0sz for idmap page in trans_ttbr0 + * src_addr start address for source pages. + * dst_addr start address for destination pages. + * copy_len Number of bytes that need to be copied */ struct kern_reloc_arg { phys_addr_t head; @@ -114,6 +120,12 @@ struct kern_reloc_arg { phys_addr_t kern_arg2; phys_addr_t kern_arg3; phys_addr_t el2_vector; + phys_addr_t trans_ttbr0; + phys_addr_t trans_ttbr1; + unsigned long trans_t0sz; + unsigned long src_addr; + unsigned long dst_addr; + unsigned long copy_len; }; #define ARCH_HAS_KIMAGE_ARCH diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 8a9475be1b62..06278611451d 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -160,6 +160,12 @@ int main(void) DEFINE(KEXEC_KRELOC_KERN_ARG2, offsetof(struct kern_reloc_arg, kern_arg2)); DEFINE(KEXEC_KRELOC_KERN_ARG3, offsetof(struct kern_reloc_arg, kern_arg3)); DEFINE(KEXEC_KRELOC_EL2_VECTOR, offsetof(struct kern_reloc_arg, el2_vector)); + DEFINE(KEXEC_KRELOC_TRANS_TTBR0, offsetof(struct kern_reloc_arg, trans_ttbr0)); + DEFINE(KEXEC_KRELOC_TRANS_TTBR1, offsetof(struct kern_reloc_arg, trans_ttbr1)); + DEFINE(KEXEC_KRELOC_TRANS_T0SZ, offsetof(struct kern_reloc_arg, trans_t0sz)); + DEFINE(KEXEC_KRELOC_SRC_ADDR, offsetof(struct kern_reloc_arg, src_addr)); + DEFINE(KEXEC_KRELOC_DST_ADDR, offsetof(struct kern_reloc_arg, dst_addr)); + DEFINE(KEXEC_KRELOC_COPY_LEN, offsetof(struct kern_reloc_arg, copy_len)); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 41d1e3ca13f8..dc1b7e5a54fb 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -21,6 +21,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -71,11 +72,91 @@ static void *kexec_page_alloc(void *arg) return page_address(page); } +/* + * Map source segments starting from src_va, and map destination + * segments starting from dst_va, and return size of copy in + * *copy_len argument. + * Relocation function essentially needs to do: + * memcpy(dst_va, src_va, copy_len); + */ +static int map_segments(struct kimage *kimage, pgd_t *pgdp, + struct trans_pgd_info *info, + unsigned long src_va, + unsigned long dst_va, + unsigned long *copy_len) +{ + unsigned long *ptr = 0; + unsigned long dest = 0; + unsigned long len = 0; + unsigned long entry, addr; + int rc; + + for (entry = kimage->head; !(entry & IND_DONE); entry = *ptr++) { + addr = entry & PAGE_MASK; + + switch (entry & IND_FLAGS) { + case IND_DESTINATION: + dest = addr; + break; + case IND_INDIRECTION: + ptr = __va(addr); + if (rc) + return rc; + break; + case IND_SOURCE: + rc = trans_pgd_map_page(info, pgdp, __va(addr), + src_va, PAGE_KERNEL); + if (rc) + return rc; + rc = trans_pgd_map_page(info, pgdp, __va(dest), + dst_va, PAGE_KERNEL); + if (rc) + return rc; + dest += PAGE_SIZE; + src_va += PAGE_SIZE; + dst_va += PAGE_SIZE; + len += PAGE_SIZE; + } + } + *copy_len = len; + + return 0; +} + +static int mmu_relocate_setup(struct kimage *kimage, void *reloc_code, + struct kern_reloc_arg *kern_reloc_arg) +{ + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; + pgd_t *trans_pgd = kexec_page_alloc(kimage); + int rc; + + if (!trans_pgd) + return -ENOMEM; + + /* idmap relocation function */ + rc = trans_pgd_idmap_page(&info, &kern_reloc_arg->trans_ttbr0, + &kern_reloc_arg->trans_t0sz, reloc_code); + if (rc) + return rc; + + kern_reloc_arg->src_addr = _PAGE_OFFSET(VA_BITS_MIN); + kern_reloc_arg->dst_addr = _PAGE_OFFSET(VA_BITS_MIN - 1); + kern_reloc_arg->trans_ttbr1 = phys_to_ttbr(__pa(trans_pgd)); + + rc = map_segments(kimage, trans_pgd, &info, kern_reloc_arg->src_addr, + kern_reloc_arg->dst_addr, &kern_reloc_arg->copy_len); + return rc; +} + int machine_kexec_post_load(struct kimage *kimage) { void *reloc_code = page_to_virt(kimage->control_code_page); struct kern_reloc_arg *kern_reloc_arg = kexec_page_alloc(kimage); long func_offset, vector_offset, reloc_size; + int rc = 0; if (!kern_reloc_arg) return -ENOMEM; @@ -95,6 +176,14 @@ int machine_kexec_post_load(struct kimage *kimage) if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) kern_reloc_arg->el2_vector = __pa(reloc_code) + vector_offset; + /* + * If relocation is not needed, we do not need to enable MMU in + * relocation routine, therefore do not create page tables for + * scenarios such as crash kernel + */ + if (!(kimage->head & IND_DONE)) + rc = mmu_relocate_setup(kimage, reloc_code, kern_reloc_arg); + kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ @@ -103,7 +192,7 @@ int machine_kexec_post_load(struct kimage *kimage) reloc_size); __flush_dcache_area(kern_reloc_arg, sizeof(struct kern_reloc_arg)); - return 0; + return rc; } /** From patchwork Mon Jan 25 19:19:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 630CBC433E0 for ; Mon, 25 Jan 2021 19:30:04 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 163FB224DF for ; Mon, 25 Jan 2021 19:30:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 163FB224DF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=rw5feXPvBaXz1B0APzX6mO65AaHsZT3yWiRJZ/a8HSE=; b=gKl+9dYYHU4ACXPgjaQixvBBl8 62AxjGVz2OqeVeL2z0hKsfiG5lXeDzanJQOgXM5bKq5+df+SYqdRP6B1UzVPVUoKEgIzFErMLUVX1 VVgPUyITaQpmp9ESdsxGZJAPhYFBIUx/Laxw6cElO/O8TRb2rZo1GPAsIeHHwn1sanvxmDzbLjbwn Nu/QWhvi3BktxOaq4d9yMznlmMDsYpKUC7ovMgODpjQU7wGSTQBkfRkh7P5LkrFVhJCU+tVun76pK IBFsqaZ1YtwwmMZyklCt29ffwxeosdyS+QPTi2eG/qAzGFqLJu+cYM3Xgqpq8uHhmvPH3g0mWcx7H A3QizG3Q==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47XB-0002X6-V0; Mon, 25 Jan 2021 19:28:22 +0000 Received: from mail-qk1-x735.google.com ([2607:f8b0:4864:20::735]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47Uv-0001Xp-E4 for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:26:03 +0000 Received: by mail-qk1-x735.google.com with SMTP id x81so10782255qkb.0 for ; Mon, 25 Jan 2021 11:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RDSr+b90kmy9FHMSi/QfI0SbQbMsMCPJRwK16BQ54DQ=; b=WxdWiV+PelSJ2780GgGioA+Au/FzkI2Ld3y9Qv3Ck/gIiT2w/EXtKoJsE+0QnLkhrF NzgL174DN5cyesZLqlpqi8wPfQWzScrpFqhGV5CV3/CcIi8+lHuuakfvuwiDRZKLYZ3q nxbpY0DJyx2V7mpAlbB6yER6hOLSOQxb0inofimIHO0nE2l6OxdL+p8TooiCMkTBEqgS aKxIWoPPMcArqYT3n2Hllj39Y5pRCseZpf4aEvhKRcGsVkbcHpvyIkHgd/77M21AOhoO jx4xb6tc/azC6m9EMYG0FlqI24127XEIEFtRtuGwkiuvC1Tn85VAmqo8jJxWldh7tVaP NTXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RDSr+b90kmy9FHMSi/QfI0SbQbMsMCPJRwK16BQ54DQ=; b=SYSD3B3VKGbH51RCIi5Az1mNUKK3ro/+rBXhjndh04HZB4ikjcWkojIfZeJy2gTk/9 F4JcGekofl9Ef9XafTu7Dfb5bM0mB6fb+yAZvTNdTZaitjcmjqiMrLe4mh+bkFDWneKp JhfJMYpn5ShhS82G0F1haMwa33uxbz6y9ikiXHI/r63xk6pjy3YaFPah+yC6slofcCqa GBrhMC9gyGeMQK/WBJDxQRaWyvcFbyW9Dobr+bM8KOU59GgwY7GLeLdmyTZxeiMER+Gr +r+KgH5pi2vum1/9sWL4d7AUd8uxZxiRgiil14uCB157vGMJHTb5kaFc4EWATYFXYjZG k5fQ== X-Gm-Message-State: AOAM533aFj14VIGpJ4inDX0O1DiG1sqvjfx336+ichtmOXqBbScBOgR3 C9WhRRtd8iTtpgJVQ+XaZ7n51kWUdKDqkg== X-Google-Smtp-Source: ABdhPJxQav/dTh+GHIW6w/wd5sxxZLx8m02Wa9EmWUQyDQjM4DNHDdovsx6PrQGMmiR59tWpk8dAcg== X-Received: by 2002:ac8:5a4b:: with SMTP id o11mr1983211qta.202.1611602394865; Mon, 25 Jan 2021 11:19:54 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:54 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 17/18] arm64: kexec: enable MMU during kexec relocation Date: Mon, 25 Jan 2021 14:19:22 -0500 Message-Id: <20210125191923.1060122-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_142601_565242_4DF2F0A5 X-CRM114-Status: GOOD ( 17.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that we have transitional page tables configured, temporarily enable MMU to allow faster relocation of segments to final destination. The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/relocate_kernel.S | 131 ++++++++++++++++++---------- 1 file changed, 87 insertions(+), 44 deletions(-) diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index c6178b1a4e60..9c60981a6911 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (C) 2020, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -14,6 +16,54 @@ #include #include +.macro tlb_invalidate + dsb sy + dsb ish + tlbi vmalle1 + dsb ish + isb +.endm + +.macro turn_off_mmu tmp1, tmp2 + mrs \tmp1, sctlr_el1 + mov_q \tmp2, SCTLR_ELx_FLAGS + bic \tmp1, \tmp1, \tmp2 + pre_disable_mmu_workaround + msr sctlr_el1, \tmp1 + isb +.endm + +.macro turn_on_mmu tmp1, tmp2 + mrs \tmp1, sctlr_el1 + mov_q \tmp2, SCTLR_ELx_FLAGS + orr \tmp1, \tmp1, \tmp2 + msr sctlr_el1, \tmp1 + ic iallu + dsb nsh + isb +.endm + +/* + * Set ttbr0 and ttbr1, called while MMU is disabled, so no need to temporarily + * set zero_page table. Invalidate TLB after new tables are set. + */ +.macro set_ttbr arg, tmp1, tmp2 + ldr \tmp1, [\arg, #KEXEC_KRELOC_TRANS_TTBR0] + msr ttbr0_el1, \tmp1 + ldr \tmp1, [\arg, #KEXEC_KRELOC_TRANS_TTBR1] + offset_ttbr1 \tmp1, \tmp2 + msr ttbr1_el1, \tmp1 + isb +.endm + +/* Set T0SZ to match the requirements of idmap page */ +.macro set_tcr_t0sz arg, tmp1, tmp2 + ldr \tmp2, [\arg, #KEXEC_KRELOC_TRANS_T0SZ] + mrs \tmp1, tcr_el1 + bfi \tmp1, \tmp2, TCR_T0SZ_OFFSET, TCR_TxSZ_WIDTH + msr tcr_el1, \tmp1 +.endm + .macro el1_sync_64 .align 7 br x4 /* Jump to new world from el2 */ @@ -36,56 +86,49 @@ * symbols arm64_relocate_new_kernel and arm64_relocate_new_kernel_end. The * machine_kexec() routine will copy arm64_relocate_new_kernel to the kexec * safe memory that has been set up to be preserved during the copy operation. + * + * This function temporarily enables MMU if kernel relocation is needed. + * Also, if we enter this function at EL2 on non-VHE kernel, we temporarily go + * to EL1 to enable MMU, and escalate back to EL2 at the end to do the jump to + * the new kernel. This is determined by presence of el2_vector. */ SYM_CODE_START(arm64_relocate_new_kernel) - /* Check if the new image needs relocation. */ - ldr x16, [x0, #KEXEC_KRELOC_HEAD] /* x16 = kimage_head */ - tbnz x16, IND_DONE_BIT, .Ldone - raw_dcache_line_size x15, x1 /* x15 = dcache line size */ -.Lloop: - and x12, x16, PAGE_MASK /* x12 = addr */ - - /* Test the entry flags. */ -.Ltest_source: - tbz x16, IND_SOURCE_BIT, .Ltest_indirection - - /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - - copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 - b .Lnext -.Ltest_indirection: - tbz x16, IND_INDIRECTION_BIT, .Ltest_destination - mov x14, x12 /* ptr = addr */ - b .Lnext -.Ltest_destination: - tbz x16, IND_DESTINATION_BIT, .Lnext - mov x13, x12 /* dest = addr */ -.Lnext: - ldr x16, [x14], #8 /* entry = *ptr++ */ - tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: - /* wait for writes from copy_page to finish */ - dsb nsh - ic iallu - dsb nsh - isb - - /* Start new image. */ - ldr x4, [x0, #KEXEC_KRELOC_ENTRY_ADDR] /* x4 = kimage_start */ + mov x20, xzr /* x20 will hold vector value */ + ldr x11, [x0, #KEXEC_KRELOC_COPY_LEN] + cbz x11, 5f /* Check if need to relocate */ + ldr x20, [x0, #KEXEC_KRELOC_EL2_VECTOR] + cbz x20, 2f /* need to reduce to EL1? */ + msr vbar_el2, x20 /* el2_vector present, means */ + adr x1, 2f /* we will do copy in el1 but */ + msr elr_el2, x1 /* do final jump from el2 */ + eret /* Reduce to EL1 */ +2: set_tcr_t0sz x0, x1, x2 /* Set t0sz for idmaped page */ + set_ttbr x0, x1, x2 /* Set our page tables */ + tlb_invalidate + ldr x1, [x0, #KEXEC_KRELOC_DST_ADDR]; /* arg is not idmapped so */ + ldr x2, [x0, #KEXEC_KRELOC_SRC_ADDR]; /* read before MMU is on */ + turn_on_mmu x3, x4 /* Turn MMU back on */ + mov x12, x1 /* x12 dst backup */ +3: copy_page x1, x2, x3, x4, x5, x6, x7, x8, x9, x10 + sub x11, x11, #PAGE_SIZE + cbnz x11, 3b /* page copy loop */ + raw_dcache_line_size x2, x3 /* x2 = dcache line size */ + sub x3, x2, #1 /* x3 = dcache_size - 1 */ + bic x12, x12, x3 +4: dc cvau, x12 /* Flush D-cache */ + add x12, x12, x2 + cmp x12, x1 /* Compare to dst + len */ + b.ne 4b /* D-cache flush loop */ + turn_off_mmu x1, x2 /* Turn off MMU */ + tlb_invalidate /* Invalidate TLB */ +5: ldr x4, [x0, #KEXEC_KRELOC_ENTRY_ADDR] /* x4 = kimage_start */ ldr x3, [x0, #KEXEC_KRELOC_KERN_ARG3] ldr x2, [x0, #KEXEC_KRELOC_KERN_ARG2] ldr x1, [x0, #KEXEC_KRELOC_KERN_ARG1] ldr x0, [x0, #KEXEC_KRELOC_KERN_ARG0] /* x0 = dtb address */ - br x4 + cbnz x20, 6f /* need to escalate to el2? */ + br x4 /* Jump to new world */ +6: hvc #0 /* enters kexec_el1_sync */ SYM_CODE_END(arm64_relocate_new_kernel) /* el2 vectors - switch el2 here while we restore the memory image. */ From patchwork Mon Jan 25 19:19:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 12044169 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A056C433DB for ; Mon, 25 Jan 2021 19:26:35 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B41D21D79 for ; Mon, 25 Jan 2021 19:26:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B41D21D79 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe :List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:To:From: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=FW7j+sr8nxFh0l9iYgl3T9N+auDkpex0XGogkteu7I8=; b=wFX1wVPnOHMNGGQbxsoztqiYID hMUlFL6wyaECpzZBBRAXvvMtrc3RlPjXtMfJ758+RFXdaNu4FSNEZyEvMdSKGlKS53Y1sDUQ/gaP0 9fUgnG8KBdrwpDlrQ+/bIEdeD/XXsYRLQZFQWVc0O9sgQbJoR7/qBI0Dp1CfApZyxpFGBeF8juLI5 qQnXxh5b84T+HO0vekvEsh97lhqioabw3Se4UwLXGq9QeqSDQ4mnDw5yG7RrUF/pzn9G3Cga95Ac3 6yO+3sKmNrQtDuAE5Hq+RtkRaI+26ySCyzelGLA0LtPqqhh2lZ3o0kTHiCcZX8+SWYQD0WnEPPQnH 6TpGmH4A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47TF-0000jC-0l; Mon, 25 Jan 2021 19:24:17 +0000 Received: from mail-qv1-f42.google.com ([209.85.219.42]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l47P4-0007G8-D0 for linux-arm-kernel@lists.infradead.org; Mon, 25 Jan 2021 19:20:18 +0000 Received: by mail-qv1-f42.google.com with SMTP id h21so6715993qvb.8 for ; Mon, 25 Jan 2021 11:19:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=bntzwWb0HnF+pu1wTRlxa4nGPd0MURNpYR8uZbvPwl4=; b=aznWjElVYsHK1zXw38ZeQN4JK2jZE1LCcfMo91RU+SnIfejFZOlOFF6GnXfHtUxTuB nx+vd0X1j3aObV6HjWaRTYR3YFz3OMIKiMzQsFdP8O//rc3JyHEGvZ1eza4Hx9FhM9Xu MLPWDXzEokhcrZo7iPjKXIf8uyiVouOu6eWmBrQdBUyASn920C6UxGCP11uwpZJnT8Aq c0PQ22aHJcWhVfHa+IuSyf3iqMVoSgtM7PDqmk74MOJ5xqFyGUnmJ1Dw4Bg7hGJGR+Jq o7Vrk/rs5TNslgwP46rBg/KriUXV461e/WvbyR+EfD1Cn1cQnLm6i4tN3YvEpNTWjQwh psfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bntzwWb0HnF+pu1wTRlxa4nGPd0MURNpYR8uZbvPwl4=; b=B61oFOHxiMcW+D7j7kn2JQltMI1ozfz0rJIXYsAMO1BEMo5D7H9/v0n/VFVasAvPti MpHL7JcEXwcx4lXEXag+3qaS+JjvqPS1hjlv1473ihPxqLGhFailVTC8IbyuBwKwsknI qBJdbCREzP7mZ32ehYo7yToULUi1xwYqVmtshlZu7EioubKWZUrOOVzixjzag4uoxecz tuSoUk7ZgLanZ+0DzUzvu0WNuUnAtfZBpX7FgSdlwzKzy1dmdGIaiRKlDNgvjzONagBy Y9IDlnttt6lnX4Gw+JTesbjrXhs5NyppQTG1erkWckcpR8tcNtZJUmlUO54F4xtJmQ+x bTlQ== X-Gm-Message-State: AOAM5339jmYJjfqW5QtP7KoSQPRSPGd6HVUwgsejyZkrc9A2oD5zMToa qZuGWV0261zW7547wsBZLwq4wg== X-Google-Smtp-Source: ABdhPJytuTHdBIVuiybjpt5hI8b1ljcElIFMcTl+j/BWrIjw/SnfAX/Yl/DjO9FsYiUxo5xvziJ1Kw== X-Received: by 2002:a05:6214:138e:: with SMTP id g14mr2241683qvz.7.1611602396408; Mon, 25 Jan 2021 11:19:56 -0800 (PST) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id s6sm9047638qtx.63.2021.01.25.11.19.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Jan 2021 11:19:55 -0800 (PST) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, maz@kernel.org, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, linux-mm@kvack.org, mark.rutland@arm.com, steve.capper@arm.com, rfontana@redhat.com, tglx@linutronix.de, selindag@gmail.com, tyhicks@linux.microsoft.com Subject: [PATCH v10 18/18] arm64: kexec: remove head from relocation argument Date: Mon, 25 Jan 2021 14:19:23 -0500 Message-Id: <20210125191923.1060122-19-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210125191923.1060122-1-pasha.tatashin@soleen.com> References: <20210125191923.1060122-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210125_141958_511223_CFABDA53 X-CRM114-Status: GOOD ( 11.50 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now, that relocation is done using virtual addresses, reloc_arg->head is not needed anymore. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 2 -- arch/arm64/kernel/asm-offsets.c | 1 - arch/arm64/kernel/machine_kexec.c | 1 - 3 files changed, 4 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 049cde429b1b..2fa4109bd582 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -97,7 +97,6 @@ extern const char arm64_kexec_el2_vectors[]; /* * kern_reloc_arg is passed to kernel relocation function as an argument. - * head kimage->head, allows to traverse through relocation segments. * entry_addr kimage->start, where to jump from relocation function (new * kernel, or purgatory entry address). * kern_arg0 first argument to kernel is its dtb address. The other @@ -113,7 +112,6 @@ extern const char arm64_kexec_el2_vectors[]; * copy_len Number of bytes that need to be copied */ struct kern_reloc_arg { - phys_addr_t head; phys_addr_t entry_addr; phys_addr_t kern_arg0; phys_addr_t kern_arg1; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 06278611451d..94f050ad6471 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -153,7 +153,6 @@ int main(void) BLANK(); #endif #ifdef CONFIG_KEXEC_CORE - DEFINE(KEXEC_KRELOC_HEAD, offsetof(struct kern_reloc_arg, head)); DEFINE(KEXEC_KRELOC_ENTRY_ADDR, offsetof(struct kern_reloc_arg, entry_addr)); DEFINE(KEXEC_KRELOC_KERN_ARG0, offsetof(struct kern_reloc_arg, kern_arg0)); DEFINE(KEXEC_KRELOC_KERN_ARG1, offsetof(struct kern_reloc_arg, kern_arg1)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index dc1b7e5a54fb..c2dff232a85b 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -168,7 +168,6 @@ int machine_kexec_post_load(struct kimage *kimage) memcpy(reloc_code, __relocate_new_kernel_start, reloc_size); kimage->arch.kern_reloc = __pa(reloc_code) + func_offset; kimage->arch.kern_reloc_arg = __pa(kern_reloc_arg); - kern_reloc_arg->head = kimage->head; kern_reloc_arg->entry_addr = kimage->start; kern_reloc_arg->kern_arg0 = kimage->arch.dtb_mem;