From patchwork Tue Jul 16 16:56:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11046507 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09C06912 for ; Tue, 16 Jul 2019 16:57:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB18D2861E for ; Tue, 16 Jul 2019 16:57:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DEDB528671; Tue, 16 Jul 2019 16:57:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 624CE2861E for ; Tue, 16 Jul 2019 16:57:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6AVueCBwdDpMpYlXnq32e+c/N3PDbscAUi1NFjVcR68=; b=BPKVKPC6hcl4za owTSBPoQa1bA7LVxdb4RPZVSvCOcsjy0Pcmz4GFIWotuqy/vkrkLaMNaPw6YjfcAzA0DhgR7MfExW 1QMl4iqrpKRdRuds5WDvcs4Zv1kZI9ST5q29ElY/HuTX543uhGlkdyBQaJ5L8LVMRIYhgdPN1yL26 40T2fX8p49MdAzNIC29ELrdaAFiW12QNn3bNm9uzPxL6UWdYMTlFXJX41Ntsl/C712WfZcjroaQbH VlUgFTFfKxtZHlz+d8ZC3swir39+/PGAewEF7AVbx6ChRtUMEZKWJB0EZd5Ul+Krk3irobE5CFVpZ KdfDRwU3hbgXHQ/oZtOg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hnQlP-0005LP-R2; Tue, 16 Jul 2019 16:57:15 +0000 Received: from mail-qt1-x841.google.com ([2607:f8b0:4864:20::841]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hnQkv-00052R-FU for linux-arm-kernel@lists.infradead.org; Tue, 16 Jul 2019 16:56:48 +0000 Received: by mail-qt1-x841.google.com with SMTP id y26so20307352qto.4 for ; Tue, 16 Jul 2019 09:56:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=JkgYB201+YgtZ67xz124QalImKwHhD6hCX1vEDo1XAU=; b=h5Lfrj4oD0WIqU6ygkd/Fq6YtgSA5kUklS4HcoiluHDChhEtO4DfF3FXAOHZoNSeCm Qhej/p8oi+GphDHnwLxKOaFOAJeBkg6l9mlc/+E4PJtYlKxvBn+HtZ4Ta4dpZVmxhbZX 1K9RtH/Q50Ei12r27O9HiDA+aOe1WIV7OpE+Ur5ro2lt2huOfTy2h/vYCnccPex7x35l bxf3Hq39Q/gK7xyaA+Ok5qq5QKv5KwayqGxpnxBPP38fc9PySfALkm1xG8plhfjIY5au Ubi4eL+bANK8Z/oL4+ZVTiLwR63VOBupKwcL+wUi6xtjo6aMfBD77X+qKxDGAflTHmNX oCUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JkgYB201+YgtZ67xz124QalImKwHhD6hCX1vEDo1XAU=; b=HlY/mmWxdJLdeN62EyvaVYWSbHdIp2ZTX0RrB/DsX8pf2lsG7J6vY91Xum2tSNMJRh gFBrzKhMLQ6j36yrd1NiA7e2dU8K89k3HhCk29XLui+/JjoGQMF8TfJ+JFY7+iYQJpx6 zOQ9WEdGuIfx2kVYbT+PmISIyKCtW4cTqb18ZcNWXkgFeOvOG/oVwqAniQeDTqxa0ND0 T6BzpZwMQmb1079FdzVPXh7v7i/OYpNrQYEKHIldGvelQfFPMDqSrsMvQv/NEQzumCuc QPbQBIt5sisOgBW5mfxe2nyB9Z+JkhHc92CuRBL36xBHk25CZy8ygRWnqXkaKmElL91V lnkQ== X-Gm-Message-State: APjAAAWqe6/dvQxv3HgJtzA98WqYaIED1Kaa9L9hUxUfpJfF55/RL2WF RuYRDxdafKfZ2/UKKsxEdqY= X-Google-Smtp-Source: APXvYqwZZYB7JTSxjflHH6PzEmuWW65i2dKDOtRvxBkvqU4u3mcg8TDxomQhWhsJDugWob8iCPoiBw== X-Received: by 2002:ac8:2b49:: with SMTP id 9mr24808489qtv.343.1563296204311; Tue, 16 Jul 2019 09:56:44 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id f20sm8519538qkh.15.2019.07.16.09.56.43 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 16 Jul 2019 09:56:43 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [RFC v1 1/4] arm64, mm: identity mapped page table Date: Tue, 16 Jul 2019 12:56:38 -0400 Message-Id: <20190716165641.6990-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190716165641.6990-1-pasha.tatashin@soleen.com> References: <20190716165641.6990-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190716_095645_518362_B6E211C9 X-CRM114-Status: GOOD ( 14.37 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Created identiy mapped page table that maps 1 to 1 virtual to physical addresses. Similarly to x86, this table can be used in kasan, hibernate, and kexec. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/ident_map.h | 26 ++++++++ arch/arm64/mm/Makefile | 1 + arch/arm64/mm/ident_map.c | 99 ++++++++++++++++++++++++++++++ 3 files changed, 126 insertions(+) create mode 100644 arch/arm64/include/asm/ident_map.h create mode 100644 arch/arm64/mm/ident_map.c diff --git a/arch/arm64/include/asm/ident_map.h b/arch/arm64/include/asm/ident_map.h new file mode 100644 index 000000000000..1bb9fcd27368 --- /dev/null +++ b/arch/arm64/include/asm/ident_map.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2019, Microsoft Corporation. + * Pavel Tatashin + */ + +#ifndef _ASM_IDENT_MAP_H +#define _ASM_IDENT_MAP_H + +#include +#include + +struct ident_map_info { + void * (*alloc_pgt_page)(void *); /* allocate a page */ + void *alloc_arg; /* arg. for alloc_pgt_page */ + unsigned long page_flags; /* PMD or PUD flags */ + unsigned long offset; /* ident mapping offset */ + bool pud_pages; /* PUD level huge pages */ +}; + +int ident_map_pgd_populate(struct ident_map_info *info, + phys_addr_t pgd_page, + phys_addr_t addr, + phys_addr_t end); + +#endif /* _ASM_ARM64_IDENT_MAP_H */ diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 849c1df3d214..dfa5a074a360 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -5,6 +5,7 @@ obj-y := dma-mapping.o extable.o fault.o init.o \ context.o proc.o pageattr.o obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_ARM64_PTDUMP_CORE) += dump.o +obj-$(CONFIG_KEXEC_CORE) += ident_map.o obj-$(CONFIG_ARM64_PTDUMP_DEBUGFS) += ptdump_debugfs.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o diff --git a/arch/arm64/mm/ident_map.c b/arch/arm64/mm/ident_map.c new file mode 100644 index 000000000000..bcfff5e2573b --- /dev/null +++ b/arch/arm64/mm/ident_map.c @@ -0,0 +1,99 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2019, Microsoft Corporation. + * Pavel Tatashin + */ + +#include +#include + +/* Initialize PMD size huge entries in page table */ +static void ident_map_pmd_init(struct ident_map_info *info, + phys_addr_t pmd_page, phys_addr_t addr, + phys_addr_t end) +{ + const unsigned long flags = info->page_flags; + const unsigned long offset = info->offset; + pmd_t *pmdp = (pmd_t *)__va(pmd_page) + pmd_index(addr); + + addr &= PMD_MASK; + for (; addr < end; addr += PMD_SIZE, pmdp++) { + set_pmd(pmdp, __pmd(__phys_to_pmd_val(addr - offset) | flags)); + } +} + +/* Initialize PUD size huge entries in page table */ +static void ident_map_pud_init(struct ident_map_info *info, + phys_addr_t pud_page, phys_addr_t addr, + phys_addr_t end) +{ + const unsigned long flags = info->page_flags; + const unsigned long offset = info->offset; + pud_t *pudp = (pud_t *)__va(pud_page) + pud_index(addr); + + addr &= PUD_MASK; + for (; addr < end; addr += PUD_SIZE, pudp++) { + set_pud(pudp, __pud(__phys_to_pud_val(addr - offset) | flags)); + } +} + +/* Populate PUD level with PMD entries */ +static int ident_map_pud_populate(struct ident_map_info *info, + phys_addr_t pud_page, phys_addr_t addr, + phys_addr_t end) +{ + pud_t *pudp = (pud_t *)__va(pud_page) + pud_index(addr); + phys_addr_t pmd_page, next; + + for (; addr < end; addr = next, pudp++) { + next = pud_addr_end(addr, end); + if (pud_none(*pudp)) { + void *pmd = info->alloc_pgt_page(info->alloc_arg); + + if (!pmd) + return -ENOMEM; + + clear_page(pmd); + __pud_populate(pudp, __pa(pmd), PUD_TYPE_TABLE); + } + pmd_page = __pud_to_phys(*pudp); + ident_map_pmd_init(info, pmd_page, addr, next); + } + + return 0; +} + +/* Populate identify mapped page table with physical range[addr, end) */ +int ident_map_pgd_populate(struct ident_map_info *info, + phys_addr_t pgd_page, phys_addr_t addr, + phys_addr_t end) +{ + const bool pud_pages = info->pud_pages; + pgd_t *pgdp = (pgd_t *)__va(pgd_page) + pgd_index(addr); + phys_addr_t pud_page, next; + + for (; addr < end; addr = next, pgdp++) { + next = pgd_addr_end(addr, end); + if (pgd_none(*pgdp)) { + void *pud = info->alloc_pgt_page(info->alloc_arg); + + if (!pud) + return -ENOMEM; + + clear_page(pud); + __pgd_populate(pgdp, __pa(pud), PUD_TYPE_TABLE); + } + pud_page = __pgd_to_phys(*pgdp); + if (pud_pages) { + ident_map_pud_init(info, pud_page, addr, next); + } else { + int rv = ident_map_pud_populate(info, pud_page, addr, + next); + + if (rv) + return rv; + } + } + + return 0; +} From patchwork Tue Jul 16 16:56:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11046509 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3FD5F912 for ; Tue, 16 Jul 2019 16:57:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2D6F928673 for ; Tue, 16 Jul 2019 16:57:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2168428675; Tue, 16 Jul 2019 16:57:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7BEFF28671 for ; Tue, 16 Jul 2019 16:57:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8CDvET7iFYGl+wmNsiHWc/q+N3zgqBbyKRPZLJAUxO0=; b=nfnoud7so2dn56 RBaYBHEw2MAn8KqnWJavu6wRU+g5IVobS7ier81dWkURDGQ78exAYRFNPNdbJok6Q8p6OV6VJvkmC OOvXUA3fu2ycKkqRx86zbI3VR1jms1K3OxZo5i9c4yGsMGEzcS1pM8wP9hb4mHo8EzksYVBa+3fs2 Ik1TX1wcMBYoQgK6JGhFFGFM59ZnhrWlUnFhM+ciVFc4OId6Yzqp2T6yX1OlX0B/2v/XZhEppSnUi hRYnTk2MY5UaHauCq5SPohDQd+QNe3pBtIT3dNSYUDAaACIUwGSft0HG+A0y7Lm2Z5DDzwkHgOt/K gvddraQP9jlhLd+rIVdw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hnQll-0005hU-SA; Tue, 16 Jul 2019 16:57:38 +0000 Received: from mail-qt1-x842.google.com ([2607:f8b0:4864:20::842]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hnQkw-00053R-UV for linux-arm-kernel@lists.infradead.org; Tue, 16 Jul 2019 16:56:48 +0000 Received: by mail-qt1-x842.google.com with SMTP id d17so20255363qtj.8 for ; Tue, 16 Jul 2019 09:56:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=EifO61K65QbWAJ6xpvhOah9bA7Qm2CvZMBmlIJekn+0=; b=idL/2gxy1CAm62HCYqxM5BClPT9M4sH4sKI6TrJoDPJCf2MiSdvmWabARetWUf3w1e iLVb6WIYwP5AjN2cIxv4h5a/dXC1HSfIq/EowkRDV/VcrwVXFilBnt+w4BX/wJSeNTac VyO4uTItzt3qHiEZV1+Cipw1CKfeXeHp4CuapXhbkQG5CU5UI6UQiqrQmRW6A4HfkYH5 kB+ZyqB3ioM+rxuyRjt7E0sXi1Gw71iKFcmO1Vy4zAAGlf2PsetNRiLVlfApLPbn9+g0 yelseVL4ypdMFaIAuiYKSa557KSo3N7gKd7xeCmlxODv06M1IU+zX/u6dMsGi8SfWUcu Qqgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=EifO61K65QbWAJ6xpvhOah9bA7Qm2CvZMBmlIJekn+0=; b=L5OI55Ff0+WgojrBPEoR6tkXDciG/OfiRueRna39DAWOs2ct7kr3wHg/6xqyaZaqaf Wf8eS97shc6esDQKVR6RvLvsCg5L5jSyRFzMmb9852VTI6UIpEM95dsas4H8X/ITkcNe R1ZZfjwBk4eHEhFf/j9Hi+KalvHg5mMH+Ad9T+xdxP+12mf70dksLCtgS9d+DCUHjlxB ZPbG0Nf/xMTBGYUM1788xEETOJ2uzbB6YOAAfRcRHVPp8fKSJffBVDcYyY2P6CGxieEJ 6doMz++iUAnUHqoe/6aHeDHu1eov4faPFnm4YjpWCQn/UpAGF0vwJAR4xEAf0LU7VjJq H2kQ== X-Gm-Message-State: APjAAAWDcAMqvZQtY65oBMud8Ys9RW9wUperFLI8hbnTWiCzwbvXLCxL mPVIsvjbHxhcs967Lkt/XZQ= X-Google-Smtp-Source: APXvYqzHhhYiAKJdjA+sOKDaBPu/cH7wsljYIAcV0+pVKa8zDrnxHDQQygDK1MnOobsI2gFvewAVfA== X-Received: by 2002:aed:24d9:: with SMTP id u25mr24126760qtc.111.1563296205591; Tue, 16 Jul 2019 09:56:45 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id f20sm8519538qkh.15.2019.07.16.09.56.44 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 16 Jul 2019 09:56:44 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [RFC v1 2/4] arm64, kexec: interface preparation for mmu enabled kexec Date: Tue, 16 Jul 2019 12:56:39 -0400 Message-Id: <20190716165641.6990-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190716165641.6990-1-pasha.tatashin@soleen.com> References: <20190716165641.6990-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190716_095647_027966_8A129269 X-CRM114-Status: GOOD ( 16.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Currently cpu_install_idmap() is used to install page table during kexec switch over to purgatory. We soon will be using our own page table, that maps the whole physical range (and might be even more, i.e if new DTB describes a bigger physical range or mem= parameter limited physical range in the current kernel). Make kimage_arch to be always part of arm64. Add relocate_kern and kexec_pgtable verctors to this struct, as we won't be able to rely on a single control page anymore. Copy relocation function in machine_kexec_prepare(), and setup page table there as well (for now idmap_pg_dir). Cleanup call to cpu_soft_restart by removing ugly ifdefs. When kimage->arch.dtb_mem is not set, it is 0 anyway. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 5 +-- arch/arm64/kernel/cpu-reset.h | 7 ++++- arch/arm64/kernel/machine_kexec.c | 52 ++++++++++++++----------------- 3 files changed, 32 insertions(+), 32 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 12a561a54128..ef2d2442b890 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,14 +90,15 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif -#ifdef CONFIG_KEXEC_FILE #define ARCH_HAS_KIMAGE_ARCH - struct kimage_arch { void *dtb; unsigned long dtb_mem; + void *relocate_kern; + pgd_t *kexec_pgtable; }; +#ifdef CONFIG_KEXEC_FILE extern const struct kexec_file_ops kexec_image_ops; struct kimage; diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..c795811587f0 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -14,6 +14,7 @@ void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, unsigned long arg0, unsigned long arg1, unsigned long arg2); static inline void __noreturn cpu_soft_restart(unsigned long entry, + pgd_t *kexec_pgtable, unsigned long arg0, unsigned long arg1, unsigned long arg2) @@ -24,7 +25,11 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, is_hyp_mode_available(); restart = (void *)__pa_symbol(__cpu_soft_restart); - cpu_install_idmap(); + cpu_set_reserved_ttbr0(); + local_flush_tlb_all(); + write_sysreg(phys_to_ttbr(virt_to_phys(kexec_pgtable)), ttbr0_el1); + isb(); + restart(el2_switch, entry, arg0, arg1, arg2); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 0df8493624e0..f4565eb01d09 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -42,6 +42,8 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" arch.kexec_pgtable: %p\n", kimage->arch.kexec_pgtable); + pr_debug(" arch.relocate_kern: %p\n", kimage->arch.relocate_kern); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -67,13 +69,24 @@ void machine_kexec_cleanup(struct kimage *kimage) */ int machine_kexec_prepare(struct kimage *kimage) { - kexec_image_info(kimage); + void *reloc_buf = page_address(kimage->control_code_page); if (kimage->type != KEXEC_TYPE_CRASH && cpus_are_stuck_in_kernel()) { pr_err("Can't kexec: CPUs are stuck in the kernel.\n"); return -EBUSY; } + /* + * Copy arm64_relocate_new_kernel to the buffer for use after the kernel + * is shut down. + */ + memcpy(reloc_buf, arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + + kimage->arch.relocate_kern = reloc_buf; + kimage->arch.kexec_pgtable = lm_alias(idmap_pg_dir); + kexec_image_info(kimage); + return 0; } @@ -143,8 +156,6 @@ static void kexec_segment_flush(const struct kimage *kimage) */ void machine_kexec(struct kimage *kimage) { - phys_addr_t reboot_code_buffer_phys; - void *reboot_code_buffer; bool in_kexec_crash = (kimage == kexec_crash_image); bool stuck_cpus = cpus_are_stuck_in_kernel(); @@ -155,32 +166,17 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - reboot_code_buffer_phys = page_to_phys(kimage->control_code_page); - reboot_code_buffer = phys_to_virt(reboot_code_buffer_phys); - kexec_image_info(kimage); - - pr_debug("%s:%d: control_code_page: %p\n", __func__, __LINE__, - kimage->control_code_page); - pr_debug("%s:%d: reboot_code_buffer_phys: %pa\n", __func__, __LINE__, - &reboot_code_buffer_phys); - pr_debug("%s:%d: reboot_code_buffer: %p\n", __func__, __LINE__, - reboot_code_buffer); pr_debug("%s:%d: relocate_new_kernel: %p\n", __func__, __LINE__, arm64_relocate_new_kernel); pr_debug("%s:%d: relocate_new_kernel_size: 0x%lx(%lu) bytes\n", __func__, __LINE__, arm64_relocate_new_kernel_size, arm64_relocate_new_kernel_size); - /* - * Copy arm64_relocate_new_kernel to the reboot_code_buffer for use - * after the kernel is shut down. - */ - memcpy(reboot_code_buffer, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - /* Flush the reboot_code_buffer in preparation for its execution. */ - __flush_dcache_area(reboot_code_buffer, arm64_relocate_new_kernel_size); + /* Flush the relocate_kern in preparation for its execution. */ + __flush_dcache_area(kimage->arch.relocate_kern, + arm64_relocate_new_kernel_size); /* * Although we've killed off the secondary CPUs, we don't update @@ -188,7 +184,7 @@ void machine_kexec(struct kimage *kimage) * need to avoid flush_icache_range(), which will attempt to IPI * the offline CPUs. Therefore, we must use the __* variant here. */ - __flush_icache_range((uintptr_t)reboot_code_buffer, + __flush_icache_range((uintptr_t)kimage->arch.relocate_kern, arm64_relocate_new_kernel_size); /* Flush the kimage list and its buffers. */ @@ -204,7 +200,7 @@ void machine_kexec(struct kimage *kimage) /* * cpu_soft_restart will shutdown the MMU, disable data caches, then - * transfer control to the reboot_code_buffer which contains a copy of + * transfer control to the relocate_kern which contains a copy of * the arm64_relocate_new_kernel routine. arm64_relocate_new_kernel * uses physical addressing to relocate the new image to its final * position and transfers control to the image entry point when the @@ -214,12 +210,10 @@ void machine_kexec(struct kimage *kimage) * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(reboot_code_buffer_phys, kimage->head, kimage->start, -#ifdef CONFIG_KEXEC_FILE - kimage->arch.dtb_mem); -#else - 0); -#endif + cpu_soft_restart(__pa(kimage->arch.relocate_kern), + kimage->arch.kexec_pgtable, + kimage->head, kimage->start, + kimage->arch.dtb_mem); BUG(); /* Should never get here. */ } From patchwork Tue Jul 16 16:56:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11046511 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 60A931395 for ; Tue, 16 Jul 2019 16:58:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4CA3D28671 for ; Tue, 16 Jul 2019 16:58:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 40F4C28673; Tue, 16 Jul 2019 16:58:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CB02528671 for ; Tue, 16 Jul 2019 16:58:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZeqmXoZanzimYxLPc6BwczdjAjBbl4jb6FJTaJ7s4p8=; b=cAQz1ZRaj4LYLM wDUXPTkS1UucgGFWlNZxEL6BNJBchZCmg02J232QvPWjhHPC0dJ8v8KKFocqf+tg+8hLeWpai620X vQuskJFRphLG2niPY7ZbyRkkWdR8UJNvRbsspXZ0tWWoKj3nNm7crhkGa1+0k2uHITX8MHK+w5rY7 pxmHI45+J9vG2+SQXI2hqeY05d+FJPZjxoW0/RyU8GeOi8OaEdGQvk3vMDUVj6P0FxY5vZz/WYCF5 M1IrrGd8Uvz+Er/QtpKhKyB2uVrQ1QZ9Dui0ocec63UUTPL81t91Fk7toi3QX9sn7+z9EwR7ipSWI j2Bhmv+7Pk8muXMmVT0A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hnQmC-00060I-0Q; Tue, 16 Jul 2019 16:58:04 +0000 Received: from mail-qk1-x743.google.com ([2607:f8b0:4864:20::743]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hnQkx-00053s-R2 for linux-arm-kernel@lists.infradead.org; Tue, 16 Jul 2019 16:56:51 +0000 Received: by mail-qk1-x743.google.com with SMTP id a27so15147257qkk.5 for ; Tue, 16 Jul 2019 09:56:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5D8mlvVawCjEgddelLAduhKCIQh0CPiyGgQp446bGsE=; b=VxsqpYcitDOLAtpg1Jls5stspr2tgCl0PGz2J1Z817sVMFGcuo/roOKCaavlnYeATr jU6YTyCAGZdlqYJ/jAs5NtTWvKRhJMmjirZcQE+Iucd/gw8X+OddfHUsoLTZ8sMYS/3c ve/dJt/iFA5AYuQOfT8c21qt+EjJ268NVoprOSpHjzlTm8z+pnUOXbMTcz0Wg8M1xsbA VBgDOJX8ALLB3Bwq5D6MxZszG6wXjnwReOXDz93IVfYtFF6y2ddjcZH7FZb1wO7Km03h cbJW0XLKEZ9fLZEWNShJLv0RqMAIYlex+//VRDwPg40RaMG+UAF8sT158AGKpfuS8DXz p6jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5D8mlvVawCjEgddelLAduhKCIQh0CPiyGgQp446bGsE=; b=LW/8/i0ZxKoMhX6DjTzpSQUmxBqqCroU534kFZChh5AJf/L2WFn4oZus5Mwp85Pi83 tT3l0vpmS6FS+GK7mhxe+xIagFJmEwV5V8U5ImeTd31Uu9CXkaoMKsnacg3HdCUs3AEy l0AZMeR8qxDuVH+N5P4AKXeKfwgMl/0SIA/+LYdgYlQg7zDelFei2/sYKKg6X4dC4KpH u4LEls75f+ShOUx4mzEgmzFrbkZaqmNCMKpfgjWDFfUSZQmKJcykJx5IZEd8iKqyGFqX um/ZpPaOtT1HgYlRpb2LEH1qHHBzkrxzzQCWEpJcNatT3xwf//zc48Gb7/eg9Hdw3Ziw uGbg== X-Gm-Message-State: APjAAAXsXs3iQMeznH34n0ckrxt1tFqO/RC3En6qfAfRACzXPnJHOpOE LL5V9A5nlR+JLGAr1HmsfEg= X-Google-Smtp-Source: APXvYqzvQ0JQLAGkRNhRdaqxRVuO7Hvjn0EZosyRHwPjvqeoogbj3QaW6JZ3XDhFK6dpEIze4WbX/w== X-Received: by 2002:a05:620a:69c:: with SMTP id f28mr22093343qkh.274.1563296206729; Tue, 16 Jul 2019 09:56:46 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id f20sm8519538qkh.15.2019.07.16.09.56.45 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 16 Jul 2019 09:56:46 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [RFC v1 3/4] arm64, kexec: add kexec's own identity page table Date: Tue, 16 Jul 2019 12:56:40 -0400 Message-Id: <20190716165641.6990-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190716165641.6990-1-pasha.tatashin@soleen.com> References: <20190716165641.6990-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190716_095647_875357_5DD325D9 X-CRM114-Status: GOOD ( 14.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Allocate and configure identity page table to be used for kexec reboot. Note, for now we still have MMU disabled during kernel relocation phase, so this table is still used the same as idmap_pg_dir was used. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/machine_kexec.c | 78 ++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index f4565eb01d09..60433c264178 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -20,6 +21,7 @@ #include #include #include +#include #include "cpu-reset.h" @@ -55,6 +57,77 @@ static void _kexec_image_info(const char *func, int line, } } +/* Allocates pages for kexec page table */ +static void *kexec_pgtable_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + return page_address(page); +} + +/* + * Create identity mapped page table for kexec purposes. The flags that are used + * in this page table are the same as what is set in __create_page_tables. The + * page table is needed for performance reasons. Without it, kernel relocation + * is rather slow, because when MMU is off, d-caching is disabled as well. + */ +static int +kexec_create_pgtable(struct kimage *kimage) +{ + void *pgd_page = kexec_pgtable_alloc(kimage); + phys_addr_t kexec_pgtable; + int rv, i; + struct memblock_region *reg; + struct ident_map_info info = { + .alloc_pgt_page = kexec_pgtable_alloc, + .alloc_arg = kimage, + .page_flags = PMD_SECT_VALID | PMD_SECT_AF | PMD_SECT_S | + PMD_ATTRINDX(MT_NORMAL), + .offset = 0, + .pud_pages = false, + }; + + if (!pgd_page) + return -ENOMEM; + + clear_page(pgd_page); + kexec_pgtable = __pa(pgd_page); + + for_each_memblock(memory, reg) { + phys_addr_t mstart = reg->base; + phys_addr_t mend = reg->base + reg->size; + + rv = ident_map_pgd_populate(&info, kexec_pgtable, mstart, mend); + if (rv) + return rv; + } + + /* + * It is possible new kernel knows of some physical addresses that this + * kernel does not know: for example a different device tree might + * provide information of a memory region, or memory could have been + * reduced via mem= kernel parameter. + * This is why also unconditionally map new kernel segments, even though + * most likely this is redundant. + */ + for (i = 0; i < kimage->nr_segments; i++) { + phys_addr_t mstart = kimage->segment[i].mem; + phys_addr_t mend = mstart + kimage->segment[i].memsz; + + rv = ident_map_pgd_populate(&info, kexec_pgtable, mstart, mend); + if (rv) + return rv; + } + + kimage->arch.kexec_pgtable = pgd_page; + + return 0; +} + void machine_kexec_cleanup(struct kimage *kimage) { /* Empty routine needed to avoid build errors. */ @@ -70,6 +143,7 @@ void machine_kexec_cleanup(struct kimage *kimage) int machine_kexec_prepare(struct kimage *kimage) { void *reloc_buf = page_address(kimage->control_code_page); + int rv; if (kimage->type != KEXEC_TYPE_CRASH && cpus_are_stuck_in_kernel()) { pr_err("Can't kexec: CPUs are stuck in the kernel.\n"); @@ -84,10 +158,10 @@ int machine_kexec_prepare(struct kimage *kimage) arm64_relocate_new_kernel_size); kimage->arch.relocate_kern = reloc_buf; - kimage->arch.kexec_pgtable = lm_alias(idmap_pg_dir); + rv = kexec_create_pgtable(kimage); kexec_image_info(kimage); - return 0; + return rv; } /** From patchwork Tue Jul 16 16:56:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11046513 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 613F91395 for ; Tue, 16 Jul 2019 16:58:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DE6C2866D for ; Tue, 16 Jul 2019 16:58:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 41B4028672; Tue, 16 Jul 2019 16:58:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D5B102866D for ; Tue, 16 Jul 2019 16:58:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Q3+e80Y/FyztU0yuOHA2jYYdDFbcf8M2P5OqwkZ4yig=; b=LNu/p+tL0jcqol trWKGKJXJPy8AEHr7PxJ0fbcDDG9ZTtcZW66ecrtKq7MK6UQofDHfvPAHMi86KC2osvtFUjDXvuHb enxeoPGpqmlDhCH4UKRBGEl4+HVPOArw4T2CsGaNyLkierPYe1UGPBqNOfXUJL8Qq+eMCnqqxLIrd 9J9SdvpjZ4b4XEfzrAwvLm4kdAKjnhu77lPJzQWHX48dIbUCsyQNN8IWKaj3IZutBOMp5Ci71z1uN JHEqcDJW/6KR5d/onbrHwoBhKhmSzUE/WOBdD/DRSG7Tq8HGgrOHa3YGjTWY1HHbe+NV8SJD8Aw7O FfXOfma3nlhGseG08+2A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1hnQmZ-0006Jx-0Y; Tue, 16 Jul 2019 16:58:27 +0000 Received: from mail-qt1-x844.google.com ([2607:f8b0:4864:20::844]) by bombadil.infradead.org with esmtps (Exim 4.92 #3 (Red Hat Linux)) id 1hnQkz-00055H-J1 for linux-arm-kernel@lists.infradead.org; Tue, 16 Jul 2019 16:56:51 +0000 Received: by mail-qt1-x844.google.com with SMTP id 44so20246024qtg.11 for ; Tue, 16 Jul 2019 09:56:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=OHreDCpotkEUucxoE5EiEKrcc9yzecv+duVH9vOORKc=; b=GGIQw/kgAUl/eG3d2NKYO7rrigP3xCd6L3ITkDfbZyMEUnoEj0yVHLsV2LlbgRYbC+ wxGDoUKvbsOD49xDxxVjVMA/cVW24Ta7j9hj3+wxoFsCV7bE/6zX1BcHliTfmRJa2HL6 t1GMulrmuUdkYdtWJR3tWcDBbewttH11SYty1lYTYPi9/xqOKf63ADEYQ5vRWxW8OT6H VHjxbQVnM+5FFok1c8bap5vm+E40C8t2yYrwiy22m4L+k24TpV5SxJbfpoBHfRpCqzb2 H6dm0a1hIjm8g/mZThetxdugoRf8Pcyv6ZduRuCZRIwQmu9srFbLpacSM3DPtY1OABO9 4vlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OHreDCpotkEUucxoE5EiEKrcc9yzecv+duVH9vOORKc=; b=dhhBgqPBYCTciL99jeda9JHgecemgsvsYyCuOs0KLrYs/s95CJziNZI0GshMc7DK6H Ck1yibY6OK/ZacIHAIKjZrEOXjPd4H5jNW7wntCvTSdLYSZAwFvvd7Xpc7Q4DvzJeNUV 2Pu/sK+vwBxl1RpXFeLmJS5L7zWIeCJDraRJRINT7RTdoOjhY5IhoC2k6KI+kkYs5kOX VVXWpX3ecJQ8aJrVz5EOZM8aej0Xle9x5xwkyPoNLtnYz45NSzAJocyfESO/hmko1Ul5 LsGbApUXXxaDC0O62LaU2PMGRIw4CiLQQaPcy+pdZgpZQ+zaPBmcr71slYbpRwjlTWDd Z8iw== X-Gm-Message-State: APjAAAUuYFffq6/aU2vw3msegcyT7djpCwFYGou9vjud4dsNQH91zS2W gphnRtmzi16CDpTZjNEUQbA= X-Google-Smtp-Source: APXvYqwxJ0A/FHhU5OVW66jrPFnnHKm79duNOHFiKTe25GUCXmo7/DTY10UCE+L1XIgV2bRdwMnQ6w== X-Received: by 2002:ac8:41d1:: with SMTP id o17mr23857397qtm.17.1563296207876; Tue, 16 Jul 2019 09:56:47 -0700 (PDT) Received: from localhost.localdomain (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id f20sm8519538qkh.15.2019.07.16.09.56.46 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Tue, 16 Jul 2019 09:56:47 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [RFC v1 4/4] arm64: Keep MMU on while kernel is being relocated Date: Tue, 16 Jul 2019 12:56:41 -0400 Message-Id: <20190716165641.6990-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190716165641.6990-1-pasha.tatashin@soleen.com> References: <20190716165641.6990-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190716_095649_697406_C379A0FC X-CRM114-Status: UNSURE ( 9.57 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP It is inefficient to do kernel relocation with MMU disabled. This is because if MMU is disabled, dcache must also be disabled. Now, that we have identity page table we can disable MMU after relocation is completed. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/cpu-reset.S | 8 ------- arch/arm64/kernel/relocate_kernel.S | 36 ++++++++++++++++++----------- 2 files changed, 22 insertions(+), 22 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 6ea337d464c4..d5cfc17b8e1f 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -30,14 +30,6 @@ * flat identity mapping. */ ENTRY(__cpu_soft_restart) - /* Clear sctlr_el1 flags. */ - mrs x12, sctlr_el1 - ldr x13, =SCTLR_ELx_FLAGS - bic x12, x12, x13 - pre_disable_mmu_workaround - msr sctlr_el1, x12 - isb - cbz x0, 1f // el2_switch? mov x0, #HVC_SOFT_RESTART hvc #0 // no return diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index c1d7db71a726..e2724fedd082 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -36,18 +36,6 @@ ENTRY(arm64_relocate_new_kernel) mov x14, xzr /* x14 = entry ptr */ mov x13, xzr /* x13 = copy dest */ - /* Clear the sctlr_el2 flags. */ - mrs x0, CurrentEL - cmp x0, #CurrentEL_EL2 - b.ne 1f - mrs x0, sctlr_el2 - ldr x1, =SCTLR_ELx_FLAGS - bic x0, x0, x1 - pre_disable_mmu_workaround - msr sctlr_el2, x0 - isb -1: - /* Check if the new image needs relocation. */ tbnz x16, IND_DONE_BIT, .Ldone @@ -63,10 +51,10 @@ ENTRY(arm64_relocate_new_kernel) add x20, x0, #PAGE_SIZE sub x1, x15, #1 bic x0, x0, x1 -2: dc ivac, x0 +1: dc ivac, x0 add x0, x0, x15 cmp x0, x20 - b.lo 2b + b.lo 1b dsb sy mov x20, x13 @@ -104,6 +92,26 @@ ENTRY(arm64_relocate_new_kernel) dsb nsh isb + /* Clear sctlr_el1 flags. */ + mrs x12, sctlr_el1 + ldr x13, =SCTLR_ELx_FLAGS + bic x12, x12, x13 + pre_disable_mmu_workaround + msr sctlr_el1, x12 + isb + + /* Clear the sctlr_el2 flags. */ + mrs x0, CurrentEL + cmp x0, #CurrentEL_EL2 + b.ne 2f + mrs x0, sctlr_el2 + ldr x1, =SCTLR_ELx_FLAGS + bic x0, x0, x1 + pre_disable_mmu_workaround + msr sctlr_el2, x0 + isb +2: + /* Start new image. */ mov x0, x18 mov x1, xzr