From patchwork Mon Sep 23 20:34:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157567 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 12B3917D4 for ; Mon, 23 Sep 2019 20:34:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D2FFB21783 for ; Mon, 23 Sep 2019 20:34:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="g7DE7rX2" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D2FFB21783 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BF1B76B0281; Mon, 23 Sep 2019 16:34:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BA39C6B0282; Mon, 23 Sep 2019 16:34:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A92A56B0283; Mon, 23 Sep 2019 16:34:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id 82D986B0281 for ; Mon, 23 Sep 2019 16:34:41 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 328CA909A for ; Mon, 23 Sep 2019 20:34:41 +0000 (UTC) X-FDA: 75967338762.21.pot99_8028266012f24 X-Spam-Summary: 2,0,0,ca7a6013a2a7d720,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3653:3867:3868:3871:3872:3874:4321:5007:6261:6653:6737:9390:10004:11026:11658:11914:12048:12296:12297:12517:12519:12555:12663:12895:12986:13069:13095:13311:13357:13894:14040:14181:14384:14394:14721:21080:21325:21433:21444:21451:21611:21627:30054:30069,0,RBL:209.85.210.194:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Do mainCach X-HE-Tag: pot99_8028266012f24 X-Filterd-Recvd-Size: 4380 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:40 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id q12so9821800pff.9 for ; Mon, 23 Sep 2019 13:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=KZlSM1+HYlx5AabYJySSHWoh4S6JQNg+E/eloF1WuIo=; b=g7DE7rX2uCp8tfoUWCXCLl034/utkvLPBdVIKc8+powWUExFWSItdszP8Cm/yR3bQA INyWFFWB57bbdD2iHNW1/fZENA+1VBlqYaMxRwKUwsr13wORyC7Njz/SoqYlIMCXWndy +2lPVOCdcRmjy3opEnn14+U9nd/vOCxsE7gGnmOczmK8wF6uSKK9ePWHSOSG4IdNhWL4 i6B0jQ3c/Hc+3IQ4XVvxJF0T0VSgEARneLSQ8kgrBRVlnEy0OVPrm3FAAWvjOBu96um6 TJyjAuN1XqnAdwdM/bythEqz6HBUc9ctqw1AsAPVwpa9SaVxY5Z1wKyBAHCKwQ0/s4aD R+OA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KZlSM1+HYlx5AabYJySSHWoh4S6JQNg+E/eloF1WuIo=; b=hHvanH+GSdRtyhcKRqAVn8uEfMRaKtuHGMugjArR4jvr2oCnxGzyAAY6NtYOQaNj1D jYOvoBnaT4p8XOzD23aGpAe9m/NAG0vRETvC+ITCoyUSmC3M6LIDEWrD/gnEZNkt9aAV /lkzai8hd+UYQd39Db5zJHk3EM85zVmSrBpwbsRVlkuRada0qVl1GfDCqDU5b92puvQt CoCFtqG4bUWTqQvk8NiDHWgFia+VNHOeCdf5KANNmMpYdWejoCWOVaDBUj0XUqvufmJV At1gtj4txY/pqIKf1IjoLLMTgtxdSTyDc7e/F15fu53APr+PMwCXj1k0zW6tnTUq1TzS kj5g== X-Gm-Message-State: APjAAAWJTy9A/ZfUweVvOTxBTn3j5DLj0lRM13vt8sa2iYc0WkovjwKf MtWV/rHRL5YUxW7DNhM5g9Lf+Q== X-Google-Smtp-Source: APXvYqy145da1OUiQVlRDiBy/OIzvCU32t+7Zhz/DSbXEWYlU+3nmVsNL564Bq6FlkTk3IW8uBpmsA== X-Received: by 2002:a63:c947:: with SMTP id y7mr1755459pgg.345.1569270879589; Mon, 23 Sep 2019 13:34:39 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:38 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 01/17] kexec: quiet down kexec reboot Date: Mon, 23 Sep 2019 16:34:11 -0400 Message-Id: <20190923203427.294286-2-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Here is a regular kexec command sequence and output: ===== $ kexec --reuse-cmdline -i --load Image $ kexec -e [ 161.342002] kexec_core: Starting new kernel Welcome to Buildroot buildroot login: ===== Even when "quiet" kernel parameter is specified, "kexec_core: Starting new kernel" is printed. This message has KERN_EMERG level, but there is no emergency, it is a normal kexec operation, so quiet it down to appropriate KERN_NOTICE. Machines that have slow console baud rate benefit from less output. Signed-off-by: Pavel Tatashin Reviewed-by: Simon Horman Acked-by: Dave Young --- kernel/kexec_core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index d5870723b8ad..2c5b72863b7b 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -1169,7 +1169,7 @@ int kernel_kexec(void) * CPU hotplug again; so re-enable it here. */ cpu_hotplug_enable(); - pr_emerg("Starting new kernel\n"); + pr_notice("Starting new kernel\n"); machine_shutdown(); } From patchwork Mon Sep 23 20:34:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157569 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AEA6817EE for ; Mon, 23 Sep 2019 20:34:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7118F2168B for ; Mon, 23 Sep 2019 20:34:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="NPdkNIm0" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7118F2168B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7FD9C6B0283; Mon, 23 Sep 2019 16:34:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 75EC36B0284; Mon, 23 Sep 2019 16:34:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64CFA6B0285; Mon, 23 Sep 2019 16:34:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0124.hostedemail.com [216.40.44.124]) by kanga.kvack.org (Postfix) with ESMTP id 44BEF6B0283 for ; Mon, 23 Sep 2019 16:34:44 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id D3911824376A for ; Mon, 23 Sep 2019 20:34:43 +0000 (UTC) X-FDA: 75967338846.18.bag95_809055872913b X-Spam-Summary: 2,0,0,2f5e1160caee3331,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:3138:3139:3140:3141:3142:3352:3865:3866:3867:3870:3871:3872:4250:5007:6117:6119:6261:6653:6737:7901:7903:10004:11026:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12895:13069:13153:13228:13311:13357:13894:14093:14096:14181:14384:14394:14721:21080:21444:21451:21627:21740:30036:30054,0,RBL:209.85.214.194:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5 ,0.5,0.5 X-HE-Tag: bag95_809055872913b X-Filterd-Recvd-Size: 4802 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:43 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id q24so6979161plr.13 for ; Mon, 23 Sep 2019 13:34:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3zTijyt8B1oVtT9JT6yCKls7nJQIDBWuYolaWDAK7ic=; b=NPdkNIm0wrZOKAEjjebCfOXhVt9BA9UfqiHQ6RuQplwNC6RxIM1b11Cd8A5afWNV07 O6nrgyrp1NE11jqQQCyB9u5TPHar+d5IM2aGGaZIYmpdW8dv7YMiMNhr1NSeN1HdA17Q qA5tfNc/q4JkZV2xI+YaOcH/q+gypgvpqre3Jkl1CqhZ7D587jfQ0Spvdkvxzjv8zD2N CRZ62EB12Q0TusTaPfzTP14sUgiy/v1N1+PCc2ulUmZTK1P7yRF5UBuYnlTdZVPkmV33 LOXk1q2Arz2juRQsSya+z99tARblhJgNSrG3P6W3AXLNVCUaP7uV2C1cIZnhzMYJ/rur HjSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3zTijyt8B1oVtT9JT6yCKls7nJQIDBWuYolaWDAK7ic=; b=N+82zSoBqlJRXlQhh8+K82d0Hb9YVD3YfDlsz2nS/v5ExXvMAwExjF3805mLkJ7wy+ HQZY36nKhRbU1u2n3Pmxfv6JvbQXwU0CjGce7SJrgWpGQtfvCteCD+cpCn++PWCL5ZQb ZuU0lT6Cf3KUtuouilrxiizcX0y65WbeZXL7+rdudNXh25GTNjQX7z10DO6ZkMFr+dHe 8G3FylMgV0REE8u4LTm/H97I7J/bdXeWT60EPM34kHGzxUL51N9Le5rJ9KyauAapnI2m 6gpgSx79eEHrx2RIOxc7mPPYSHwOkDSTX5rpqunDrIorY6tIm3e37KMlfxvLqnzwpcxH IPSw== X-Gm-Message-State: APjAAAWVgWw1pH5ZLtFNdc25xJwxLZ8vnmNO7pvbLmfZr+cO8rxZuRUB vsU4pjnEdchA4CYSR7ylLHJXTg== X-Google-Smtp-Source: APXvYqyM+tozPszKt/AHnpXbGQE8m+WydlkTbUy8bM7XQTkOKWP6qGml1eU2fXHZcpiqnRJpO7UwYA== X-Received: by 2002:a17:902:ba89:: with SMTP id k9mr1747223pls.44.1569270882293; Mon, 23 Sep 2019 13:34:42 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:41 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 02/17] arm64: hibernate: pass the allocated pgdp to ttbr0 Date: Mon, 23 Sep 2019 16:34:12 -0400 Message-Id: <20190923203427.294286-3-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ttbr0 should be set to the beginning of pgdp, however, currently in create_safe_exec_page it is set to pgdp after pgd_offset_raw(), which works by accident. Fixes: 0194e760f7d2 ("arm64: hibernate: avoid potential TLB conflict") Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hibernate.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index e0a7fce0e01c..d52f69462c8f 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -201,6 +201,7 @@ static int create_safe_exec_page(void *src_start, size_t length, gfp_t mask) { int rc = 0; + pgd_t *trans_pgd; pgd_t *pgdp; pud_t *pudp; pmd_t *pmdp; @@ -215,7 +216,8 @@ static int create_safe_exec_page(void *src_start, size_t length, memcpy((void *)dst, src_start, length); __flush_icache_range(dst, dst + length); - pgdp = pgd_offset_raw(allocator(mask), dst_addr); + trans_pgd = allocator(mask); + pgdp = pgd_offset_raw(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { pudp = allocator(mask); if (!pudp) { @@ -262,7 +264,7 @@ static int create_safe_exec_page(void *src_start, size_t length, */ cpu_set_reserved_ttbr0(); local_flush_tlb_all(); - write_sysreg(phys_to_ttbr(virt_to_phys(pgdp)), ttbr0_el1); + write_sysreg(phys_to_ttbr(virt_to_phys(trans_pgd)), ttbr0_el1); isb(); *phys_dst_addr = virt_to_phys((void *)dst); From patchwork Mon Sep 23 20:34:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157571 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5359D17EE for ; Mon, 23 Sep 2019 20:34:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 15D0021783 for ; Mon, 23 Sep 2019 20:34:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="oWTaiXQU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15D0021783 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0119D6B0285; Mon, 23 Sep 2019 16:34:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F02CB6B0287; Mon, 23 Sep 2019 16:34:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D81A46B0285; Mon, 23 Sep 2019 16:34:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id AB54C6B0285 for ; Mon, 23 Sep 2019 16:34:46 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 5B067180AD808 for ; Mon, 23 Sep 2019 20:34:46 +0000 (UTC) X-FDA: 75967338972.05.guide72_80eb32a7c3519 X-Spam-Summary: 2,0,0,f945fe4e6ec46617,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1534:1541:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:2896:3138:3139:3140:3141:3142:3352:3867:3870:3871:3872:3874:4321:5007:6117:6119:6120:6261:6653:6737:7901:7903:10004:11026:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12895:13069:13161:13229:13311:13357:13870:13894:14096:14181:14384:14394:14721:21080:21444:21451:21627:30054:30070,0,RBL:209.85.214.194:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5, 0.5,0.5, X-HE-Tag: guide72_80eb32a7c3519 X-Filterd-Recvd-Size: 4278 Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:45 +0000 (UTC) Received: by mail-pl1-f194.google.com with SMTP id u12so6996104pls.12 for ; Mon, 23 Sep 2019 13:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=4PNhEWMnxTPqUB+Gwx2nqJhU+kkzlGq5UWgg81CPAOk=; b=oWTaiXQUOW/n+bZ3aNPNA4OtwzBJkIcKJ7w8UHidzdwXhLyBh7H6XeCFOH4hrLa0TF 5tO48HD+yimpY3hC6jgRBluPwrr5tgxt77/q6RSoQIg8t/tRCWOU6l8oYNCNvsXRKB4D Z8cHhyCyQ6gPEYAU2u5cdpMqDzvOZEnFOvri1Ewy4jmdOYjAh+x8UWdyU39v2SL5rPSV K4dkoFFYKtiBQ1BjRZfz7omkXdFczekyPXcpEcNWyxECRtdRYNCDtUFUDOsq8wHx+W9f xkjwmwwuLO8WLQ/ZZtLJsCija11LteHo6qyLnvLelyuoP2vORsoEtQA4U9vpzDPIcO1V 1N7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4PNhEWMnxTPqUB+Gwx2nqJhU+kkzlGq5UWgg81CPAOk=; b=VRVdUcyJlZzSDs/6nDRRJeMc2unA/BJWbZeveMZ7X/BYJN7NDilIfbta6C7jLZAVAY 8gqRePnTTE2VNQxpv7sXpVyAHrIFZEEM9EpF8PfTJqmWOFvFv1Pd9jeMBay31X4wwXyN LY3hGtDBMAmrK9jak5PRZKl+TDqsR+EgFH++/ZLYTdjhgAuVjFiEi4GnxquiTVE6iMYi KaDJ4586RSXrGDnJaL2SBESYcBFrdaU6f+s30VQ3gJncMNnv1l+Arzx/yxQ5/O4nBTLV jtZgMMoyhpLTh7DChZAyZGBzGvanN14mTLgqTAdxRB3PzfMaV5AIybn+qCw2AwqlXY/h KBKA== X-Gm-Message-State: APjAAAWgMctI/FhTJajesIqpPXmSdPe0OwNP4bPvwDc4azb19fkpjG+q d74tVBZyKEDMJOF0uetYSZg3qA== X-Google-Smtp-Source: APXvYqzyh38AtgfcuB6R0U6M0N80Ipw7rS/QgbnW4pA75DgBWBEPXi74l8ts5ld5MRP7duUrSaZIBA== X-Received: by 2002:a17:902:8492:: with SMTP id c18mr1658221plo.279.1569270884919; Mon, 23 Sep 2019 13:34:44 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:44 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 03/17] arm64: hibernate: check pgd table allocation Date: Mon, 23 Sep 2019 16:34:13 -0400 Message-Id: <20190923203427.294286-4-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a bug in create_safe_exec_page(), when page table is allocated it is not checked that table is allocated successfully: But it is dereferenced in: pgd_none(READ_ONCE(*pgdp)). Check that allocation was successful. Fixes: 82869ac57b5d ("arm64: kernel: Add support for hibernate/suspend-to-disk") Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hibernate.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index d52f69462c8f..ef46ce66d7e8 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -217,6 +217,11 @@ static int create_safe_exec_page(void *src_start, size_t length, __flush_icache_range(dst, dst + length); trans_pgd = allocator(mask); + if (!trans_pgd) { + rc = -ENOMEM; + goto out; + } + pgdp = pgd_offset_raw(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { pudp = allocator(mask); From patchwork Mon Sep 23 20:34:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157573 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DC9717D4 for ; Mon, 23 Sep 2019 20:34:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 20C5120673 for ; Mon, 23 Sep 2019 20:34:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="dFemtziU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 20C5120673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 16C416B0287; Mon, 23 Sep 2019 16:34:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 11E186B0288; Mon, 23 Sep 2019 16:34:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 032376B0289; Mon, 23 Sep 2019 16:34:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0233.hostedemail.com [216.40.44.233]) by kanga.kvack.org (Postfix) with ESMTP id D180F6B0287 for ; Mon, 23 Sep 2019 16:34:49 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 48FDC180AD808 for ; Mon, 23 Sep 2019 20:34:49 +0000 (UTC) X-FDA: 75967339098.19.chalk06_814f482299d5b X-Spam-Summary: 2,0,0,9abbc74ac1698c44,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1542:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3874:4117:4321:4419:5007:6117:6119:6261:6653:6737:7903:10004:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12895:12986:13894:14181:14394:14721:21063:21080:21433:21444:21451:21627:30012:30054,0,RBL:209.85.214.196:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5, Netcheck X-HE-Tag: chalk06_814f482299d5b X-Filterd-Recvd-Size: 6274 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf22.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:48 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id q15so6991138pll.11 for ; Mon, 23 Sep 2019 13:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MH284WpfdMmTT4sRM0t5AqX3XpYou6gu7tWPymm7DjQ=; b=dFemtziUUFaRYULYpBmu8NJmI/dbD4cbh1SYA7qB7chGTOR6LFk7fQr6OFdm2Yx62O 3dpbS1qsBx8c9nCZNRgwfg9bKOijVMBFikwwnEjYXpVcTl4t9HBqGQBnjciA7ttafi8d KDKRj/LLWiGmuAfNEqz8+vSSUcdJjH/QdS1UQxIoe1gBHiZcgrte8geeaSyAdZTwwgLr ne2TUvQVTeDJ520F9gX/mMBrJUUtKsJzX7KjCXEYNUuUVdoGWzR/nWBrwoLWgDp/YPlr 6/2j4PssNh9GleF3Uvl7WCswZ7cirfpyTjmuaAZPVr6MymjPpiGuXB2sVm1Q3Hym4IFq cVvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MH284WpfdMmTT4sRM0t5AqX3XpYou6gu7tWPymm7DjQ=; b=EYUbNaL1+nV+bVxVRnxPB55ydFaFRNQgQlEl5EjwzzJgSc1KkRuSl97BsVWyEXDQ96 7UHwInf1D4AxXDjPzMgceKkqSgsSqI7KY2NjAuMHOiwtpFBMgzgfWVp5/hNBsLdjOxT5 quo9g6+t8DYPO2R5RdsLltzjW8fe+uQa/X8N52BGLr0P7/jSGOyP5T7SNrFHyzUJbDIG CmZ2Epl1HZUtrTVa2tvCEabeGFXUB21VqwqWE5TIeiJrZKV7EdnBPuzcZZk3JCoXk3SJ 94oDqyMcNHPuBidjGl9YD7SwQbnZHGNFYUgnp8tdk43rKfFnZEZjWdvJZ9utGCao5ueO NIJw== X-Gm-Message-State: APjAAAVzehPlJWwgPSFT9+bmUsG2AHdlw+8Tg+Vi8jnHxf/GwzGttJPq Dua9/6BUpLVYx7JgBk2RxALlkg== X-Google-Smtp-Source: APXvYqzsQjO1TnwmDcDODrVrxdzYDe42E+P66K/SdBySMPGcvfbOVfLz14Ppu7ESXE+9kNDFkqZ90w== X-Received: by 2002:a17:902:8bca:: with SMTP id r10mr1657503plo.43.1569270887591; Mon, 23 Sep 2019 13:34:47 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:46 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 04/17] arm64: hibernate: use get_safe_page directly Date: Mon, 23 Sep 2019 16:34:14 -0400 Message-Id: <20190923203427.294286-5-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: create_safe_exec_page() uses hibernate's allocator to create a set of page table to map a single page that will contain the relocation code. Remove the allocator related arguments, and use get_safe_page directly, as it is done in other local functions in this file to simplify function prototype. Removing this function pointer makes it easier to refactor the code later. Signed-off-by: Pavel Tatashin Reviewed-by: Matthias Brugger --- arch/arm64/kernel/hibernate.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index ef46ce66d7e8..34297716643f 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -196,9 +196,7 @@ EXPORT_SYMBOL(arch_hibernation_header_restore); */ static int create_safe_exec_page(void *src_start, size_t length, unsigned long dst_addr, - phys_addr_t *phys_dst_addr, - void *(*allocator)(gfp_t mask), - gfp_t mask) + phys_addr_t *phys_dst_addr) { int rc = 0; pgd_t *trans_pgd; @@ -206,7 +204,7 @@ static int create_safe_exec_page(void *src_start, size_t length, pud_t *pudp; pmd_t *pmdp; pte_t *ptep; - unsigned long dst = (unsigned long)allocator(mask); + unsigned long dst = get_safe_page(GFP_ATOMIC); if (!dst) { rc = -ENOMEM; @@ -216,7 +214,7 @@ static int create_safe_exec_page(void *src_start, size_t length, memcpy((void *)dst, src_start, length); __flush_icache_range(dst, dst + length); - trans_pgd = allocator(mask); + trans_pgd = (void *)get_safe_page(GFP_ATOMIC); if (!trans_pgd) { rc = -ENOMEM; goto out; @@ -224,7 +222,7 @@ static int create_safe_exec_page(void *src_start, size_t length, pgdp = pgd_offset_raw(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { - pudp = allocator(mask); + pudp = (void *)get_safe_page(GFP_ATOMIC); if (!pudp) { rc = -ENOMEM; goto out; @@ -234,7 +232,7 @@ static int create_safe_exec_page(void *src_start, size_t length, pudp = pud_offset(pgdp, dst_addr); if (pud_none(READ_ONCE(*pudp))) { - pmdp = allocator(mask); + pmdp = (void *)get_safe_page(GFP_ATOMIC); if (!pmdp) { rc = -ENOMEM; goto out; @@ -244,7 +242,7 @@ static int create_safe_exec_page(void *src_start, size_t length, pmdp = pmd_offset(pudp, dst_addr); if (pmd_none(READ_ONCE(*pmdp))) { - ptep = allocator(mask); + ptep = (void *)get_safe_page(GFP_ATOMIC); if (!ptep) { rc = -ENOMEM; goto out; @@ -530,8 +528,7 @@ int swsusp_arch_resume(void) */ rc = create_safe_exec_page(__hibernate_exit_text_start, exit_size, (unsigned long)hibernate_exit, - &phys_hibernate_exit, - (void *)get_safe_page, GFP_ATOMIC); + &phys_hibernate_exit); if (rc) { pr_err("Failed to create safe executable page for hibernate_exit code.\n"); goto out; From patchwork Mon Sep 23 20:34:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157575 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40CA417D4 for ; Mon, 23 Sep 2019 20:34:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F3CE720882 for ; Mon, 23 Sep 2019 20:34:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="T60+YUHd" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3CE720882 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7D5A06B0289; Mon, 23 Sep 2019 16:34:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7AC836B028A; Mon, 23 Sep 2019 16:34:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6733D6B028B; Mon, 23 Sep 2019 16:34:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0045.hostedemail.com [216.40.44.45]) by kanga.kvack.org (Postfix) with ESMTP id 427936B0289 for ; Mon, 23 Sep 2019 16:34:52 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id D20C5181AC9BA for ; Mon, 23 Sep 2019 20:34:51 +0000 (UTC) X-FDA: 75967339182.17.match77_81b47d17b6762 X-Spam-Summary: 2,0,0,18dfd46a75ccea71,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1543:1711:1730:1747:1777:1792:2194:2198:2199:2200:2393:2559:2562:2731:2914:3138:3139:3140:3141:3142:3355:3622:3865:3866:3867:3868:3870:3871:3874:4118:4321:4605:5007:6117:6119:6261:6653:6737:7875:7903:10004:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12683:12895:12986:13894:14110:14181:14394:14721:21080:21121:21444:21451:21627:30012:30054,0,RBL:209.85.214.193:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none ,Bayesia X-HE-Tag: match77_81b47d17b6762 X-Filterd-Recvd-Size: 7062 Received: from mail-pl1-f193.google.com (mail-pl1-f193.google.com [209.85.214.193]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:51 +0000 (UTC) Received: by mail-pl1-f193.google.com with SMTP id q24so6979341plr.13 for ; Mon, 23 Sep 2019 13:34:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=54zhWvY2wVoS8DJjdyrocZPF+2/4Isg+VJgEApzMseg=; b=T60+YUHd1pT2vUT24jaOnMItUPfL3L23oM9QlMa+jsQhTAMmE/Y6MOdB9kPjxsU2YE s0+Pj9I26CF7T263iX4+bryVjK3tjDXFjfcOSBZdy7hV7MpKa4gjlFgld8gCis5i0WKu DA5x0mdvLTvptlXKZYlRoL1Td1daAq/CiJc4sgJPCp/DOfklSjrpaiuYytsyCwDbIPep RrMtL9f00tDwSIjrtVkqS7eAQK9SjIWmyh9i5SqhoiJMe5ly6HuMgRWYiOWumRtRxhtF MIQP2OUZMi20ZuYed85vWrpjF7HIlaVilmAfvJ0kbcEB92CCSNdKtM1cYEJ0Wqv69SHa Igrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=54zhWvY2wVoS8DJjdyrocZPF+2/4Isg+VJgEApzMseg=; b=Lo49RXwxZwWGw0CnDwp5yCRxSeBj2MQxRrBIZe/zMkhTXFLujRjX4E3T69GAxQEmUt RZr6fUb+8YoOdkfr/Ajwp03lq0FjGyI0dpnNkPqRVJ/xkUuz/MmQDq9hE64BH7bboeZr 7IA20vrSKnbx7JkcbM5Nzsn1+qc6MfZzlIyz+g6ty11pHnQl3yggUofYTmk903G4OB95 L4ltXlp1/pqs/E7pGTThZfwASG/2QH6ViITiJV6PtwLwOvrozruJTlGejZnQBqfJGdxM G/G1Mj+APim022zWxZfZWIUbUD2NlDeWr6Xz1TdrmvVBboFYkJlDQAjJFQIRIUv2hFBK OFJg== X-Gm-Message-State: APjAAAXspm9BE6ZHhjroGzDrngzMRxOqizHLV27qLsRPbGIKiEPra/8L 5Ba1+uhJ4odCek1PTz8edcDciA== X-Google-Smtp-Source: APXvYqwUHTTCL6RSrWrnHViIB5bAHeudUhgSk3W0cjjZa1o+jEKfZ6t0y0OfB7SpczrqbsafTznlZg== X-Received: by 2002:a17:902:a58a:: with SMTP id az10mr1704614plb.42.1569270890214; Mon, 23 Sep 2019 13:34:50 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:49 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 05/17] arm64: hibernate: remove gotos as they are not needed Date: Mon, 23 Sep 2019 16:34:15 -0400 Message-Id: <20190923203427.294286-6-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Usually, gotos are used to handle cleanup after exception, but in case of create_safe_exec_page and swsusp_arch_resume there are no clean-ups. So, simply return the errors directly. Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/kernel/hibernate.c | 49 ++++++++++++----------------------- 1 file changed, 17 insertions(+), 32 deletions(-) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 34297716643f..83c41a2f8400 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -198,7 +198,6 @@ static int create_safe_exec_page(void *src_start, size_t length, unsigned long dst_addr, phys_addr_t *phys_dst_addr) { - int rc = 0; pgd_t *trans_pgd; pgd_t *pgdp; pud_t *pudp; @@ -206,47 +205,37 @@ static int create_safe_exec_page(void *src_start, size_t length, pte_t *ptep; unsigned long dst = get_safe_page(GFP_ATOMIC); - if (!dst) { - rc = -ENOMEM; - goto out; - } + if (!dst) + return -ENOMEM; memcpy((void *)dst, src_start, length); __flush_icache_range(dst, dst + length); trans_pgd = (void *)get_safe_page(GFP_ATOMIC); - if (!trans_pgd) { - rc = -ENOMEM; - goto out; - } + if (!trans_pgd) + return -ENOMEM; pgdp = pgd_offset_raw(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { pudp = (void *)get_safe_page(GFP_ATOMIC); - if (!pudp) { - rc = -ENOMEM; - goto out; - } + if (!pudp) + return -ENOMEM; pgd_populate(&init_mm, pgdp, pudp); } pudp = pud_offset(pgdp, dst_addr); if (pud_none(READ_ONCE(*pudp))) { pmdp = (void *)get_safe_page(GFP_ATOMIC); - if (!pmdp) { - rc = -ENOMEM; - goto out; - } + if (!pmdp) + return -ENOMEM; pud_populate(&init_mm, pudp, pmdp); } pmdp = pmd_offset(pudp, dst_addr); if (pmd_none(READ_ONCE(*pmdp))) { ptep = (void *)get_safe_page(GFP_ATOMIC); - if (!ptep) { - rc = -ENOMEM; - goto out; - } + if (!ptep) + return -ENOMEM; pmd_populate_kernel(&init_mm, pmdp, ptep); } @@ -272,8 +261,7 @@ static int create_safe_exec_page(void *src_start, size_t length, *phys_dst_addr = virt_to_phys((void *)dst); -out: - return rc; + return 0; } #define dcache_clean_range(start, end) __flush_dcache_area(start, (end - start)) @@ -482,7 +470,7 @@ static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, */ int swsusp_arch_resume(void) { - int rc = 0; + int rc; void *zero_page; size_t exit_size; pgd_t *tmp_pg_dir; @@ -498,12 +486,11 @@ int swsusp_arch_resume(void) tmp_pg_dir = (pgd_t *)get_safe_page(GFP_ATOMIC); if (!tmp_pg_dir) { pr_err("Failed to allocate memory for temporary page tables.\n"); - rc = -ENOMEM; - goto out; + return -ENOMEM; } rc = copy_page_tables(tmp_pg_dir, PAGE_OFFSET, PAGE_END); if (rc) - goto out; + return rc; /* * We need a zero page that is zero before & after resume in order to @@ -512,8 +499,7 @@ int swsusp_arch_resume(void) zero_page = (void *)get_safe_page(GFP_ATOMIC); if (!zero_page) { pr_err("Failed to allocate zero page.\n"); - rc = -ENOMEM; - goto out; + return -ENOMEM; } /* @@ -531,7 +517,7 @@ int swsusp_arch_resume(void) &phys_hibernate_exit); if (rc) { pr_err("Failed to create safe executable page for hibernate_exit code.\n"); - goto out; + return rc; } /* @@ -558,8 +544,7 @@ int swsusp_arch_resume(void) resume_hdr.reenter_kernel, restore_pblist, resume_hdr.__hyp_stub_vectors, virt_to_phys(zero_page)); -out: - return rc; + return 0; } int hibernate_resume_nonboot_cpu_disable(void) From patchwork Mon Sep 23 20:34:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157579 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B5EE01599 for ; Mon, 23 Sep 2019 20:34:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 81D0A20673 for ; Mon, 23 Sep 2019 20:34:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="TyuvnBIn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81D0A20673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1E2936B028B; Mon, 23 Sep 2019 16:34:55 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1919E6B028C; Mon, 23 Sep 2019 16:34:55 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 088426B028D; Mon, 23 Sep 2019 16:34:54 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0024.hostedemail.com [216.40.44.24]) by kanga.kvack.org (Postfix) with ESMTP id C9B576B028B for ; Mon, 23 Sep 2019 16:34:54 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 76A3F840F for ; Mon, 23 Sep 2019 20:34:54 +0000 (UTC) X-FDA: 75967339308.01.club62_82182b063323b X-Spam-Summary: 2,0,0,3c01e78ece683316,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1542:1711:1730:1747:1777:1792:2194:2199:2393:2559:2562:3138:3139:3140:3141:3142:3353:3743:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:5007:6119:6120:6261:6653:6737:7901:7903:10004:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12895:13161:13229:13894:14096:14181:14394:14721:21080:21444:21627:30003:30025:30054:30070,0,RBL:209.85.210.193:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5 ,0.5,Net X-HE-Tag: club62_82182b063323b X-Filterd-Recvd-Size: 5506 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:53 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id x127so9831061pfb.7 for ; Mon, 23 Sep 2019 13:34:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1SIbn4I8MDCXYckkp2+GPJdKwVMA/7ROubKTL+86cvE=; b=TyuvnBIn7GTrkJOJF6O1n2kUKVjWTl/Jvq/eyl04IDGEV6GdE55xcYD+uXPR2CJYE2 Fj6EcXdXduuUt1EhJLwSvO6fcyU4oSLHJ45WMKqPaiy+ElKfo9uflXCaBjVesMZXOA4c kAyU9Y/8s4SMeBT7mZta59+mTzqWJztbv/LdkfvoSNXRFaHh1FBDK4B7ofrT3Ltjadvv Os8RakD0DNjJTCwhFlW+dpkos4GKwNH2Th5ge+OFkLPGMjqdwuQlsCwu58U0vwZfK3MJ HHNMp+Uxvbrj+Xlu5ejnfqQOXwNtQ2cUig111vDTMd10LCfxD60w2qwEwgUwnuRpN5pv C+zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1SIbn4I8MDCXYckkp2+GPJdKwVMA/7ROubKTL+86cvE=; b=jF3UPTmekoSbrPQDUyAp6SkvHgQgXyxB+gOvV3Qpn9z3B3xmUwYZdkb7kKdETMv4wa 9vALFKpFUmIxlEbflqDEMaWuBv7mOj9rvVivjnKGFzdtJN0UYly/NLYLDqPCnC4yu+9x kmVJhc8KqF/nEKaSHWbMjWBBGIC6e9p+b1tjhQ320w7zXY3wQmSX2WpdogXkdDutBnDu zdX4KWgWiJK1olpakEGlTq008yzXasTo1cSz/Y/X69ykr4Oi7eqkmhlBZtru9DTQdO8M FINRlZPpFdveusQvszdHh6w8AEvRSBv6zm1ZBtP79qBAlC4ag7r7VNHCIN0lYr/kYCoR 1iTw== X-Gm-Message-State: APjAAAXDmQwnG1+ED5Yr0mknp2nl/4jeX/oxeu3eTReUpHyuaPVseGp6 4j5WxNPHn9/P2EEUt7BGkBopDw== X-Google-Smtp-Source: APXvYqw9oGXCsJ6OvOp1oWJ6xLrYsWIQg7iPoDupFuStJ4lfLcvfFLOSkhGnPGaSagWjR4lvHn8LvA== X-Received: by 2002:a17:90a:360b:: with SMTP id s11mr1477696pjb.30.1569270892861; Mon, 23 Sep 2019 13:34:52 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:52 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 06/17] arm64: hibernate: rename dst to page in create_safe_exec_page Date: Mon, 23 Sep 2019 16:34:16 -0400 Message-Id: <20190923203427.294286-7-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: create_safe_exec_page() allocates a safe page and maps it at a specific location, also this function returns the physical address of newly allocated page. The destination VA, and PA are specified in arguments: dst_addr, phys_dst_addr However, within the function it uses "dst" which has unsigned long type, but is actually a pointers in the current virtual space. This is confusing to read. Rename dst to more appropriate page (page that is created), and also change its time to "void *" Signed-off-by: Pavel Tatashin Reviewed-by: James Morse --- arch/arm64/kernel/hibernate.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 83c41a2f8400..1ca8af685e96 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -198,18 +198,18 @@ static int create_safe_exec_page(void *src_start, size_t length, unsigned long dst_addr, phys_addr_t *phys_dst_addr) { + void *page = (void *)get_safe_page(GFP_ATOMIC); pgd_t *trans_pgd; pgd_t *pgdp; pud_t *pudp; pmd_t *pmdp; pte_t *ptep; - unsigned long dst = get_safe_page(GFP_ATOMIC); - if (!dst) + if (!page) return -ENOMEM; - memcpy((void *)dst, src_start, length); - __flush_icache_range(dst, dst + length); + memcpy(page, src_start, length); + __flush_icache_range((unsigned long)page, (unsigned long)page + length); trans_pgd = (void *)get_safe_page(GFP_ATOMIC); if (!trans_pgd) @@ -240,7 +240,7 @@ static int create_safe_exec_page(void *src_start, size_t length, } ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(dst), PAGE_KERNEL_EXEC)); + set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); /* * Load our new page tables. A strict BBM approach requires that we @@ -259,7 +259,7 @@ static int create_safe_exec_page(void *src_start, size_t length, write_sysreg(phys_to_ttbr(virt_to_phys(trans_pgd)), ttbr0_el1); isb(); - *phys_dst_addr = virt_to_phys((void *)dst); + *phys_dst_addr = virt_to_phys(page); return 0; } From patchwork Mon Sep 23 20:34:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157581 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4CA331599 for ; Mon, 23 Sep 2019 20:34:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EFBD206C2 for ; Mon, 23 Sep 2019 20:34:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="LwEAxQjM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EFBD206C2 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8C0776B028D; Mon, 23 Sep 2019 16:34:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8A0176B028E; Mon, 23 Sep 2019 16:34:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7ADA46B028F; Mon, 23 Sep 2019 16:34:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 560766B028D for ; Mon, 23 Sep 2019 16:34:57 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 00FB6824376A for ; Mon, 23 Sep 2019 20:34:56 +0000 (UTC) X-FDA: 75967339434.01.bait29_82784dc31ad02 X-Spam-Summary: 2,0,0,530b172311e5c975,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:973:982:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1534:1541:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3352:3870:3871:3874:3876:4321:4605:5007:6261:6653:6737:10004:11026:11473:11657:11658:11914:12043:12048:12297:12517:12519:12555:12895:12986:13069:13161:13229:13311:13357:13894:14181:14384:14394:14721:21080:21444:21451:21627:30054,0,RBL:209.85.210.195:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk ,SPF:fp, X-HE-Tag: bait29_82784dc31ad02 X-Filterd-Recvd-Size: 4622 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by imf41.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:56 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id y5so9845693pfo.4 for ; Mon, 23 Sep 2019 13:34:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=nna15T41FBOo8C0ScgrwWvHxD4xBB4A7rtr2LOcD0mg=; b=LwEAxQjMu7W1QVZ23k/g1S4Eyx/tFZWSFOlnPakoYnP0ZAU2dmXFAKO2Oxrbqk1VaX AfgCnMNHzL9+qHkNVurT7ZHFnvl4gqQXCtMOx1UyGRVHrhSxv9+npn68KE8GC7IT+RRe SmLG9/jB22QvYfRuUHH82/bA6S9fNOUiYGPTxCFxiHwjEU4rXZsEjXnLQsfgv+gclC7b BD1+Q3J8Jry4/pY7Bz4mzVK5LteVKPXJkGwjkNv08PKILtB+xRyQ1C9cCMUZZroxQmy5 0cFI7Ks3jDLpcpzUqeRn7R7xXfc84q8H5qJoaryaJ8zt4fmCPEkoTA40UJ1vjXj2y2et d8Cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nna15T41FBOo8C0ScgrwWvHxD4xBB4A7rtr2LOcD0mg=; b=PD1S3eHHYIfr+AoiTvX5vr+IJBWQIGQKVfh23/FZ/Tht8YfR9WUMQN81NL8hlFaYr8 EizTEl/JLav+U1eibpXxndFp7Sqi4adKxsglt40f+rkyJh5gZf+xKbfBiGXIVe8u8Vup S3Vks3xAdr0GJoU0PKbf1iLOhhhvPg70WrnJnZRjX2iaPJwczwta+VjEL7NN+T5hE23M 6VUgk9r8zJ8ZV82YxPsDveKSKLqkgiI8q3Ae2U7jJ6IhQ2bDNRObdp8Y+AM2XNEaDWhS Ah6Mb4eGq3obQ+UN+Ih2EXvdvNQUzDxZTH7kjlzoeMmODCchSC5FhX9CLeC1O5oyNL0D h1ow== X-Gm-Message-State: APjAAAVu+4C3jREsE6iKY/LPCj6n4pUoMFw8WVyGEZ4HxIBsD3F86xeJ MJqEUN9yIApCTh1nCcgvz1HuZg== X-Google-Smtp-Source: APXvYqwnL0GEjIshKzO5kl0urcaVAH5d910rW/z2mm7V4txKhm3HL8CIVxbIbYZBpIjpI74fhCfRHw== X-Received: by 2002:a65:5543:: with SMTP id t3mr1763983pgr.242.1569270895507; Mon, 23 Sep 2019 13:34:55 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:54 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 07/17] arm64: hibernate: add PUD_SECT_RDONLY Date: Mon, 23 Sep 2019 16:34:17 -0400 Message-Id: <20190923203427.294286-8-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is PMD_SECT_RDONLY that is used in pud_* function which is confusing. Signed-off-by: Pavel Tatashin Acked-by: James Morse --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/kernel/hibernate.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 3df60f97da1f..756a1dfb4f55 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -110,6 +110,7 @@ #define PUD_TABLE_BIT (_AT(pudval_t, 1) << 1) #define PUD_TYPE_MASK (_AT(pudval_t, 3) << 0) #define PUD_TYPE_SECT (_AT(pudval_t, 1) << 0) +#define PUD_SECT_RDONLY (_AT(pudval_t, 1) << 7) /* AP[2] */ /* * Level 2 descriptor (PMD). diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 1ca8af685e96..ce60bceed357 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -436,7 +436,7 @@ static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, return -ENOMEM; } else { set_pud(dst_pudp, - __pud(pud_val(pud) & ~PMD_SECT_RDONLY)); + __pud(pud_val(pud) & ~PUD_SECT_RDONLY)); } } while (dst_pudp++, src_pudp++, addr = next, addr != end); From patchwork Mon Sep 23 20:34:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157583 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ED2111599 for ; Mon, 23 Sep 2019 20:35:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AEBC821841 for ; Mon, 23 Sep 2019 20:35:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="lHmp4caK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AEBC821841 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 711526B028F; Mon, 23 Sep 2019 16:35:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6EA246B0290; Mon, 23 Sep 2019 16:35:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 589D46B0291; Mon, 23 Sep 2019 16:35:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 367606B028F for ; Mon, 23 Sep 2019 16:35:00 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id CF444181AC9BA for ; Mon, 23 Sep 2019 20:34:59 +0000 (UTC) X-FDA: 75967339518.08.gate06_82e15083fca1e X-Spam-Summary: 2,0,0,bc95658e3fd7c052,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:69:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1544:1605:1711:1730:1747:1777:1792:2194:2196:2199:2200:2393:2553:2559:2562:2693:2903:2914:3138:3139:3140:3141:3142:3865:3866:3867:3868:3871:3872:3874:4118:4250:4321:4385:4605:5007:6117:6119:6261:6653:6737:7903:9592:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12895:13161:13180:13229:13894:14181:14394:14721:21080:21325:21433:21444:21451:21627:30003:30012:30054:30079:30090,0,RBL:209.85.215.194:@so leen.com X-HE-Tag: gate06_82e15083fca1e X-Filterd-Recvd-Size: 7670 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:34:59 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id a3so8615615pgm.13 for ; Mon, 23 Sep 2019 13:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=O0CBEozBtTGW2pERGsJgZuZ1bw9J0cp5B10jh4ayzug=; b=lHmp4caKuhpmRRvFqoxk+Dv5szLbANKDLUEWmrgxe8QsZvtX+S/e4/dgG63d+dz+gj aI23C8K/DoJFcMMTmzwnqOAAjS3aLIOEvZcvivZQQE4WuKhh6Iv6jngNjFd7iRwU/Fcn fhVsrRLCV2BVXysC36+MW9luGtaag0iC2gAz0kIkqm4GNWv9cZPmQEPhxposYUN9Glo1 Os5dkLWoufY5xru5S0scCgOEFTHzcHtm1hSD8md81Kapc5ORK+W9GzSZ2UFkcxAR/Gop ZAlkNeDevYn1xV06eVLnKVqvBlWGolA3Dmoh+DbXwEqMluD8vbujBOJaPlST/ZS2Mnxq +aWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=O0CBEozBtTGW2pERGsJgZuZ1bw9J0cp5B10jh4ayzug=; b=PFfaaniMp77MG0QdIJKlIW9viC7hsfSGmopaMCNs13sLkNJUuf50TAWEjb/uH+/GjX tzO/U1DOaZLl7H9Vv78MW5eCC6LsFp0eDjLsues6QGU8/XJWolRJ+GFrmijP7CsO6GUP FyZwBMpQjxdBqDX7B2k5RYBb7/x5aYK0p065V44r76UZD9T7g/RGSlXh9Hb+hsa10irV Vl7UCAis2OKeyF0pr0bWOM2OesNHJYZ1pPqzlM2vxh8lY1lB8sFVUDDp/zFwd2E0DJF4 VvVyVzu98dh5v5UO8xfG/Z7BScFth6rocgkfkHK0Yxd7li+FB5QABHTI39kbMwHb7Alx VnGQ== X-Gm-Message-State: APjAAAXYt4Q4kteXgTGg6oz9ceu2KDeek7XMaSge96mO2l1KGAOXiBSV EWQqRWct5CZWRY9uxeOQMBfBXhSSz+w= X-Google-Smtp-Source: APXvYqw5yiDBk357lvxCwhrR5IIZRZiMUdx+N3wsBSQAzrnQ0hKR9ris0Yk1ISvpya8YgsUvM4p3rw== X-Received: by 2002:a63:ea14:: with SMTP id c20mr1691039pgi.185.1569270898184; Mon, 23 Sep 2019 13:34:58 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:34:57 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 08/17] arm64: hibernate: add trans_pgd public functions Date: Mon, 23 Sep 2019 16:34:18 -0400 Message-Id: <20190923203427.294286-9-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: trans_pgd_create_copy() and trans_pgd_map_page() are going to be the basis for new shared code that handles page tables for cases which are between kernels: kexec, and hibernate. Note: Eventually, get_safe_page() will be moved into a function pointer passed via argument, but for now keep it as is. Signed-off-by: Pavel Tatashin --- arch/arm64/kernel/hibernate.c | 88 ++++++++++++++++++++++++----------- 1 file changed, 60 insertions(+), 28 deletions(-) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index ce60bceed357..ded9034b9d39 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -182,39 +182,15 @@ int arch_hibernation_header_restore(void *addr) } EXPORT_SYMBOL(arch_hibernation_header_restore); -/* - * Copies length bytes, starting at src_start into an new page, - * perform cache maintentance, then maps it at the specified address low - * address as executable. - * - * This is used by hibernate to copy the code it needs to execute when - * overwriting the kernel text. This function generates a new set of page - * tables, which it loads into ttbr0. - * - * Length is provided as we probably only want 4K of data, even on a 64K - * page system. - */ -static int create_safe_exec_page(void *src_start, size_t length, - unsigned long dst_addr, - phys_addr_t *phys_dst_addr) +int trans_pgd_map_page(pgd_t *trans_pgd, void *page, + unsigned long dst_addr, + pgprot_t pgprot) { - void *page = (void *)get_safe_page(GFP_ATOMIC); - pgd_t *trans_pgd; pgd_t *pgdp; pud_t *pudp; pmd_t *pmdp; pte_t *ptep; - if (!page) - return -ENOMEM; - - memcpy(page, src_start, length); - __flush_icache_range((unsigned long)page, (unsigned long)page + length); - - trans_pgd = (void *)get_safe_page(GFP_ATOMIC); - if (!trans_pgd) - return -ENOMEM; - pgdp = pgd_offset_raw(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { pudp = (void *)get_safe_page(GFP_ATOMIC); @@ -242,6 +218,44 @@ static int create_safe_exec_page(void *src_start, size_t length, ptep = pte_offset_kernel(pmdp, dst_addr); set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); + return 0; +} + +/* + * Copies length bytes, starting at src_start into an new page, + * perform cache maintenance, then maps it at the specified address low + * address as executable. + * + * This is used by hibernate to copy the code it needs to execute when + * overwriting the kernel text. This function generates a new set of page + * tables, which it loads into ttbr0. + * + * Length is provided as we probably only want 4K of data, even on a 64K + * page system. + */ +static int create_safe_exec_page(void *src_start, size_t length, + unsigned long dst_addr, + phys_addr_t *phys_dst_addr) +{ + void *page = (void *)get_safe_page(GFP_ATOMIC); + pgd_t *trans_pgd; + int rc; + + if (!page) + return -ENOMEM; + + memcpy(page, src_start, length); + __flush_icache_range((unsigned long)page, (unsigned long)page + length); + + trans_pgd = (void *)get_safe_page(GFP_ATOMIC); + if (!trans_pgd) + return -ENOMEM; + + rc = trans_pgd_map_page(trans_pgd, page, dst_addr, + PAGE_KERNEL_EXEC); + if (rc) + return rc; + /* * Load our new page tables. A strict BBM approach requires that we * ensure that TLBs are free of any entries that may overlap with the @@ -462,6 +476,24 @@ static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, return 0; } +int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, + unsigned long end) +{ + int rc; + pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); + + if (!trans_pgd) { + pr_err("Failed to allocate memory for temporary page tables.\n"); + return -ENOMEM; + } + + rc = copy_page_tables(trans_pgd, start, end); + if (!rc) + *dst_pgdp = trans_pgd; + + return rc; +} + /* * Setup then Resume from the hibernate image using swsusp_arch_suspend_exit(). * @@ -488,7 +520,7 @@ int swsusp_arch_resume(void) pr_err("Failed to allocate memory for temporary page tables.\n"); return -ENOMEM; } - rc = copy_page_tables(tmp_pg_dir, PAGE_OFFSET, PAGE_END); + rc = trans_pgd_create_copy(&tmp_pg_dir, PAGE_OFFSET, PAGE_END); if (rc) return rc; From patchwork Mon Sep 23 20:34:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157585 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 746DA1747 for ; Mon, 23 Sep 2019 20:35:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2752F20673 for ; Mon, 23 Sep 2019 20:35:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="CPIvhWyU" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2752F20673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B55826B0292; Mon, 23 Sep 2019 16:35:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B2CCE6B0293; Mon, 23 Sep 2019 16:35:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F4A26B0294; Mon, 23 Sep 2019 16:35:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0153.hostedemail.com [216.40.44.153]) by kanga.kvack.org (Postfix) with ESMTP id 713606B0292 for ; Mon, 23 Sep 2019 16:35:03 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 0131C9095 for ; Mon, 23 Sep 2019 20:35:02 +0000 (UTC) X-FDA: 75967339686.25.heart74_8346529ca0838 X-Spam-Summary: 2,0,0,7eb47309058ddf91,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:4:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1381:1431:1437:1515:1605:1730:1747:1777:1792:2194:2198:2199:2200:2393:2538:2553:2559:2562:2640:2693:2731:2896:2898:2914:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4605:5007:6117:6119:6261:6653:6737:6755:7688:7903:8784:9592:10004:11026:11232:11473:11657:11658:11914:12043:12048:12219:12291:12296:12297:12438:12517:12519:12555:12895:12986:13141:13230:13255:13894:14096:14394:21080:21325:21433:21444:21451:21627:30003:30012:30054:30067:30 069:3007 X-HE-Tag: heart74_8346529ca0838 X-Filterd-Recvd-Size: 17958 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf35.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:02 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id w10so8639894pgj.7 for ; Mon, 23 Sep 2019 13:35:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=0EOgOzC/oEWFwgj/ixSy1OIk0JJ0SJsV3TAjT5pKIqM=; b=CPIvhWyUZcRrOERzBXnm8NVod3Lz0EekzpI+HT9/0qPDAN5sJrr0ccuT6cd26HaYQw 4HqLwkvwtvsW16PThfn5CcaxcCOvoeYG7kRROoLTmGXPLcATsR+jb/Tu9Q9XsKBTtL6e 7UKwoy4roT127UNaG+wULaMJ/QudC8UP8dfV53g53ZVOdkNLeE01siF3zr6UQX5IIO4M 7YYK8QXwDo7GtoUnzHx6xgJ/LCqsVbmaE1hJFBbLOMsWDjZy7NNZlbwZO9rszXKdUzTw /BAlV8cHslYklDY1DvlksKSQjC8Wf8u0g4iuLwrFshPFXDRPmRKGqZ8oRP2VrDcUAU8c gY/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0EOgOzC/oEWFwgj/ixSy1OIk0JJ0SJsV3TAjT5pKIqM=; b=ISv/GwG/ODPqcEPrMaPoiX1TZI27zACcedFlV83b+LeTY03J3uZKcSaA0W5DOSDsYW KG++ppLv1P3TsCpOykTyH/b1g0AzcmWbOWer3IfzVwQOC572cq6Z/qhb/JXqPq08Nb6O mI8XklFiripYCbHdePeU0PqkPHV6PnOIY4zHPCwhoo5RUELq76SIZUe+Vc0UCzLtnuEs eYSv3SM/R0l+HenA7hLjXIv3Rhu/aH3xFFAr1z4EkcH6IL6ncTYPKw6Oz7GcQpKRpO6G AqwjAOzYIFGmRzFHyNYuqxyLItEperz8IDl5UYtsnUC6W4sHJmOWW9F0ssxSc79XvVZn Pyzw== X-Gm-Message-State: APjAAAXZ9BNEkqB/CUvKI6de5azQKDoRiYcajG8GFp5ikAgb/icCPI6o k/qOtHeSfhc2hxy2lm9QFqLMgQ== X-Google-Smtp-Source: APXvYqzk4PEiRcRY0wzdMVTx/a35uezaICJ1+dmTU6m9Qd/bDXD3whUmiEY04msSdaMk0eJWp3ROMg== X-Received: by 2002:a17:90a:a403:: with SMTP id y3mr1348810pjp.69.1569270900881; Mon, 23 Sep 2019 13:35:00 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.34.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:00 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 09/17] arm64: hibernate: move page handling function to new trans_pgd.c Date: Mon, 23 Sep 2019 16:34:19 -0400 Message-Id: <20190923203427.294286-10-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, that we abstracted the required functions move them to a new home. Later, we will generalize these function in order to be useful outside of hibernation. Signed-off-by: Pavel Tatashin --- arch/arm64/Kconfig | 4 + arch/arm64/include/asm/trans_pgd.h | 20 +++ arch/arm64/kernel/hibernate.c | 199 +------------------------- arch/arm64/mm/Makefile | 1 + arch/arm64/mm/trans_pgd.c | 219 +++++++++++++++++++++++++++++ 5 files changed, 245 insertions(+), 198 deletions(-) create mode 100644 arch/arm64/include/asm/trans_pgd.h create mode 100644 arch/arm64/mm/trans_pgd.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 37c610963eee..789b90b80e48 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1017,6 +1017,10 @@ config CRASH_DUMP For more details see Documentation/admin-guide/kdump/kdump.rst +config TRANS_TABLE + def_bool y + depends on HIBERNATION || KEXEC_CORE + config XEN_DOM0 def_bool y depends on XEN diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h new file mode 100644 index 000000000000..c7b5402b7d87 --- /dev/null +++ b/arch/arm64/include/asm/trans_pgd.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * Copyright (c) 2019, Microsoft Corporation. + * Pavel Tatashin + */ + +#ifndef _ASM_TRANS_TABLE_H +#define _ASM_TRANS_TABLE_H + +#include +#include + +int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, + unsigned long end); + +int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, + pgprot_t pgprot); + +#endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index ded9034b9d39..d6346ad23f87 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -16,7 +16,6 @@ #define pr_fmt(x) "hibernate: " x #include #include -#include #include #include #include @@ -31,14 +30,12 @@ #include #include #include -#include -#include -#include #include #include #include #include #include +#include #include /* @@ -182,45 +179,6 @@ int arch_hibernation_header_restore(void *addr) } EXPORT_SYMBOL(arch_hibernation_header_restore); -int trans_pgd_map_page(pgd_t *trans_pgd, void *page, - unsigned long dst_addr, - pgprot_t pgprot) -{ - pgd_t *pgdp; - pud_t *pudp; - pmd_t *pmdp; - pte_t *ptep; - - pgdp = pgd_offset_raw(trans_pgd, dst_addr); - if (pgd_none(READ_ONCE(*pgdp))) { - pudp = (void *)get_safe_page(GFP_ATOMIC); - if (!pudp) - return -ENOMEM; - pgd_populate(&init_mm, pgdp, pudp); - } - - pudp = pud_offset(pgdp, dst_addr); - if (pud_none(READ_ONCE(*pudp))) { - pmdp = (void *)get_safe_page(GFP_ATOMIC); - if (!pmdp) - return -ENOMEM; - pud_populate(&init_mm, pudp, pmdp); - } - - pmdp = pmd_offset(pudp, dst_addr); - if (pmd_none(READ_ONCE(*pmdp))) { - ptep = (void *)get_safe_page(GFP_ATOMIC); - if (!ptep) - return -ENOMEM; - pmd_populate_kernel(&init_mm, pmdp, ptep); - } - - ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); - - return 0; -} - /* * Copies length bytes, starting at src_start into an new page, * perform cache maintenance, then maps it at the specified address low @@ -339,161 +297,6 @@ int swsusp_arch_suspend(void) return ret; } -static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) -{ - pte_t pte = READ_ONCE(*src_ptep); - - if (pte_valid(pte)) { - /* - * Resume will overwrite areas that may be marked - * read only (code, rodata). Clear the RDONLY bit from - * the temporary mappings we use during restore. - */ - set_pte(dst_ptep, pte_mkwrite(pte)); - } else if (debug_pagealloc_enabled() && !pte_none(pte)) { - /* - * debug_pagealloc will removed the PTE_VALID bit if - * the page isn't in use by the resume kernel. It may have - * been in use by the original kernel, in which case we need - * to put it back in our copy to do the restore. - * - * Before marking this entry valid, check the pfn should - * be mapped. - */ - BUG_ON(!pfn_valid(pte_pfn(pte))); - - set_pte(dst_ptep, pte_mkpresent(pte_mkwrite(pte))); - } -} - -static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, - unsigned long end) -{ - pte_t *src_ptep; - pte_t *dst_ptep; - unsigned long addr = start; - - dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC); - if (!dst_ptep) - return -ENOMEM; - pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); - dst_ptep = pte_offset_kernel(dst_pmdp, start); - - src_ptep = pte_offset_kernel(src_pmdp, start); - do { - _copy_pte(dst_ptep, src_ptep, addr); - } while (dst_ptep++, src_ptep++, addr += PAGE_SIZE, addr != end); - - return 0; -} - -static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, - unsigned long end) -{ - pmd_t *src_pmdp; - pmd_t *dst_pmdp; - unsigned long next; - unsigned long addr = start; - - if (pud_none(READ_ONCE(*dst_pudp))) { - dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); - if (!dst_pmdp) - return -ENOMEM; - pud_populate(&init_mm, dst_pudp, dst_pmdp); - } - dst_pmdp = pmd_offset(dst_pudp, start); - - src_pmdp = pmd_offset(src_pudp, start); - do { - pmd_t pmd = READ_ONCE(*src_pmdp); - - next = pmd_addr_end(addr, end); - if (pmd_none(pmd)) - continue; - if (pmd_table(pmd)) { - if (copy_pte(dst_pmdp, src_pmdp, addr, next)) - return -ENOMEM; - } else { - set_pmd(dst_pmdp, - __pmd(pmd_val(pmd) & ~PMD_SECT_RDONLY)); - } - } while (dst_pmdp++, src_pmdp++, addr = next, addr != end); - - return 0; -} - -static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, - unsigned long end) -{ - pud_t *dst_pudp; - pud_t *src_pudp; - unsigned long next; - unsigned long addr = start; - - if (pgd_none(READ_ONCE(*dst_pgdp))) { - dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC); - if (!dst_pudp) - return -ENOMEM; - pgd_populate(&init_mm, dst_pgdp, dst_pudp); - } - dst_pudp = pud_offset(dst_pgdp, start); - - src_pudp = pud_offset(src_pgdp, start); - do { - pud_t pud = READ_ONCE(*src_pudp); - - next = pud_addr_end(addr, end); - if (pud_none(pud)) - continue; - if (pud_table(pud)) { - if (copy_pmd(dst_pudp, src_pudp, addr, next)) - return -ENOMEM; - } else { - set_pud(dst_pudp, - __pud(pud_val(pud) & ~PUD_SECT_RDONLY)); - } - } while (dst_pudp++, src_pudp++, addr = next, addr != end); - - return 0; -} - -static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, - unsigned long end) -{ - unsigned long next; - unsigned long addr = start; - pgd_t *src_pgdp = pgd_offset_k(start); - - dst_pgdp = pgd_offset_raw(dst_pgdp, start); - do { - next = pgd_addr_end(addr, end); - if (pgd_none(READ_ONCE(*src_pgdp))) - continue; - if (copy_pud(dst_pgdp, src_pgdp, addr, next)) - return -ENOMEM; - } while (dst_pgdp++, src_pgdp++, addr = next, addr != end); - - return 0; -} - -int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, - unsigned long end) -{ - int rc; - pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); - - if (!trans_pgd) { - pr_err("Failed to allocate memory for temporary page tables.\n"); - return -ENOMEM; - } - - rc = copy_page_tables(trans_pgd, start, end); - if (!rc) - *dst_pgdp = trans_pgd; - - return rc; -} - /* * Setup then Resume from the hibernate image using swsusp_arch_suspend_exit(). * diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile index 849c1df3d214..f3002f1d0e61 100644 --- a/arch/arm64/mm/Makefile +++ b/arch/arm64/mm/Makefile @@ -6,6 +6,7 @@ obj-y := dma-mapping.o extable.o fault.o init.o \ obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_ARM64_PTDUMP_CORE) += dump.o obj-$(CONFIG_ARM64_PTDUMP_DEBUGFS) += ptdump_debugfs.o +obj-$(CONFIG_TRANS_TABLE) += trans_pgd.o obj-$(CONFIG_NUMA) += numa.o obj-$(CONFIG_DEBUG_VIRTUAL) += physaddr.o KASAN_SANITIZE_physaddr.o += n diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c new file mode 100644 index 000000000000..5ac712b92439 --- /dev/null +++ b/arch/arm64/mm/trans_pgd.c @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: GPL-2.0 + +/* + * Transitional page tables for kexec and hibernate + * + * This file derived from: arch/arm64/kernel/hibernate.c + * + * Copyright (c) 2019, Microsoft Corporation. + * Pavel Tatashin + * + */ + +/* + * Transitional tables are used during system transferring from one world to + * another: such as during hibernate restore, and kexec reboots. During these + * phases one cannot rely on page table not being overwritten. This is because + * hibernate and kexec can overwrite the current page tables during transition. + */ + +#include +#include +#include +#include +#include +#include +#include + +static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) +{ + pte_t pte = READ_ONCE(*src_ptep); + + if (pte_valid(pte)) { + /* + * Resume will overwrite areas that may be marked + * read only (code, rodata). Clear the RDONLY bit from + * the temporary mappings we use during restore. + */ + set_pte(dst_ptep, pte_mkwrite(pte)); + } else if (debug_pagealloc_enabled() && !pte_none(pte)) { + /* + * debug_pagealloc will removed the PTE_VALID bit if + * the page isn't in use by the resume kernel. It may have + * been in use by the original kernel, in which case we need + * to put it back in our copy to do the restore. + * + * Before marking this entry valid, check the pfn should + * be mapped. + */ + BUG_ON(!pfn_valid(pte_pfn(pte))); + + set_pte(dst_ptep, pte_mkpresent(pte_mkwrite(pte))); + } +} + +static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, + unsigned long end) +{ + pte_t *src_ptep; + pte_t *dst_ptep; + unsigned long addr = start; + + dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC); + if (!dst_ptep) + return -ENOMEM; + pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); + dst_ptep = pte_offset_kernel(dst_pmdp, start); + + src_ptep = pte_offset_kernel(src_pmdp, start); + do { + _copy_pte(dst_ptep, src_ptep, addr); + } while (dst_ptep++, src_ptep++, addr += PAGE_SIZE, addr != end); + + return 0; +} + +static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, + unsigned long end) +{ + pmd_t *src_pmdp; + pmd_t *dst_pmdp; + unsigned long next; + unsigned long addr = start; + + if (pud_none(READ_ONCE(*dst_pudp))) { + dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); + if (!dst_pmdp) + return -ENOMEM; + pud_populate(&init_mm, dst_pudp, dst_pmdp); + } + dst_pmdp = pmd_offset(dst_pudp, start); + + src_pmdp = pmd_offset(src_pudp, start); + do { + pmd_t pmd = READ_ONCE(*src_pmdp); + + next = pmd_addr_end(addr, end); + if (pmd_none(pmd)) + continue; + if (pmd_table(pmd)) { + if (copy_pte(dst_pmdp, src_pmdp, addr, next)) + return -ENOMEM; + } else { + set_pmd(dst_pmdp, + __pmd(pmd_val(pmd) & ~PMD_SECT_RDONLY)); + } + } while (dst_pmdp++, src_pmdp++, addr = next, addr != end); + + return 0; +} + +static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, + unsigned long end) +{ + pud_t *dst_pudp; + pud_t *src_pudp; + unsigned long next; + unsigned long addr = start; + + if (pgd_none(READ_ONCE(*dst_pgdp))) { + dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC); + if (!dst_pudp) + return -ENOMEM; + pgd_populate(&init_mm, dst_pgdp, dst_pudp); + } + dst_pudp = pud_offset(dst_pgdp, start); + + src_pudp = pud_offset(src_pgdp, start); + do { + pud_t pud = READ_ONCE(*src_pudp); + + next = pud_addr_end(addr, end); + if (pud_none(pud)) + continue; + if (pud_table(pud)) { + if (copy_pmd(dst_pudp, src_pudp, addr, next)) + return -ENOMEM; + } else { + set_pud(dst_pudp, + __pud(pud_val(pud) & ~PUD_SECT_RDONLY)); + } + } while (dst_pudp++, src_pudp++, addr = next, addr != end); + + return 0; +} + +static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, + unsigned long end) +{ + unsigned long next; + unsigned long addr = start; + pgd_t *src_pgdp = pgd_offset_k(start); + + dst_pgdp = pgd_offset_raw(dst_pgdp, start); + do { + next = pgd_addr_end(addr, end); + if (pgd_none(READ_ONCE(*src_pgdp))) + continue; + if (copy_pud(dst_pgdp, src_pgdp, addr, next)) + return -ENOMEM; + } while (dst_pgdp++, src_pgdp++, addr = next, addr != end); + + return 0; +} + +int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, + unsigned long end) +{ + int rc; + pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); + + if (!trans_pgd) { + pr_err("Failed to allocate memory for temporary page tables.\n"); + return -ENOMEM; + } + + rc = copy_page_tables(trans_pgd, start, end); + if (!rc) + *dst_pgdp = trans_pgd; + + return rc; +} + +int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, + pgprot_t pgprot) +{ + pgd_t *pgdp; + pud_t *pudp; + pmd_t *pmdp; + pte_t *ptep; + + pgdp = pgd_offset_raw(trans_pgd, dst_addr); + if (pgd_none(READ_ONCE(*pgdp))) { + pudp = (void *)get_safe_page(GFP_ATOMIC); + if (!pudp) + return -ENOMEM; + pgd_populate(&init_mm, pgdp, pudp); + } + + pudp = pud_offset(pgdp, dst_addr); + if (pud_none(READ_ONCE(*pudp))) { + pmdp = (void *)get_safe_page(GFP_ATOMIC); + if (!pmdp) + return -ENOMEM; + pud_populate(&init_mm, pudp, pmdp); + } + + pmdp = pmd_offset(pudp, dst_addr); + if (pmd_none(READ_ONCE(*pmdp))) { + ptep = (void *)get_safe_page(GFP_ATOMIC); + if (!ptep) + return -ENOMEM; + pmd_populate_kernel(&init_mm, pmdp, ptep); + } + + ptep = pte_offset_kernel(pmdp, dst_addr); + set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); + + return 0; +} From patchwork Mon Sep 23 20:34:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157587 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61F3B1668 for ; Mon, 23 Sep 2019 20:35:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 21D7720673 for ; Mon, 23 Sep 2019 20:35:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="Jk9emTcb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21D7720673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B55FE6B0294; Mon, 23 Sep 2019 16:35:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B072A6B0296; Mon, 23 Sep 2019 16:35:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 95C616B0295; Mon, 23 Sep 2019 16:35:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0128.hostedemail.com [216.40.44.128]) by kanga.kvack.org (Postfix) with ESMTP id 763F66B0293 for ; Mon, 23 Sep 2019 16:35:05 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 2173F180AD808 for ; Mon, 23 Sep 2019 20:35:05 +0000 (UTC) X-FDA: 75967339770.27.sky23_83a472cbd9630 X-Spam-Summary: 2,0,0,68ed252c379d3f6e,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:2:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1431:1437:1515:1535:1605:1606:1730:1747:1777:1792:1978:2194:2199:2393:2559:2562:2897:2899:3138:3139:3140:3141:3142:3865:3867:3868:3870:3871:3872:3874:4119:4250:4321:4605:5007:6117:6261:6653:6737:7875:8603:10004:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12895:13894:14096:14394:21080:21444:21451:21627:21796:30003:30036:30054:30075,0,RBL:209.85.210.193:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Baye sian:0.5 X-HE-Tag: sky23_83a472cbd9630 X-Filterd-Recvd-Size: 8541 Received: from mail-pf1-f193.google.com (mail-pf1-f193.google.com [209.85.210.193]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:04 +0000 (UTC) Received: by mail-pf1-f193.google.com with SMTP id q5so9803427pfg.13 for ; Mon, 23 Sep 2019 13:35:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RIb8E41xezTeI4CoJm74LLND2GVI+qEZuMMibtzyOTc=; b=Jk9emTcbpnEPuZ4AUFtAn1jGByVB9ywpSlHfGhEy2nlPzIdxAObdfYH/kDGmLI0Rtj BcqkpWnWYXn/hIpAlle4EU4z/hk4hsN6RZVnjan7CPwebcgBQQwdFi0EoiiwsFsR7vbp ZYv5OFKUnRgQsB/jF4h30z/c0SRXIoT8lFSb+Kky2JYybksRPVi1Uo2g7Vk2y24SoAE4 /OJEllswVfKOsKOq3/Pd5zjE8raop/2w/7qTEfVLVnet0NGPLTw3nrLmIqkA+/7Xb+AH S4ZfsNtTFPmWeAIfzWBsghd77UMoXG1/VrotEUku0e7qDI6PU8uQRvhIK3u/W1zLEb9G JzeA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RIb8E41xezTeI4CoJm74LLND2GVI+qEZuMMibtzyOTc=; b=MbjmnrwXq7W4FIdvCxeHkaT13xs/5215N74/7Gzb2H+d536zoaJoVTQJrYH2t0g1i/ +7pQG1M+rGrvPKTuBZShD+g/6RSrDtN76crnagVRyx71JlYuSjL+D/4hRp/6jG8UdTtc eoqXdWFVSHlMW6Utmg3Zwrqhbslg/pt4uaUkfa2QrxeONC0+ktd6TT1I208D+5s0u+OD GIxVwAYBdhnsYMo2cPMN03NTudvkFSQzgZFsCNkjYCzpPWt66JUj3y+yIUi8qTZb/38B pZkogsHFaa2QZ6Qh/QSCdL9GzAgh7r0VT9KgQYuCJkjgGjuD+vfMAixZSzv5mwbttrF/ VbYA== X-Gm-Message-State: APjAAAXJCSPfpig7z1ntZSBMCS9rjVVwXnULBc4HhKUQC12o2Pde7QlN S3nZWDmdSHmvN4mtwhKmdOk2aw== X-Google-Smtp-Source: APXvYqyyLPMVdXB9BvWIObEkR0Me1bGPISDbYutiuZpzfJnAdLVAMnCxrmYt0llJqDvygSPPV38npw== X-Received: by 2002:a63:4142:: with SMTP id o63mr1832434pga.426.1569270903510; Mon, 23 Sep 2019 13:35:03 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:02 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 10/17] arm64: trans_pgd: make trans_pgd_map_page generic Date: Mon, 23 Sep 2019 16:34:20 -0400 Message-Id: <20190923203427.294286-11-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: kexec is going to use a different allocator, so make trans_pgd_map_page to accept allocator as an argument, and also kexec is going to use a different map protection, so also pass it via argument. Signed-off-by: Pavel Tatashin Reviewed-by: Matthias Brugger --- arch/arm64/include/asm/trans_pgd.h | 18 ++++++++++++++++-- arch/arm64/kernel/hibernate.c | 12 +++++++++++- arch/arm64/mm/trans_pgd.c | 27 +++++++++++++++++++++------ 3 files changed, 48 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index c7b5402b7d87..bb38f73aa7aa 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -11,10 +11,24 @@ #include #include +/* + * trans_alloc_page + * - Allocator that should return exactly one zeroed page, if this + * allocator fails, trans_pgd returns -ENOMEM error. + * + * trans_alloc_arg + * - Passed to trans_alloc_page as an argument + */ + +struct trans_pgd_info { + void * (*trans_alloc_page)(void *arg); + void *trans_alloc_arg; +}; + int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, unsigned long end); -int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, - pgprot_t pgprot); +int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, + void *page, unsigned long dst_addr, pgprot_t pgprot); #endif /* _ASM_TRANS_TABLE_H */ diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index d6346ad23f87..7f0a5e152c45 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -179,6 +179,11 @@ int arch_hibernation_header_restore(void *addr) } EXPORT_SYMBOL(arch_hibernation_header_restore); +static void *hibernate_page_alloc(void *arg) +{ + return (void *)get_safe_page((gfp_t)(unsigned long)arg); +} + /* * Copies length bytes, starting at src_start into an new page, * perform cache maintenance, then maps it at the specified address low @@ -195,6 +200,11 @@ static int create_safe_exec_page(void *src_start, size_t length, unsigned long dst_addr, phys_addr_t *phys_dst_addr) { + struct trans_pgd_info trans_info = { + .trans_alloc_page = hibernate_page_alloc, + .trans_alloc_arg = (void *)GFP_ATOMIC, + }; + void *page = (void *)get_safe_page(GFP_ATOMIC); pgd_t *trans_pgd; int rc; @@ -209,7 +219,7 @@ static int create_safe_exec_page(void *src_start, size_t length, if (!trans_pgd) return -ENOMEM; - rc = trans_pgd_map_page(trans_pgd, page, dst_addr, + rc = trans_pgd_map_page(&trans_info, trans_pgd, page, dst_addr, PAGE_KERNEL_EXEC); if (rc) return rc; diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 5ac712b92439..1142dde8c02f 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -25,6 +25,11 @@ #include #include +static void *trans_alloc(struct trans_pgd_info *info) +{ + return info->trans_alloc_page(info->trans_alloc_arg); +} + static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) { pte_t pte = READ_ONCE(*src_ptep); @@ -180,8 +185,18 @@ int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, return rc; } -int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, - pgprot_t pgprot) +/* + * Add map entry to trans_pgd for a base-size page at PTE level. + * info: contains allocator and its argument + * trans_pgd: page table in which new map is added. + * page: page to be mapped. + * dst_addr: new VA address for the pages + * pgprot: protection for the page. + * + * Returns 0 on success, and -ENOMEM on failure. + */ +int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, + void *page, unsigned long dst_addr, pgprot_t pgprot) { pgd_t *pgdp; pud_t *pudp; @@ -190,7 +205,7 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, pgdp = pgd_offset_raw(trans_pgd, dst_addr); if (pgd_none(READ_ONCE(*pgdp))) { - pudp = (void *)get_safe_page(GFP_ATOMIC); + pudp = trans_alloc(info); if (!pudp) return -ENOMEM; pgd_populate(&init_mm, pgdp, pudp); @@ -198,7 +213,7 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, pudp = pud_offset(pgdp, dst_addr); if (pud_none(READ_ONCE(*pudp))) { - pmdp = (void *)get_safe_page(GFP_ATOMIC); + pmdp = trans_alloc(info); if (!pmdp) return -ENOMEM; pud_populate(&init_mm, pudp, pmdp); @@ -206,14 +221,14 @@ int trans_pgd_map_page(pgd_t *trans_pgd, void *page, unsigned long dst_addr, pmdp = pmd_offset(pudp, dst_addr); if (pmd_none(READ_ONCE(*pmdp))) { - ptep = (void *)get_safe_page(GFP_ATOMIC); + ptep = trans_alloc(info); if (!ptep) return -ENOMEM; pmd_populate_kernel(&init_mm, pmdp, ptep); } ptep = pte_offset_kernel(pmdp, dst_addr); - set_pte(ptep, pfn_pte(virt_to_pfn(page), PAGE_KERNEL_EXEC)); + set_pte(ptep, pfn_pte(virt_to_pfn(page), pgprot)); return 0; } From patchwork Mon Sep 23 20:34:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD3C41599 for ; Mon, 23 Sep 2019 20:35:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8D2E3214DA for ; Mon, 23 Sep 2019 20:35:11 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="K7zeKlOy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8D2E3214DA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E8DB36B0296; Mon, 23 Sep 2019 16:35:08 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E3AC66B0297; Mon, 23 Sep 2019 16:35:08 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D2D766B0298; Mon, 23 Sep 2019 16:35:08 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id A8B676B0296 for ; Mon, 23 Sep 2019 16:35:08 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id 3226C181AC9BA for ; Mon, 23 Sep 2019 20:35:08 +0000 (UTC) X-FDA: 75967339896.04.help37_841479b563237 X-Spam-Summary: 2,0,0,6e52c3887e65187a,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:2:41:69:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:2898:2899:2914:3138:3139:3140:3141:3142:3865:3866:3867:3868:3871:3874:4049:4120:4321:4605:5007:6261:6653:6737:10004:11026:11657:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12683:12895:13894:14110:14394:21080:21325:21444:21451:21611:21627:30036:30054:30070:30075:30079,0,RBL:209.85.214.196:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0 .5,0.5,N X-HE-Tag: help37_841479b563237 X-Filterd-Recvd-Size: 9841 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:07 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id e5so7000327pls.9 for ; Mon, 23 Sep 2019 13:35:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=VFJvvULlApx2IXv/Kz+u5IAN/WZ7ZN12CZD/AhXv5tU=; b=K7zeKlOyuVbfaJM9f2HkRaMTHAYXYPBFf7ZdwVlehakJOdLiPs8lP0RAZWhJj4EYBP PhyKAtKOH9dkyp6M7wYpr7cX8h+iWREc5K6J4GHspK8BOjQbl8EcYZbXIcF9es8S5PXj pacARn0ENPoGiqhwhLVE+Hs8EnRRU/2+TryiR0F/4/+Se2cGfk76IU1fKkJr2utwKPCH swrtENf/Eub9badnJ+S4LAn+dxJdho8uypOf7m/EBvIWyidj46rI8qh2jHHNjvUfX/XN lHm4aHUBdBYOpiGIrKlQwZKQbINSFeTLHrlLcSuDpV/mTA1JFzp8MvYSOgrQYZkrdUZ6 a7Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VFJvvULlApx2IXv/Kz+u5IAN/WZ7ZN12CZD/AhXv5tU=; b=YbnBUJMa4SZzrb92eusTgg6ISzOac2Mx3bmj7RWhF6XxjV6uyKAU6UlOXIOPAi0GUP CZaJHvhOvGhQhOz8j4R3t8NdCAKpLMPcXnKR55W6o2tSTd+LjQPeQV0hEDLLcmenf3ae /C4UkjWS2gsBh4iU5c7sx9omXSZtEWGr6ATixM9faKbO2athN2e5xRzRmze3XC8PVFwv faRGv5w1fGNG7nV3BAUjRygduZAGevmtA6aBRIWlfjtv/B5qfA3wggxW6SPfF09FbPce Xfp+N/JOkpkrVk4wCoS4sYyKi10GL7CZLot2ATY4nPUBiiWyDj0A0ca6Zl25RemYeN1k CPlQ== X-Gm-Message-State: APjAAAU6PBXlocOOKSVX6JeBO1ohv/GHGNs1sfAwoynSzRv7OU6HRDiO EnbMgSk6AP0RhybrxhCHy6j7Cg== X-Google-Smtp-Source: APXvYqxZRRJfknWu5/RwKPd05GrIV0tdCB7qimxV6vcsVXLj+pAFybjBV6ZkK10HAiKYcn3odMhXVA== X-Received: by 2002:a17:902:b704:: with SMTP id d4mr1640872pls.204.1569270906435; Mon, 23 Sep 2019 13:35:06 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:05 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 11/17] arm64: trans_pgd: pass allocator trans_pgd_create_copy Date: Mon, 23 Sep 2019 16:34:21 -0400 Message-Id: <20190923203427.294286-12-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make trans_pgd_create_copy and its subroutines to use allocator that is passed as an argument Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/trans_pgd.h | 4 +-- arch/arm64/kernel/hibernate.c | 7 ++++- arch/arm64/mm/trans_pgd.c | 44 ++++++++++++++++++------------ 3 files changed, 35 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/trans_pgd.h b/arch/arm64/include/asm/trans_pgd.h index bb38f73aa7aa..56613e83aa53 100644 --- a/arch/arm64/include/asm/trans_pgd.h +++ b/arch/arm64/include/asm/trans_pgd.h @@ -25,8 +25,8 @@ struct trans_pgd_info { void *trans_alloc_arg; }; -int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, - unsigned long end); +int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **trans_pgd, + unsigned long start, unsigned long end); int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, void *page, unsigned long dst_addr, pgprot_t pgprot); diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index 7f0a5e152c45..038d8214df8d 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -322,6 +322,10 @@ int swsusp_arch_resume(void) phys_addr_t phys_hibernate_exit; void __noreturn (*hibernate_exit)(phys_addr_t, phys_addr_t, void *, void *, phys_addr_t, phys_addr_t); + struct trans_pgd_info trans_info = { + .trans_alloc_page = hibernate_page_alloc, + .trans_alloc_arg = (void *)GFP_ATOMIC, + }; /* * Restoring the memory image will overwrite the ttbr1 page tables. @@ -333,7 +337,8 @@ int swsusp_arch_resume(void) pr_err("Failed to allocate memory for temporary page tables.\n"); return -ENOMEM; } - rc = trans_pgd_create_copy(&tmp_pg_dir, PAGE_OFFSET, PAGE_END); + rc = trans_pgd_create_copy(&trans_info, &tmp_pg_dir, PAGE_OFFSET, + PAGE_END); if (rc) return rc; diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index 1142dde8c02f..df3a10d36f62 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -57,14 +57,14 @@ static void _copy_pte(pte_t *dst_ptep, pte_t *src_ptep, unsigned long addr) } } -static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, - unsigned long end) +static int copy_pte(struct trans_pgd_info *info, pmd_t *dst_pmdp, + pmd_t *src_pmdp, unsigned long start, unsigned long end) { pte_t *src_ptep; pte_t *dst_ptep; unsigned long addr = start; - dst_ptep = (pte_t *)get_safe_page(GFP_ATOMIC); + dst_ptep = trans_alloc(info); if (!dst_ptep) return -ENOMEM; pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); @@ -78,8 +78,8 @@ static int copy_pte(pmd_t *dst_pmdp, pmd_t *src_pmdp, unsigned long start, return 0; } -static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, - unsigned long end) +static int copy_pmd(struct trans_pgd_info *info, pud_t *dst_pudp, + pud_t *src_pudp, unsigned long start, unsigned long end) { pmd_t *src_pmdp; pmd_t *dst_pmdp; @@ -87,7 +87,7 @@ static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, unsigned long addr = start; if (pud_none(READ_ONCE(*dst_pudp))) { - dst_pmdp = (pmd_t *)get_safe_page(GFP_ATOMIC); + dst_pmdp = trans_alloc(info); if (!dst_pmdp) return -ENOMEM; pud_populate(&init_mm, dst_pudp, dst_pmdp); @@ -102,7 +102,7 @@ static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, if (pmd_none(pmd)) continue; if (pmd_table(pmd)) { - if (copy_pte(dst_pmdp, src_pmdp, addr, next)) + if (copy_pte(info, dst_pmdp, src_pmdp, addr, next)) return -ENOMEM; } else { set_pmd(dst_pmdp, @@ -113,7 +113,8 @@ static int copy_pmd(pud_t *dst_pudp, pud_t *src_pudp, unsigned long start, return 0; } -static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, +static int copy_pud(struct trans_pgd_info *info, pgd_t *dst_pgdp, + pgd_t *src_pgdp, unsigned long start, unsigned long end) { pud_t *dst_pudp; @@ -122,7 +123,7 @@ static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, unsigned long addr = start; if (pgd_none(READ_ONCE(*dst_pgdp))) { - dst_pudp = (pud_t *)get_safe_page(GFP_ATOMIC); + dst_pudp = trans_alloc(info); if (!dst_pudp) return -ENOMEM; pgd_populate(&init_mm, dst_pgdp, dst_pudp); @@ -137,7 +138,7 @@ static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, if (pud_none(pud)) continue; if (pud_table(pud)) { - if (copy_pmd(dst_pudp, src_pudp, addr, next)) + if (copy_pmd(info, dst_pudp, src_pudp, addr, next)) return -ENOMEM; } else { set_pud(dst_pudp, @@ -148,8 +149,8 @@ static int copy_pud(pgd_t *dst_pgdp, pgd_t *src_pgdp, unsigned long start, return 0; } -static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, - unsigned long end) +static int copy_page_tables(struct trans_pgd_info *info, pgd_t *dst_pgdp, + unsigned long start, unsigned long end) { unsigned long next; unsigned long addr = start; @@ -160,25 +161,34 @@ static int copy_page_tables(pgd_t *dst_pgdp, unsigned long start, next = pgd_addr_end(addr, end); if (pgd_none(READ_ONCE(*src_pgdp))) continue; - if (copy_pud(dst_pgdp, src_pgdp, addr, next)) + if (copy_pud(info, dst_pgdp, src_pgdp, addr, next)) return -ENOMEM; } while (dst_pgdp++, src_pgdp++, addr = next, addr != end); return 0; } -int trans_pgd_create_copy(pgd_t **dst_pgdp, unsigned long start, - unsigned long end) +/* + * Create trans_pgd and copy linear map. + * info: contains allocator and its argument + * dst_pgdp: new page table that is created, and to which map is copied. + * start: Start of the interval (inclusive). + * end: End of the interval (exclusive). + * + * Returns 0 on success, and -ENOMEM on failure. + */ +int trans_pgd_create_copy(struct trans_pgd_info *info, pgd_t **dst_pgdp, + unsigned long start, unsigned long end) { int rc; - pgd_t *trans_pgd = (pgd_t *)get_safe_page(GFP_ATOMIC); + pgd_t *trans_pgd = trans_alloc(info); if (!trans_pgd) { pr_err("Failed to allocate memory for temporary page tables.\n"); return -ENOMEM; } - rc = copy_page_tables(trans_pgd, start, end); + rc = copy_page_tables(info, trans_pgd, start, end); if (!rc) *dst_pgdp = trans_pgd; From patchwork Mon Sep 23 20:34:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1B001668 for ; Mon, 23 Sep 2019 20:35:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AE17620673 for ; Mon, 23 Sep 2019 20:35:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="WnAw2fGS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AE17620673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 53CFA6B0297; Mon, 23 Sep 2019 16:35:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5189A6B0298; Mon, 23 Sep 2019 16:35:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DB896B0299; Mon, 23 Sep 2019 16:35:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0241.hostedemail.com [216.40.44.241]) by kanga.kvack.org (Postfix) with ESMTP id 1CA1E6B0297 for ; Mon, 23 Sep 2019 16:35:11 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 66DA0824376A for ; Mon, 23 Sep 2019 20:35:10 +0000 (UTC) X-FDA: 75967339980.25.unit64_846d33e08d04e X-Spam-Summary: 2,0,0,8f0dfa64c0221d0f,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1542:1711:1730:1747:1777:1792:2393:2559:2562:3138:3139:3140:3141:3142:3353:3865:3871:3872:3874:5007:6117:6261:6653:6737:7903:10004:11026:11658:11914:12048:12297:12517:12519:12555:12895:13255:13894:14096:14181:14394:14721:21080:21433:21444:21627:30012:30036:30054:30075,0,RBL:209.85.215.195:@soleen.com:.lbl8.mailshell.net-62.2.175.100 66.100.201.201,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Cus tom_rule X-HE-Tag: unit64_846d33e08d04e X-Filterd-Recvd-Size: 5691 Received: from mail-pg1-f195.google.com (mail-pg1-f195.google.com [209.85.215.195]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:09 +0000 (UTC) Received: by mail-pg1-f195.google.com with SMTP id w10so8640132pgj.7 for ; Mon, 23 Sep 2019 13:35:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zqjz6RgOj4sr7JAtiYy2U5c4q561GRonF/uToZbvMtQ=; b=WnAw2fGSNE/myDLdhhX6BmHSTPEX7HnB3kfPVBIxFAmKlx5HFTEJTBGDR9Ljm7tmxX qFvbtBtRiGT/YrC/Vi002qJlWTVCBmg6080ME+eu+yEA4GFpI8xlnK4Mvg2E4e2m1lVJ +sXMy+qKAuSJsy+y4dHLIeJpXBGKcdl//fP/LTzqvMfmw6emFrRTpHCeRZmTpv3e4ur6 0cw7gBF8XsAdK2VpUuOEYTeOv2cV6Vwatca3LnJjVSSwOXuyiFUivMYZBjDbvOPP6ZR3 vzHol7eDJe4E/Eu5sGe6fbEEsVu1bhIpt6EyQtoHHOjmRQqFE7W28Gk0GzEmTxwwTCFv e9WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zqjz6RgOj4sr7JAtiYy2U5c4q561GRonF/uToZbvMtQ=; b=R4xIwAfPFd9yvZMDgRX1+g8YrZU5631jxiKGaiXpaOVNjStjUtawo8lCbWkDKz48YK F/TGvIN546/Vmtsc2alUMsu47ybJAMP1QQwdUHZxFudhoLVIS+pNeBOfiyiQK7xh8jbU 51r8su4FGp0cLx61d57u3TvPfoW0J+4n7N/56wcyBQGm95wSTmR+l/acNx/dIIy66bvL de2PCpSLQfuas134x2mlRmflTDP7eWW7RrgocOwJ20Q0eWekA7DAs9O7IbaXvPWqQLZq 0sc8xsyYI44lyKHlYqm3LzuY1VCXV9K4hVIBQp+fNXD631/2Dsh26aJXvcr1tJmf2sHD jJTA== X-Gm-Message-State: APjAAAUhbB2+M+oigKQx2rIMWeAdBFWEZZQIrECriZejy3M6hwD+IsiI 1ojaNcZQN9eGiWrq89v/+jN4nA== X-Google-Smtp-Source: APXvYqzXWJHRickfcJWQxs+CgMxHvl/b0k8/aUu4O+dX/YeWwklhNW/J0DTVkn8q5vOK+EICXnoj3Q== X-Received: by 2002:a62:7d8c:: with SMTP id y134mr1547755pfc.257.1569270909101; Mon, 23 Sep 2019 13:35:09 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:08 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 12/17] arm64: trans_pgd: pass NULL instead of init_mm to *_populate functions Date: Mon, 23 Sep 2019 16:34:22 -0400 Message-Id: <20190923203427.294286-13-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: trans_pgd_* should be independent from mm context because the tables that are created by this code are used when there are no mm context around, as it is between kernels. Simply replace mm_init's with NULL. Signed-off-by: Pavel Tatashin --- arch/arm64/mm/trans_pgd.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/mm/trans_pgd.c b/arch/arm64/mm/trans_pgd.c index df3a10d36f62..2b02a646101f 100644 --- a/arch/arm64/mm/trans_pgd.c +++ b/arch/arm64/mm/trans_pgd.c @@ -67,7 +67,7 @@ static int copy_pte(struct trans_pgd_info *info, pmd_t *dst_pmdp, dst_ptep = trans_alloc(info); if (!dst_ptep) return -ENOMEM; - pmd_populate_kernel(&init_mm, dst_pmdp, dst_ptep); + pmd_populate_kernel(NULL, dst_pmdp, dst_ptep); dst_ptep = pte_offset_kernel(dst_pmdp, start); src_ptep = pte_offset_kernel(src_pmdp, start); @@ -90,7 +90,7 @@ static int copy_pmd(struct trans_pgd_info *info, pud_t *dst_pudp, dst_pmdp = trans_alloc(info); if (!dst_pmdp) return -ENOMEM; - pud_populate(&init_mm, dst_pudp, dst_pmdp); + pud_populate(NULL, dst_pudp, dst_pmdp); } dst_pmdp = pmd_offset(dst_pudp, start); @@ -126,7 +126,7 @@ static int copy_pud(struct trans_pgd_info *info, pgd_t *dst_pgdp, dst_pudp = trans_alloc(info); if (!dst_pudp) return -ENOMEM; - pgd_populate(&init_mm, dst_pgdp, dst_pudp); + pgd_populate(NULL, dst_pgdp, dst_pudp); } dst_pudp = pud_offset(dst_pgdp, start); @@ -218,7 +218,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, pudp = trans_alloc(info); if (!pudp) return -ENOMEM; - pgd_populate(&init_mm, pgdp, pudp); + pgd_populate(NULL, pgdp, pudp); } pudp = pud_offset(pgdp, dst_addr); @@ -226,7 +226,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, pmdp = trans_alloc(info); if (!pmdp) return -ENOMEM; - pud_populate(&init_mm, pudp, pmdp); + pud_populate(NULL, pudp, pmdp); } pmdp = pmd_offset(pudp, dst_addr); @@ -234,7 +234,7 @@ int trans_pgd_map_page(struct trans_pgd_info *info, pgd_t *trans_pgd, ptep = trans_alloc(info); if (!ptep) return -ENOMEM; - pmd_populate_kernel(&init_mm, pmdp, ptep); + pmd_populate_kernel(NULL, pmdp, ptep); } ptep = pte_offset_kernel(pmdp, dst_addr); From patchwork Mon Sep 23 20:34:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157595 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E83491599 for ; Mon, 23 Sep 2019 20:35:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B4F2020673 for ; Mon, 23 Sep 2019 20:35:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="hECEW80O" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B4F2020673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AE1946B0299; Mon, 23 Sep 2019 16:35:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AB92C6B029B; Mon, 23 Sep 2019 16:35:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D1EE6B0299; Mon, 23 Sep 2019 16:35:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id 737D96B0299 for ; Mon, 23 Sep 2019 16:35:13 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with SMTP id 1FAA9824376A for ; Mon, 23 Sep 2019 20:35:13 +0000 (UTC) X-FDA: 75967340106.07.hall47_84d008d660a63 X-Spam-Summary: 2,0,0,aef3499f1d8f702d,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:41:355:379:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1542:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2559:2562:2731:2914:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3872:3874:4321:4385:4605:5007:6261:6653:6737:10004:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12895:12986:13215:13229:13894:14096:14181:14394:14721:21063:21080:21325:21444:21627:21740:30054,0,RBL:209.85.210.194:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0. 5,0.5,0. X-HE-Tag: hall47_84d008d660a63 X-Filterd-Recvd-Size: 5663 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:12 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id x127so9831771pfb.7 for ; Mon, 23 Sep 2019 13:35:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kCOV/SKGyCS/idoYOjNn3DaQRL2Ie2fUru8T4/v0tiY=; b=hECEW80OowucfYu/52sXrIDun1tK80j3pQv+b2fbBgabumiVdSJyvqrVXtrioJldNd urMLV5iHCV3KA3E/4qiZfwr6Z9WIteQzsU21H1akcamSGkDklOdO0o22VbxIYwDU4OII YXT7LW+oVJg2xvLIWYmceBLLXADhdoVm/He4N2R4NnICbF5ZpRvuBYeCnVE7KlBMmSy8 sX4PsszBAesbNLnSIu9ancRbbrLfOiQySP1b1fnOWoPRtK4xURhLl+EDRFi/KjeM7c7i Rl8+8rDh0FeE7pU0wB4hm2pFfl4L7VoTGrmZJoK4y/ViGTtjNcJFT+ZwkhsfY4SBhnn5 Phvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kCOV/SKGyCS/idoYOjNn3DaQRL2Ie2fUru8T4/v0tiY=; b=O0+5eaxyy++ewJlCXLnE4Asl8K2gtoVuLkaqpek7ACnkXPfNwYraZ7rir4HHPdOci6 qLJ+Nj7U5EzSxpBdU4wky/duWPavk5PcNS4LmetH/RxUJGWemUyKHxbMCuOzKmP8MXLy QNWt5WvSM6eYxYmKrDrXRGCrIh04YIlj8LW/qAx5vjBwoNppokLQxLg/tFq7lA4RSF6c Wj9WJ2rgMybkRQ4gNqIMPllpfYTQXTT1SHrkB5QnBKneGlmNTL2MA9JSZYDH8+koY5Fz wEIUgoPelo74eivES043Cva4qUaYXaZj/1Nc4M4uSCJdS2sd54tHhd3d5YgfdNtN0V29 qvuA== X-Gm-Message-State: APjAAAVbwEquTEy7zf/6SkovElDeA+VTICtprXQ4OTt5pqqjRQZpnxEg 6dBO1+h+iYN05cI+b6VIbtWWwdlSbTw= X-Google-Smtp-Source: APXvYqyh0pHM+T1vgCjrUaqwaXzl27erGlVnBJX3ECpzLMx0ZfHUDTjGJiel8N46eTWPvVx5gcFZhA== X-Received: by 2002:aa7:953c:: with SMTP id c28mr1610359pfp.106.1569270911771; Mon, 23 Sep 2019 13:35:11 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:11 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 13/17] kexec: add machine_kexec_post_load() Date: Mon, 23 Sep 2019 16:34:23 -0400 Message-Id: <20190923203427.294286-14-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is the same as machine_kexec_prepare(), but is called after segments are loaded. This way, can do processing work with already loaded relocation segments. One such example is arm64: it has to have segments loaded in order to create a page table, but it cannot do it during kexec time, because at that time allocations won't be possible anymore. Signed-off-by: Pavel Tatashin Acked-by: Dave Young --- kernel/kexec.c | 4 ++++ kernel/kexec_core.c | 6 ++++++ kernel/kexec_file.c | 4 ++++ kernel/kexec_internal.h | 2 ++ 4 files changed, 16 insertions(+) diff --git a/kernel/kexec.c b/kernel/kexec.c index 1b018f1a6e0d..27b71dc7b35a 100644 --- a/kernel/kexec.c +++ b/kernel/kexec.c @@ -159,6 +159,10 @@ static int do_kexec_load(unsigned long entry, unsigned long nr_segments, kimage_terminate(image); + ret = machine_kexec_post_load(image); + if (ret) + goto out; + /* Install the new kernel and uninstall the old */ image = xchg(dest_image, image); diff --git a/kernel/kexec_core.c b/kernel/kexec_core.c index 2c5b72863b7b..8360645d1bbe 100644 --- a/kernel/kexec_core.c +++ b/kernel/kexec_core.c @@ -587,6 +587,12 @@ static void kimage_free_extra_pages(struct kimage *image) kimage_free_page_list(&image->unusable_pages); } + +int __weak machine_kexec_post_load(struct kimage *image) +{ + return 0; +} + void kimage_terminate(struct kimage *image) { if (*image->entry != 0) diff --git a/kernel/kexec_file.c b/kernel/kexec_file.c index b8cc032d5620..cb531d768114 100644 --- a/kernel/kexec_file.c +++ b/kernel/kexec_file.c @@ -391,6 +391,10 @@ SYSCALL_DEFINE5(kexec_file_load, int, kernel_fd, int, initrd_fd, kimage_terminate(image); + ret = machine_kexec_post_load(image); + if (ret) + goto out; + /* * Free up any temporary buffers allocated which are not needed * after image has been loaded diff --git a/kernel/kexec_internal.h b/kernel/kexec_internal.h index 48aaf2ac0d0d..39d30ccf8d87 100644 --- a/kernel/kexec_internal.h +++ b/kernel/kexec_internal.h @@ -13,6 +13,8 @@ void kimage_terminate(struct kimage *image); int kimage_is_destination_range(struct kimage *image, unsigned long start, unsigned long end); +int machine_kexec_post_load(struct kimage *image); + extern struct mutex kexec_mutex; #ifdef CONFIG_KEXEC_FILE From patchwork Mon Sep 23 20:34:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157597 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A66B01668 for ; Mon, 23 Sep 2019 20:35:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 65254207FD for ; Mon, 23 Sep 2019 20:35:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="jux3OpGp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 65254207FD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C41EA6B029B; Mon, 23 Sep 2019 16:35:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BCD5B6B029C; Mon, 23 Sep 2019 16:35:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A22D66B029D; Mon, 23 Sep 2019 16:35:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 7F1256B029B for ; Mon, 23 Sep 2019 16:35:16 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 2DC3F180AD806 for ; Mon, 23 Sep 2019 20:35:16 +0000 (UTC) X-FDA: 75967340232.27.cave38_853cbb844a149 X-Spam-Summary: 2,0,0,8b9b46e698e9e6c2,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:2:41:69:355:379:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1381:1437:1515:1535:1606:1730:1747:1777:1792:2393:2553:2559:2562:2693:3138:3139:3140:3141:3142:3355:3622:3865:3866:3867:3868:3870:3871:4119:4321:4605:5007:6261:6653:6737:7875:7903:9592:10004:11026:11473:11657:11658:11914:12043:12048:12294:12296:12297:12438:12517:12519:12555:12683:12895:13894:14096:14394:21063:21080:21433:21444:21451:21627:30054:30056:30079:30090,0,RBL:209.85.210.196:@soleen.com:.lbl8.mailshell.net-62.2.0.100 66.100.201.201,CacheIP:none,Bayesian:0 .5,0.5,0 X-HE-Tag: cave38_853cbb844a149 X-Filterd-Recvd-Size: 8350 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:15 +0000 (UTC) Received: by mail-pf1-f196.google.com with SMTP id v4so5219866pff.6 for ; Mon, 23 Sep 2019 13:35:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=FUw6qa2MYJdKp/gIXzP4SLM1N30xDLUwNEhq/71i4ng=; b=jux3OpGpnyRelCm06XQFKJxAtiCgQjXAU3yBuk4hiMuoEu9NAtJTYWv+FZ+dfztsVy DFOdweakcZOZQtBN2eXpsNoPf9Qe1YFmCu8EsodDkfK7QBbz0RdmnyVM2hMpH9vVfTqD DBZv9iWRyd1m9IwdDoScnkJkMph9MLTQ2z3sg98y5dVQU6cZRSxgDubHwe7/ZbEE9tc2 lDRqKVwuMHeI8x2KzdjBNUeuRoeQjSzbpL9VygINigNYWO0DrI37mtooeYchu4c30lbz f5SxcOFNW5wUoQesL8cthTvZ7AefKaazjxFLtfNMjWFJ73gsuIF8aYhssM4sGDUNHfcL KiBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FUw6qa2MYJdKp/gIXzP4SLM1N30xDLUwNEhq/71i4ng=; b=lAcTcsGyWylHVBUvgQ4Rem+Wz0b3ezgOfsv3OfolKxn2UCjrdYvIMeJQJnrtj2TLVG hy6MsG2OALNj8jxSxYjYDmevFxgI+LCuKTlkK45rXC4WwSoX7pKV3Wvvi4XNYYTNHnaN LmcGUnV43odcHtC7XLZ994bXPGMMoACXCYWrxWrUfryENcE0FrAm1RwUOSSpPOdnW5Nr sMskPED1yyP7hNtzA5YJ7yNtdffx1MBrHZbqtnlzOt0y2iGOrqdidUueTR09Q4cT2BLg WqOup6cGmAgFCjME2JUBQMMsd9NhACnfO8bKwdajPGPprUoa+TPxIoUMkUFT4zlkCSRO /nVg== X-Gm-Message-State: APjAAAUaEk+0tpNt5VFPSeBv4Wda4hRxoDznGB6sxvYvm0Baf++4dwmC 7fbnQFqEijti6v6IpWasfuVUMA== X-Google-Smtp-Source: APXvYqxCTgz3dICM/EAcruCJjgncpNyB5h7Y/fv5wHwmR3EVjnzautTzShDb2tQwkjaazbLE8xsUFw== X-Received: by 2002:a17:90a:330e:: with SMTP id m14mr1370234pjb.25.1569270914523; Mon, 23 Sep 2019 13:35:14 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:13 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 14/17] arm64: kexec: move relocation function setup and clean up Date: Mon, 23 Sep 2019 16:34:24 -0400 Message-Id: <20190923203427.294286-15-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, kernel relocation function is configured in machine_kexec() at the time of kexec reboot by using control_code_page. This operation, however, is more logical to be done during kexec_load, and thus remove from reboot time. Move, setup of this function to newly added machine_kexec_post_load(). In addition, do some cleanup: add infor about reloction function to kexec_image_info(), and remove extra messages from machine_kexec(). Make dtb_mem, always available, if CONFIG_KEXEC_FILE is not configured dtb_mem is set to zero anyway. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 3 +- arch/arm64/kernel/machine_kexec.c | 49 +++++++++++-------------------- 2 files changed, 19 insertions(+), 33 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 12a561a54128..d15ca1ca1e83 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,14 +90,15 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif -#ifdef CONFIG_KEXEC_FILE #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { void *dtb; unsigned long dtb_mem; + unsigned long kern_reloc; }; +#ifdef CONFIG_KEXEC_FILE extern const struct kexec_file_ops kexec_image_ops; struct kimage; diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 0df8493624e0..9b41da50e6f7 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -42,6 +42,7 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" start: %lx\n", kimage->start); pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); + pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -58,6 +59,19 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } +int machine_kexec_post_load(struct kimage *kimage) +{ + unsigned long kern_reloc; + + kern_reloc = page_to_phys(kimage->control_code_page); + memcpy(__va(kern_reloc), arm64_relocate_new_kernel, + arm64_relocate_new_kernel_size); + kimage->arch.kern_reloc = kern_reloc; + + kexec_image_info(kimage); + return 0; +} + /** * machine_kexec_prepare - Prepare for a kexec reboot. * @@ -67,8 +81,6 @@ void machine_kexec_cleanup(struct kimage *kimage) */ int machine_kexec_prepare(struct kimage *kimage) { - kexec_image_info(kimage); - if (kimage->type != KEXEC_TYPE_CRASH && cpus_are_stuck_in_kernel()) { pr_err("Can't kexec: CPUs are stuck in the kernel.\n"); return -EBUSY; @@ -143,8 +155,7 @@ static void kexec_segment_flush(const struct kimage *kimage) */ void machine_kexec(struct kimage *kimage) { - phys_addr_t reboot_code_buffer_phys; - void *reboot_code_buffer; + void *reboot_code_buffer = phys_to_virt(kimage->arch.kern_reloc); bool in_kexec_crash = (kimage == kexec_crash_image); bool stuck_cpus = cpus_are_stuck_in_kernel(); @@ -155,30 +166,8 @@ void machine_kexec(struct kimage *kimage) WARN(in_kexec_crash && (stuck_cpus || smp_crash_stop_failed()), "Some CPUs may be stale, kdump will be unreliable.\n"); - reboot_code_buffer_phys = page_to_phys(kimage->control_code_page); - reboot_code_buffer = phys_to_virt(reboot_code_buffer_phys); - kexec_image_info(kimage); - pr_debug("%s:%d: control_code_page: %p\n", __func__, __LINE__, - kimage->control_code_page); - pr_debug("%s:%d: reboot_code_buffer_phys: %pa\n", __func__, __LINE__, - &reboot_code_buffer_phys); - pr_debug("%s:%d: reboot_code_buffer: %p\n", __func__, __LINE__, - reboot_code_buffer); - pr_debug("%s:%d: relocate_new_kernel: %p\n", __func__, __LINE__, - arm64_relocate_new_kernel); - pr_debug("%s:%d: relocate_new_kernel_size: 0x%lx(%lu) bytes\n", - __func__, __LINE__, arm64_relocate_new_kernel_size, - arm64_relocate_new_kernel_size); - - /* - * Copy arm64_relocate_new_kernel to the reboot_code_buffer for use - * after the kernel is shut down. - */ - memcpy(reboot_code_buffer, arm64_relocate_new_kernel, - arm64_relocate_new_kernel_size); - /* Flush the reboot_code_buffer in preparation for its execution. */ __flush_dcache_area(reboot_code_buffer, arm64_relocate_new_kernel_size); @@ -214,12 +203,8 @@ void machine_kexec(struct kimage *kimage) * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(reboot_code_buffer_phys, kimage->head, kimage->start, -#ifdef CONFIG_KEXEC_FILE - kimage->arch.dtb_mem); -#else - 0); -#endif + cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, + kimage->arch.dtb_mem); BUG(); /* Should never get here. */ } From patchwork Mon Sep 23 20:34:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 821F71668 for ; Mon, 23 Sep 2019 20:35:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 351FE214DA for ; Mon, 23 Sep 2019 20:35:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="EAs0+f0e" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 351FE214DA Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9AA5A6B029D; Mon, 23 Sep 2019 16:35:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9595B6B029F; Mon, 23 Sep 2019 16:35:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 849146B02A0; Mon, 23 Sep 2019 16:35:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 5464F6B029D for ; Mon, 23 Sep 2019 16:35:19 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id E66E3840F for ; Mon, 23 Sep 2019 20:35:18 +0000 (UTC) X-FDA: 75967340316.11.spoon09_85a300e6e903d X-Spam-Summary: 2,0,0,6830419e1109edce,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:1:41:69:355:379:421:541:800:960:966:973:988:989:1260:1311:1314:1345:1359:1381:1431:1437:1500:1515:1605:1730:1747:1777:1792:2194:2196:2199:2200:2393:2553:2559:2562:2637:2693:2892:2896:3138:3139:3140:3141:3142:3622:3865:3866:3867:3868:3870:3871:3872:3874:4250:4321:4385:4605:5007:6117:6119:6261:6653:6737:7903:8603:8784:9010:9592:10004:10226:11026:11233:11473:11657:11658:11914:12043:12048:12295:12296:12297:12438:12517:12519:12555:12679:12895:12986:13149:13161:13229:13230:13894:13972:14394:21063:21080:21324:21325:21444:21451:21627:2174 0:21796: X-HE-Tag: spoon09_85a300e6e903d X-Filterd-Recvd-Size: 14982 Received: from mail-pf1-f195.google.com (mail-pf1-f195.google.com [209.85.210.195]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:18 +0000 (UTC) Received: by mail-pf1-f195.google.com with SMTP id 205so9853264pfw.2 for ; Mon, 23 Sep 2019 13:35:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=uw6s1yEhhfH1B/xKucsl67sYBjMXvbaEFciAMxIuXLw=; b=EAs0+f0eb4V7RymDbO+ruQZqm9GrneWGmejFqNJXLxPT/wy4qbWLf7e8Rm2w7cCl4B npAsZjtxbOVouw4a8y8rl818Wi2kSXuerOwk84c3QqsmgDeJLKnTkddRwBMI356zgalV wO1NFsNrKo1+AnxxdIKJ6VQG6OFlq8kg46Fg+F0DZFQVYWVJyk48gJe5E7zsWej1fKNs pup4zs/LkTfsvGTPKyAXDfyR4oUkIFOh1Y8tAKczEpx0hJt4ouakVVrFJAz5BS6xIUCH dPADc+fYsRrMBR2i/Lh1NY1KseY7M9Ec/s8QM2BjJ+Arw6IgB53JoyrIEYAxsQDWY6Cr lFYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uw6s1yEhhfH1B/xKucsl67sYBjMXvbaEFciAMxIuXLw=; b=SReDCW6cbFGkp8YGY9LoZAXFbeoF3qXCCXjlvfzz5oEztdTERoc7JldkKsb2tfyq3r fjqYGHYfLERdl5mC2nKOjtFlU0m+ZEwE85HYs1dnL4UJpkw7kRKklNf2AwrrVqZ3B9dS Jwf86jciztyV/CI7s3gEj6QWnETk7QANynfigDt+jAYiApEfsNZ6bfg7d/iYFlU2mly8 trXX0N1N3ZibHw153YZtmHwwDSCJ9VGm5pJOavJmNETbcvFOtHZ/2LZ1WAPCcvoQprFO RUM1LkUe0fTAjxCvIxaIX1ODaGfdJuRTIYvEGVH1w/FTJbPgpzNMLHSiTOYOHsLLVMrT pCCg== X-Gm-Message-State: APjAAAUPFI84aeg19Cbe+2ckfSqKJxu0ye33JwsVVJU5tgxRf/Zuf02g 78W3zY+BX9xX0ykO9xB5tOoUxQ== X-Google-Smtp-Source: APXvYqxjRKRX+IBqPOgBNyAxC9U2z1VmniLULpLQiwYJBvozZu/oVQe3txdx6M/4xYvriPiCI1SFhQ== X-Received: by 2002:a17:90a:dd43:: with SMTP id u3mr1388358pjv.98.1569270917196; Mon, 23 Sep 2019 13:35:17 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:16 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 15/17] arm64: kexec: add expandable argument to relocation function Date: Mon, 23 Sep 2019 16:34:25 -0400 Message-Id: <20190923203427.294286-16-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, kexec relocation function (arm64_relocate_new_kernel) accepts the following arguments: head: start of array that contains relocation information. entry: entry point for new kernel or purgatory. dtb_mem: first and only argument to entry. The number of arguments cannot be easily expended, because this function is also called from HVC_SOFT_RESTART, which preserves only three arguments. And, also arm64_relocate_new_kernel is written in assembly but called without stack, thus no place to move extra arguments to free registers. Soon, we will need to pass more arguments: once we enable MMU we will need to pass information about page tables. Another benefit of allowing this function to accept more arguments, is that kernel can actually accept up to 4 arguments (x0-x3), however currently only one is used, but if in the future we will need for more (for example, pass information about when previous kernel exited to have a precise measurement in time spent in purgatory), we won't be easilty do that if arm64_relocate_new_kernel can't accept more arguments. So, add a new struct: kern_reloc_arg, and place it in kexec safe page (i.e memory that is not overwritten during relocation). Thus, make arm64_relocate_new_kernel to only take one argument, that contains all the needed information. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 18 ++++++ arch/arm64/kernel/asm-offsets.c | 9 +++ arch/arm64/kernel/cpu-reset.S | 4 +- arch/arm64/kernel/cpu-reset.h | 8 +-- arch/arm64/kernel/machine_kexec.c | 29 +++++++++- arch/arm64/kernel/relocate_kernel.S | 88 ++++++++++------------------- 6 files changed, 87 insertions(+), 69 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d15ca1ca1e83..d5b79d4c7fae 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,12 +90,30 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +/* + * kern_reloc_arg is passed to kernel relocation function as an argument. + * head kimage->head, allows to traverse through relocation segments. + * entry_addr kimage->start, where to jump from relocation function (new + * kernel, or purgatory entry address). + * kern_arg0 first argument to kernel is its dtb address. The other + * arguments are currently unused, and must be set to 0 + */ +struct kern_reloc_arg { + unsigned long head; + unsigned long entry_addr; + unsigned long kern_arg0; + unsigned long kern_arg1; + unsigned long kern_arg2; + unsigned long kern_arg3; +}; + #define ARCH_HAS_KIMAGE_ARCH struct kimage_arch { void *dtb; unsigned long dtb_mem; unsigned long kern_reloc; + unsigned long kern_reloc_arg; }; #ifdef CONFIG_KEXEC_FILE diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 214685760e1c..900394907fd8 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -23,6 +23,7 @@ #include #include #include +#include int main(void) { @@ -126,6 +127,14 @@ int main(void) #ifdef CONFIG_ARM_SDE_INTERFACE DEFINE(SDEI_EVENT_INTREGS, offsetof(struct sdei_registered_event, interrupted_regs)); DEFINE(SDEI_EVENT_PRIORITY, offsetof(struct sdei_registered_event, priority)); +#endif +#ifdef CONFIG_KEXEC_CORE + DEFINE(KRELOC_HEAD, offsetof(struct kern_reloc_arg, head)); + DEFINE(KRELOC_ENTRY_ADDR, offsetof(struct kern_reloc_arg, entry_addr)); + DEFINE(KRELOC_KERN_ARG0, offsetof(struct kern_reloc_arg, kern_arg0)); + DEFINE(KRELOC_KERN_ARG1, offsetof(struct kern_reloc_arg, kern_arg1)); + DEFINE(KRELOC_KERN_ARG2, offsetof(struct kern_reloc_arg, kern_arg2)); + DEFINE(KRELOC_KERN_ARG3, offsetof(struct kern_reloc_arg, kern_arg3)); #endif return 0; } diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 6ea337d464c4..64c78a42919f 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -43,9 +43,7 @@ ENTRY(__cpu_soft_restart) hvc #0 // no return 1: mov x18, x1 // entry - mov x0, x2 // arg0 - mov x1, x3 // arg1 - mov x2, x4 // arg2 + mov x0, x2 // arg br x18 ENDPROC(__cpu_soft_restart) diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..7a8720ff186f 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -11,12 +11,10 @@ #include void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, - unsigned long arg0, unsigned long arg1, unsigned long arg2); + unsigned long arg); static inline void __noreturn cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) + unsigned long arg) { typeof(__cpu_soft_restart) *restart; @@ -25,7 +23,7 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, restart = (void *)__pa_symbol(__cpu_soft_restart); cpu_install_idmap(); - restart(el2_switch, entry, arg0, arg1, arg2); + restart(el2_switch, entry, arg); unreachable(); } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 9b41da50e6f7..fb6138a1c9ff 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -43,6 +43,7 @@ static void _kexec_image_info(const char *func, int line, pr_debug(" head: %lx\n", kimage->head); pr_debug(" nr_segments: %lu\n", kimage->nr_segments); pr_debug(" kern_reloc: %pa\n", &kimage->arch.kern_reloc); + pr_debug(" kern_reloc_arg: %pa\n", &kimage->arch.kern_reloc_arg); for (i = 0; i < kimage->nr_segments; i++) { pr_debug(" segment[%lu]: %016lx - %016lx, 0x%lx bytes, %lu pages\n", @@ -59,14 +60,39 @@ void machine_kexec_cleanup(struct kimage *kimage) /* Empty routine needed to avoid build errors. */ } +/* Allocates pages for kexec page table */ +static void *kexec_page_alloc(void *arg) +{ + struct kimage *kimage = (struct kimage *)arg; + struct page *page = kimage_alloc_control_pages(kimage, 0); + + if (!page) + return NULL; + + memset(page_address(page), 0, PAGE_SIZE); + + return page_address(page); +} + int machine_kexec_post_load(struct kimage *kimage) { unsigned long kern_reloc; + struct kern_reloc_arg *kern_reloc_arg; kern_reloc = page_to_phys(kimage->control_code_page); memcpy(__va(kern_reloc), arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); + + kern_reloc_arg = kexec_page_alloc(kimage); + if (!kern_reloc_arg) + return -ENOMEM; + kimage->arch.kern_reloc = kern_reloc; + kimage->arch.kern_reloc_arg = __pa(kern_reloc_arg); + + kern_reloc_arg->head = kimage->head; + kern_reloc_arg->entry_addr = kimage->start; + kern_reloc_arg->kern_arg0 = kimage->arch.dtb_mem; kexec_image_info(kimage); return 0; @@ -203,8 +229,7 @@ void machine_kexec(struct kimage *kimage) * userspace (kexec-tools). * In kexec_file case, the kernel starts directly without purgatory. */ - cpu_soft_restart(kimage->arch.kern_reloc, kimage->head, kimage->start, - kimage->arch.dtb_mem); + cpu_soft_restart(kimage->arch.kern_reloc, kimage->arch.kern_reloc_arg); BUG(); /* Should never get here. */ } diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index c1d7db71a726..d352faf7cbe6 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -8,7 +8,7 @@ #include #include - +#include #include #include #include @@ -17,86 +17,58 @@ /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * - * The memory that the old kernel occupies may be overwritten when coping the + * The memory that the old kernel occupies may be overwritten when copying the * new image to its final location. To assure that the * arm64_relocate_new_kernel routine which does that copy is not overwritten, * all code and data needed by arm64_relocate_new_kernel must be between the * symbols arm64_relocate_new_kernel and arm64_relocate_new_kernel_end. The * machine_kexec() routine will copy arm64_relocate_new_kernel to the kexec - * control_code_page, a special page which has been set up to be preserved - * during the copy operation. + * safe memory that has been set up to be preserved during the copy operation. */ ENTRY(arm64_relocate_new_kernel) - - /* Setup the list loop variables. */ - mov x18, x2 /* x18 = dtb address */ - mov x17, x1 /* x17 = kimage_start */ - mov x16, x0 /* x16 = kimage_head */ - raw_dcache_line_size x15, x0 /* x15 = dcache line size */ - mov x14, xzr /* x14 = entry ptr */ - mov x13, xzr /* x13 = copy dest */ - /* Clear the sctlr_el2 flags. */ - mrs x0, CurrentEL - cmp x0, #CurrentEL_EL2 + mrs x2, CurrentEL + cmp x2, #CurrentEL_EL2 b.ne 1f - mrs x0, sctlr_el2 + mrs x2, sctlr_el2 ldr x1, =SCTLR_ELx_FLAGS - bic x0, x0, x1 + bic x2, x2, x1 pre_disable_mmu_workaround - msr sctlr_el2, x0 + msr sctlr_el2, x2 isb -1: - - /* Check if the new image needs relocation. */ +1: /* Check if the new image needs relocation. */ + ldr x16, [x0, #KRELOC_HEAD] /* x16 = kimage_head */ tbnz x16, IND_DONE_BIT, .Ldone - + raw_dcache_line_size x15, x1 /* x15 = dcache line size */ .Lloop: and x12, x16, PAGE_MASK /* x12 = addr */ - /* Test the entry flags. */ .Ltest_source: tbz x16, IND_SOURCE_BIT, .Ltest_indirection /* Invalidate dest page to PoC. */ - mov x0, x13 - add x20, x0, #PAGE_SIZE + mov x2, x13 + add x20, x2, #PAGE_SIZE sub x1, x15, #1 - bic x0, x0, x1 -2: dc ivac, x0 - add x0, x0, x15 - cmp x0, x20 + bic x2, x2, x1 +2: dc ivac, x2 + add x2, x2, x15 + cmp x2, x20 b.lo 2b dsb sy - mov x20, x13 - mov x21, x12 - copy_page x20, x21, x0, x1, x2, x3, x4, x5, x6, x7 - - /* dest += PAGE_SIZE */ - add x13, x13, PAGE_SIZE + copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 b .Lnext - .Ltest_indirection: tbz x16, IND_INDIRECTION_BIT, .Ltest_destination - - /* ptr = addr */ - mov x14, x12 + mov x14, x12 /* ptr = addr */ b .Lnext - .Ltest_destination: tbz x16, IND_DESTINATION_BIT, .Lnext - - /* dest = addr */ - mov x13, x12 - + mov x13, x12 /* dest = addr */ .Lnext: - /* entry = *ptr++ */ - ldr x16, [x14], #8 - - /* while (!(entry & DONE)) */ - tbz x16, IND_DONE_BIT, .Lloop - + ldr x16, [x14], #8 /* entry = *ptr++ */ + tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ .Ldone: /* wait for writes from copy_page to finish */ dsb nsh @@ -105,18 +77,16 @@ ENTRY(arm64_relocate_new_kernel) isb /* Start new image. */ - mov x0, x18 - mov x1, xzr - mov x2, xzr - mov x3, xzr - br x17 - -ENDPROC(arm64_relocate_new_kernel) + ldr x4, [x0, #KRELOC_ENTRY_ADDR] /* x4 = kimage_start */ + ldr x3, [x0, #KRELOC_KERN_ARG3] + ldr x2, [x0, #KRELOC_KERN_ARG2] + ldr x1, [x0, #KRELOC_KERN_ARG1] + ldr x0, [x0, #KRELOC_KERN_ARG0] /* x0 = dtb address */ + br x4 +END(arm64_relocate_new_kernel) .ltorg - .align 3 /* To keep the 64-bit values below naturally aligned. */ - .Lcopy_end: .org KEXEC_CONTROL_PAGE_SIZE From patchwork Mon Sep 23 20:34:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF4391599 for ; Mon, 23 Sep 2019 20:35:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9E51220673 for ; Mon, 23 Sep 2019 20:35:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="OGxUtTFa" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E51220673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 69B766B02A0; Mon, 23 Sep 2019 16:35:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6539C6B02A1; Mon, 23 Sep 2019 16:35:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ED976B02A2; Mon, 23 Sep 2019 16:35:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 2D2FE6B02A0 for ; Mon, 23 Sep 2019 16:35:22 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with SMTP id C122A181AC9BA for ; Mon, 23 Sep 2019 20:35:21 +0000 (UTC) X-FDA: 75967340442.29.join95_860974df1dd3a X-Spam-Summary: 2,0,0,53213cff6ed49cee,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:1:2:41:355:379:541:800:960:973:988:989:1260:1311:1314:1345:1359:1381:1431:1437:1500:1515:1605:1730:1747:1777:1792:2194:2199:2393:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3874:4052:4250:4321:4605:5007:6119:6261:6653:6737:7875:7903:8603:9036:9592:10004:11026:11473:11657:11658:11914:12043:12048:12291:12295:12296:12297:12438:12517:12519:12555:12895:12986:13161:13229:13548:13894:14096:14394:21063:21080:21433:21444:21451:21627:30003:30034:30054:30070:30075:30079,0,RBL:209.85.215.194:@soleen.com:.lbl8.mailshe ll.net-6 X-HE-Tag: join95_860974df1dd3a X-Filterd-Recvd-Size: 13146 Received: from mail-pg1-f194.google.com (mail-pg1-f194.google.com [209.85.215.194]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:20 +0000 (UTC) Received: by mail-pg1-f194.google.com with SMTP id 4so8623111pgm.12 for ; Mon, 23 Sep 2019 13:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1RZkZY1cm3fyZ1mRNLzSw31HBeKwrW9anL0qGMNckK8=; b=OGxUtTFa9LaWeK9KWDL+UrpALkAR0k4TOpQ/i5LOqHeEZUnm9DzsAJH0AJoYBMJyCq AJAkfoRHvjs7vrby58uPHeZVHQuHzxJq0zIilBoi5UL1euBtHrkenJI7AY4dD4Uaz0JG 8CuvUY9Rr1En5VdqB9AF0arxAtsqsFfospIeoHu5RcflKy5E9k3SY/Eee/yh/L4kivDM Sy6QctDd+iWsKf7muddwV5gbQEEhohuxP2PlbCdVQwr411Vs9fyZ8JS0yTNvuQMDtDhs e3tT8fIoTpvCURZVTF+CWRAUVCytkY835vqmtVOOIxqQdoDVApa6XffZ96s6fnKGUq5f KFww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1RZkZY1cm3fyZ1mRNLzSw31HBeKwrW9anL0qGMNckK8=; b=MPqDgMRMJbC3zCTC6Wjq0ffH7fp+pMc3DaDdB/NtBOn5l6KxJ63NaIzNYJEr4iyEF6 6RtxpQGaJ6oaZOZ4uUS4mDJGOxd70jmpWiKCcoijFAYfZywswDWsr3l4MxZp735XL+5g jNJ3OlC+u3Xs5jlKb+pGOZtvzkcBnPummPKosCf+N7hoGedmETq+ho5q116o6DuPNsBb qwKBRpSLv7wcrFGq7Lw/DOoVv6m5iU1U5oSZFXZn23Y+TLpF0E2z1c1bWaRH8Z9U9gmv x0X6LtBAufG3/Fm8yicSLHrokdRn0TPVpPYwjOF/iCVfrMfTthlfJcrAaYCpbuvUW5zK w6lQ== X-Gm-Message-State: APjAAAWlXaZofS4B5g5Jy5APrCPLuPIUhJ+aN9kZQqt98w7+5V8uMktD XAtixzT6iSlSXk69RgiChDki7A== X-Google-Smtp-Source: APXvYqySVpP3jOsDcUDVSEo8IRqXfNNcCJU5Z6f4uLaJN3tO9ol6cdCc0j5ahgjr6Yxtc3Lb9w5hEw== X-Received: by 2002:a63:a54c:: with SMTP id r12mr1758722pgu.5.1569270919910; Mon, 23 Sep 2019 13:35:19 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:19 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 16/17] arm64: kexec: configure trans_pgd page table for kexec Date: Mon, 23 Sep 2019 16:34:26 -0400 Message-Id: <20190923203427.294286-17-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Configure a page table located in kexec-safe memory that has the following mappings: 1. identity mapping for text of relocation function with executable permission. 2. identity mapping for argument for relocation function. 3. linear mappings for all source ranges 4. linear mappings for all destination ranges. Also, configure el2_vector, that is used to jump to new kernel from EL2 on non-VHE kernels. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 32 +++++++ arch/arm64/kernel/asm-offsets.c | 6 ++ arch/arm64/kernel/machine_kexec.c | 125 ++++++++++++++++++++++++++-- arch/arm64/kernel/relocate_kernel.S | 16 +++- 4 files changed, 170 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index d5b79d4c7fae..450d8440f597 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -90,6 +90,23 @@ static inline void crash_prepare_suspend(void) {} static inline void crash_post_resume(void) {} #endif +#if defined(CONFIG_KEXEC_CORE) +/* Global variables for the arm64_relocate_new_kernel routine. */ +extern const unsigned char arm64_relocate_new_kernel[]; +extern const unsigned long arm64_relocate_new_kernel_size; + +/* Body of the vector for escalating to EL2 from relocation routine */ +extern const unsigned char kexec_el1_sync[]; +extern const unsigned long kexec_el1_sync_size; + +#define KEXEC_EL2_VECTOR_TABLE_SIZE 2048 +#define KEXEC_EL2_SYNC_OFFSET (KEXEC_EL2_VECTOR_TABLE_SIZE / 2) + +#endif + +#define KEXEC_SRC_START PAGE_OFFSET +#define KEXEC_DST_START (PAGE_OFFSET + \ + ((UL(0xffffffffffffffff) - PAGE_OFFSET) >> 1) + 1) /* * kern_reloc_arg is passed to kernel relocation function as an argument. * head kimage->head, allows to traverse through relocation segments. @@ -97,6 +114,15 @@ static inline void crash_post_resume(void) {} * kernel, or purgatory entry address). * kern_arg0 first argument to kernel is its dtb address. The other * arguments are currently unused, and must be set to 0 + * trans_ttbr0 idmap for relocation function and its argument + * trans_ttbr1 linear map for source/destination addresses. + * el2_vector If present means that relocation routine will go to EL1 + * from EL2 to do the copy, and then back to EL2 to do the jump + * to new world. This vector contains only the final jump + * instruction at KEXEC_EL2_SYNC_OFFSET. + * src_addr linear map for source pages. + * dst_addr linear map for destination pages. + * copy_len Number of bytes that need to be copied */ struct kern_reloc_arg { unsigned long head; @@ -105,6 +131,12 @@ struct kern_reloc_arg { unsigned long kern_arg1; unsigned long kern_arg2; unsigned long kern_arg3; + unsigned long trans_ttbr0; + unsigned long trans_ttbr1; + unsigned long el2_vector; + unsigned long src_addr; + unsigned long dst_addr; + unsigned long copy_len; }; #define ARCH_HAS_KIMAGE_ARCH diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 900394907fd8..7c2ba09a8ceb 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -135,6 +135,12 @@ int main(void) DEFINE(KRELOC_KERN_ARG1, offsetof(struct kern_reloc_arg, kern_arg1)); DEFINE(KRELOC_KERN_ARG2, offsetof(struct kern_reloc_arg, kern_arg2)); DEFINE(KRELOC_KERN_ARG3, offsetof(struct kern_reloc_arg, kern_arg3)); + DEFINE(KRELOC_TRANS_TTBR0, offsetof(struct kern_reloc_arg, trans_ttbr0)); + DEFINE(KRELOC_TRANS_TTBR1, offsetof(struct kern_reloc_arg, trans_ttbr1)); + DEFINE(KRELOC_EL2_VECTOR, offsetof(struct kern_reloc_arg, el2_vector)); + DEFINE(KRELOC_SRC_ADDR, offsetof(struct kern_reloc_arg, src_addr)); + DEFINE(KRELOC_DST_ADDR, offsetof(struct kern_reloc_arg, dst_addr)); + DEFINE(KRELOC_COPY_LEN, offsetof(struct kern_reloc_arg, copy_len)); #endif return 0; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index fb6138a1c9ff..71479013dd24 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -20,13 +20,10 @@ #include #include #include +#include #include "cpu-reset.h" -/* Global variables for the arm64_relocate_new_kernel routine. */ -extern const unsigned char arm64_relocate_new_kernel[]; -extern const unsigned long arm64_relocate_new_kernel_size; - /** * kexec_image_info - For debugging output. */ @@ -74,15 +71,124 @@ static void *kexec_page_alloc(void *arg) return page_address(page); } +/* + * Map source segments starting from KEXEC_SRC_START, and map destination + * segments starting from KEXEC_DST_START, and return size of copy in + * *copy_len argument. + * Relocation function essentially needs to do: + * memcpy(KEXEC_DST_START, KEXEC_SRC_START, copy_len); + */ +static int map_segments(struct kimage *kimage, pgd_t *pgdp, + struct trans_pgd_info *info, + unsigned long *copy_len) +{ + unsigned long *ptr = 0; + unsigned long dest = 0; + unsigned long src_va = KEXEC_SRC_START; + unsigned long dst_va = KEXEC_DST_START; + unsigned long len = 0; + unsigned long entry, addr; + int rc; + + for (entry = kimage->head; !(entry & IND_DONE); entry = *ptr++) { + addr = entry & PAGE_MASK; + + switch (entry & IND_FLAGS) { + case IND_DESTINATION: + dest = addr; + break; + case IND_INDIRECTION: + ptr = __va(addr); + if (rc) + return rc; + break; + case IND_SOURCE: + rc = trans_pgd_map_page(info, pgdp, __va(addr), + src_va, PAGE_KERNEL); + if (rc) + return rc; + rc = trans_pgd_map_page(info, pgdp, __va(dest), + dst_va, PAGE_KERNEL); + if (rc) + return rc; + dest += PAGE_SIZE; + src_va += PAGE_SIZE; + dst_va += PAGE_SIZE; + len += PAGE_SIZE; + } + } + *copy_len = len; + + return 0; +} + +static int mmu_relocate_setup(struct kimage *kimage, unsigned long kern_reloc, + struct kern_reloc_arg *kern_reloc_arg) +{ + struct trans_pgd_info info = { + .trans_alloc_page = kexec_page_alloc, + .trans_alloc_arg = kimage, + }; + + pgd_t *trans_ttbr0 = kexec_page_alloc(kimage); + pgd_t *trans_ttbr1 = kexec_page_alloc(kimage); + int rc; + + if (!trans_ttbr0 || !trans_ttbr1) + return -ENOMEM; + + rc = map_segments(kimage, trans_ttbr1, &info, + &kern_reloc_arg->copy_len); + if (rc) + return rc; + + /* Map relocation function va == pa */ + rc = trans_pgd_map_page(&info, trans_ttbr0, __va(kern_reloc), + kern_reloc, PAGE_KERNEL_EXEC); + if (rc) + return rc; + + /* Map relocation function argument va == pa */ + rc = trans_pgd_map_page(&info, trans_ttbr0, kern_reloc_arg, + __pa(kern_reloc_arg), PAGE_KERNEL); + if (rc) + return rc; + + kern_reloc_arg->trans_ttbr0 = phys_to_ttbr(__pa(trans_ttbr0)); + kern_reloc_arg->trans_ttbr1 = phys_to_ttbr(__pa(trans_ttbr1)); + kern_reloc_arg->src_addr = KEXEC_SRC_START; + kern_reloc_arg->dst_addr = KEXEC_DST_START; + + return 0; +} + int machine_kexec_post_load(struct kimage *kimage) { + unsigned long el2_vector = 0; unsigned long kern_reloc; struct kern_reloc_arg *kern_reloc_arg; + int rc = 0; + + /* + * Sanity check that relocation function + el2_vector fit into one + * page. + */ + if (arm64_relocate_new_kernel_size > KEXEC_EL2_VECTOR_TABLE_SIZE) { + pr_err("can't fit relocation function and el2_vector in one page"); + return -ENOMEM; + } kern_reloc = page_to_phys(kimage->control_code_page); memcpy(__va(kern_reloc), arm64_relocate_new_kernel, arm64_relocate_new_kernel_size); + /* Setup vector table only when EL2 is available, but no VHE */ + if (is_hyp_mode_available() && !is_kernel_in_hyp_mode()) { + el2_vector = kern_reloc + KEXEC_EL2_VECTOR_TABLE_SIZE; + memcpy(__va(el2_vector + KEXEC_EL2_SYNC_OFFSET), kexec_el1_sync, + kexec_el1_sync_size); + } + kern_reloc_arg = kexec_page_alloc(kimage); if (!kern_reloc_arg) return -ENOMEM; @@ -92,10 +198,19 @@ int machine_kexec_post_load(struct kimage *kimage) kern_reloc_arg->head = kimage->head; kern_reloc_arg->entry_addr = kimage->start; + kern_reloc_arg->el2_vector = el2_vector; kern_reloc_arg->kern_arg0 = kimage->arch.dtb_mem; + /* + * If relocation is not needed, we do not need to enable MMU in + * relocation routine, therefore do not create page tables for + * scenarios such as crash kernel + */ + if (!(kimage->head & IND_DONE)) + rc = mmu_relocate_setup(kimage, kern_reloc, kern_reloc_arg); + kexec_image_info(kimage); - return 0; + return rc; } /** diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index d352faf7cbe6..14243a678277 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -83,17 +83,25 @@ ENTRY(arm64_relocate_new_kernel) ldr x1, [x0, #KRELOC_KERN_ARG1] ldr x0, [x0, #KRELOC_KERN_ARG0] /* x0 = dtb address */ br x4 +.ltorg +.Larm64_relocate_new_kernel_end: END(arm64_relocate_new_kernel) -.ltorg +ENTRY(kexec_el1_sync) + br x4 /* Jump to new world from el2 */ +.Lkexec_el1_sync_end: +END(kexec_el1_sync) + .align 3 /* To keep the 64-bit values below naturally aligned. */ -.Lcopy_end: .org KEXEC_CONTROL_PAGE_SIZE - /* * arm64_relocate_new_kernel_size - Number of bytes to copy to the * control_code_page. */ .globl arm64_relocate_new_kernel_size arm64_relocate_new_kernel_size: - .quad .Lcopy_end - arm64_relocate_new_kernel + .quad .Larm64_relocate_new_kernel_end - arm64_relocate_new_kernel + +.globl kexec_el1_sync_size +kexec_el1_sync_size: + .quad .Lkexec_el1_sync_end - kexec_el1_sync From patchwork Mon Sep 23 20:34:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pasha Tatashin X-Patchwork-Id: 11157603 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 414E61599 for ; Mon, 23 Sep 2019 20:35:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0022D20673 for ; Mon, 23 Sep 2019 20:35:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=soleen.com header.i=@soleen.com header.b="lM7DPRQ7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0022D20673 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2A9E56B02A3; Mon, 23 Sep 2019 16:35:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 22F826B02A4; Mon, 23 Sep 2019 16:35:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CFAD6B02A5; Mon, 23 Sep 2019 16:35:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0141.hostedemail.com [216.40.44.141]) by kanga.kvack.org (Postfix) with ESMTP id E0DB96B02A3 for ; Mon, 23 Sep 2019 16:35:24 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with SMTP id 8CB929095 for ; Mon, 23 Sep 2019 20:35:24 +0000 (UTC) X-FDA: 75967340568.12.bite53_86755d3d99116 X-Spam-Summary: 2,0,0,e0426a069abd3392,d41d8cd98f00b204,pasha.tatashin@soleen.com,:pasha.tatashin@soleen.com:jmorris@namei.org:sashal@kernel.org:ebiederm@xmission.com:kexec@lists.infradead.org:linux-kernel@vger.kernel.org:corbet@lwn.net:catalin.marinas@arm.com:will@kernel.org:linux-arm-kernel@lists.infradead.org:marc.zyngier@arm.com:james.morse@arm.com:vladimir.murzin@arm.com:matthias.bgg@gmail.com:bhsharma@redhat.com::mark.rutland@arm.com,RULES_HIT:1:2:41:69:355:379:421:541:800:960:968:973:988:989:1260:1311:1314:1345:1359:1381:1431:1437:1515:1605:1730:1747:1777:1792:2194:2199:2393:2538:2559:2562:2693:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:4051:4250:4321:4605:5007:6117:6119:6261:6653:6737:7688:7875:7903:8603:9036:9592:10004:10226:11026:11232:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12895:12986:13161:13229:13894:13972:14394:21063:21080:21324:21325:21444:21451:21627:21740:21966:30003:30034:30054:30067:30070:30079:30 089,0,RB X-HE-Tag: bite53_86755d3d99116 X-Filterd-Recvd-Size: 11494 Received: from mail-pf1-f194.google.com (mail-pf1-f194.google.com [209.85.210.194]) by imf12.hostedemail.com (Postfix) with ESMTP for ; Mon, 23 Sep 2019 20:35:23 +0000 (UTC) Received: by mail-pf1-f194.google.com with SMTP id y22so9851154pfr.3 for ; Mon, 23 Sep 2019 13:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=T59+jejKmTBkrYoU4MBF6HUfmWY8hcdgE9D78tkOOv0=; b=lM7DPRQ7e6v/hvLmBSit4hb8WJAUQx0/C5HrI3WytOGmcYX89DmSX517hOHUmnAu1t spchMnp9U7xL1ILB+UX5oG2w6LaR4VPABfm6u2X9x64KKL8WSOfD6NZIhHHP0b7wnQiL Nrr3rWLF+NG4PRZ6E0/sugdRZUHUnJ5wvuC7gn8TmupG9Etcvn+Ao4RT5dnzPB15u8hj egdUdiEPEAw/9BwoWuPk5bOCx8tw1NspHOI1ZGilyFzEaTPH5NHqhQNwumTjvykb2v+f bTsJPtEcHp1G/IZKJBjyX40xF/13nN2OaRL6vsfV/3MNyDCrbOFIy2Rcj+S9YdFNhOZO PZvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=T59+jejKmTBkrYoU4MBF6HUfmWY8hcdgE9D78tkOOv0=; b=JHki27aBG23u48bwXpxT8zCLIFNo6zVYh0BeVfO4FFAD2eEXy6U9+sjdyigh2l5+QQ iO/wJDlwVnOLRx46EXrMvC0WIS5w5j7jVu9n99qzCB69z8k9fDDnWFVdsDGDnwcWQc7n AkuCiPVzJxKvFbgKsbct86Qr/lvNkfZMfGlABSi46ii4xaJHleLyINF9KbkIRuCqFNJs Im0itK5TFEiJ5oMDY+B5DT6nUF/DrB/JG8M2vJ3aeY7nspuoh1iWURXGPjtAkLi+qUwz yLtDpB7MdSh3hyv2ICk+hREEf24bkGJErvvU1QpCSqpJ1udnrEpqCnwkrOuN3FyaqgH6 hy+w== X-Gm-Message-State: APjAAAXpNC4JKqwuSponNH8wgUsrJXJV6yXTnrEgP7y52qWJCMUgxUeC YCb1wRWviAvKncg8i49RFppB0g== X-Google-Smtp-Source: APXvYqyjVk+NWaKTkuYcPYdlCcWwblDa45ibd88Lp4XFxrTYVEoZa2FEFlN3gx0GzatF1lPRsqzZlw== X-Received: by 2002:a65:41c2:: with SMTP id b2mr1745651pgq.320.1569270922760; Mon, 23 Sep 2019 13:35:22 -0700 (PDT) Received: from xakep.corp.microsoft.com (c-73-69-118-222.hsd1.nh.comcast.net. [73.69.118.222]) by smtp.gmail.com with ESMTPSA id n29sm12798676pgm.4.2019.09.23.13.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 13:35:22 -0700 (PDT) From: Pavel Tatashin To: pasha.tatashin@soleen.com, jmorris@namei.org, sashal@kernel.org, ebiederm@xmission.com, kexec@lists.infradead.org, linux-kernel@vger.kernel.org, corbet@lwn.net, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, marc.zyngier@arm.com, james.morse@arm.com, vladimir.murzin@arm.com, matthias.bgg@gmail.com, bhsharma@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com Subject: [PATCH v5 17/17] arm64: kexec: enable MMU during kexec relocation Date: Mon, 23 Sep 2019 16:34:27 -0400 Message-Id: <20190923203427.294286-18-pasha.tatashin@soleen.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20190923203427.294286-1-pasha.tatashin@soleen.com> References: <20190923203427.294286-1-pasha.tatashin@soleen.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, that we have transitional page tables configured, temporarily enable MMU to allow faster relocation of segments to final destination. The performance data: for a moderate size kernel + initramfs: 25M the relocation was taking 0.382s, with enabled MMU it now takes 0.019s only or x20 improvement. The time is proportional to the size of relocation, therefore if initramfs is larger, 100M it could take over a second. Also, remove reloc_arg->head, as it is not needed anymore once MMU is enabled. Signed-off-by: Pavel Tatashin --- arch/arm64/include/asm/kexec.h | 2 - arch/arm64/kernel/asm-offsets.c | 1 - arch/arm64/kernel/machine_kexec.c | 1 - arch/arm64/kernel/relocate_kernel.S | 136 +++++++++++++++++----------- 4 files changed, 84 insertions(+), 56 deletions(-) diff --git a/arch/arm64/include/asm/kexec.h b/arch/arm64/include/asm/kexec.h index 450d8440f597..ad81ed3e5751 100644 --- a/arch/arm64/include/asm/kexec.h +++ b/arch/arm64/include/asm/kexec.h @@ -109,7 +109,6 @@ extern const unsigned long kexec_el1_sync_size; ((UL(0xffffffffffffffff) - PAGE_OFFSET) >> 1) + 1) /* * kern_reloc_arg is passed to kernel relocation function as an argument. - * head kimage->head, allows to traverse through relocation segments. * entry_addr kimage->start, where to jump from relocation function (new * kernel, or purgatory entry address). * kern_arg0 first argument to kernel is its dtb address. The other @@ -125,7 +124,6 @@ extern const unsigned long kexec_el1_sync_size; * copy_len Number of bytes that need to be copied */ struct kern_reloc_arg { - unsigned long head; unsigned long entry_addr; unsigned long kern_arg0; unsigned long kern_arg1; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 7c2ba09a8ceb..13ad00b1b90f 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -129,7 +129,6 @@ int main(void) DEFINE(SDEI_EVENT_PRIORITY, offsetof(struct sdei_registered_event, priority)); #endif #ifdef CONFIG_KEXEC_CORE - DEFINE(KRELOC_HEAD, offsetof(struct kern_reloc_arg, head)); DEFINE(KRELOC_ENTRY_ADDR, offsetof(struct kern_reloc_arg, entry_addr)); DEFINE(KRELOC_KERN_ARG0, offsetof(struct kern_reloc_arg, kern_arg0)); DEFINE(KRELOC_KERN_ARG1, offsetof(struct kern_reloc_arg, kern_arg1)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 71479013dd24..970892970bab 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -196,7 +196,6 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.kern_reloc = kern_reloc; kimage->arch.kern_reloc_arg = __pa(kern_reloc_arg); - kern_reloc_arg->head = kimage->head; kern_reloc_arg->entry_addr = kimage->start; kern_reloc_arg->el2_vector = el2_vector; kern_reloc_arg->kern_arg0 = kimage->arch.dtb_mem; diff --git a/arch/arm64/kernel/relocate_kernel.S b/arch/arm64/kernel/relocate_kernel.S index 14243a678277..96ff6760bd9c 100644 --- a/arch/arm64/kernel/relocate_kernel.S +++ b/arch/arm64/kernel/relocate_kernel.S @@ -4,6 +4,8 @@ * * Copyright (C) Linaro. * Copyright (C) Huawei Futurewei Technologies. + * Copyright (c) 2019, Microsoft Corporation. + * Pavel Tatashin */ #include @@ -14,6 +16,49 @@ #include #include +/* Invalidae TLB */ +.macro tlb_invalidate + dsb sy + dsb ish + tlbi vmalle1 + dsb ish + isb +.endm + +/* Turn-off mmu at level specified by sctlr */ +.macro turn_off_mmu sctlr, tmp1, tmp2 + mrs \tmp1, \sctlr + ldr \tmp2, =SCTLR_ELx_FLAGS + bic \tmp1, \tmp1, \tmp2 + pre_disable_mmu_workaround + msr \sctlr, \tmp1 + isb +.endm + +/* Turn-on mmu at level specified by sctlr */ +.macro turn_on_mmu sctlr, tmp1, tmp2 + mrs \tmp1, \sctlr + ldr \tmp2, =SCTLR_ELx_FLAGS + orr \tmp1, \tmp1, \tmp2 + msr \sctlr, \tmp1 + ic iallu + dsb nsh + isb +.endm + +/* + * Set ttbr0 and ttbr1, called while MMU is disabled, so no need to temporarily + * set zero_page table. Invalidate TLB after new tables are set. + */ +.macro set_ttbr arg, tmp + ldr \tmp, [\arg, #KRELOC_TRANS_TTBR0] + msr ttbr0_el1, \tmp + ldr \tmp, [\arg, #KRELOC_TRANS_TTBR1] + offset_ttbr1 \tmp + msr ttbr1_el1, \tmp + isb +.endm + /* * arm64_relocate_new_kernel - Put a 2nd stage image in place and boot it. * @@ -24,65 +69,52 @@ * symbols arm64_relocate_new_kernel and arm64_relocate_new_kernel_end. The * machine_kexec() routine will copy arm64_relocate_new_kernel to the kexec * safe memory that has been set up to be preserved during the copy operation. + * + * This function temporarily enables MMU if kernel relocation is needed. + * Also, if we enter this function at EL2 on non-VHE kernel, we temporarily go + * to EL1 to enable MMU, and escalate back to EL2 at the end to do the jump to + * the new kernel. This is determined by presence of el2_vector. */ ENTRY(arm64_relocate_new_kernel) - /* Clear the sctlr_el2 flags. */ - mrs x2, CurrentEL - cmp x2, #CurrentEL_EL2 + mrs x1, CurrentEL + cmp x1, #CurrentEL_EL2 b.ne 1f - mrs x2, sctlr_el2 - ldr x1, =SCTLR_ELx_FLAGS - bic x2, x2, x1 - pre_disable_mmu_workaround - msr sctlr_el2, x2 - isb -1: /* Check if the new image needs relocation. */ - ldr x16, [x0, #KRELOC_HEAD] /* x16 = kimage_head */ - tbnz x16, IND_DONE_BIT, .Ldone - raw_dcache_line_size x15, x1 /* x15 = dcache line size */ -.Lloop: - and x12, x16, PAGE_MASK /* x12 = addr */ - /* Test the entry flags. */ -.Ltest_source: - tbz x16, IND_SOURCE_BIT, .Ltest_indirection - - /* Invalidate dest page to PoC. */ - mov x2, x13 - add x20, x2, #PAGE_SIZE - sub x1, x15, #1 - bic x2, x2, x1 -2: dc ivac, x2 - add x2, x2, x15 - cmp x2, x20 - b.lo 2b - dsb sy - - copy_page x13, x12, x1, x2, x3, x4, x5, x6, x7, x8 - b .Lnext -.Ltest_indirection: - tbz x16, IND_INDIRECTION_BIT, .Ltest_destination - mov x14, x12 /* ptr = addr */ - b .Lnext -.Ltest_destination: - tbz x16, IND_DESTINATION_BIT, .Lnext - mov x13, x12 /* dest = addr */ -.Lnext: - ldr x16, [x14], #8 /* entry = *ptr++ */ - tbz x16, IND_DONE_BIT, .Lloop /* while (!(entry & DONE)) */ -.Ldone: - /* wait for writes from copy_page to finish */ - dsb nsh - ic iallu - dsb nsh - isb - - /* Start new image. */ - ldr x4, [x0, #KRELOC_ENTRY_ADDR] /* x4 = kimage_start */ + turn_off_mmu sctlr_el2, x1, x2 /* Turn off MMU at EL2 */ +1: mov x20, xzr /* x20 will hold vector value */ + ldr x11, [x0, #KRELOC_COPY_LEN] + cbz x11, 5f /* Check if need to relocate */ + ldr x20, [x0, #KRELOC_EL2_VECTOR] + cbz x20, 2f /* need to reduce to EL1? */ + msr vbar_el2, x20 /* el2_vector present, means */ + adr x1, 2f /* we will do copy in el1 but */ + msr elr_el2, x1 /* do final jump from el2 */ + eret /* Reduce to EL1 */ +2: set_ttbr x0, x1 /* Set our page tables */ + tlb_invalidate + turn_on_mmu sctlr_el1, x1, x2 /* Turn MMU back on */ + ldr x1, [x0, #KRELOC_DST_ADDR]; + ldr x2, [x0, #KRELOC_SRC_ADDR]; + mov x12, x1 /* x12 dst backup */ +3: copy_page x1, x2, x3, x4, x5, x6, x7, x8, x9, x10 + sub x11, x11, #PAGE_SIZE + cbnz x11, 3b /* page copy loop */ + raw_dcache_line_size x2, x3 /* x2 = dcache line size */ + sub x3, x2, #1 /* x3 = dcache_size - 1 */ + bic x12, x12, x3 +4: dc cvau, x12 /* Flush D-cache */ + add x12, x12, x2 + cmp x12, x1 /* Compare to dst + len */ + b.ne 4b /* D-cache flush loop */ + turn_off_mmu sctlr_el1, x1, x2 /* Turn off MMU */ + tlb_invalidate /* Invalidate TLB */ +5: ldr x4, [x0, #KRELOC_ENTRY_ADDR] /* x4 = kimage_start */ ldr x3, [x0, #KRELOC_KERN_ARG3] ldr x2, [x0, #KRELOC_KERN_ARG2] ldr x1, [x0, #KRELOC_KERN_ARG1] ldr x0, [x0, #KRELOC_KERN_ARG0] /* x0 = dtb address */ - br x4 + cbnz x20, 6f /* need to escalate to el2? */ + br x4 /* Jump to new world */ +6: hvc #0 /* enters kexec_el1_sync */ .ltorg .Larm64_relocate_new_kernel_end: END(arm64_relocate_new_kernel)