From patchwork Mon Apr 11 09:48:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12808922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0DE96C433EF for ; Mon, 11 Apr 2022 10:15:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Jj/RQo2t4pwtnOPrWmUPiWH6Hd4xaFzyvFjAU22Sr0U=; b=cMJ620ACLMNnND 4V2kXyYMf1czFYcxrm1T+bKqESlriFta9ifhtI+wSx/fZRgSZwroMMvdn0Wf/It3enWd0DNPndftn aQ1oNnYoLu0w8GeT/ZvMM3TAwDdTw94Lrwt3qR5wo/NM+Q0DacRJU6mJl6paX08MZWJ3MHGtWePLg nf/t13IAPahLVnp/YsOuDWkWR2hFsEXiONIajg3voyGfRSDP/Akb/hUJjtvsBMYWfUkPTCcBm8mV3 di20UO+m5pwJJgSjMipq49b48ynqSNJUpUUEm84IYQOzueXLBv3mA0IvdLktk2Q3pWbhmCaCYfibl R6DUqj5lqp3iMY+/DpcA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ndr3N-008Dm4-2w; Mon, 11 Apr 2022 10:13:49 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ndqg6-0082ec-UX for linux-arm-kernel@lists.infradead.org; Mon, 11 Apr 2022 09:49:48 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 728B3B811CC; Mon, 11 Apr 2022 09:49:45 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30A23C385AD; Mon, 11 Apr 2022 09:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1649670585; bh=XxI2AF5xD7GY4EBoXrk7Kfh7spE/Yd2cWMa8RrYU8ig=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OMfDo/w27ac+Zmx7u5TbY0enbL1V8QgtgQu+POfQSNfvCHJc8OVsdhHz0DzwwIigg hSRAGiHKg12uKIyyoaPgjJGnnrEjTcfcEkhrq33KmexE08/m5AGIb8F04gP+ne4MjX kWt3zJKKrJuFdjKs9Eg2UjKEGhixu6YjCX/sYokHJ4XCcWtT1XnyHCmdiq8aBiCEE9 +sDaejeZR3KGrYdFe//bZjHoWLQtGJx1jNBsP7MPuRatFsUTFlnnvfAy3TGxU9O2xQ vdsUGlwIq2HwDH+krwa3r++rjznpOQAaoBx35E6FsQEPXg5l67lxBEw4d5XXJchTUu J+drsu1wRy3lw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-efi@vger.kernel.org, Ard Biesheuvel , Marc Zyngier , Will Deacon , Mark Rutland , Kees Cook , Catalin Marinas , Mark Brown Subject: [PATCH v3 26/30] arm64: head: avoid cache invalidation when entering with the MMU on Date: Mon, 11 Apr 2022 11:48:20 +0200 Message-Id: <20220411094824.4176877-27-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220411094824.4176877-1-ardb@kernel.org> References: <20220411094824.4176877-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1782; h=from:subject; bh=XxI2AF5xD7GY4EBoXrk7Kfh7spE/Yd2cWMa8RrYU8ig=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBiU/lgbxCxfVzMmOVUpp7oJCq0arLzzk6ZpS/U7ovI vsF/ammJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYlP5YAAKCRDDTyI5ktmPJMdhC/ 9scjqLQCdqy+vMsxBUhfgC/i7tIgDvfjYGcXusyIZIdzZAuTAsf0tbxBpZ+gXtq9jGISpEDXzp18n2 duSpaEYNBlRjxHwtk4oFpgC9Hmgs+hjy7w2j4N/uuT+XCFJH4zwNBb7x+mhPzUBOzZ+hPPrYut6+df O+337jwEvIS4SjnMvtFcZmUdZV1vTyykvpiRESyWjduFibIkfKMdCvyv5+8Xs9ScZAb2ChtsMNhJ4w 3N8vwSWSf4yekXBAKyIe29StocbdT2hy5AEVwHIZCJ/l3aeaj7yr1ah7nn1PRmegaNSSunbroQgVFX AtdBy+BvWUNjaUupfprEdoPPS3HMUTg9No8qYiF1e2aGEeViBs5NCY02ibhXyL0A49KXnasSJNJF4M g+1bpoDIY7HxlC9TthsgBcQJYS+8nfNdBkAa7OPrUUEsUk0Om5XyOYpGrGF43QQgOCqvh/Tg1SzYNj 6yEHv9uBgVvVN7NMr53s2bRmDHl9L3vRu+lLopHnCxLa0= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220411_024947_353128_2DF9E2D6 X-CRM114-Status: GOOD ( 13.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we enter with the MMU on, there is no need for explicit cache invalidation for stores to memory, as they will be coherent with the caches. Let's take advantage of this, and create the ID map with the MMU still enabled if that is how we entered, and avoid any cache invalidation calls in that case. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 1ff474701e99..4a05f4480207 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -92,9 +92,9 @@ SYM_CODE_START(primary_entry) bl record_mmu_state bl preserve_boot_args + bl create_idmap bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 - bl create_idmap /* * The following calls CPU setup code, see arch/arm64/mm/proc.S for @@ -127,11 +127,13 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x22, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: ret SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(clear_page_tables) @@ -366,12 +368,13 @@ SYM_FUNC_START_LOCAL(create_idmap) * accesses (MMU disabled), invalidate those tables again to * remove any speculatively loaded cache lines. */ + cbnz x22, 0f // skip cache invalidation if MMU is on dmb sy adrp x0, init_idmap_pg_dir adrp x1, init_idmap_pg_end bl dcache_inval_poc - ret x28 +0: ret x28 SYM_FUNC_END(create_idmap) SYM_FUNC_START_LOCAL(create_kernel_mapping)