From patchwork Thu Jun 30 14:42:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12901950 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 20CF2C43334 for ; Thu, 30 Jun 2022 14:44:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LT9+xsxwNvxx+5uMp9T7WX+yF3f8BvgFl6/dA+veXDE=; b=PoD5shKRvk/D7I BCStXg1W0dVBw5QVIMFee5c9WFyrhFGZUe3YgVEi+NT0n1yb6f4K7QaBPrEcJEfoTx8PdzwPPaHTG /ID1ittYI0uvxhzggOKGpXE59aaYDO7xfZN6WYdU2Q2/W2qLCu+GOzAauToFzh3uYp5VzVBIbGmHH duOoFFZpr+OEYPYILUQr8oF+xoaRPZuRfnOUl+rcwG3AO1CkmgL7qXZtESs5k/rlj6X4B34JjscwL UjB6Gu4a7Br7DFMMRgAjMAZUBmuF82IE/4xN42h22KMqBsIF/u3qHzGqTF9nY0Hk5fg5DeoXJBn4H un6z2FxPQ4VzcjZV5LHA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vOB-0007tI-NC; Thu, 30 Jun 2022 14:43:27 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o6vNU-0007g1-63 for linux-arm-kernel@lists.infradead.org; Thu, 30 Jun 2022 14:42:45 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C68A06231F; Thu, 30 Jun 2022 14:42:43 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AAA31C341CB; Thu, 30 Jun 2022 14:42:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656600163; bh=TwP65EAjFILSHRHy/ydIST6zDHt+Ko5p7xzLC5G/PHw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kNL9UgnMF8Sk9alm+LLDllU3dqGAI3YSfkVSBIz6Ju1TK/ApfcsgGR2/sziFfo/SD /LsNonZJTCIYz2bC5+FR6mk4btN94mrzbzpiNt8JX88OiE66aJNiPwsnm9k73IaqtS 4lhE2nsTqFRZK0t/rZVMvSvdOQ0PavZmeaj6RxvZSvut5TdR8Zqy0iouQ6c8Cp+Duc zUyFMGh6rwRIRfKjtzXhOzSQ7WtFixiYCiF81+8G7dvPYriwkYNgsvjiYQUnIm6NA2 CNkJjAdqAmctHQlx2uz7OJONryvNVV9DztV6IgTbx53eCQY2v7ZAiHYfRliuIq9FQV EP8q57BNlte7A== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Marc Zyngier , =?utf-8?q?Pierre-Cl=C3=A9ment_Tosi?= , Quentin Perret , Mark Rutland Subject: [PATCH 4/6] arm64: head: avoid cache invalidation when entering with the MMU on Date: Thu, 30 Jun 2022 16:42:28 +0200 Message-Id: <20220630144230.2332555-5-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220630144230.2332555-1-ardb@kernel.org> References: <20220630144230.2332555-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1782; h=from:subject; bh=TwP65EAjFILSHRHy/ydIST6zDHt+Ko5p7xzLC5G/PHw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivbZQNCLRZcOyp7euLrckrC6C93tSZUwKV/WoulmF EtOUeuuJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr22UAAKCRDDTyI5ktmPJEBhDA C5ROrsu0Sz/8VTjBUeoEaUILbFs+BtEexMTVIWbIhmqwC6WG4HMvQ0SnGgbooQoyCsCeCpQvkb3R1U HRh4VpKJNBGxq+2OTYdL6oyhDNfnRDzI3WZW9Eozp3eFP27/42KICcfF5yILJVFFYVevZ+QHxxKgh0 UZPUOxJDErT02lUK5LVfKTMvm43f0jOswMadhkYQOSL6qwSkuKZz4XWK1mROlA1H6stGXneorW3f8T e74qIoiumyxs9HGpXdC0uHpJpL4Hy34zH6twBER/sxxU/3CwuQzlTLEU/jyTlIQwvM89fHrtc2iitf 6tP85EvoFQ3YE+Wy+iOGSr1YDu95pZv/C8kvjPHl/TjGMTEcAPAYqEXla+Hcvm0D6sq3FAn0Tb/PjS pOwCD7j8K5Nx77rXQb53NunwYftNhEbnnKZp4O/y0Skn2dS7SPjC7LtjkHC4e5WenzhVS5NQ6rGjNa yjWmEvqi7PyPidHst2MLCOb1A+XHwMPE/ioecGF1o7lEc= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220630_074244_341134_CB65C305 X-CRM114-Status: GOOD ( 12.69 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we enter with the MMU on, there is no need for explicit cache invalidation for stores to memory, as they will be coherent with the caches. Let's take advantage of this, and create the ID map with the MMU still enabled if that is how we entered, and avoid any cache invalidation calls in that case. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index a79c842395ee..42fc7e980b35 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -88,9 +88,9 @@ SYM_CODE_START(primary_entry) bl record_mmu_state bl preserve_boot_args + bl create_idmap bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 - bl create_idmap /* * The following calls CPU setup code, see arch/arm64/mm/proc.S for @@ -130,11 +130,13 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: ret SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(clear_page_tables) @@ -371,12 +373,13 @@ SYM_FUNC_START_LOCAL(create_idmap) * accesses (MMU disabled), invalidate those tables again to * remove any speculatively loaded cache lines. */ + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy adrp x0, init_idmap_pg_dir adrp x1, init_idmap_pg_end bl dcache_inval_poc - ret x28 +0: ret x28 SYM_FUNC_END(create_idmap) SYM_FUNC_START_LOCAL(create_kernel_mapping)