From patchwork Tue Oct 18 11:04:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13010269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A112DC433FE for ; Tue, 18 Oct 2022 11:06:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=t5vHaYcKvv6rh0cOds7siqlUh9eUsobxLU1TJr/POTw=; b=ZvfvTfkJwaiOWJ suhU0YEyJ9W/yMJkrs0J/8uVQD0PUKeZNcbwxFfxEkn6vUHKVIePJLHfM6YK3p1WHm2UoA1NG1oX9 DDA3NyYxGcOhzo55LLuO1sGPVKrWUixowntW+Q07A7lq6gIIxZunDPebk2nDKVAzmNoEENiT7LUNZ MllMWcDMaOORcbtq+0Ch4/bNUBiPwxNXJJqMuiL3s82JU6gk9HT5NGMBBBD7WNWYn0HbK/sUeCXHP dti+lOq1K9Mo2TKLRoQjYRCGLhQaogOCI/L65eWZ8bKIbpg8dmlfQfqjt8lTFwR7pPZkPhe7v9rUE zHxRdctlQUdfdiTGm0pA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1okkPt-005zHW-Qm; Tue, 18 Oct 2022 11:05:50 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1okkPA-005ySR-MJ for linux-arm-kernel@lists.infradead.org; Tue, 18 Oct 2022 11:05:06 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id CEE0CB81E85; Tue, 18 Oct 2022 11:05:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D03C0C433D6; Tue, 18 Oct 2022 11:04:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666091099; bh=XyJL/8SEF32qYGNUVw1eQPMZ3OctMavcgBrjB2KO9Fw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=drHA4IKwrRO/yg42Rr+HJDZB5FhJHBzm4Yl/1inBn+YGl8WafkIB3yGPVcUX73y5H bY39w39IvRY4dkWTEU3zA68cWrId5E3wETGVMD14g15sYLezxUhRiQTy/1HaY02ALM OIsyu9K3UV/UFDeoVxezyBkWjhzwEahTGA3fUHNH0iZ2B6uihnXna5gGQxn7Yf+RQZ CTFf7u7kKfLsVppXSZYIAThV4NEqvF7fru2U9pFXABqLTI4MXoMMzjfEUIX2vGmd6G QblFIzyCv8x8CgiKkZi1VlkwqMxNs6GPmqDfH38lRLfdQ6IpRVOV4/anKjbZb5zf+5 8UuSPWflP4Dvg== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: linux-efi@vger.kernel.org, keescook@chromium.org, Ard Biesheuvel , Will Deacon , Catalin Marinas , Marc Zyngier , Mark Rutland Subject: [PATCH v4 4/6] arm64: head: avoid cache invalidation when entering with the MMU on Date: Tue, 18 Oct 2022 13:04:39 +0200 Message-Id: <20221018110441.3855148-5-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221018110441.3855148-1-ardb@kernel.org> References: <20221018110441.3855148-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1782; i=ardb@kernel.org; h=from:subject; bh=XyJL/8SEF32qYGNUVw1eQPMZ3OctMavcgBrjB2KO9Fw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjTohF5kWHwyv/QI5b27eYUOy19fcWWtdWHZdU2aUQ R7bmEyqJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY06IRQAKCRDDTyI5ktmPJAxOC/ 0cT2Sj7Y9pvPUeeCLm4X+r7iMyGxhGtcUTjlruFJR/cPWBaCIYM5DICNFPFzznQOTsjozpSG3jd7Bp c9lD/woQ724tptXFWBaxQlCBOllqrwE7ndgMxO5OgfkA9pK4xJFer0exBk3hgdGGULdegnQ1hnHztz ALmab4kiINJf8N/mjjm6PFNYdGCl1s52lMhk6N8oW+9OFhjHWN3SGwuXrNAA5sqwwEG2bac8NYHMSR lbqBabpx64PWZ8BrTX/UTp1sBenuVLNvdReHOwMq0HV4kGHY5m5LQgkHGW6NF0lBJDL6KhTReNhIic h7vy+xJYfp3vngb+VKp/zZeVW7lSTTKZTZrYfqTG11sANrBtLPtRRBhuBu0SnnsuGLqNNm5MRGw9KF Huyew7mEiq4iIKY+OD20z0DsOgnjR0bz2iMmmFcdj7HdeNWKRgusOZ/sGhdE7TUS/4AOm4bE3Ogklb S8VFjP+DXyXOWSkPBXh8P7KL1KaaVyaHfJMHempQ4SbcU= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221018_040505_044678_CD47EFC9 X-CRM114-Status: GOOD ( 12.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org If we enter with the MMU on, there is no need for explicit cache invalidation for stores to memory, as they will be coherent with the caches. Let's take advantage of this, and create the ID map with the MMU still enabled if that is how we entered, and avoid any cache invalidation calls in that case. Signed-off-by: Ard Biesheuvel --- arch/arm64/kernel/head.S | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 643797b21a1c..5de2ba3539a8 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -89,9 +89,9 @@ SYM_CODE_START(primary_entry) bl record_mmu_state bl preserve_boot_args + bl create_idmap bl init_kernel_el // w0=cpu_boot_mode mov x20, x0 - bl create_idmap /* * The following calls CPU setup code, see arch/arm64/mm/proc.S for @@ -134,11 +134,13 @@ SYM_CODE_START_LOCAL(preserve_boot_args) stp x21, x1, [x0] // x0 .. x3 at kernel entry stp x2, x3, [x0, #16] + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy // needed before dc ivac with // MMU off add x1, x0, #0x20 // 4 x 8 bytes b dcache_inval_poc // tail call +0: ret SYM_CODE_END(preserve_boot_args) SYM_FUNC_START_LOCAL(clear_page_tables) @@ -375,12 +377,13 @@ SYM_FUNC_START_LOCAL(create_idmap) * accesses (MMU disabled), invalidate those tables again to * remove any speculatively loaded cache lines. */ + cbnz x19, 0f // skip cache invalidation if MMU is on dmb sy adrp x0, init_idmap_pg_dir adrp x1, init_idmap_pg_end bl dcache_inval_poc - ret x28 +0: ret x28 SYM_FUNC_END(create_idmap) SYM_FUNC_START_LOCAL(create_kernel_mapping)