From patchwork Thu May 20 12:43:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 444CBC433ED for ; Thu, 20 May 2021 12:47:51 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB85061184 for ; Thu, 20 May 2021 12:47:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB85061184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZPBAThwdR9OpwBG/KIwqgCMNtigGoiEHgJmy4cUUE6E=; b=FmMYATssdKFf3e +YajSq2JfDrB2I31sOrHPqzX0HKQG5drmaHhjXsokT5Cgi9cjM+jvxepM8D8JXD7eVqqUvYZcnypx WbAicBFbe8KlF9V0EYMDIRtPWrXUTY12lef5STf++4s+zFDtD4KXNXb+A+ohljA78P4NleQ07xG31 99I79JH4WS2xovHEnn8NaeGazPZAuV4SlnA0rAlLFx1BQaTtVWAU2/536YcNzjm5S48Ri3S2MLj+I bOcgMQTq6Ex+0KbXJq9smCyf8b87/8cytMV0G122ieQlVojBFm+tQK6VLcIvdNBPnKdRLs/PUBYJS QfxJHZ20DtNr65o1CCRw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji44-000ut0-CZ; Thu, 20 May 2021 12:46:12 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2K-000tzE-Dl for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=y9sZY7i6M6Si+BnsNu0zYtWboK89Av3bx/Kh9fRDw9I=; b=HexGiq/T6m0I/rNKxADpUjUDZE 0e88IEUM33gUweIoEN+o4MbXRdE45ZI42TvSctEFLdT7PufA8PrCYhxmDzenwJpWc/CUUf/xbzUVA Kbbvnkc/lE64MRGwHKFvzx/xYP5gBzABwB5241rSnYv6yjJIviKBgiDmdbpgrWA2xpuZh+pPTQo4S +ruMgZ+8zXmOxFtAtQNNhUo4G0fJFpy/ZePzVl/ECpMTkv7Vsbe3tszmccaxUTV779xe4aC4gmlEn w3zJudHdilQrGUGHqYvV3xBd6YqPy6iI1J4nzJj7J8fwcfABlLPC48ebG+8ISsRfI9wQEjijiKUQh 8EoUqs7A==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2H-00GKdF-MT for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:23 +0000 Received: by mail-wr1-x449.google.com with SMTP id f19-20020adfb6130000b02901121afc9a31so1899196wre.10 for ; Thu, 20 May 2021 05:44:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=y9sZY7i6M6Si+BnsNu0zYtWboK89Av3bx/Kh9fRDw9I=; b=IKYB7v0eEan/h1zlFMckoa1jPIUgQ1zeQ0W5irTPL4a6Q+Xayl+DPb+tu0DqgFA1Uv DDqscsoasR8wB7Mazx/L/4auUXiJAAi62twHJjc5zlpu2eRO7bJg+QZ3RqSFad+dIjkE D5PETTUINRRZZETyrEtYiNArJzIivHMg/lmGuysQBTootdqR/jovLpyk4GgFCvMWsd/a J1UnFJJk5AZOfpO0xnf/ljOGo6u9ecmCK3PhqZneJtXkx3WY4RBM8OH3mYDSd6gNvRR7 rZqywabHBBRcR+oti48ANgQoa08eL2LB/IfhpqstIqoE+sKXv1drHe2axLidMvUIf+p6 dBuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=y9sZY7i6M6Si+BnsNu0zYtWboK89Av3bx/Kh9fRDw9I=; b=eDXmiM3GW+dlPwGw4iKzDnk7OPazhFsOMInFbxc3wUqi1oSNGRgMrbi/jo/YzvhkCO DtfU4//Ha1DoO+0AhXFaq7jWuummBaBQZSfJVui0W02+o4rvXQvI9BUs8XgTda0XoYTd BsmtkogDqSEITAijzyocVxVOI96swBlPWNu93lJrQOfg+euEc0fJhO7vc1tpPWTZC8UL l+hONskDrYb741gvqOLTbigdj2g39f2Gh/aNoQ0/BCkwWIAwPM/AKSZmXN1s/mahXdoy FqOY/luXjIOrSMegaawkqvASCX8cTa95ANYj8WKdfa7Od/c3TpQdtIXYPwJSRUEggDOb fPFg== X-Gm-Message-State: AOAM532Nm0Rld+72lM92cCfJQFVJ+6EcYBYcUQB/3zWUHi5JtrFPeps+ Qm0NzapZW590kFeL4UlwNOsEOQdGwJsRAo1t+Ek+1Lbct65cjNyabSaXNe+9LiXCUBYba+xTc+u PzxMAsGVio1qlJlSwnt4LGWoedTfIj6MSlskhVJTvQ+pz3Y9FxJ1ufWXi8AXrs/SgdiMrXtBeMF DfCQ== X-Google-Smtp-Source: ABdhPJxNt3K2lx5C1BC/WQzLQjDOTOzqw/XidTTgo9R7BLM6RsgqX17EsRrudBRVZc9ySO5jhZhXgWFOXw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:4b92:: with SMTP id e18mr3472289wmp.71.1621514658992; Thu, 20 May 2021 05:44:18 -0700 (PDT) Date: Thu, 20 May 2021 13:43:53 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-6-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 05/18] arm64: Do not enable uaccess for flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054421_756426_DF8A5B5D X-CRM114-Status: GOOD ( 13.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __flush_icache_range works on the kernel linear map, and doesn't need uaccess. The existing code is a side-effect of its current implementation with __flush_cache_user_range fallthrough. Instead of fallthrough to share the code, use a common macro for the two where the caller specifies an optional fixup label if user access is needed. If provided, this label would be used to generate an extable entry. No functional change intended. Possible performance impact due to the reduced number of instructions. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba Acked-by: Mark Rutland Acked-by: Catalin Marinas --- arch/arm64/mm/cache.S | 64 +++++++++++++++++++++++++++---------------- 1 file changed, 41 insertions(+), 23 deletions(-) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 5ff8dfa86975..c6bc3b8138e1 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -14,6 +14,41 @@ #include #include +/* + * __flush_cache_range(start,end) [fixup] + * + * Ensure that the I and D caches are coherent within specified region. + * This is typically used when code has been written to a memory region, + * and will be executed. + * + * - start - virtual start address of region + * - end - virtual end address of region + * - fixup - optional label to branch to on user fault + */ +.macro __flush_cache_range, fixup +alternative_if ARM64_HAS_CACHE_IDC + dsb ishst + b .Ldc_skip_\@ +alternative_else_nop_endif + dcache_line_size x2, x3 + sub x3, x2, #1 + bic x4, x0, x3 +.Ldc_loop_\@: +user_alt "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE, \fixup + add x4, x4, x2 + cmp x4, x1 + b.lo .Ldc_loop_\@ + dsb ish + +.Ldc_skip_\@: +alternative_if ARM64_HAS_CACHE_DIC + isb + b .Lic_skip_\@ +alternative_else_nop_endif + invalidate_icache_by_line x0, x1, x2, x3, \fixup +.Lic_skip_\@: +.endm + /* * flush_icache_range(start,end) * @@ -25,7 +60,9 @@ * - end - virtual end address of region */ SYM_FUNC_START(__flush_icache_range) - /* FALLTHROUGH */ + __flush_cache_range + ret +SYM_FUNC_END(__flush_icache_range) /* * __flush_cache_user_range(start,end) @@ -39,34 +76,15 @@ SYM_FUNC_START(__flush_icache_range) */ SYM_FUNC_START(__flush_cache_user_range) uaccess_ttbr0_enable x2, x3, x4 -alternative_if ARM64_HAS_CACHE_IDC - dsb ishst - b 7f -alternative_else_nop_endif - dcache_line_size x2, x3 - sub x3, x2, #1 - bic x4, x0, x3 -1: -user_alt "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE, 9f - add x4, x4, x2 - cmp x4, x1 - b.lo 1b - dsb ish -7: -alternative_if ARM64_HAS_CACHE_DIC - isb - b 8f -alternative_else_nop_endif - invalidate_icache_by_line x0, x1, x2, x3, 9f -8: mov x0, #0 + __flush_cache_range 2f + mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 ret -9: +2: mov x0, #-EFAULT b 1b -SYM_FUNC_END(__flush_icache_range) SYM_FUNC_END(__flush_cache_user_range) /*