From patchwork Mon May 24 08:29:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BEE5C2B9F7 for ; Mon, 24 May 2021 23:33:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58B7A6140B for ; Mon, 24 May 2021 23:33:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58B7A6140B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=wVMtUvUYxLvKNC4Rj5DwEoLGgHJL0m1wHHMRP3mmfdo=; b=u8AqjH5fLx0eiB4SChTYH4wUrw ZdTIQh1aRcSRvdYzPIzrWsxIwCoAKhr5P1yjVjbSSPFLs8w7AzzTWjDKh8QyZ/41ppsR6u8NVMlkM /hzgnm4Y6hTLacpKwRsOYFIUytJ9O2pMhHdCiLR+TFSlXYS8G0YU+zNnvHX7mD4ur+f+ZYgXlvswa eCh6yu5rEr599U7MjA8yWqYAfadL1NIMnJbIJc56/kVIey/zvLx2z7CbpB9CvswPpoqFRf1uRtXCT ueQBF5jcbx3xU721C+LAy3dllDS1GMcP+KeO4m+OBrFe7AQ8ZYxFx3YHlO4xqiubueBzIvbJ/uxm0 6x3ExHlQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llK2d-002LnT-Rg; Mon, 24 May 2021 23:31:25 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yZ-000lq4-Jt for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:17 +0000 Received: by mail-wr1-x44a.google.com with SMTP id 2-20020adf94020000b0290110481f75ddso12736899wrq.21 for ; Mon, 24 May 2021 01:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9jxTZq434paFFRmnyHWZee+kMcfmIDCH9ZCbQKUPMzw=; b=aWtHnlLznXAwts7AVfLTQE816WjV27QJPc8pNbx7KS69ORaCSxEuWK9MgPNTulhhPS VvZHAdF8+Oy30FtyzyD74qV+NZBWxlDUtuLrt6ACOFJLUz9Im+tK0l3NjRANxtq9abxZ x8KVvSHe7yVPLJBqLnu0dtDYhw3rQfg2fS+bHnsMxU3JkdgldG+VdPQzLhDgUvh+imGO ccGOiTsruHrcyGMb/2IUGS+OUvuHBsM4iSE3V9fFgET4t18RnYtJ4ZrCOIAKq0lOMbXI 1VAhybo+l2VFyTtoA+UKdqK5twRQwiJrclpFDUU9fi1T1vHHFvBC7HlJSy1hPcgjYLYL D+rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9jxTZq434paFFRmnyHWZee+kMcfmIDCH9ZCbQKUPMzw=; b=Q4J+Aal5WtEGtViXxVgfOlHcoGU/EiPT2ha48x2aoeWKlvxUD/3mlragpkqnGsmv1j uwgeCMqophJGutgLWfEPBsWnT9kPbP6QfuJT5K/yleiqQWHwLLbsybEI1XnozJGm0Jlh GLkaAdA+U55bqsXV8sWSbVc9298s+1Jzz+nP/sQWJGl3z0AiEAwHGDKDMDHdZLKldsOW RzYqif0e17jRaqd49V9nJlc4tUOS//jfauro3dAZj4SIYfu23waFLtS3m3c4JKtxGUgz MBvuwYLJUKYkmamq6+HBcGPXCskZ38DcDqdptOudWBeWCqvO6bp7rBHnP7JOc5b5Uq4T ZvUA== X-Gm-Message-State: AOAM530MlEzXB3sDaQtv/NfkT8l/P7mhXSQjCXDyIcRvOUVNDcGiMO5c 2EFdIhN2rf5zU2Ijk9vJIxtrhzb5+9FwWJeL8l0/uTRRoF6jUqhmWxE9B/4TnLbV5Y1wCaoyTKj GHdA70VIe+4Wft0uBSgfzFNmOu9yAs/J0z5AnYXaHsyRXOgZV+7uZNk9zZpS+TKFu/6Tx9AsckS 7kwg== X-Google-Smtp-Source: ABdhPJxYdIRO25TTljnN1hlc7ZL1Lp+EElnE5FHoKQqH7KXTPbzsE5oSbUbZhRJH27/NVwqws+Hrn5K6Yg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:7703:: with SMTP id t3mr19075782wmi.99.1621845011968; Mon, 24 May 2021 01:30:11 -0700 (PDT) Date: Mon, 24 May 2021 09:29:47 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-5-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 04/18] arm64: Do not enable uaccess for flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013015_674965_4BA93108 X-CRM114-Status: GOOD ( 13.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __flush_icache_range works on kernel addresses, and doesn't need uaccess. The existing code is a side-effect of its current implementation with __flush_cache_user_range fallthrough. Instead of fallthrough to share the code, use a common macro for the two where the caller specifies an optional fixup label if user access is needed. If provided, this label would be used to generate an extable entry. Simplify the code to use dcache_by_line_op, instead of replicating much of its functionality. No functional change intended. Possible performance impact due to the reduced number of instructions. Reported-by: Catalin Marinas Reported-by: Will Deacon Reported-by: Mark Rutland Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Link: https://lore.kernel.org/linux-arm-kernel/20210521121846.GB1040@C02TD0UTHF1T.local/ Signed-off-by: Fuad Tabba Acked-by: Catalin Marinas --- arch/arm64/mm/cache.S | 57 ++++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 2d881f34dd9d..7c54bcbf5a36 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -14,6 +14,34 @@ #include #include +/* + * __flush_cache_range(start,end) [fixup] + * + * Ensure that the I and D caches are coherent within specified region. + * This is typically used when code has been written to a memory region, + * and will be executed. + * + * - start - virtual start address of region + * - end - virtual end address of region + * - fixup - optional label to branch to on user fault + */ +.macro __flush_cache_range, fixup +alternative_if ARM64_HAS_CACHE_IDC + dsb ishst + b .Ldc_skip_\@ +alternative_else_nop_endif + mov x2, x0 + sub x3, x1, x0 + dcache_by_line_op cvau, ish, x2, x3, x4, x5, \fixup +.Ldc_skip_\@: +alternative_if ARM64_HAS_CACHE_DIC + isb + b .Lic_skip_\@ +alternative_else_nop_endif + invalidate_icache_by_line x0, x1, x2, x3, \fixup +.Lic_skip_\@: +.endm + /* * flush_icache_range(start,end) * @@ -25,7 +53,9 @@ * - end - virtual end address of region */ SYM_FUNC_START(__flush_icache_range) - /* FALLTHROUGH */ + __flush_cache_range + ret +SYM_FUNC_END(__flush_icache_range) /* * __flush_cache_user_range(start,end) @@ -39,34 +69,15 @@ SYM_FUNC_START(__flush_icache_range) */ SYM_FUNC_START(__flush_cache_user_range) uaccess_ttbr0_enable x2, x3, x4 -alternative_if ARM64_HAS_CACHE_IDC - dsb ishst - b 7f -alternative_else_nop_endif - dcache_line_size x2, x3 - sub x3, x2, #1 - bic x4, x0, x3 -1: -user_alt 9f, "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE - add x4, x4, x2 - cmp x4, x1 - b.lo 1b - dsb ish -7: -alternative_if ARM64_HAS_CACHE_DIC - isb - b 8f -alternative_else_nop_endif - invalidate_icache_by_line x0, x1, x2, x3, 9f -8: mov x0, #0 + __flush_cache_range 2f + mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 ret -9: +2: mov x0, #-EFAULT b 1b -SYM_FUNC_END(__flush_icache_range) SYM_FUNC_END(__flush_cache_user_range) /*