From patchwork Tue May 11 14:42:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D3B5C433B4 for ; Tue, 11 May 2021 14:44:52 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D1B4461288 for ; Tue, 11 May 2021 14:44:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1B4461288 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=bnfS/T2Uk217KHAqg0X+E2D+llVBz6OvRKVWGR06V/8=; b=MF+WwVmmv/Wq/Z gepFFBlhtslv01cMOLetwa0dFpxxKG7ZB57Pf0X1LLY8C/d+kpkSXKCkyqbWZ/EsfmhXi9FXbbXQk 7lx+Kkh+JXQOh5EtIH+SDPfkftUihy8xFUuXF0MOOmovxrbDg15uPJeY1M4vHNBZ4kOX1wMA4KtOj BdUaXSywPaF8/SvtAer3Na3Znj0Cs9ukJtrHEb1gZ9qHP3ohXyoRLj1qYxdXcGjgBBBGiyM6+oP7m DY/Kk/bXm6EKsXP2ycJGFNlYTzPXYVs2VlsYMX/CF54uZGNxDmm0T1FE3X627RMy0VYsnWApzxNRO s52rRQmH/hZp58WBD/Zg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbO-000TFz-32; Tue, 11 May 2021 14:43:14 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbC-000TEG-7Q for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=7iA7EGNqY9uoFmeZlT72qQlEQKhYwcgbhe01v0kzgRc=; b=3YAqBFJx0IOUrj5HlGc/6Nmqo3 UKnpD9qmaZOEBR1AWgNjZa2kuOYtFFDsAdGt9lk/7sf+zdmRXuLPhqwpEfbvRW2sPWAH4dV+hY12J ECLUK8udBD+uOGT5xidzSdHYVI7YImGfNppSeGARPgSTZDYTN9jemYNUUk9F1nQU6bwKoN0hcnNuB ilDhYiZ7PXq1ls1jCWLdsSTDDEOBYcpUAPyAbgfNgu7tqFaefrucJDjQGJGV8zGJQEYUcvE1FaMC6 g1poQm8UagfJANWECAU/bbyzkreYhozzFVkYwjR0QWDFwjPdwXqVKMTvcnjUSAVukrk8S3Um5fX5B I0jnK3hg==; Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTb9-009gUH-9q for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:01 +0000 Received: by mail-wm1-x349.google.com with SMTP id s66-20020a1ca9450000b0290149fce15f03so708261wme.9 for ; Tue, 11 May 2021 07:42:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7iA7EGNqY9uoFmeZlT72qQlEQKhYwcgbhe01v0kzgRc=; b=QAnSCWTr1GStjc2oA8zWh39v7SAZUFZxd9ChlwOFzLo4efXHbjrfMod+fRwrsrC8B+ f5jZ6WcPBKqrYcGlfXtwzVoxtTwp+GRUMMajoI8uv/QouU31p8rBBnnH38OFDAuqGQRv yQIDNx0Rg8EAQLyyhXOVeit9EPNWOOeAvmr6E4Fwx04fG/8bgJ8ULuxrx1bHgJHH+Du6 pNIGLui8GBFjWLmJFMmcFuRST9Ma+BsAf1CzqJGdJBHRbmpDcPFnih9BXibJD4MdLecR NibSb4Nhb7oEdTiQc5+HkWiGTJCcKX4KsFZMwa3tZ7a+BJcSsXduVBw1ClHHbSx7Rqs7 bnzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7iA7EGNqY9uoFmeZlT72qQlEQKhYwcgbhe01v0kzgRc=; b=Q2+e4Otj+icGaohZfArU7DVnDxvHAVl719qXhZ38vi1sP2Yh0zEpSJFSOiwe3GlAbc 3s3en360Ft3LtPgNG57SaISjbzybaLAzaQa1LDEwiVey7syIjbli8ejhoeKfPkT2DkPo D5xZmlr7P8lpP/nsjmEMUg5AHb9EeA6n+9esJZkpeYCB32xxJ79D/eRN2rDpYBogtDTe qV5w6QZNB/07iSSgHJGtlZaDhpz4ZsBistDmMjegy1vx12Eq4xfLIlBeH4KvNBDcI1wG odjylZrhZZ6KnvlmPtOur5T1J7s2BZkYT/AB6rc4qCGp6Yi1m0/bKMOOizgQnO7pxLXE yG9Q== X-Gm-Message-State: AOAM533MLQRzt8hDa1zD4jEzK8GFhhPlCxNZD2SMZ7ae732ySV5zaKb0 Lez27P3O/7xVxR+J9454i/SpJjjqmIHUhCASxk3ULeazrgnWMAS6ZZ5wHu/mtHOkU+WJrA5QM2x jn+ZMRz9NEouj9ZZ/c6BBBTYrtN2zHWabMWPEzoZTfJmDJBany9wrNSneTA4vTSkHPW8YFHMcBW Dx0g== X-Google-Smtp-Source: ABdhPJxWcIEZbGBf8WTZe75GSE1VQHMwUXwOK9r41NDXVuSMR9mdO2sRBAa6Xn1bI2jcjh7nJMouOwZudg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:cbc4:: with SMTP id n4mr6047808wmi.153.1620744176749; Tue, 11 May 2021 07:42:56 -0700 (PDT) Date: Tue, 11 May 2021 15:42:40 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-2-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 01/13] arm64: Do not enable uaccess for flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074259_367247_14846F31 X-CRM114-Status: GOOD ( 13.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __flush_icache_range works on the kernel linear map, and doesn't need uaccess. The existing code is a side-effect of its current implementation with __flush_cache_user_range fallthrough. Instead of fallthrough to share the code, use a common macro for the two where the caller can specify whether user-space access is needed. No functional change intended. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/assembler.h | 13 ++++-- arch/arm64/mm/cache.S | 64 +++++++++++++++++++++--------- 2 files changed, 54 insertions(+), 23 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8418c1bd8f04..6ff7a3a3b238 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -426,16 +426,21 @@ alternative_endif * Macro to perform an instruction cache maintenance for the interval * [start, end) * - * start, end: virtual addresses describing the region - * label: A label to branch to on user fault. - * Corrupts: tmp1, tmp2 + * start, end: virtual addresses describing the region + * needs_uaccess: might access user space memory + * label: label to branch to on user fault (if needs_uaccess) + * Corrupts: tmp1, tmp2 */ - .macro invalidate_icache_by_line start, end, tmp1, tmp2, label + .macro invalidate_icache_by_line start, end, tmp1, tmp2, needs_uaccess, label icache_line_size \tmp1, \tmp2 sub \tmp2, \tmp1, #1 bic \tmp2, \start, \tmp2 9997: + .if \needs_uaccess USER(\label, ic ivau, \tmp2) // invalidate I line PoU + .else + ic ivau, \tmp2 + .endif add \tmp2, \tmp2, \tmp1 cmp \tmp2, \end b.lo 9997b diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 2d881f34dd9d..092f73acdf9a 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -15,30 +15,20 @@ #include /* - * flush_icache_range(start,end) + * __flush_cache_range(start,end) [needs_uaccess] * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, * and will be executed. * - * - start - virtual start address of region - * - end - virtual end address of region + * - start - virtual start address of region + * - end - virtual end address of region + * - needs_uaccess - (macro parameter) might access user space memory */ -SYM_FUNC_START(__flush_icache_range) - /* FALLTHROUGH */ - -/* - * __flush_cache_user_range(start,end) - * - * Ensure that the I and D caches are coherent within specified region. - * This is typically used when code has been written to a memory region, - * and will be executed. - * - * - start - virtual start address of region - * - end - virtual end address of region - */ -SYM_FUNC_START(__flush_cache_user_range) +.macro __flush_cache_range, needs_uaccess + .if \needs_uaccess uaccess_ttbr0_enable x2, x3, x4 + .endif alternative_if ARM64_HAS_CACHE_IDC dsb ishst b 7f @@ -47,7 +37,11 @@ alternative_else_nop_endif sub x3, x2, #1 bic x4, x0, x3 1: + .if \needs_uaccess user_alt 9f, "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE + .else +alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE + .endif add x4, x4, x2 cmp x4, x1 b.lo 1b @@ -58,15 +52,47 @@ alternative_if ARM64_HAS_CACHE_DIC isb b 8f alternative_else_nop_endif - invalidate_icache_by_line x0, x1, x2, x3, 9f + invalidate_icache_by_line x0, x1, x2, x3, \needs_uaccess, 9f 8: mov x0, #0 1: + .if \needs_uaccess uaccess_ttbr0_disable x1, x2 + .endif ret + + .if \needs_uaccess 9: mov x0, #-EFAULT b 1b + .endif +.endm + +/* + * flush_icache_range(start,end) + * + * Ensure that the I and D caches are coherent within specified region. + * This is typically used when code has been written to a memory region, + * and will be executed. + * + * - start - virtual start address of region + * - end - virtual end address of region + */ +SYM_FUNC_START(__flush_icache_range) + __flush_cache_range needs_uaccess=0 SYM_FUNC_END(__flush_icache_range) + +/* + * __flush_cache_user_range(start,end) + * + * Ensure that the I and D caches are coherent within specified region. + * This is typically used when code has been written to a memory region, + * and will be executed. + * + * - start - virtual start address of region + * - end - virtual end address of region + */ +SYM_FUNC_START(__flush_cache_user_range) + __flush_cache_range needs_uaccess=1 SYM_FUNC_END(__flush_cache_user_range) /* @@ -86,7 +112,7 @@ alternative_else_nop_endif uaccess_ttbr0_enable x2, x3, x4 - invalidate_icache_by_line x0, x1, x2, x3, 2f + invalidate_icache_by_line x0, x1, x2, x3, 1, 2f mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 From patchwork Tue May 11 14:42:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 304D5C433ED for ; Tue, 11 May 2021 14:44:54 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9558561288 for ; Tue, 11 May 2021 14:44:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9558561288 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZhSQlSIOM9VcjdGxgK+lrjhgxfdZnUph0yjXjxGN8mQ=; b=NJ2YKcTGbJxstz 5hVrdEglEUXAe3zGtFpM1j3rWw1dy9W3Rfpre/ts6b3ewTUDOL4tLDfbADY3fNcMHADdOoUJOOUNq EbHmv3EuTLhZknPu52Z2O/2+L+Z994Rlltf6z4Crh4DxFAvxUpxnxQ2OoiTZ6kSW5Vsn47XuWJalO VdEO8nIKQCfCXbxj2s7XZdRITAUr1jah2nic+Vk3FrMMTISZvNsqsSYPhxKcKdPSrn1A89TKPiU3H 5uP5lsSwZgbu0pGboyAd/caHH/y08w9Izk9E2MHT+/W56jCrNqac0It0xXNAKotA+6V/EFkiGk1K0 aaG/Ka2YTwrWEcCjyqyA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTba-000TIB-5h; Tue, 11 May 2021 14:43:26 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbE-000TEi-1E for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0ZoRB0Udm2EWMuty7SBfrM43lsjufH1WUmYkNXcjwEU=; b=FQ7gxZyyh7DvR9ui8COsx5n/Qo vCrpKP74xFR7wboptbbRRvQuroT6TbScNJB+sTtF1cQR0fJGWgizb4ZUK2oY2djkPP1CxCgmEEhSN izzt51EEYmgIDmcdaZDiuSac3vijPjb8TdJNebThA5r8G+PcX6PyvV/LOm0QDwYXxjy059ndAo3de xjalfkeSnIq3yAWWAERuW5j5Ur+BzZXPLVS7vtXuK/AERWxeEL3ynOVCk1HmMYYgRQ684qs3uk32g WD44w/lVoWtbBUi4bn9euVnelJ+0Aof6ipzoX1VJFmEcs07Sc0TPYaWK/L3eToFAjm0UrAKxwaZe1 0oWUZVGw==; Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbB-009gUl-EA for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:02 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id b24-20020a0cb3d80000b02901e78b82d74aso5566978qvf.20 for ; Tue, 11 May 2021 07:42:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=0ZoRB0Udm2EWMuty7SBfrM43lsjufH1WUmYkNXcjwEU=; b=niJm6rde0gZpJ5M4Ffe3BIGD5Mv5uruqnAAIkRc6Ly/WE3kVMIQuPXYBhXk+4cvIY/ 0/6chg3cRK3riX+c5ZBUXwiqu0YML3w1tCcUuFb5R+OWEqKocF0GuD7gG2KOLSlCRQwf dE6Wnuga7gdWfr4eX4vD8pjeMDfRwsQRQLuxFh2Lhd2tGBg4s9qLwRIW93rTdkZOo73a p4BfnuckbEnIoJbT9/X7mp+pE0eE+8zh76Ad/vYFH4I/1G+c52RXVGHPQYBkSuyJ2VAU QOivRxtdoxTTocyCaEV9CkJQuUi8wd4FsfyaF1WtRIneQQQLNJNe37NKJUXLRGMAJxyS sjng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0ZoRB0Udm2EWMuty7SBfrM43lsjufH1WUmYkNXcjwEU=; b=gA9z1PgWR68sI4a+BHeuUic1PF/vhThc1VtSPQzNpi6RONpSVKM66OYuGapSq6GwuV jxk2zLWHWuNfnyD4D800X23fyScxofL1Nm9tMfxr2aLzMkDvXwXVghhVEoFqt2DP9Ae1 5KMYVKyWcJU6hmFaB+g8euBAPQOVrdy6o1WaZcgxqeJiA0ILXrq+omT6ZgcSPz9bqHg+ 2Z7IxupH2hcA1X+XDJqUKBaNA0RMhGNcrvA0Ga/ZGunbsmiGS+KaV9iWCCjEcLvD9n2v 2+wN9bUjZQ+0HKLwIwepury+Tn8GRQbLbhfyXO94+nfQAoZkdkYfsyoY8FTfNzPAXxX5 C18Q== X-Gm-Message-State: AOAM532nXTI+LyCsXT94veOx5gd3IdxNXeYZ1TzW9VKIzeYsmQRuse2P XRw3jKgAM3UnnILPTn2jBWcv6tXpspzLEoT3bIFr8g6TDaQTJl1iBcvBoS4bGbeCgC17KM3CgnF w2UJuQ/rug9b7LX5vnBzC3GPqOMz8V7lo5NbHoiG+Fgo+8VtOfDGSDMyEfA64nx7p2JBFDeV+v3 GFkQ== X-Google-Smtp-Source: ABdhPJyeLXD/afiyFzs9BtYJrL6HZBtwURrFcwE+DYdfkWBCAT/YT7DzyBPMnHsedHYldkBuukUFRt5dXA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:f0c9:: with SMTP id d9mr30066890qvl.3.1620744178762; Tue, 11 May 2021 07:42:58 -0700 (PDT) Date: Tue, 11 May 2021 15:42:41 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-3-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 02/13] arm64: Do not enable uaccess for invalidate_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074301_495312_911BBDA3 X-CRM114-Status: GOOD ( 11.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org invalidate_icache_range() works on the kernel linear map, and doesn't need uaccess. Remove the code that toggles uaccess_ttbr0_enable, as well as the code that emits an entry into the exception table (via the macro invalidate_icache_by_line). No functional change intended. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/mm/cache.S | 11 +---------- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 52e5c1623224..a586afa84172 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -57,7 +57,7 @@ * - size - region size */ extern void __flush_icache_range(unsigned long start, unsigned long end); -extern int invalidate_icache_range(unsigned long start, unsigned long end); +extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); extern void __inval_dcache_area(void *addr, size_t len); extern void __clean_dcache_area_poc(void *addr, size_t len); diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 092f73acdf9a..6babaaf34f17 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -105,21 +105,12 @@ SYM_FUNC_END(__flush_cache_user_range) */ SYM_FUNC_START(invalidate_icache_range) alternative_if ARM64_HAS_CACHE_DIC - mov x0, xzr isb ret alternative_else_nop_endif - uaccess_ttbr0_enable x2, x3, x4 - - invalidate_icache_by_line x0, x1, x2, x3, 1, 2f - mov x0, xzr -1: - uaccess_ttbr0_disable x1, x2 + invalidate_icache_by_line x0, x1, x2, x3, 0, 0f ret -2: - mov x0, #-EFAULT - b 1b SYM_FUNC_END(invalidate_icache_range) /* From patchwork Tue May 11 14:42:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90DF6C433ED for ; Tue, 11 May 2021 14:45:02 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E6B4C61396 for ; Tue, 11 May 2021 14:45:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6B4C61396 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gk3kMM/tzllavDwxtWm/AllivSZJqMNnw5SYesgyuTk=; b=Uq17GU++LELDWY 3N7dWj5HmPVx7hZf9Yb4WBlDj+iAdv/Cq8TAbnjAJIERnBT3lLMPUw9zBXmXlj1DCMiyV+sK0SFB4 iRM3FXu0u4/G02fQngLPKtCT/Dewg6qx5+CslKz64zw9G2pUti+vnKBr0XxVREi16lyDBquU5wZ65 fBpw9CUw/kMNYslcLsn7CBBr65JxFHEEXogouKyrR0NBO1BL2XeVq1Azef5mMxAMCORQQR0xb6f+q g0A7ePFa0tzlrcTPW/e7rT1pHTfzpZWsMBRIHQnQRK02D/PkjkoUZ2HYSDicnOv1nzGdkWmtf2XkI W8F60ZFzGEpU7XFX8nJg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbk-000TK3-UU; Tue, 11 May 2021 14:43:37 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbG-000TEs-At for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:06 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=X8uS5WuD5KRPXhFVQzC7JRTXmBEm1qdgTenJ1j4LBVI=; b=E0xpaLjZhOUMzyOvQxnZmzJymq 8Em1AfdZa29alvfX6APq1CrIBo7wiRae5wJLiAlhlpUTdUpVPz5RgcnfSr6AJUGM8bDHSx/BQazQq BkOJldFj+ByDeiUc+b67HcNVDq7NU9uros6R/9MtPANHV2m4yR7ci7MZqYAIH5W0DUyDxDX0WW0LY 66fD8+rmX/fBmytzT9HR6vixWEhkITS1sOQNh82MngoRtXKZ/s33TMP5/+MwY/sJdiAsvlT2b31pK dQb85r9wq5xzhREjC3HaYWpws6GTDLIxKIoUYBzBcX8Y3ajMbXR9isixoMosgcH8OgZTK/i17gBV+ vy/GquVg==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbD-009gUo-O4 for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:05 +0000 Received: by mail-wr1-x44a.google.com with SMTP id c13-20020a5d6ccd0000b029010ec741b84bso178475wrc.23 for ; Tue, 11 May 2021 07:43:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=X8uS5WuD5KRPXhFVQzC7JRTXmBEm1qdgTenJ1j4LBVI=; b=ah9Kk27SQ9bettLiZEHIO4ngZzwIwZ2CT18IeEEU7Egr2j9B+udmscNQHVRrG2NfBO GuaD9prqShAN1m5+9DIDEgtuipLljxNL1LCsiWJ9Q+JE0RteRZ2pQsSJhK92/vfXhNlt gAMld6P4OCsnjwDw5jEKWxWXFyD2jMQKvesCVN1zHtzyruxYJP7ffT0Su+lFGK7qNAX6 PRIGkUnkCwScccVIAmJXl8pKxaXt6xllb+hrgKk0TJg7VG36dZ48lWbXbD4cov4l4vV0 eFRxvch5lwiYVpwLydOYrB4v6AyL6T/pCpdWYyh0ygcBWf6spqs99emRPqieX8iTFiS8 KlSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=X8uS5WuD5KRPXhFVQzC7JRTXmBEm1qdgTenJ1j4LBVI=; b=kADuvVnF1+HYnbW6EGftE41hGuOho2fH2KKwOxS1DnboZLEn0Qx1YOJFkqnsd8ZM5j OReCOw3bkLUsjH84rMVshh+oUtlnHSkwq0ULZQnyrvCOQjFxHx587XLf4I5u5FaBDghp eKS3wc+N8+VzM3TkegXWyPV5yCj+f/EZZH5MxGing1fJYNJru/MFUVU+2wlnQhGUSBBJ 0qLpD98FgQhsNfsJ5SzMAYbGdZJsCQTNgTiMPfRnQSGmb+rGmO5TmYldVRILRt35fw2l +p/gzTaf12rPLbV36VRrfxFAFq36L4Kz3kDGj1qWtgYkuocf4xml3yyRmNb+mtputrDP 58sA== X-Gm-Message-State: AOAM532dqQpGUwP6wDKpShkkv6d31/MhpK7XvG70hka8MAKCK1zyt2TR fvShxS9Wh7R8zNiUnbUfaWQ8LSVfH33E+ATT0i/9ILCtlS81UliKB6zaCJCKc5L4VoGjgLfjn+K bohwh+RWq+fFYM1wlZiAHeLxwUIUZ7bEhq/jHYuUz10peZPWR4Oqab1dSZ3ywYu77Atoz/A8bGS y2fw== X-Google-Smtp-Source: ABdhPJwyZOtmJdffSAOboLrZmkY9x3d6QJ0ERUh9ojGDzsDluuuCQ5qTYbBxpR381rT963FjAamIAlZWFw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c74d:: with SMTP id w13mr32288041wmk.25.1620744180847; Tue, 11 May 2021 07:43:00 -0700 (PDT) Date: Tue, 11 May 2021 15:42:42 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-4-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 03/13] arm64: Downgrade flush_icache_range to invalidate From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074303_800963_9352CC5B X-CRM114-Status: GOOD ( 10.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since __flush_dcache_area is called right before, invalidate_icache_range is sufficient in this case. No functional change intended. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba --- arch/arm64/kernel/machine_kexec.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..001ffbfc645b 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -70,8 +70,9 @@ int machine_kexec_post_load(struct kimage *kimage) /* Flush the reloc_code in preparation for its execution. */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + invalidate_icache_range((uintptr_t)reloc_code, + (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); return 0; } From patchwork Tue May 11 14:42:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B644C433ED for ; Tue, 11 May 2021 14:45:37 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6C26661288 for ; Tue, 11 May 2021 14:45:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C26661288 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=YT6M8TPtCn1e4ajnA8ynUr0GBBHexyHPi6Vdsn7u6e8=; b=V9Ju4FZSTO6zNG 6ntnCvvImNr4wtJ54oCmyDaCkwV4ndbgDqw58zdAF8E7UvYAxmPtUynrsouweg5RJzSe/5Nz1CM97 GyO0zTOsPMkjEUZFdfVwfYQfOh9DELrUrolWoQAfde0HYwhKpFW+Znt1Bq7DP+0bVvrkAXH8Oc/mT tIFfsf8zAH7qq1EvCvRsB0e6wA9oU1EuDfQ9eIN30XneyRjblgx4lxyHlzZ9M45GX59FcjZiDeArV eevT1igL4mmseCnJQE/vr0XOMdeTGPO7HtuTvKmK0vqd3Zg3HsWTYdJnR+O8lN/g4S/4aH7uC8vrZ 0vUNgSPm5OZO/xkTfpqw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbz-000TNh-6C; Tue, 11 May 2021 14:43:51 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbI-000TFD-38 for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=m/UmUoFHs6dE/qmIWg03H9xxYPy7qfidkg1ny1Uz5lo=; b=gQO8T1rHL0j3qqeLitNyaRrNOe 81+FWJ+s6kzQu/epYBP1/Whs9nm39fxe+lCwAD7cwwW69uu9srrlpgsI2Kjwz7qhk9BDky5Sti6kV cj/Nqd/5iPdiy0jG750lYXUqwremAdkFM1oxpe/jJWEAm17TOFvwQ1J9VDONIsFum+y8S1QrBKP4g 1PfZBbxNyQGrWZOQ60oujG+LZg2MHHtzNnE1iHObek1MbCdTpKteNNCGDnrOC5aHDSO0g6Jga3FeP 4yz0iNjUIlBwfVESpVVsFt6GptDfNzHzBoZ4t9X3EiDz3/ard8C8GQHOfLzTncgWfofmfzlLHfgJj GiWDc74A==; Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbF-009gV4-K8 for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:06 +0000 Received: by mail-qv1-xf49.google.com with SMTP id d11-20020a0cdb0b0000b02901c0da4391d5so15716915qvk.12 for ; Tue, 11 May 2021 07:43:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=m/UmUoFHs6dE/qmIWg03H9xxYPy7qfidkg1ny1Uz5lo=; b=pB+9I2jN/yjPoYYJURvAPDYZpvmfMzVl69HyreCcGUzwX4y5e5ujHoae06oVgT4UfB 1guHC8Ri19vHyuK2CuNm+cCcQT+YjFuPO8uV5EIGbMVKaQ9bKNGt+Rr3NvgKiK5wkfQp Xn3H6HFIi5LoGMBgWIkXcy+p3KesIK3As4uWZwZ82E6IBt2mMoldN3NiNGwZoNScLhmb eYaZ2OpA6sGaESi9aXO5WmxCE+K6FGhSUqzVVG4B4z4MVQtna7YmfG10JGt6+GuBj1zG TR+X9w06RdAVLeZ0j5x4UuXFpqF+vb/ZWtMHkgt0l4nhe6dUwn/EfzO+gLKMYsnAAsLY Mnpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=m/UmUoFHs6dE/qmIWg03H9xxYPy7qfidkg1ny1Uz5lo=; b=De91ie086JP9F2vqdRF9rWfcEKN4AQDmXUiQXwNoJOTY66gx+gzlgXuM5PzgNQjQvU n8OJemK5lG3tGWqcfZA1zsaxIwua/O6ymGwgMPjj3ofaFECqgEslwfRnnt1B42zD6hbB viw0ZNBLUI0ZsedFoZOWcRdzNxUu8QO97kqP+ycMCNSTomgfVMMJb0EnWOQ5lbNDuFN3 vxsPj1IOQvW79WPAG2MmiyQmbZUmeY0CrWRYjXPM0T+rM2eU1MZ9bzHo8YrcsfzJAhcQ LhErgvUyMf2u8YDfxHa7AMBIh6MYY7+G2eH47bVKZuNg3aRLm+5QkYKnq7JOf1uc5H80 gNDA== X-Gm-Message-State: AOAM533Zz2zjg1J/KKkO2Ij7Xh8RUwvmI4/gkELHGWjbr2lO6BUHK1dn aRVbvh4sZnfv/o0OSk3+tC8A22KnaEgLCPGpt4Y56Q8FfDANTS/KTsd93tpNUyk6HbeK5A4oKcs yijUrY3Z9g4EoeJmlKJld10T6DTY5laxrGGqtO956FMqHTmd3OAAo54DTXhz1h8/yJymDDFRMjo ptwA== X-Google-Smtp-Source: ABdhPJyvqG7S5/6iVZkeBEl9vcrGwXJOLs8M9ukh4GfoTv6eqhy1D/NTUWtjvK2dOCxCdNlS7GpNPAgKfQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:ad4:4441:: with SMTP id l1mr29203377qvt.1.1620744183016; Tue, 11 May 2021 07:43:03 -0700 (PDT) Date: Tue, 11 May 2021 15:42:43 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-5-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 04/13] arm64: Move documentation of dcache_by_line_op From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074305_682324_999732AE X-CRM114-Status: GOOD ( 10.79 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The comment describing the macro dcache_by_line_op is placed right before the previous macro of the one it describes, which is a bit confusing. Move it to the macro it describes (dcache_by_line_op). No functional change intended. Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/assembler.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6ff7a3a3b238..2bcfc5fdfafd 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -375,6 +375,14 @@ alternative_cb_end bfi \tcr, \tmp0, \pos, #3 .endm + .macro __dcache_op_workaround_clean_cache, op, kaddr +alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE + dc \op, \kaddr +alternative_else + dc civac, \kaddr +alternative_endif + .endm + /* * Macro to perform a data cache maintenance for the interval * [kaddr, kaddr + size) @@ -385,14 +393,6 @@ alternative_cb_end * size: size of the region * Corrupts: kaddr, size, tmp1, tmp2 */ - .macro __dcache_op_workaround_clean_cache, op, kaddr -alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE - dc \op, \kaddr -alternative_else - dc civac, \kaddr -alternative_endif - .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 dcache_line_size \tmp1, \tmp2 add \size, \kaddr, \size From patchwork Tue May 11 14:42:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDE1FC433ED for ; Tue, 11 May 2021 14:45:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3ABC561288 for ; Tue, 11 May 2021 14:45:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3ABC561288 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tZOXcYbrIlNCGQht/Og5PWDXymg31T3N/ZcCPuvAKoc=; b=OjRuZ8w1tMbEaU VWqVuhuKxyEW1SQOwyYh4neNOeOudpSuD0K49fDJmRbVqcwwR/24INfeTdHktwKNrCrA1P1AnTUqF Zge7RvnWdtGxvK/amZ8S7po4biGMKB+ulFp8Q3xaCNucmIEKLm+qDo8s71Fy0YyjmOgDx98L2sYwG CuQdVEVxxv7MeHL2DtbchyAn7I0kf3ReEyRuRmWORVU1qood71hcj1vw13WMft4smikEU66uvN9PI eDo/kmsGDxNsfix8SdiPMgN0bG1BLlOHA9ply/u7kSY8s8vULybug8Q9dFwIc/3FGCRsLLrq9XltQ +qIBlmykYy5HkU+A3Eww==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTcD-000TSI-Hx; Tue, 11 May 2021 14:44:05 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbK-000TFU-Fh for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=U5sDBAAq2htgUdldX/vcAGUnxgHxgz2qSCkNBv1Jrjw=; b=KKVPtR7IFL6QRb70aod6hHtN/I 0M04tplm5MB5p2Ha0g6zxUR3W8KAKU1WdOqspp54IWCNWo0ZrsklUkx11Zv5AQQc5p/yt/rn09E6P MdUvLJc6KPA/67rzT5vfxCsn1cIW1dUE5OMDtkD3HLTwHQJBHhc28ALtqmXq7PSkhvwNrnHCwc3cp 6OhAzRmnlXxFx69YtW3NSvH8Zgd4Hk5XPwtnDbfUphUh5kvcPsnGJpDchcnsrGIZL26lmwiFDqhJF jpm14lg1bJt5vtmdf3Zh3lCLvS1GJbbQijvG5SrtV6JV4icn7+9o4JbHzl6VLLQ1UtxftwwhgkYna PaEvwEpw==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbH-009gVX-QN for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:09 +0000 Received: by mail-wr1-x44a.google.com with SMTP id d20-20020adfc3d40000b029010e1a640bbfso8866013wrg.21 for ; Tue, 11 May 2021 07:43:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=U5sDBAAq2htgUdldX/vcAGUnxgHxgz2qSCkNBv1Jrjw=; b=I2adHtPlOeK1f51JqVfS2af2hzYEhTcSlir7qV4R2qEaqZB6dZA8+0qYBBp3a5QV2p IfJiNSkj0IRKz50jLeaM9G0dxw7oWyFrKhgdMcc/6p7GQOCnEjojuhbD6+I26q6b8MRG CK1w1QnuT11X7AcWBrjieqOQ8vcc8OEV4Nq3++dnydKSbbMFXekN/b8wIjVlwpitEadb ViGwUaTSrciNKeD/eliO08D5VZ/FN5sRqz0I7eLvAz0gGL8lbeD7LcGX+qcTOrZUY9GR 3TQqladDIvWheFuO+KNnwmNyE1LNiZhPYnsNPgM/Mvi1S7129kaZp8isQ3s3m9oxXZxb 3cYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=U5sDBAAq2htgUdldX/vcAGUnxgHxgz2qSCkNBv1Jrjw=; b=DnZgBhFvNVMuOdkICXOQu544ACiwvdOhzxy1EZOfKbBIEkbumywolDBpFMPaj9W7KB 6hKPCeNKPGQXrDD9mHynZTESgEaxvv4z433qEVEyFg/fdm8XnJ5EJMJRafzXJNcVh7tK 3mA0VZyj+L8lPHiLYAFAxHuR/8zXuJzk5EidQkTIw4yVIIiS/K0VeWjGrhCvERL55a5g Af8u0aRrPjC8b6hHzVqXvd90pne7ShES94GcgA4rL0QEaJc7P6i7IwR800E8H5gpBn76 aDFsWlLzsMl7T7t+PhHQgOekOLEsTDUuJ1w5iZ2T7HoI6gkkHKz5thYRsNAQF9jbnqCC 0oGQ== X-Gm-Message-State: AOAM531Ectm8jIooOxVF9gfqQWa7nnWGRTCFijXua8ia39OQvUZJDFum 7isBcEVafUERT4EMfLbuYUQ+7y//vLzNhLgwaoLZNkbU8jDn2bEc5OZTtaOBvrMUdZ2/qDhbGBR lMBYPLTZZDQNMvmxZ7BYjiRS/Tq3JsTDtekgN99mS3ZNy6ZtgEP1aPp54dsXV9bot2T67v+Ou1S nc1A== X-Google-Smtp-Source: ABdhPJxKSjUsLU7aqmEg4fMD16LL/JagIPC6upQMTu/WxFW0UmSpD3nTuc9SmNcMfc9csJ7plDHZGYUN4A== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:35cc:: with SMTP id r12mr567048wmq.1.1620744185161; Tue, 11 May 2021 07:43:05 -0700 (PDT) Date: Tue, 11 May 2021 15:42:44 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-6-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 05/13] arm64: __inval_dcache_area to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074307_876841_C5DC5882 X-CRM114-Status: GOOD ( 14.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. Because the code is shared with __dma_inv_area, it changes the parameters for that as well. However, __dma_inv_area, is local to cache.S so no other users are affected. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/head.S | 5 +---- arch/arm64/mm/cache.S | 16 +++++++++------- arch/arm64/mm/flush.c | 2 +- 4 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index a586afa84172..157234706817 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -59,7 +59,7 @@ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); -extern void __inval_dcache_area(void *addr, size_t len); +extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pop(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 96873dfa67fd..8df0ac8d9123 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -117,7 +117,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) dmb sy // needed before dc ivac with // MMU off - mov x1, #0x20 // 4 x 8 bytes + add x1, x0, #0x20 // 4 x 8 bytes b __inval_dcache_area // tail call SYM_CODE_END(preserve_boot_args) @@ -268,7 +268,6 @@ SYM_FUNC_START_LOCAL(__create_page_tables) */ adrp x0, init_pg_dir adrp x1, init_pg_end - sub x1, x1, x0 bl __inval_dcache_area /* @@ -382,12 +381,10 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x0, idmap_pg_dir adrp x1, idmap_pg_end - sub x1, x1, x0 bl __inval_dcache_area adrp x0, init_pg_dir adrp x1, init_pg_end - sub x1, x1, x0 bl __inval_dcache_area ret x28 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 6babaaf34f17..64507944b461 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -146,25 +146,24 @@ alternative_else_nop_endif SYM_FUNC_END(__clean_dcache_area_pou) /* - * __inval_dcache_area(kaddr, size) + * __inval_dcache_area(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are invalidated. Any partial lines at the ends of the interval are * also cleaned to PoC to prevent data loss. * - * - kaddr - kernel address - * - size - size in question + * - start - kernel start address of region + * - end - kernel end address of region */ SYM_FUNC_START_LOCAL(__dma_inv_area) SYM_FUNC_START_PI(__inval_dcache_area) /* FALLTHROUGH */ /* - * __dma_inv_area(start, size) + * __dma_inv_area(start, end) * - start - virtual start address of region - * - size - size in question + * - end - virtual end address of region */ - add x1, x1, x0 dcache_line_size x2, x3 sub x3, x2, #1 tst x1, x3 // end cache line aligned? @@ -245,8 +244,10 @@ SYM_FUNC_END_PI(__dma_flush_area) * - dir - DMA direction */ SYM_FUNC_START_PI(__dma_map_area) + add x1, x0, x1 cmp w2, #DMA_FROM_DEVICE b.eq __dma_inv_area + sub x1, x1, x0 b __dma_clean_area SYM_FUNC_END_PI(__dma_map_area) @@ -257,6 +258,7 @@ SYM_FUNC_END_PI(__dma_map_area) * - dir - DMA direction */ SYM_FUNC_START_PI(__dma_unmap_area) + add x1, x0, x1 cmp w2, #DMA_TO_DEVICE b.ne __dma_inv_area ret diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index ac485163a4a7..4e3505c2bea6 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -88,7 +88,7 @@ EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); void arch_invalidate_pmem(void *addr, size_t size) { - __inval_dcache_area(addr, size); + __inval_dcache_area((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif From patchwork Tue May 11 14:42:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 433CBC433ED for ; Tue, 11 May 2021 14:46:06 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A7C9461396 for ; Tue, 11 May 2021 14:46:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7C9461396 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xj3PpF/r1LL6iy8749qPHrLHwnrUQe4VcDEHRlneEUQ=; b=Tf/yE51yEdsqeo xzwAkK4NLQs10EXjYVEX1YoCVmpV5+QtafSpRaLR3R7lRD0lYtC6h/dMU2rxENaDOValTltrnQEwH Rm1TBp3hbXVa2asw4hJ3euWsXi23oeia4y/8Y25NWkUNVL9rst/B6nF0zUu1XKITh315ani78zNvL CqbyQBTkGfHSpzZAP2VTYWqq5IVKSP8SRbGkdZcbl6agqM2qf3wbq19/hS20dOPN7Ms49DlJGvniz f1ZDmUEhKBfCR512L9Qt4sewcfvVWBJZuOOskhBwFDaMvkkv8JU/3tgF3x+SUDwz+IF8pdbmqL6nn uyQ6fDdEznVbIA8CJwTw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTcS-000TYW-Hf; Tue, 11 May 2021 14:44:20 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbN-000TFs-5o for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=MbwhDKk5EA11jKAulpTBnY/IRT4L9GaSTz51nKVhpbI=; b=tXMQBrBHB2NtQo7tVmZcXHpPj8 mrUXhYv0RFo1CAKGGuHZ+tzXrymHTJOU6Hz7PaiQXM6hlfnqiKHnTEBbowMl46oL5yFY6T9F4mF5v dHfhzvO5BeYf1eI63zfdqtx5jvW3W+FH0cGSX7uy8OKW83aqjUH8hBJRncQU3QblmIoWNOPLhiD+N cDyiVUeU5vDU7nEF/FT4GXPGKKcsUl8d3NSBLqpywbPzxfA6FNPhMIntHxmwk0AOAp04LC9N3QWl4 MMiIoM9w6Izevdz+rj9tC4qeeCR07jmNfLb5JujFVKRY7Umvc6lPAqgDGOZvWz6rNJs8TVqBlFTGB XqvncZMA==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbK-009gVo-FJ for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:11 +0000 Received: by mail-wr1-x449.google.com with SMTP id x10-20020adfc18a0000b029010d83c83f2aso8861870wre.8 for ; Tue, 11 May 2021 07:43:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=MbwhDKk5EA11jKAulpTBnY/IRT4L9GaSTz51nKVhpbI=; b=m/TLs69EaPBriQhil/U1MOAulA4IV0+IjRupBWBrMDt6T/OakQJ+488JHc+2r3yRYY IsMLPkz1jNw3xgblKVTDNOir7J+3TuwyIAlqTwNZuX5aIzFHhpGRCPzGFjgRRWNbYJct wQdBjOmj7xwpQtP3TD1b9lbBswfMUKhMLKv7DrUulaSNZa4OZZMVxxH1n1KTodhuLMNh lTO8c0TeSTqzS9sCug0ec8w2RXawRfziK+bZf4u3S/L1y/J6umcJ5S1fyr3lQPd+3Eco ++JV47wT/23dpdRiyQppGPzAdI0K7ZSHrKSp4MRwygs1kqRkUYgjK/lYdYBgITtNnh92 CT3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=MbwhDKk5EA11jKAulpTBnY/IRT4L9GaSTz51nKVhpbI=; b=tSV3exRsFoLZe4S9qQdoOGjnI9MtjGh2ek2P/27W3xPmzbd1w/XwtIqqiTztDAeslv /RPyymbEZDcCy8RFr3dffKOOAz9ciYhX2GsUn0pW4vsps6rLhcWTVgHcjxgFf7Y8U5Ab ggfClBhwP/vtufvCIZ/1LkWtvKavOa/XdbTTD0aAEYneQ2TiBG9Dlvdlg64i2Q0eR3uu s/65Bvl/slXwjAmMudHIvO2Feys6d+vF4FcWGPHEJHE8SZcuYWL8V2y+DeNkdtQeO8SC d0XHwpGkuVUknEn8TdgvHcILxksvyMDb9/Df+tt5bNeqd9zfdkiURs12C64RMcB1rIE8 13iQ== X-Gm-Message-State: AOAM530/1p4rYJIXRccqO+b2igPSFZJLV2yXyZw6TFvx4ccBYcs6QAHw cxSYXzl/gK1/N2738rWstL/LdM+hsDVKWZqBBCZ9H6KNc9Py7zUeYuzRvxigTG2EgdNdkCIVIXR 0XhJdFRYgF5BAaCH9gGD3Byzi7bGmhMYSNaU5Vi5L4HrVEZW+73uRq2Xjd9RXiyECedzkafSZsn fTpw== X-Google-Smtp-Source: ABdhPJz+5MbONf+W6LytZA7RcQzCraMaDlxaDgamS5lHWzpfTyL79FYkofomvDIcMyWHXg795hDnboP0WA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:4ba3:: with SMTP id e35mr32614634wmp.47.1620744187730; Tue, 11 May 2021 07:43:07 -0700 (PDT) Date: Tue, 11 May 2021 15:42:45 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-7-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 06/13] arm64: dcache_by_line_op to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074310_533320_B7869680 X-CRM114-Status: GOOD ( 12.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/assembler.h | 27 +++++++++++++-------------- arch/arm64/kvm/hyp/nvhe/cache.S | 1 + arch/arm64/mm/cache.S | 5 +++++ 3 files changed, 19 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 2bcfc5fdfafd..3f75a600e6c0 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -385,39 +385,38 @@ alternative_endif /* * Macro to perform a data cache maintenance for the interval - * [kaddr, kaddr + size) + * [start, end) * * op: operation passed to dc instruction * domain: domain used in dsb instruciton - * kaddr: starting virtual address of the region - * size: size of the region - * Corrupts: kaddr, size, tmp1, tmp2 + * start: starting virtual address of the region + * end: end virtual address of the region + * Corrupts: start, end, tmp1, tmp2 */ - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2 dcache_line_size \tmp1, \tmp2 - add \size, \kaddr, \size sub \tmp2, \tmp1, #1 - bic \kaddr, \kaddr, \tmp2 + bic \start, \start, \tmp2 9998: .ifc \op, cvau - __dcache_op_workaround_clean_cache \op, \kaddr + __dcache_op_workaround_clean_cache \op, \start .else .ifc \op, cvac - __dcache_op_workaround_clean_cache \op, \kaddr + __dcache_op_workaround_clean_cache \op, \start .else .ifc \op, cvap - sys 3, c7, c12, 1, \kaddr // dc cvap + sys 3, c7, c12, 1, \start // dc cvap .else .ifc \op, cvadp - sys 3, c7, c13, 1, \kaddr // dc cvadp + sys 3, c7, c13, 1, \start // dc cvadp .else - dc \op, \kaddr + dc \op, \start .endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 - cmp \kaddr, \size + add \start, \start, \tmp1 + cmp \start, \end b.lo 9998b dsb \domain .endm diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 36cef6915428..3bcfa3cac46f 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -8,6 +8,7 @@ #include SYM_FUNC_START_PI(__flush_dcache_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 64507944b461..c801ebaf418f 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -123,6 +123,7 @@ SYM_FUNC_END(invalidate_icache_range) * - size - size in question */ SYM_FUNC_START_PI(__flush_dcache_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) @@ -141,6 +142,7 @@ alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif + add x1, x0, x1 dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret SYM_FUNC_END(__clean_dcache_area_pou) @@ -202,6 +204,7 @@ SYM_FUNC_START_PI(__clean_dcache_area_poc) * - start - virtual start address of region * - size - size in question */ + add x1, x0, x1 dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_poc) @@ -220,6 +223,7 @@ SYM_FUNC_START_PI(__clean_dcache_area_pop) alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif + add x1, x0, x1 dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_pop) @@ -233,6 +237,7 @@ SYM_FUNC_END_PI(__clean_dcache_area_pop) * - size - size in question */ SYM_FUNC_START_PI(__dma_flush_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__dma_flush_area) From patchwork Tue May 11 14:42:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A026C433B4 for ; Tue, 11 May 2021 14:46:42 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 99C2061263 for ; Tue, 11 May 2021 14:46:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 99C2061263 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Lj7bMjzv2gIqhlT/JbWzRFclEC1qoIRZsfpeV6yH6pQ=; b=pOZScRqUsDWg+I GPPzRFdgDzqmUVsbL0upnSPufKcTcm4eIbuuFX0+EZTXXcdgyLP+CtpazbG6SNdHZ5Az2K/UH4bPn BJggviwJDXZZ6jcqITfQ3swJ1rkP2HgCU3IGt2DvsD5inyD1snoAiPwRfHt2Mr8XN6VkedDg4if0P IJRv5cU1ev1EYdWctU3se6Qe3FmMaHGkpcjGc79aIbpwF6NsaW3ZgNzwzfiBe+qxAV0XDP+AFrvIi gVlgdXhVVmpfCXdJ+Dyne8rJHuYvAbXEj8K8rnkRWuicVYvadDqTk8HE4BMYUOK3QRuq+XQwdw1LT 7zX1YSzsxosH13Ino2yw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTck-000Tel-Fm; Tue, 11 May 2021 14:44:38 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbP-000TGH-Uz for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=UOEUMHPa3mnJbiJNT4416nJjncD93qQjHi9Ew1sBHXM=; b=SYLgYu31iKIp/DM21NrbF7dcxy 6mHL424L9KWNAExYNS/q3vL847HL3750oIo7aFujrr4RrOuhTPddNFQhuDhzVrl3heqYLyt3TtZUM 3UW1jCteE7o1tiOToeEhMCMEBA9IPkfTfVHrhnnhluhMmnFk3qeqjNzyz9xu5/5Cq0SGFJAZ9Whhb twcMudW2a5/vi2u36tPaWLQZxdKvVnpBkxZNDodLdIJut5n58szNSqGhv9ZLBjMAMxJSjcNK+Ehw3 5qPRb6OPd/QT9SoNZzzeGCVPdRjp4vUUzH/VWBaiJEMt3rp6qASkrKi3PYbio8TyTkwuQ3Ew06OMX hheu2YkQ==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbM-009gW4-K6 for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:14 +0000 Received: by mail-wr1-x44a.google.com with SMTP id v5-20020adf9e450000b029010e708f05b3so4547439wre.6 for ; Tue, 11 May 2021 07:43:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=UOEUMHPa3mnJbiJNT4416nJjncD93qQjHi9Ew1sBHXM=; b=aiICGcKLGHD4I8Kh7LvBZUm4bSHRcs+XvxPPQJo4WVrRAkguO7faIoDUs5jhDvAV2b XJ9TId5BxydWWO7JC4PO3Vh4Fd7SGFpocX0kMdSkpt9LKh7nPil3ZPiOYtI4drb5XtG7 rEQFToo0dis0QKpRn7BHne36wJjSEKQfh9HchT3ZBwUrhs9u8jI9Kp+C1jGdJicUMYxv cJMp/MEcZV3tN8f9vfB5CAWTtdrS3QeT5Ma8f5cAGOoXKjX2vom6EHNnj7IjgvER+Pcg JDUZ27une1SXCw0YYFnYU4tddR8Dmw2BjNPZmOGX53zG438XjBXC8Dm6h/GRYTUEDrwr nv3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=UOEUMHPa3mnJbiJNT4416nJjncD93qQjHi9Ew1sBHXM=; b=QLvQyj8iFaPMVol1emeL51hZCZy0J2FycYiguSXvgY6XtVfgziJdYfeFq4CNmrHO6X w8cBB5SfdJ2Ahgm74pmA1m4G6r34WnV+xbIlzPKijoZGgaiIRwoPCGanJy+MI89nv7nq boNQEv1zmJY5daaf/Q+AO8A3PG51mVmIPhF0MBhdl+PDdPpNI8YgoNRuYwYzV+/Z4HCp 6irHJqy5wMoeriQVUW5Y5bpe7J40E72d+XnYbfb0rNHEXeHXzSNZuykzOrnErk/rU0pO 5YDDG9IYi+QbvUaV++8WupuBK3rSTi0UDfx9LEhfkPuUHc0xeFy4E6aInfYWM9AoWIqB fM4A== X-Gm-Message-State: AOAM531VQq90kT1T5pFLa59G+kLTCMZ3hB+nQukb1Te49Gn5Dl9gp3i5 39ctVV6SYl11lmZKarSRj2CuoAZTw5F5aElebFdwmwXwEoYa9FtVo55Sw7aaSPIwbJ4sV7Iu8xt QeDjAVz1gsFXBKV8UuBgzaVvlMMu8Z1mJB5AzzYDpahHpGTM6yw9L7YSK6Y0y0PMLGgbTKkrz4t ZW5g== X-Google-Smtp-Source: ABdhPJyeWtrY8P62dHMAQVYIFQYEvKP4VycNBZ4vonAiRGJSLO96RJhD1u8Kqrl1BD56Gl2R/n5KCmSScg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:4242:: with SMTP id r2mr5921497wmm.140.1620744190463; Tue, 11 May 2021 07:43:10 -0700 (PDT) Date: Tue, 11 May 2021 15:42:46 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-8-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 07/13] arm64: __flush_dcache_area to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074312_692806_E08E91C5 X-CRM114-Status: GOOD ( 23.05 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/arch_gicv3.h | 3 ++- arch/arm64/include/asm/cacheflush.h | 8 ++++---- arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 3 ++- arch/arm64/kernel/hibernate.c | 18 +++++++++++------- arch/arm64/kernel/idreg-override.c | 3 ++- arch/arm64/kernel/kaslr.c | 12 +++++++++--- arch/arm64/kernel/machine_kexec.c | 20 +++++++++++++------- arch/arm64/kernel/smp.c | 8 ++++++-- arch/arm64/kernel/smp_spin_table.c | 7 ++++--- arch/arm64/kvm/hyp/nvhe/cache.S | 1 - arch/arm64/kvm/hyp/nvhe/setup.c | 3 ++- arch/arm64/kvm/hyp/pgtable.c | 13 ++++++++++--- arch/arm64/mm/cache.S | 9 ++++----- 14 files changed, 70 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 934b9be582d2..ed1cc9d8e6df 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -124,7 +124,8 @@ static inline u32 gic_read_rpr(void) #define gic_read_lpir(c) readq_relaxed(c) #define gic_write_lpir(v, c) writeq_relaxed(v, c) -#define gic_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l)) +#define gic_flush_dcache_to_poc(a,l) \ + __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) #define gits_read_baser(c) readq_relaxed(c) #define gits_write_baser(v, c) writeq_relaxed(v, c) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 157234706817..695f88864784 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -50,15 +50,15 @@ * - start - virtual start address * - end - virtual end address * - * __flush_dcache_area(kaddr, size) + * __flush_dcache_area(start, end) * * Ensure that the data held in page is written back. - * - kaddr - page address - * - size - region size + * - start - virtual start address + * - end - virtual end address */ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); -extern void __flush_dcache_area(void *addr, size_t len); +extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pop(void *addr, size_t len); diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 3578aba9c608..0ae2397076fd 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -137,7 +137,7 @@ void efi_virtmap_unload(void); static inline void efi_capsule_flush_cache_range(void *addr, int size) { - __flush_dcache_area(addr, size); + __flush_dcache_area((unsigned long)addr, (unsigned long)addr + size); } #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 25ed956f9af1..33293d5855af 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -180,7 +180,8 @@ static inline void *__kvm_vector_slot2addr(void *base, struct kvm; -#define kvm_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l)) +#define kvm_flush_dcache_to_poc(a,l) \ + __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..b40ddce71507 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -240,8 +240,6 @@ static int create_safe_exec_page(void *src_start, size_t length, return 0; } -#define dcache_clean_range(start, end) __flush_dcache_area(start, (end - start)) - #ifdef CONFIG_ARM64_MTE static DEFINE_XARRAY(mte_pages); @@ -383,13 +381,18 @@ int swsusp_arch_suspend(void) ret = swsusp_save(); } else { /* Clean kernel core startup/idle code to PoC*/ - dcache_clean_range(__mmuoff_data_start, __mmuoff_data_end); - dcache_clean_range(__idmap_text_start, __idmap_text_end); + __flush_dcache_area((unsigned long)__mmuoff_data_start, + (unsigned long)__mmuoff_data_end); + __flush_dcache_area((unsigned long)__idmap_text_start, + (unsigned long)__idmap_text_end); /* Clean kvm setup code to PoC? */ if (el2_reset_needed()) { - dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); - dcache_clean_range(__hyp_text_start, __hyp_text_end); + __flush_dcache_area( + (unsigned long)__hyp_idmap_text_start, + (unsigned long)__hyp_idmap_text_end); + __flush_dcache_area((unsigned long)__hyp_text_start, + (unsigned long)__hyp_text_end); } swsusp_mte_restore_tags(); @@ -474,7 +477,8 @@ int swsusp_arch_resume(void) * The hibernate exit text contains a set of el2 vectors, that will * be executed at el2 with the mmu off in order to reload hyp-stub. */ - __flush_dcache_area(hibernate_exit, exit_size); + __flush_dcache_area((unsigned long)hibernate_exit, + (unsigned long)hibernate_exit + exit_size); /* * KASLR will cause the el2 vectors to be in a different location in diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index e628c8ce1ffe..3dd515baf526 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -237,7 +237,8 @@ asmlinkage void __init init_feature_override(void) for (i = 0; i < ARRAY_SIZE(regs); i++) { if (regs[i]->override) - __flush_dcache_area(regs[i]->override, + __flush_dcache_area((unsigned long)regs[i]->override, + (unsigned long)regs[i]->override + sizeof(*regs[i]->override)); } } diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 341342b207f6..49cccd03cb37 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -72,7 +72,9 @@ u64 __init kaslr_early_init(void) * we end up running with module randomization disabled. */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - __flush_dcache_area(&module_alloc_base, sizeof(module_alloc_base)); + __flush_dcache_area((unsigned long)&module_alloc_base, + (unsigned long)&module_alloc_base + + sizeof(module_alloc_base)); /* * Try to map the FDT early. If this fails, we simply bail, @@ -170,8 +172,12 @@ u64 __init kaslr_early_init(void) module_alloc_base += (module_range * (seed & ((1 << 21) - 1))) >> 21; module_alloc_base &= PAGE_MASK; - __flush_dcache_area(&module_alloc_base, sizeof(module_alloc_base)); - __flush_dcache_area(&memstart_offset_seed, sizeof(memstart_offset_seed)); + __flush_dcache_area((unsigned long)&module_alloc_base, + (unsigned long)&module_alloc_base + + sizeof(module_alloc_base)); + __flush_dcache_area((unsigned long)&memstart_offset_seed, + (unsigned long)&memstart_offset_seed + + sizeof(memstart_offset_seed)); return offset; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 001ffbfc645b..4cada9000acf 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -69,7 +69,9 @@ int machine_kexec_post_load(struct kimage *kimage) kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area((unsigned long)reloc_code, + (unsigned long)reloc_code + + arm64_relocate_new_kernel_size); invalidate_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); @@ -103,16 +105,18 @@ static void kexec_list_flush(struct kimage *kimage) for (entry = &kimage->head; ; entry++) { unsigned int flag; - void *addr; + unsigned long addr; /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); + __flush_dcache_area((unsigned long)entry, + (unsigned long)entry + + sizeof(kimage_entry_t)); flag = *entry & IND_FLAGS; if (flag == IND_DONE) break; - addr = phys_to_virt(*entry & PAGE_MASK); + addr = (unsigned long)phys_to_virt(*entry & PAGE_MASK); switch (flag) { case IND_INDIRECTION: @@ -121,7 +125,7 @@ static void kexec_list_flush(struct kimage *kimage) break; case IND_SOURCE: /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); + __flush_dcache_area(addr, addr + PAGE_SIZE); break; case IND_DESTINATION: break; @@ -148,8 +152,10 @@ static void kexec_segment_flush(const struct kimage *kimage) kimage->segment[i].memsz, kimage->segment[i].memsz / PAGE_SIZE); - __flush_dcache_area(phys_to_virt(kimage->segment[i].mem), - kimage->segment[i].memsz); + __flush_dcache_area( + (unsigned long)phys_to_virt(kimage->segment[i].mem), + (unsigned long)phys_to_virt(kimage->segment[i].mem) + + kimage->segment[i].memsz); } } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index dcd7041b2b07..5fcdee331087 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -122,7 +122,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) secondary_data.task = idle; secondary_data.stack = task_stack_page(idle) + THREAD_SIZE; update_cpu_boot_status(CPU_MMU_OFF); - __flush_dcache_area(&secondary_data, sizeof(secondary_data)); + __flush_dcache_area((unsigned long)&secondary_data, + (unsigned long)&secondary_data + + sizeof(secondary_data)); /* Now bring the CPU into our world */ ret = boot_secondary(cpu, idle); @@ -143,7 +145,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) pr_crit("CPU%u: failed to come online\n", cpu); secondary_data.task = NULL; secondary_data.stack = NULL; - __flush_dcache_area(&secondary_data, sizeof(secondary_data)); + __flush_dcache_area((unsigned long)&secondary_data, + (unsigned long)&secondary_data + + sizeof(secondary_data)); status = READ_ONCE(secondary_data.status); if (status == CPU_MMU_OFF) status = READ_ONCE(__early_cpu_boot_status); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index c45a83512805..58d804582a35 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -36,7 +36,7 @@ static void write_pen_release(u64 val) unsigned long size = sizeof(secondary_holding_pen_release); secondary_holding_pen_release = val; - __flush_dcache_area(start, size); + __flush_dcache_area((unsigned long)start, (unsigned long)start + size); } @@ -90,8 +90,9 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(pa_holding_pen, release_addr); - __flush_dcache_area((__force void *)release_addr, - sizeof(*release_addr)); + __flush_dcache_area((__force unsigned long)release_addr, + (__force unsigned long)release_addr + + sizeof(*release_addr)); /* * Send an event to wake up the secondary CPU. diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 3bcfa3cac46f..36cef6915428 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -8,7 +8,6 @@ #include SYM_FUNC_START_PI(__flush_dcache_area) - add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 7488f53b0aa2..5dffe928f256 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -134,7 +134,8 @@ static void update_nvhe_init_params(void) for (i = 0; i < hyp_nr_cpus; i++) { params = per_cpu_ptr(&kvm_init_params, i); params->pgd_pa = __hyp_pa(pkvm_pgtable.pgd); - __flush_dcache_area(params, sizeof(*params)); + __flush_dcache_area((unsigned long)params, + (unsigned long)params + sizeof(*params)); } } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index c37c1dc4feaf..10d2f04013d4 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -839,8 +839,11 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, stage2_put_pte(ptep, mmu, addr, level, mm_ops); if (need_flush) { - __flush_dcache_area(kvm_pte_follow(pte, mm_ops), - kvm_granule_size(level)); + kvm_pte_t *pte_follow = kvm_pte_follow(pte, mm_ops); + + __flush_dcache_area((unsigned long)pte_follow, + (unsigned long)pte_follow + + kvm_granule_size(level)); } if (childp) @@ -988,11 +991,15 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct kvm_pgtable *pgt = arg; struct kvm_pgtable_mm_ops *mm_ops = pgt->mm_ops; kvm_pte_t pte = *ptep; + kvm_pte_t *pte_follow; if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pgt, pte)) return 0; - __flush_dcache_area(kvm_pte_follow(pte, mm_ops), kvm_granule_size(level)); + pte_follow = kvm_pte_follow(pte, mm_ops); + __flush_dcache_area((unsigned long)pte_follow, + (unsigned long)pte_follow + + kvm_granule_size(level)); return 0; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index c801ebaf418f..72a80d19e2d1 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -114,16 +114,15 @@ alternative_else_nop_endif SYM_FUNC_END(invalidate_icache_range) /* - * __flush_dcache_area(kaddr, size) + * __flush_dcache_area(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned and invalidated to the PoC. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_PI(__flush_dcache_area) - add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) From patchwork Tue May 11 14:42:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25692C43461 for ; Tue, 11 May 2021 14:46:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7B4FC61288 for ; Tue, 11 May 2021 14:46:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B4FC61288 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2Sdooi2QcdmlhtZHOFY/AHvw4MP6ZFj7IrEA6rEayPI=; b=onAB2pyp0it0BR 5O1AbFLUbg80dViMyBQFUl82CC+fp4ppeu/jfSoUVbenr9V5bByq/iWoi7bHjjDoxqGmR1MtpIcku 2HTl0Hs3uUn73w7yCwdQa7gAA8ZCek3i53uA72HDk11J8uJtTAigbxA3Kxs4zEms0ZtHS9BUINIrH wI8VkTNJvnq505R3yADz4j3uimD3CanTHWywETSM54qgN2P0hiET7yqK31iGzv33Jwh37FF1IH4s0 cej4CJKZ9ipMvp5khgdglDn8hpPxztGdh7uVO5USmBMIikH30xA/MH1MjftgrlMguxz2U+eWPGGJ6 GcGCY2HrnLItLU/yENhA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTd4-000TmG-Kv; Tue, 11 May 2021 14:45:00 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbR-000TGM-PH for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:17 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=VBdXd8jMHa8YfSUmZm1o+mEO+YnY6b9mKc8xcuxjBRY=; b=xoWNBtOVKNF9MIEBjNALjsajwK N+Ru1qL2rf0OiTMFIb9x4PvRYRJAMsDpzGP8O6bIozs4DYRL2966oRkuAqwhjG632ln4mWxKj47XL roCxHm80N8CexGVMSNWquw7OwSIX895Ie7gx8NNas5JC0uXAlD3IYP8CX50Z1tJZLpcptp066lb1j nImBcipaDp+rWlj50Wd4Kstbwf3EwJY+Y2qnehfZIPqO1nrqecZa2WICEwBq4OQU0OtSjnmedEYvd wsXsAVuLWMofGts/JGq4buVtwLDivneD6W4z+WcwvavdhmsC7iW9HBIhmWFhyL2YydZ5ixMWLL/S/ g5f7ublQ==; Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbP-009gWL-4q for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:16 +0000 Received: by mail-qk1-x74a.google.com with SMTP id l19-20020a37f5130000b02902e3dc23dc92so14544994qkk.15 for ; Tue, 11 May 2021 07:43:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=VBdXd8jMHa8YfSUmZm1o+mEO+YnY6b9mKc8xcuxjBRY=; b=g+ZaSMpmi2TM36VsxdwexFhWbBe8Bw60MbPoZT7G63redQXnq9rSLGaDcW47YwYeRd ydy/96UqSIi4wiZeKn3qkSL6Vkfx5qmMZ/D+wEzaYPGsht9fYfZfQTCJtFM8/sZXPoiK DLP/iGG27BWHRgQaXIFV2S7Sl2yQYyPGYkQXlTqlGLBVdykTAcAfdNnJKWrsDItjDKV4 ABJNlZB9p4R5dgkI1NNnugzG3spL5/pKAnfDvHtYiepAmpeUks4E4cyRblkcPqgOsbi6 rCixGGKmm9ag2F1ybe8gwmuQpQ5fpEEFG4wb3ECWd7jjyj6oon7orgwIpoQMYpex7q67 N8OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=VBdXd8jMHa8YfSUmZm1o+mEO+YnY6b9mKc8xcuxjBRY=; b=VhxYoodr/a2Ioa5eeyEdeG65+trUvx3DYZtXvcvYW4J6YnFh3vC+lr57bNVY+J64Rc MH1qsgIfOcV2JYBclN4Jx/2Tac50EB7V4BI7bear/zk3xdLXq0M+iqFpJdYY1EArQw03 QYE+jfGlMNSLaHlCi0Catjw0J5qMUL91OxWhsGHssPAFAS+BgeDnWhj8zQDYkfcopi0j Yb5y5kV3pUkEt9eIayDsHvXl385uxuosaxivXdiaCJ4D6YcQJlbNVmSWrAruIoN/wuUq 0qzHohVSIMtZcheTGIQ0V+X6LmMKx/1wK3owkjhXENCpeiaPrjrmAdpIfQjVIJvB2flZ yMkg== X-Gm-Message-State: AOAM530a3ssIayinFBFZHeJtuGxUKDHHMa3ScWP1PamSaL0VZVB0NEQU RGkR1Gw/G51SI22Fe8gMYILaAHhdBSxj+s16wxn2zvjQ8uC2u9CnG+zoT2pE8DOyLL8HCvLGFPr LP0nKj+yYqjEPnlNdm6tXzHw3ZjBG9aN9rZ4XIU1UcQLtMe3u3pMv0beoTCdj5s7uKEgkQNpDSZ y1KA== X-Google-Smtp-Source: ABdhPJyJh8z84GYWTpI++SYM9+Ah+B4gXm++CY/8pCB4MBSI3ejosVWRI19fP8gVjXaFG1J9bjcBpUM6oQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:6214:30a:: with SMTP id i10mr3072651qvu.42.1620744192737; Tue, 11 May 2021 07:43:12 -0700 (PDT) Date: Tue, 11 May 2021 15:42:47 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-9-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 08/13] arm64: __clean_dcache_area_poc to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074315_208847_7D821431 X-CRM114-Status: GOOD ( 14.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. Because the code is shared with __dma_clean_area, it changes the parameters for that as well. However, __dma_clean_area, is local to cache.S so no other users are affected. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/efi-entry.S | 5 +++-- arch/arm64/mm/cache.S | 16 +++++++--------- 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 695f88864784..3255878d6f30 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -60,7 +60,7 @@ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); -extern void __clean_dcache_area_poc(void *addr, size_t len); +extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 0073b24b5d25..72e6a580290a 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -28,6 +28,7 @@ SYM_CODE_START(efi_enter_kernel) * stale icache entries from before relocation. */ ldr w1, =kernel_size + add x1, x0, x1 bl __clean_dcache_area_poc ic ialluis @@ -36,7 +37,7 @@ SYM_CODE_START(efi_enter_kernel) * so that we can safely disable the MMU and caches. */ adr x0, 0f - ldr w1, 3f + adr x1, 3f bl __clean_dcache_area_poc 0: /* Turn off Dcache and MMU */ @@ -65,4 +66,4 @@ SYM_CODE_START(efi_enter_kernel) mov x3, xzr br x19 SYM_CODE_END(efi_enter_kernel) -3: .long . - 0b +3: diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 72a80d19e2d1..7ddf6ff65b15 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -186,24 +186,23 @@ SYM_FUNC_END_PI(__inval_dcache_area) SYM_FUNC_END(__dma_inv_area) /* - * __clean_dcache_area_poc(kaddr, size) + * __clean_dcache_area_poc(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoC. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_LOCAL(__dma_clean_area) SYM_FUNC_START_PI(__clean_dcache_area_poc) /* FALLTHROUGH */ /* - * __dma_clean_area(start, size) + * __dma_clean_area(start, end) * - start - virtual start address of region - * - size - size in question + * - end - virtual end address of region */ - add x1, x0, x1 dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_poc) @@ -219,10 +218,10 @@ SYM_FUNC_END(__dma_clean_area) * - size - size in question */ SYM_FUNC_START_PI(__clean_dcache_area_pop) + add x1, x0, x1 alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif - add x1, x0, x1 dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_pop) @@ -251,7 +250,6 @@ SYM_FUNC_START_PI(__dma_map_area) add x1, x0, x1 cmp w2, #DMA_FROM_DEVICE b.eq __dma_inv_area - sub x1, x1, x0 b __dma_clean_area SYM_FUNC_END_PI(__dma_map_area) From patchwork Tue May 11 14:42:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90D41C433ED for ; Tue, 11 May 2021 14:47:06 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EC25D61263 for ; Tue, 11 May 2021 14:47:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC25D61263 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zPhzgraoeaWea9W639jQlOv7Ymh694EPV0dXQsSkT9Y=; b=YBwoXMaXWKIdCQ PPmMYSs4KRmyZFWAXd3Wl575Lp10ktZW8c10A+cpv08yFLVvBNPhFYjOjyiFJ/UMLhrPnuQZ6vDZE B4Dbw2rPZ/MM4nsiZIqOBvSPMzgLl35izu4CL7BZAHQV642vcBdZiK+Y3tEh78kVbHtpiO04R1fgY tSVqyqqM1NIqp7h6LI6RkMzu6A1fXo7/pEH8pqY56YGbE2PLnU7TuIpWRdLikJO3xaigSjTtu768S wqcA+n1zb7poJOw7Sei7Gn7aZZj8VYjyoc6lPwmu/TgBIFYwGKPLNkLmlDLmHhU2vtv2vI1JAgwBf N9ps+yy9WNtXVuEzNJ7w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTdV-000Tyw-Q1; Tue, 11 May 2021 14:45:26 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbT-000TGi-8C for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=T7VsirMUmcmfR3W7jGLVBbLf6Nqe5HLZuoCOt+ECiBU=; b=BBZbtO4CQaYviJO63rYVYsvj/q RRYj7hF9oAIpV0r6qoLIjTgaZUcSXS7i+vYePQAcAQrTuGKYX9m/H36WxkE/erg4uCuY+T3N0cPhw 3i+WWVO4UZThy1A2q4QZF7FuFoxrfyQkyQELIk14ndTrxe3ww+aQ8gkiiffXgtEMxib/Z7F5S1Twd fH1UPMzNmbmqbOFn9/1zOYVi1p5C7Z97882PVBjP0IrdKqKEATAIgkxcJJzCw5WA6bs034hyEjmIV uIIH8U3i6Xg9zxLNyVvTOX5Uw+kLafQFIc4BfrVVAqrFCGVcR/zr4J7KcwCJCQQTW98Jr5gezNnQZ cpgBZ0Mg==; Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbQ-009gWb-MK for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:18 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id s15-20020a0cdc0f0000b02901e9373e40e2so4304233qvk.17 for ; Tue, 11 May 2021 07:43:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=T7VsirMUmcmfR3W7jGLVBbLf6Nqe5HLZuoCOt+ECiBU=; b=ZmJEDh8mZaKupvJuaDkrflyEsS0/19ykAI9k9hQADe2NCMUoS4Qf/RIYaU5h23HOCC OCrWZEWd+QDiiknWgUoW4pwssyaTf96Z+70m9VSNdeHNSRIaDkJQ8BK+FdoheeEDnN7z ATuDHBSdvJC+a08Xv4KX7znXQ9CU2pPIRlcFq7yeXUnpIixWBB1aD5nkRu8/dgnPKVI0 N3j3s0vx6PwPJlPT1Tq4U/WHlnzelJn3QJGC9hAG6cJJiin43+vh354GlocKSDcGj7GE Km9VuiChxLscGCypVd80d3iM5+z65eqE+C+wu7lR6r975tXh9mmoMKqSeQmq98wHS1SJ U/Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=T7VsirMUmcmfR3W7jGLVBbLf6Nqe5HLZuoCOt+ECiBU=; b=SgI2w9GjMT3dcZhzmQIEHwVSnsl56rzWeu6W6V3OfeXuaooKkhJ4AqrZQM1owVGxzu x/GoBwqD/RQVes66hC3V6eBz3bNmnWUEkIQBF0OsZDfHOxR5BLd4qU3SRrdvZhmeI3sV bTlweHiNL5qHc+t9binLv1rPvcazKSbpyMG5yql+CPAnmEffJ5fdqBCd6mUsPaX9Fssv Ks7fQW5jFDgCr85+QGdF5hE6/EDLR8vhfI8PAr2hoyfIzJN1VdLgWJSwlSni7zEgIPgg rsdtqTc2zJmyzWP77Hcn5G/+whLDtsjClbp9iXZxGLMCHCeJG6HLjbuUJWhnHulzgv7e EFsQ== X-Gm-Message-State: AOAM533Qj47gYeT1IhOqeDsGbXCKDE3A4AImgQK47zLvO+3KP1H7w7iq k84Z6gJUVaNqKJ+6s+kPOd8O1YH7Ua9cFBYuergm1/gs6ILalNL5b7+HOgN+jEJWtXs/BKd2f8h iCMR51VtWwoBUob+SGRJaHyyMKJQJxLpAK58Ud23Xp5z2ADLpuFMcurHi8VvyFEdaasb+HDGB/5 v6gA== X-Google-Smtp-Source: ABdhPJwOXQG3Hf0k6Tho5+enpUE1dtGSNbayE1emXYQstpGgbAbIf1p2tXpRDmyMa11xqKTQCUjUYe1+AA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:9d0e:: with SMTP id m14mr29960854qvf.50.1620744194809; Tue, 11 May 2021 07:43:14 -0700 (PDT) Date: Tue, 11 May 2021 15:42:48 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-10-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 09/13] arm64: __clean_dcache_area_pop to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074316_752609_4F1ED2AA X-CRM114-Status: GOOD ( 13.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/lib/uaccess_flushcache.c | 4 ++-- arch/arm64/mm/cache.S | 9 ++++----- arch/arm64/mm/flush.c | 2 +- 4 files changed, 8 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 3255878d6f30..fa5641868d65 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -61,7 +61,7 @@ extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pop(void *addr, size_t len); +extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); extern void sync_icache_aliases(void *kaddr, unsigned long len); diff --git a/arch/arm64/lib/uaccess_flushcache.c b/arch/arm64/lib/uaccess_flushcache.c index c83bb5a4aad2..62ea989effe8 100644 --- a/arch/arm64/lib/uaccess_flushcache.c +++ b/arch/arm64/lib/uaccess_flushcache.c @@ -15,7 +15,7 @@ void memcpy_flushcache(void *dst, const void *src, size_t cnt) * barrier to order the cache maintenance against the memcpy. */ memcpy(dst, src, cnt); - __clean_dcache_area_pop(dst, cnt); + __clean_dcache_area_pop((unsigned long)dst, (unsigned long)dst + cnt); } EXPORT_SYMBOL_GPL(memcpy_flushcache); @@ -33,6 +33,6 @@ unsigned long __copy_user_flushcache(void *to, const void __user *from, rc = raw_copy_from_user(to, from, n); /* See above */ - __clean_dcache_area_pop(to, n - rc); + __clean_dcache_area_pop((unsigned long)to, (unsigned long)to + n - rc); return rc; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 7ddf6ff65b15..f35f28845691 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -209,16 +209,15 @@ SYM_FUNC_END_PI(__clean_dcache_area_poc) SYM_FUNC_END(__dma_clean_area) /* - * __clean_dcache_area_pop(kaddr, size) + * __clean_dcache_area_pop(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoP. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_PI(__clean_dcache_area_pop) - add x1, x0, x1 alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 4e3505c2bea6..5aba7fe42d4b 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -82,7 +82,7 @@ void arch_wb_cache_pmem(void *addr, size_t size) { /* Ensure order against any prior non-cacheable writes */ dmb(osh); - __clean_dcache_area_pop(addr, size); + __clean_dcache_area_pop((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); From patchwork Tue May 11 14:42:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADDA5C433B4 for ; Tue, 11 May 2021 14:47:30 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14EEA61361 for ; Tue, 11 May 2021 14:47:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14EEA61361 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5Cg5nZb4UrHjMmqTtHp3abKA6LytBGMenCdVX8DyC+A=; b=MqAq3DII2tRQnD OoF3ERI+2bRDESavKa5g+oirUBfz3ewQm53IP4kwlmODyiuqFNYHDI46wP1Q5GEx3ExwSZLC3BpMX mhGzsEqEaSfNJzTwJzzRIwMJcQiyy3VRqxmWBotX+FvBTI7PH58qLd265ggsrE6/gYh5B5lpc/2Q8 ymHR44rgg1yLFKGrbbfSH3si/glzByMC/JlVt2sN7I/folOoZNXGW2tJ1X//cRu0cmTSUiD4iU9o2 FYhw6qoV0YphpvUWbC4nb6m3KDlM7CowIgcAO06UY/0SUwk8iYCXEZTS7oXWCQA/OOS6JXayxAtBQ C6RBxVJfpVtJ17v7KA3w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTdt-000U8m-6u; Tue, 11 May 2021 14:45:49 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbV-000TH4-Go for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=aUVmLrAjgkm7TuYinwPWvv/o8H/Z5yvNrHnHRrMUfwI=; b=QB5oI9ARMtuO58h0nmEow2MXbv O2kBooxR+5TGPqg+q7ZhV3N59z32Z+jkarqIu/INchfKCQ+AtP1zlhMV3SnurMhkrE/NfDmhWRlR1 ss/dAGQTGRzO3wcm1G55uON15nqXhqFrck5voZZxErHodrOe6gw9oa9Pecd/3+3BxqzrlVCkyqIqI zV6sA05oaXOFkkMYz8sT4HdVDbYSuhHnO2IEmXShjIQQ31Ybc93tXHAKTDnLzhLlz6iGlCqvy9EfT KLjQh0zpEUsQrULlZgry32+Z+OaWhgCoIvC2i1fa0YY7aUlJQ3seX3Ottz0O0YW0A26SA+gSXEoR3 QLMOz6Ww==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbS-009gWr-Tr for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:20 +0000 Received: by mail-wr1-x449.google.com with SMTP id t5-20020adfb7c50000b029010dd0bb24cfso8850121wre.2 for ; Tue, 11 May 2021 07:43:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=aUVmLrAjgkm7TuYinwPWvv/o8H/Z5yvNrHnHRrMUfwI=; b=h14Wo4Zrgw9PVJ7A12/eWp6QE0ZCUbZD+euc4x2AWmQNGNnMyIItGPkGvTZv9zd1GP BFvVwB+q0jBCYD1X1qGmRhaIRmICo1/L+bOHsZt9aJq2g4SWcIkyFCPCvq+3jYKJLLQy LH9b+EZllyF+J0VHmv7WDwo3ZinEgT83r2N9rSQ4CMJmgUTMw19vAY18JorNzwpIn3Bl CUEFC7uF5XiMNM837ZEs23u3p+vV72C8kiq+9XwYJhKAOLzwwGlgMW41tOIUzwOvywqL tclpSI2tp15Osoc0LDtOf4WA/AEjpshkfp7c7XF6lAeGsnr/j7GM27kjpPChqCI9Ju3i sf0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=aUVmLrAjgkm7TuYinwPWvv/o8H/Z5yvNrHnHRrMUfwI=; b=onw9K87GXRpb3umOd1tP4JxLrfgF4JglG/KVdszlN8g+RcvwK/4A0X1mA727iNPaw/ hJSE112cU1ikWomcv+MtyLHzDwT72HyG+yHgESpnR558+b4ZNj4iQBYF2EH8j2WfpjJt SMXt6cME0vGJBxCCTExiS120ZiiCJ3xJoajmpfbTQpw6X5MFtJZ/CjuBdk/UWsfAbA1Q ZBsxRUDcBFPG33upOQ7i4nJNS+nx4FmDZjmdpE1rBxKjuM83/qMgox2Wfy7EJJ6A3ioS /smntng5K3yIokJm0P4oqoP/33n68CwbMXlb9408w0+cUURhHGh9JZhqbfWCsZKKS5lW 7FIA== X-Gm-Message-State: AOAM532VMtbGYMtYWeXNcEwjUQRZDD6F+ZDVfjzIbQ88RUdL+qYVUZ5e pDVBWTH2tYEOUsqTzpZu5iv34L4tlA8TrCChRrf6wvTK0axJIdzSRJCCUyxFdLprdxCUxWnWbAd JhetQgjs7Vx+yLuJ4oN1iFr2lJbiLEwBv3buus0kKNscMthFyWJsKt8128+biK84fiSLwVStSHa wZrg== X-Google-Smtp-Source: ABdhPJyiumpvsUeMisUOEBScDYLU79ywwsmqO0aF3drfIiWjjMb+9az30NPwE3gwq9iBN6Tck+z/KKqQOg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:adf:efc6:: with SMTP id i6mr39077748wrp.408.1620744196852; Tue, 11 May 2021 07:43:16 -0700 (PDT) Date: Tue, 11 May 2021 15:42:49 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-11-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 10/13] arm64: __clean_dcache_area_pou to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074318_984355_06403590 X-CRM114-Status: GOOD ( 13.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/mm/cache.S | 9 ++++----- arch/arm64/mm/flush.c | 2 +- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index fa5641868d65..f86723047315 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -62,7 +62,7 @@ extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pou(void *addr, size_t len); +extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); extern long __flush_cache_user_range(unsigned long start, unsigned long end); extern void sync_icache_aliases(void *kaddr, unsigned long len); diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index f35f28845691..d8434e57fab3 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -128,20 +128,19 @@ SYM_FUNC_START_PI(__flush_dcache_area) SYM_FUNC_END_PI(__flush_dcache_area) /* - * __clean_dcache_area_pou(kaddr, size) + * __clean_dcache_area_pou(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoU. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START(__clean_dcache_area_pou) alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif - add x1, x0, x1 dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret SYM_FUNC_END(__clean_dcache_area_pou) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 5aba7fe42d4b..a69d745fb1dc 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -19,7 +19,7 @@ void sync_icache_aliases(void *kaddr, unsigned long len) unsigned long addr = (unsigned long)kaddr; if (icache_is_aliasing()) { - __clean_dcache_area_pou(kaddr, len); + __clean_dcache_area_pou(kaddr, kaddr + len); __flush_icache_all(); } else { /* From patchwork Tue May 11 14:42:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28960C433B4 for ; Tue, 11 May 2021 14:48:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 795D161263 for ; Tue, 11 May 2021 14:48:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 795D161263 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KPmWnRecfDsymIEy6H7JhA3CqLbWvBndNhuH7ROs65o=; b=Dy7RxHB9PcNvAj 82+Z9KaQ/4L+Pwr68LkHxq+L7cwpmDHG1SD74GO/gxSzVEoXJ2ruGWJi0oUiMi0SybOo9qwr2VCkn JhEV6tbM/zhkU4T6ZsgB1AEoWqL1lG7R9RQ8ujVK2Y+NaGJfV8vzM8psHZmin0exL27NhUnuiia45 MUSf0TgM9revL0X4Msc8CSQUqmL+184czQcSH27lzjXqNn7MoMa00qvYZ1Oi3A1ZbfLudueRrgwq4 LeYkOPXzhR+3xF6eFEYyYMEJtZjHWTvpEto5wx0m8I5JZX37VAH9qlL17J/J3tHigmNRJ3/XOOPBx xLn/oH4bNGzVG3QStEBw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTeH-000UHt-VY; Tue, 11 May 2021 14:46:14 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbX-000THD-En for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=LllD6PPnuTvw+BG/LkclHvotID4IniI01gePp2bphoI=; b=jSprVUlAip+ESUgJutyrB10P6F CQc9NtakFpztJ13KGqex9yL7x7267z+awlxbFkMGyEx/QUrS0a9wW/TqaWvC/MTZLnZa2GJHTgE+/ PPnuV5BwebpnRWRbg6h+FpSnbADTNu3bg2cYrhNZGczDQL6Iw5KhMwJoiKmMPOS5iTlDSACjmnOsK HR3FgaDsk5jRSmlvKCVZcvP1wMGcTuOFGHIULexBLKnVRqoMhpN7Y827NzoGfOeAisQMxJbCCkynM 1wP2NXIDQbUzn8nqCF0EOgPHc/3Fd73HxNxeMvVaZjPnWZPfs59bLI3OjEEGzJLlmWx43NRuAGCoR c5dkhYZA==; Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbU-009gX9-Rm for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:22 +0000 Received: by mail-wm1-x349.google.com with SMTP id r10-20020a05600c2c4ab029014b601975e1so819301wmg.0 for ; Tue, 11 May 2021 07:43:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=LllD6PPnuTvw+BG/LkclHvotID4IniI01gePp2bphoI=; b=elO1CfYJwh3QlosQDKtHgJxe2Ue0w0P7+IX3Q/23iF7gg+EHiJGfnJsAhJ3z9c+qh/ emigzzkVM1cIPpg+nOIIsINycjuSxLnI2jRxQCyW/jQKBGcGdP7ZFlXuHS2cg9SNLBiW amPLmWG4VQ8duyJV+eQD0miJizDHbBgxW68cRfrxYjD7FJPxqazJVPtEwRvpcgu9jWQK 0Ea9OgBC60MjdbM2trprF1Cc9iTgMY9+FjyVFDCPnJnq7kdxg9TrO+GN64Crpa3Pxmgc OeA8XjXbK+LiBmIa54vE12ul5SFWys499QHwlD7crUeRLlqP26vt91m6m0/fzJySypOO 1h2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=LllD6PPnuTvw+BG/LkclHvotID4IniI01gePp2bphoI=; b=EKP8udFmaKdUn8AznSdr8ujzuoT9p99CO5hlOqZ+zyDEX73zBkgSu0tnWU8CNVNlQN vhLO1xUuYmkQe10dqz4YfemUQAf2E/yZjdInznfkuq99yVraV8sUfYu+DccWLzv/cYLb ZbLOy4ASNKYwwJBbRKgG3Tykh9lwn7qgbcqgEzvYUKp9z16UkQ6sIdKBx9wn82A0xIYc 8mvOKqBlMr+l/QWISasyTaXxqRH1nq8SC0fkUr/fki1bLfYEVQFFoBmMIQa3suJEkWJF /Nfzkb3qkj4wSKBnVFjYaPDT6uwfNgZAMkrwbMm9gzqR/6ictXpF/aZROG2yAcxFpN91 P+RQ== X-Gm-Message-State: AOAM532hLUDxgp8UnqENUAC6D/69LT7esGd/TmAHJsUVF/gITs+vl2nu 1MFKIQ5Gy9SJ2oL7zmSoDhNJNMFI7pQvuRQPYxTOkZl7vPbFz2WBmfUwfZLDBKkjWiNX4Kk+qAS B6G6z8/lRBzo+z0g01VOgKkyh82w7OkVRN0oKYJmIidhuUNH3OiXL3IPJdK/3GWxf2bHceqqdFk Zi1A== X-Google-Smtp-Source: ABdhPJxc0p1nltaziJtRTL/azbiUpYPS/vyGE57KbcEsKqhmzRyaqT/+SQKhbZfy5LDLcIxz5J9/yEg5Hg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c0cb:: with SMTP id s11mr33161403wmh.146.1620744198820; Tue, 11 May 2021 07:43:18 -0700 (PDT) Date: Tue, 11 May 2021 15:42:50 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-12-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 11/13] arm64: sync_icache_aliases to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074320_919784_6ACC02A2 X-CRM114-Status: GOOD ( 14.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/probes/uprobes.c | 2 +- arch/arm64/mm/flush.c | 21 +++++++++++---------- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index f86723047315..70b389a8dea5 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -64,7 +64,7 @@ extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); extern long __flush_cache_user_range(unsigned long start, unsigned long end); -extern void sync_icache_aliases(void *kaddr, unsigned long len); +extern void sync_icache_aliases(unsigned long start, unsigned long end); static inline void flush_icache_range(unsigned long start, unsigned long end) { diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c index 2c247634552b..9be668f3f034 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -21,7 +21,7 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, memcpy(dst, src, len); /* flush caches (dcache/icache) */ - sync_icache_aliases(dst, len); + sync_icache_aliases((unsigned long)dst, (unsigned long)dst + len); kunmap_atomic(xol_page_kaddr); } diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index a69d745fb1dc..143f625e7727 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -14,28 +14,26 @@ #include #include -void sync_icache_aliases(void *kaddr, unsigned long len) +void sync_icache_aliases(unsigned long start, unsigned long end) { - unsigned long addr = (unsigned long)kaddr; - if (icache_is_aliasing()) { - __clean_dcache_area_pou(kaddr, kaddr + len); + __clean_dcache_area_pou(start, end); __flush_icache_all(); } else { /* * Don't issue kick_all_cpus_sync() after I-cache invalidation * for user mappings. */ - __flush_icache_range(addr, addr + len); + __flush_icache_range(start, end); } } static void flush_ptrace_access(struct vm_area_struct *vma, struct page *page, - unsigned long uaddr, void *kaddr, - unsigned long len) + unsigned long uaddr, unsigned long start, + unsigned long end) { if (vma->vm_flags & VM_EXEC) - sync_icache_aliases(kaddr, len); + sync_icache_aliases(start, end); } /* @@ -48,7 +46,8 @@ void copy_to_user_page(struct vm_area_struct *vma, struct page *page, unsigned long len) { memcpy(dst, src, len); - flush_ptrace_access(vma, page, uaddr, dst, len); + flush_ptrace_access(vma, page, uaddr, (unsigned long)dst, + (unsigned long)dst + len); } void __sync_icache_dcache(pte_t pte) @@ -56,7 +55,9 @@ void __sync_icache_dcache(pte_t pte) struct page *page = pte_page(pte); if (!test_and_set_bit(PG_dcache_clean, &page->flags)) - sync_icache_aliases(page_address(page), page_size(page)); + sync_icache_aliases((unsigned long)page_address(page), + (unsigned long)page_address(page) + + page_size(page)); } EXPORT_SYMBOL_GPL(__sync_icache_dcache); From patchwork Tue May 11 14:42:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CDEA8C433B4 for ; Tue, 11 May 2021 14:48:29 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2DBE161263 for ; Tue, 11 May 2021 14:48:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2DBE161263 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=cQlMMsoxRrFKmBl4TkSxBydP1n/ddkMtBR9GqApkW40=; b=HiSLtKeLKHp27l Rooq2Blfdy0zQPnAGUaZqvDoV73BjAFdTvA/fdKT6jtSLMd1ShcMFkADfn0FaGhYMl4YEnTiAb2SS sVLAA8IzqI1PzReTq2aYjT5Q3VmK4zbo8kv/pigqIaI44iUgVy0tiSV7d+RWa3EbR9kIQVPz0iEga bFq+dRh/nFptodV4RzVomia6Vegz+UkNwxZHDAq9dmBtasGdEVnKMctAWDWmkq10HZLpq6KJ6IcPb k4oYS6AaVl9vtXa1Yj9hKtiKhHfyPDVXma2MWEzNq6+h5EtodAr+aUphP6NaK3QsudO2pgKBEFwyJ OEilwcbDIvrzAN578Y4A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTem-000UUE-MT; Tue, 11 May 2021 14:46:45 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbZ-000THu-2u for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:25 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=w+SJNOEqyTo2ibRFXPy7dTBPGWZnvW3GLChijud2oBg=; b=o31y7K2y9qC6kebbXakDHQDFd5 879EMkL5rGT/8WXPYQw+QHRnUxEW0U2wxJeZhM4cDzWwC0xmG5wDg8duCaCPdQBcHQ/LfuDQAAdDq 8vv5qm7OpUbaz9jqAwQLQ554Nx5cArEMR76Kc7rYgfJrbe16KH7mtp2TJPa+2H3a3ppY+3HIiGLIp 4GH62nOFFLqlDmqp9CQsIWluHxQWCtwQ5pleu6OldyH3a5gzfQQb0L3zZUsRqwYkjCS9Gl4C2wAhV V/l88tJdIbKnCl2lzUQtIANDcMF95nzI/53DPlPuy+R8SMBCCQK1Mkkl1yWq6EXfkbnf8yn0TS3aX 9v8VH40g==; Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbW-009gXP-HW for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:23 +0000 Received: by mail-qv1-xf49.google.com with SMTP id f20-20020a0caa940000b02901c5058e5813so15684240qvb.23 for ; Tue, 11 May 2021 07:43:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=w+SJNOEqyTo2ibRFXPy7dTBPGWZnvW3GLChijud2oBg=; b=BLpDZ80JomTqVO56UVs8uvFvd/x5AMo6ChHbOAcnlMv4T1APxm6ubvv1bd39yuLI/W ZD+TM0bHfQgZZSp4ChtiexbL8c+jgtyrmK5T3LFlTwR5lKY7Plb6ke5StjhbckQr+FaB oFIly+dcq1FlDJCQVtUH2gSVYYe/6/xS2SVhRDeOSJICKsmIg1r+8/d8asADvEE2KVrY 6iEJSadR29tbYFotx1iTgBEH+FDvIy3aq27i/Dk4Da/RXTny6H8R+p8CP8kbrjfci0D5 iA6BUZ+9QAkeGEs/T3oiKD4wxSrvzGUxtLmyu8xyBsiXjR15mG85ppPjhEpITmEULP6x 4uTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=w+SJNOEqyTo2ibRFXPy7dTBPGWZnvW3GLChijud2oBg=; b=pfOFSFs9zy7qcH24dhkycrc8NB1hpRBaHF7vg+zasdAZBUK3s7C3yW0AHjGbT4cOFo lIHY50w6DbxcuyEE28wN1WAkNAlNioDVL36PwRTYdOCy5aqNnb6Jn4qJBM59plUwtZOZ vat1k5cDtkm4bHsvg2MyX/4Pq5k0X2vbfxuXm7BIyoFuQaGLeOXAIlwzB4uG5C/IhPJ0 GfPUO0+q9aqOJ283zuQ4OTk4bjMBIAW4nFXX3W+Hs3eFtFazLwUuH1zG7ymmAA8wgkcw XqCSa0TsRk9zEmxhYb6ScBiPehzr7FDFljabhpcDbCb025klJfGm9SIzrxC6XeUZIHsV 45sA== X-Gm-Message-State: AOAM532mBLW0CGrPBxYIGn56ydi+yAa2XEd5j3cJXwH9evB8bqwukx9T LSpSAMwjLFbzYKDbRyMltjV/9eO8ggEQ1dtB69W3KkU9Ic0fzvIs729VCHfk2aLofmlLXjmMmBD qfuCOdWwuu5S9+uyFmgl0veBZk1cNGqWYA/179Ek5MiH6AgP4twVpQV6H0QkjWCPN0qmNNpz/gP 5lNw== X-Google-Smtp-Source: ABdhPJy0Qwp2NlYTajA16dety4GmON5jUPMr0Rwx71KHnZMxXWefN08W6wEx+Y4SPbn8NGfEom/zhWYKFw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:6214:951:: with SMTP id dn17mr29937185qvb.27.1620744200642; Tue, 11 May 2021 07:43:20 -0700 (PDT) Date: Tue, 11 May 2021 15:42:51 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-13-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 12/13] arm64: Fix cache maintenance function comments From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074322_600453_CA855924 X-CRM114-Status: GOOD ( 11.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Fix and expand comments for the cache maintenance functions in cacheflush.h. Adds comments to functions that weren't described before. Explains what the functions do using Arm Architecture Reference Manual terminology. Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 43 +++++++++++++++++++---------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 70b389a8dea5..4b91d3530013 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -30,31 +30,44 @@ * the implementation assumes non-aliasing VIPT D-cache and (aliasing) * VIPT I-cache. * - * flush_icache_range(start, end) - * - * Ensure coherency between the I-cache and the D-cache in the - * region described by start, end. + * All functions below apply to the region described by [start, end) * - start - virtual start address * - end - virtual end address * - * invalidate_icache_range(start, end) + * __flush_icache_range(start, end) * - * Invalidate the I-cache in the region described by start, end. - * - start - virtual start address - * - end - virtual end address + * Ensure coherency between the I-cache and the D-cache region to + * the Point of Unification. * * __flush_cache_user_range(start, end) * - * Ensure coherency between the I-cache and the D-cache in the - * region described by start, end. - * - start - virtual start address - * - end - virtual end address + * Ensure coherency between the I-cache and the D-cache region to + * the Point of Unification. + * Use only if the region might access user memory. + * + * invalidate_icache_range(start, end) + * + * Invalidate I-cache region to the Point of Unification. * * __flush_dcache_area(start, end) * - * Ensure that the data held in page is written back. - * - start - virtual start address - * - end - virtual end address + * Clean and invalidate D-cache region to the Point of Coherence. + * + * __inval_dcache_area(start, end) + * + * Invalidate D-cache region to the Point of Coherence. + * + * __clean_dcache_area_poc(start, end) + * + * Clean D-cache region to the Point of Coherence. + * + * __clean_dcache_area_pop(start, end) + * + * Clean D-cache region to the Point of Persistence. + * + * __clean_dcache_area_pou(start, end) + * + * Clean D-cache region to the Point of Unification. */ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); From patchwork Tue May 11 14:42:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12251283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D3BFC433ED for ; Tue, 11 May 2021 14:49:16 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9829E61364 for ; Tue, 11 May 2021 14:49:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9829E61364 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=azjp4js+BACZn8hvaXiM4tVewhTxuSDVzlHA7UBgnZM=; b=ivcpirLWA5ORcT NLkupI21qRaLI4aRvREED8pZn/mEb8DD6fQmUG5ZYM9mRYIyNQqM7kBmMXyTxU/XTyN9mi7+9Bnce fnp0sPiM8D3NNn2KKXmPRSvoPY42M4HtnjDulXrqOS2Dd0bh2Vxbnk2LJqckKlEQ1+eLdJyTMPNut qVzkc7uydT2wwUb0W2oL+DYEX3GpDuMCYsoB23M7ByriFOWABbUY4gPUpCyRQYmrurpREUEdiHL0R iq5hwJNvb7t4gwaiTClotx23Xac7lmPRthPipfyNE7Ggfnng+odvUoht4oq5X0FVuI/XUTqbqMv7h ghZpTISuffUW5o37xpnw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lgTfH-000UhV-Je; Tue, 11 May 2021 14:47:16 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbd-000TIy-HT for linux-arm-kernel@desiato.infradead.org; Tue, 11 May 2021 14:43:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Dn0/Dn3ggPaFtr+zRgDZdKfonp4+5vLLfU/OPwH8/Z0=; b=YSaIYqfyyvokhVKqUVoFs0tvZh m1aH3wA4aSVlLvv2ZMqqvAeaPss4+BLbwsUgVE1AzcUWvbDkzuKFeFSoETobfMPx7pBxVX7bUifk3 SfCvQHebiPN9Q9OAtTD70m/B5CvoKA6NsXU5ZWlvbWmrakKCFtPykZlTe6v0310tHW/HAeeISxa/o vVOjlHwW2p0eblCCak5D0t3QC8+vFONv+wuPMkGtttK1LV1V4Vh9DKei1aqgYMsCQAnFKuvEHELxp ZdPDARMGJ3J+U66DMa2RygFE9paemnAXeXsciseiuGXoK7f208n/wXqxCwos5dPq59TjHZdZc8wD+ VSAplImg==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lgTbZ-009gXh-6f for linux-arm-kernel@lists.infradead.org; Tue, 11 May 2021 14:43:28 +0000 Received: by mail-wr1-x44a.google.com with SMTP id c13-20020a5d6ccd0000b029010ec741b84bso178899wrc.23 for ; Tue, 11 May 2021 07:43:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Dn0/Dn3ggPaFtr+zRgDZdKfonp4+5vLLfU/OPwH8/Z0=; b=U1LEG7HwdphRX0oMKySJb/emCzWsHtAF5HSL+aM2VLGRQv63lmilZP+w1TtACOECd3 8p34AvT7/Lk6oaM0pJj/NMYGmD6/cZOwLOc6/W6pasLMj045k8yj4Zp8/z2lHbmkILTj GaG7CbcHM2ELz4BvFJsif6RMwCewW87NnHvZAXsjfmNFFrQvEp/Q5cP2dO9dYXBk2kFg wu2lhlbYnPVi/j0iDrNSxIG1xu9JB2NmmWO7b7SKCgY///1E+OOYLnLy1Erc9XrdJgYo OzPUyj0Ip4JAJMZ8kfoph87A9/Sk+JCtDpA7NYRUaLG6RA4pXlyeqXj6axrhlaD8K6eP 6jVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Dn0/Dn3ggPaFtr+zRgDZdKfonp4+5vLLfU/OPwH8/Z0=; b=YEITZkuhrW6wmcL0SVm31d3wGs49Y4iUdIS2fVN4n+c2o41xDLoIj53CQixcFE0eHm XV7bYZBB2OFpxVWEWjBu1C6SGxzIkL3pLhBJ5sWZHb9YcopVrWr5zWVr5QlnVgrQUqaj bTE6ismn8jA0pXiW6aLMPL2M4Z+tUwd89xwHU2QzZlF+IXaK8S8iUViKUcSP2MQQM1Yp 0eEsbx6UieBncGWT0WImzBUhW4cK2GJPnpnNgWrYbNnvjmIwz1o32phdlLIKr/XKnXwd SvePQdWtjq/U4YPpg4decZl6jDmvUkapevC2bC8/kvjltovR0Mzmiv0vhDhBVphaSvsU +3Ng== X-Gm-Message-State: AOAM531nsfydVGXF3MVMipXvWld1Az7pNxdnWZq+DKf5TJG3RlBHcX3R 1EqpC6hhsBmjExLI3s/14HTtSVfUQ32Ti8D2PjcuPsYsc/cwUYilQETHXaFPflIcPkAg+98Wtl2 jhVveOYeXF32jV5wqF0TmYGSzvFjNq2X1g9yr5zX78hLXfWqb7b/RVpQZf+W86nH9xgwwKzYK/d 2OiA== X-Google-Smtp-Source: ABdhPJw0ybsPixIrfO946pjveBEFDeJOC9Aa18SPQaye9gXOA5P8fGL/RCT9whgsdzNSU55Zg7zZffVpGQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:cb0b:: with SMTP id u11mr351668wmj.0.1620744202697; Tue, 11 May 2021 07:43:22 -0700 (PDT) Date: Tue, 11 May 2021 15:42:52 +0100 In-Reply-To: <20210511144252.3779113-1-tabba@google.com> Message-Id: <20210511144252.3779113-14-tabba@google.com> Mime-Version: 1.0 References: <20210511144252.3779113-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.607.g51e8a6a459-goog Subject: [PATCH v1 13/13] arm64: Rename arm64-internal cache maintenance functions From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210511_074325_290679_A8AC68DD X-CRM114-Status: GOOD ( 22.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Although naming across the codebase isn't that consistent, it tends to follow certain patterns. Moreover, the term "flush" isn't defined in the Arm Architecture reference manual, and might be interpreted to mean clean, invalidate, or both for a cache. Rename arm64-internal functions to make the naming internally consistent, as well as making it consistent with the Arm ARM, by clarifying whether the operation is a clean, invalidate, or both. Also specify which point the operation applies two, i.e., to the point of unification (PoU), coherence (PoC), or persistence (PoP). This commit applies the following sed transformation to all files under arch/arm64: "s/\b__flush_cache_range\b/__clean_inval_cache_pou_macro/g;"\ "s/\b__flush_icache_range\b/__clean_inval_cache_pou/g;"\ "s/\binvalidate_icache_range\b/__inval_icache_pou/g;"\ "s/\b__flush_dcache_area\b/__clean_inval_dcache_poc/g;"\ "s/\b__inval_dcache_area\b/__inval_dcache_poc/g;"\ "s/__clean_dcache_area_poc\b/__clean_dcache_poc/g;"\ "s/\b__clean_dcache_area_pop\b/__clean_dcache_pop/g;"\ "s/\b__clean_dcache_area_pou\b/__clean_dcache_pou/g;"\ "s/\b__flush_cache_user_range\b/__clean_inval_cache_user_pou/g;"\ "s/\b__flush_icache_all\b/__clean_inval_all_icache_pou/g;" Note that __clean_dcache_area_poc is deliberately missing a word boundary check to match the efistub symbols in image-vars.h. No functional change intended. Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/arch_gicv3.h | 2 +- arch/arm64/include/asm/cacheflush.h | 36 +++++++++---------- arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 6 ++-- arch/arm64/kernel/alternative.c | 2 +- arch/arm64/kernel/efi-entry.S | 4 +-- arch/arm64/kernel/head.S | 8 ++--- arch/arm64/kernel/hibernate.c | 12 +++---- arch/arm64/kernel/idreg-override.c | 2 +- arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kernel/insn.c | 2 +- arch/arm64/kernel/kaslr.c | 6 ++-- arch/arm64/kernel/machine_kexec.c | 10 +++--- arch/arm64/kernel/smp.c | 4 +-- arch/arm64/kernel/smp_spin_table.c | 4 +-- arch/arm64/kernel/sys_compat.c | 2 +- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/cache.S | 4 +-- arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 2 +- arch/arm64/kvm/hyp/pgtable.c | 4 +-- arch/arm64/lib/uaccess_flushcache.c | 4 +-- arch/arm64/mm/cache.S | 56 ++++++++++++++--------------- arch/arm64/mm/flush.c | 12 +++---- 24 files changed, 95 insertions(+), 95 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index ed1cc9d8e6df..4b7ac9098e8f 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -125,7 +125,7 @@ static inline u32 gic_read_rpr(void) #define gic_write_lpir(v, c) writeq_relaxed(v, c) #define gic_flush_dcache_to_poc(a,l) \ - __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) + __clean_inval_dcache_poc((unsigned long)(a), (unsigned long)(a)+(l)) #define gits_read_baser(c) readq_relaxed(c) #define gits_write_baser(v, c) writeq_relaxed(v, c) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 4b91d3530013..526eee4522eb 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -34,54 +34,54 @@ * - start - virtual start address * - end - virtual end address * - * __flush_icache_range(start, end) + * __clean_inval_cache_pou(start, end) * * Ensure coherency between the I-cache and the D-cache region to * the Point of Unification. * - * __flush_cache_user_range(start, end) + * __clean_inval_cache_user_pou(start, end) * * Ensure coherency between the I-cache and the D-cache region to * the Point of Unification. * Use only if the region might access user memory. * - * invalidate_icache_range(start, end) + * __inval_icache_pou(start, end) * * Invalidate I-cache region to the Point of Unification. * - * __flush_dcache_area(start, end) + * __clean_inval_dcache_poc(start, end) * * Clean and invalidate D-cache region to the Point of Coherence. * - * __inval_dcache_area(start, end) + * __inval_dcache_poc(start, end) * * Invalidate D-cache region to the Point of Coherence. * - * __clean_dcache_area_poc(start, end) + * __clean_dcache_poc(start, end) * * Clean D-cache region to the Point of Coherence. * - * __clean_dcache_area_pop(start, end) + * __clean_dcache_pop(start, end) * * Clean D-cache region to the Point of Persistence. * - * __clean_dcache_area_pou(start, end) + * __clean_dcache_pou(start, end) * * Clean D-cache region to the Point of Unification. */ -extern void __flush_icache_range(unsigned long start, unsigned long end); -extern void invalidate_icache_range(unsigned long start, unsigned long end); -extern void __flush_dcache_area(unsigned long start, unsigned long end); -extern void __inval_dcache_area(unsigned long start, unsigned long end); -extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); -extern long __flush_cache_user_range(unsigned long start, unsigned long end); +extern void __clean_inval_cache_pou(unsigned long start, unsigned long end); +extern void __inval_icache_pou(unsigned long start, unsigned long end); +extern void __clean_inval_dcache_poc(unsigned long start, unsigned long end); +extern void __inval_dcache_poc(unsigned long start, unsigned long end); +extern void __clean_dcache_poc(unsigned long start, unsigned long end); +extern void __clean_dcache_pop(unsigned long start, unsigned long end); +extern void __clean_dcache_pou(unsigned long start, unsigned long end); +extern long __clean_inval_cache_user_pou(unsigned long start, unsigned long end); extern void sync_icache_aliases(unsigned long start, unsigned long end); static inline void flush_icache_range(unsigned long start, unsigned long end) { - __flush_icache_range(start, end); + __clean_inval_cache_pou(start, end); /* * IPI all online CPUs so that they undergo a context synchronization @@ -135,7 +135,7 @@ extern void copy_to_user_page(struct vm_area_struct *, struct page *, #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1 extern void flush_dcache_page(struct page *); -static __always_inline void __flush_icache_all(void) +static __always_inline void __clean_inval_all_icache_pou(void) { if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC)) return; diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 0ae2397076fd..d1e2a4bf8def 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -137,7 +137,7 @@ void efi_virtmap_unload(void); static inline void efi_capsule_flush_cache_range(void *addr, int size) { - __flush_dcache_area((unsigned long)addr, (unsigned long)addr + size); + __clean_inval_dcache_poc((unsigned long)addr, (unsigned long)addr + size); } #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 33293d5855af..29d2aa6f3940 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -181,7 +181,7 @@ static inline void *__kvm_vector_slot2addr(void *base, struct kvm; #define kvm_flush_dcache_to_poc(a,l) \ - __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) + __clean_inval_dcache_poc((unsigned long)(a), (unsigned long)(a)+(l)) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { @@ -209,12 +209,12 @@ static inline void __invalidate_icache_guest_page(kvm_pfn_t pfn, { if (icache_is_aliasing()) { /* any kind of VIPT cache */ - __flush_icache_all(); + __clean_inval_all_icache_pou(); } else if (is_kernel_in_hyp_mode() || !icache_is_vpipt()) { /* PIPT or VPIPT at EL2 (see comment in __kvm_tlb_flush_vmid_ipa) */ void *va = page_address(pfn_to_page(pfn)); - invalidate_icache_range((unsigned long)va, + __inval_icache_pou((unsigned long)va, (unsigned long)va + size); } } diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index c906d20c7b52..ea2d52fa9a0c 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -181,7 +181,7 @@ static void __nocfi __apply_alternatives(struct alt_region *region, bool is_modu */ if (!is_module) { dsb(ish); - __flush_icache_all(); + __clean_inval_all_icache_pou(); isb(); /* Ignore ARM64_CB bit from feature mask */ diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 72e6a580290a..230506f460ec 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -29,7 +29,7 @@ SYM_CODE_START(efi_enter_kernel) */ ldr w1, =kernel_size add x1, x0, x1 - bl __clean_dcache_area_poc + bl __clean_dcache_poc ic ialluis /* @@ -38,7 +38,7 @@ SYM_CODE_START(efi_enter_kernel) */ adr x0, 0f adr x1, 3f - bl __clean_dcache_area_poc + bl __clean_dcache_poc 0: /* Turn off Dcache and MMU */ mrs x0, CurrentEL diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 8df0ac8d9123..ea0447c5010a 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -118,7 +118,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) // MMU off add x1, x0, #0x20 // 4 x 8 bytes - b __inval_dcache_area // tail call + b __inval_dcache_poc // tail call SYM_CODE_END(preserve_boot_args) /* @@ -268,7 +268,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) */ adrp x0, init_pg_dir adrp x1, init_pg_end - bl __inval_dcache_area + bl __inval_dcache_poc /* * Clear the init page tables. @@ -381,11 +381,11 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x0, idmap_pg_dir adrp x1, idmap_pg_end - bl __inval_dcache_area + bl __inval_dcache_poc adrp x0, init_pg_dir adrp x1, init_pg_end - bl __inval_dcache_area + bl __inval_dcache_poc ret x28 SYM_FUNC_END(__create_page_tables) diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b40ddce71507..ec871b24fd5b 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -210,7 +210,7 @@ static int create_safe_exec_page(void *src_start, size_t length, return -ENOMEM; memcpy(page, src_start, length); - __flush_icache_range((unsigned long)page, (unsigned long)page + length); + __clean_inval_cache_pou((unsigned long)page, (unsigned long)page + length); rc = trans_pgd_idmap_page(&trans_info, &trans_ttbr0, &t0sz, page); if (rc) return rc; @@ -381,17 +381,17 @@ int swsusp_arch_suspend(void) ret = swsusp_save(); } else { /* Clean kernel core startup/idle code to PoC*/ - __flush_dcache_area((unsigned long)__mmuoff_data_start, + __clean_inval_dcache_poc((unsigned long)__mmuoff_data_start, (unsigned long)__mmuoff_data_end); - __flush_dcache_area((unsigned long)__idmap_text_start, + __clean_inval_dcache_poc((unsigned long)__idmap_text_start, (unsigned long)__idmap_text_end); /* Clean kvm setup code to PoC? */ if (el2_reset_needed()) { - __flush_dcache_area( + __clean_inval_dcache_poc( (unsigned long)__hyp_idmap_text_start, (unsigned long)__hyp_idmap_text_end); - __flush_dcache_area((unsigned long)__hyp_text_start, + __clean_inval_dcache_poc((unsigned long)__hyp_text_start, (unsigned long)__hyp_text_end); } @@ -477,7 +477,7 @@ int swsusp_arch_resume(void) * The hibernate exit text contains a set of el2 vectors, that will * be executed at el2 with the mmu off in order to reload hyp-stub. */ - __flush_dcache_area((unsigned long)hibernate_exit, + __clean_inval_dcache_poc((unsigned long)hibernate_exit, (unsigned long)hibernate_exit + exit_size); /* diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index 3dd515baf526..6b4b5727f2db 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -237,7 +237,7 @@ asmlinkage void __init init_feature_override(void) for (i = 0; i < ARRAY_SIZE(regs); i++) { if (regs[i]->override) - __flush_dcache_area((unsigned long)regs[i]->override, + __clean_inval_dcache_poc((unsigned long)regs[i]->override, (unsigned long)regs[i]->override + sizeof(*regs[i]->override)); } diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index bcf3c2755370..14beda6a573d 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -35,7 +35,7 @@ __efistub_strnlen = __pi_strnlen; __efistub_strcmp = __pi_strcmp; __efistub_strncmp = __pi_strncmp; __efistub_strrchr = __pi_strrchr; -__efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc; +__efistub___clean_dcache_poc = __pi___clean_dcache_poc; #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) __efistub___memcpy = __pi_memcpy; diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c index 6c0de2f60ea9..11c7be09e305 100644 --- a/arch/arm64/kernel/insn.c +++ b/arch/arm64/kernel/insn.c @@ -198,7 +198,7 @@ int __kprobes aarch64_insn_patch_text_nosync(void *addr, u32 insn) ret = aarch64_insn_write(tp, insn); if (ret == 0) - __flush_icache_range((uintptr_t)tp, + __clean_inval_cache_pou((uintptr_t)tp, (uintptr_t)tp + AARCH64_INSN_SIZE); return ret; diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 49cccd03cb37..038a4cc7de93 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -72,7 +72,7 @@ u64 __init kaslr_early_init(void) * we end up running with module randomization disabled. */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - __flush_dcache_area((unsigned long)&module_alloc_base, + __clean_inval_dcache_poc((unsigned long)&module_alloc_base, (unsigned long)&module_alloc_base + sizeof(module_alloc_base)); @@ -172,10 +172,10 @@ u64 __init kaslr_early_init(void) module_alloc_base += (module_range * (seed & ((1 << 21) - 1))) >> 21; module_alloc_base &= PAGE_MASK; - __flush_dcache_area((unsigned long)&module_alloc_base, + __clean_inval_dcache_poc((unsigned long)&module_alloc_base, (unsigned long)&module_alloc_base + sizeof(module_alloc_base)); - __flush_dcache_area((unsigned long)&memstart_offset_seed, + __clean_inval_dcache_poc((unsigned long)&memstart_offset_seed, (unsigned long)&memstart_offset_seed + sizeof(memstart_offset_seed)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 4cada9000acf..0e20a789b03e 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -69,10 +69,10 @@ int machine_kexec_post_load(struct kimage *kimage) kexec_image_info(kimage); /* Flush the reloc_code in preparation for its execution. */ - __flush_dcache_area((unsigned long)reloc_code, + __clean_inval_dcache_poc((unsigned long)reloc_code, (unsigned long)reloc_code + arm64_relocate_new_kernel_size); - invalidate_icache_range((uintptr_t)reloc_code, + __inval_icache_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); @@ -108,7 +108,7 @@ static void kexec_list_flush(struct kimage *kimage) unsigned long addr; /* flush the list entries. */ - __flush_dcache_area((unsigned long)entry, + __clean_inval_dcache_poc((unsigned long)entry, (unsigned long)entry + sizeof(kimage_entry_t)); @@ -125,7 +125,7 @@ static void kexec_list_flush(struct kimage *kimage) break; case IND_SOURCE: /* flush the source pages. */ - __flush_dcache_area(addr, addr + PAGE_SIZE); + __clean_inval_dcache_poc(addr, addr + PAGE_SIZE); break; case IND_DESTINATION: break; @@ -152,7 +152,7 @@ static void kexec_segment_flush(const struct kimage *kimage) kimage->segment[i].memsz, kimage->segment[i].memsz / PAGE_SIZE); - __flush_dcache_area( + __clean_inval_dcache_poc( (unsigned long)phys_to_virt(kimage->segment[i].mem), (unsigned long)phys_to_virt(kimage->segment[i].mem) + kimage->segment[i].memsz); diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 5fcdee331087..2044210ed15a 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -122,7 +122,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) secondary_data.task = idle; secondary_data.stack = task_stack_page(idle) + THREAD_SIZE; update_cpu_boot_status(CPU_MMU_OFF); - __flush_dcache_area((unsigned long)&secondary_data, + __clean_inval_dcache_poc((unsigned long)&secondary_data, (unsigned long)&secondary_data + sizeof(secondary_data)); @@ -145,7 +145,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) pr_crit("CPU%u: failed to come online\n", cpu); secondary_data.task = NULL; secondary_data.stack = NULL; - __flush_dcache_area((unsigned long)&secondary_data, + __clean_inval_dcache_poc((unsigned long)&secondary_data, (unsigned long)&secondary_data + sizeof(secondary_data)); status = READ_ONCE(secondary_data.status); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index 58d804582a35..a946ccb9791e 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -36,7 +36,7 @@ static void write_pen_release(u64 val) unsigned long size = sizeof(secondary_holding_pen_release); secondary_holding_pen_release = val; - __flush_dcache_area((unsigned long)start, (unsigned long)start + size); + __clean_inval_dcache_poc((unsigned long)start, (unsigned long)start + size); } @@ -90,7 +90,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(pa_holding_pen, release_addr); - __flush_dcache_area((__force unsigned long)release_addr, + __clean_inval_dcache_poc((__force unsigned long)release_addr, (__force unsigned long)release_addr + sizeof(*release_addr)); diff --git a/arch/arm64/kernel/sys_compat.c b/arch/arm64/kernel/sys_compat.c index 265fe3eb1069..fdd415f8d841 100644 --- a/arch/arm64/kernel/sys_compat.c +++ b/arch/arm64/kernel/sys_compat.c @@ -41,7 +41,7 @@ __do_compat_cache_op(unsigned long start, unsigned long end) dsb(ish); } - ret = __flush_cache_user_range(start, start + chunk); + ret = __clean_inval_cache_user_pou(start, start + chunk); if (ret) return ret; diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1cb39c0803a4..edeca89405ff 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1064,7 +1064,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) stage2_unmap_vm(vcpu->kvm); else - __flush_icache_all(); + __clean_inval_all_icache_pou(); } vcpu_reset_hcr(vcpu); diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 36cef6915428..a906dd596e66 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -7,7 +7,7 @@ #include #include -SYM_FUNC_START_PI(__flush_dcache_area) +SYM_FUNC_START_PI(__clean_inval_dcache_poc) dcache_by_line_op civac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__flush_dcache_area) +SYM_FUNC_END_PI(__clean_inval_dcache_poc) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 5dffe928f256..a16719f5068d 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -134,7 +134,7 @@ static void update_nvhe_init_params(void) for (i = 0; i < hyp_nr_cpus; i++) { params = per_cpu_ptr(&kvm_init_params, i); params->pgd_pa = __hyp_pa(pkvm_pgtable.pgd); - __flush_dcache_area((unsigned long)params, + __clean_inval_dcache_poc((unsigned long)params, (unsigned long)params + sizeof(*params)); } } diff --git a/arch/arm64/kvm/hyp/nvhe/tlb.c b/arch/arm64/kvm/hyp/nvhe/tlb.c index 83dc3b271bc5..184c9c7c13bd 100644 --- a/arch/arm64/kvm/hyp/nvhe/tlb.c +++ b/arch/arm64/kvm/hyp/nvhe/tlb.c @@ -104,7 +104,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, * you should be running with VHE enabled. */ if (icache_is_vpipt()) - __flush_icache_all(); + __clean_inval_all_icache_pou(); __tlb_switch_to_host(&cxt); } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 10d2f04013d4..fb2613f458de 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -841,7 +841,7 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (need_flush) { kvm_pte_t *pte_follow = kvm_pte_follow(pte, mm_ops); - __flush_dcache_area((unsigned long)pte_follow, + __clean_inval_dcache_poc((unsigned long)pte_follow, (unsigned long)pte_follow + kvm_granule_size(level)); } @@ -997,7 +997,7 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, return 0; pte_follow = kvm_pte_follow(pte, mm_ops); - __flush_dcache_area((unsigned long)pte_follow, + __clean_inval_dcache_poc((unsigned long)pte_follow, (unsigned long)pte_follow + kvm_granule_size(level)); return 0; diff --git a/arch/arm64/lib/uaccess_flushcache.c b/arch/arm64/lib/uaccess_flushcache.c index 62ea989effe8..b1a6d9823864 100644 --- a/arch/arm64/lib/uaccess_flushcache.c +++ b/arch/arm64/lib/uaccess_flushcache.c @@ -15,7 +15,7 @@ void memcpy_flushcache(void *dst, const void *src, size_t cnt) * barrier to order the cache maintenance against the memcpy. */ memcpy(dst, src, cnt); - __clean_dcache_area_pop((unsigned long)dst, (unsigned long)dst + cnt); + __clean_dcache_pop((unsigned long)dst, (unsigned long)dst + cnt); } EXPORT_SYMBOL_GPL(memcpy_flushcache); @@ -33,6 +33,6 @@ unsigned long __copy_user_flushcache(void *to, const void __user *from, rc = raw_copy_from_user(to, from, n); /* See above */ - __clean_dcache_area_pop((unsigned long)to, (unsigned long)to + n - rc); + __clean_dcache_pop((unsigned long)to, (unsigned long)to + n - rc); return rc; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index d8434e57fab3..2df7212de799 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -15,7 +15,7 @@ #include /* - * __flush_cache_range(start,end) [needs_uaccess] + * __clean_inval_cache_pou_macro(start,end) [needs_uaccess] * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -25,7 +25,7 @@ * - end - virtual end address of region * - needs_uaccess - (macro parameter) might access user space memory */ -.macro __flush_cache_range, needs_uaccess +.macro __clean_inval_cache_pou_macro, needs_uaccess .if \needs_uaccess uaccess_ttbr0_enable x2, x3, x4 .endif @@ -77,12 +77,12 @@ alternative_else_nop_endif * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__flush_icache_range) - __flush_cache_range needs_uaccess=0 -SYM_FUNC_END(__flush_icache_range) +SYM_FUNC_START(__clean_inval_cache_pou) + __clean_inval_cache_pou_macro needs_uaccess=0 +SYM_FUNC_END(__clean_inval_cache_pou) /* - * __flush_cache_user_range(start,end) + * __clean_inval_cache_user_pou(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -91,19 +91,19 @@ SYM_FUNC_END(__flush_icache_range) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__flush_cache_user_range) - __flush_cache_range needs_uaccess=1 -SYM_FUNC_END(__flush_cache_user_range) +SYM_FUNC_START(__clean_inval_cache_user_pou) + __clean_inval_cache_pou_macro needs_uaccess=1 +SYM_FUNC_END(__clean_inval_cache_user_pou) /* - * invalidate_icache_range(start,end) + * __inval_icache_pou(start,end) * * Ensure that the I cache is invalid within specified region. * * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(invalidate_icache_range) +SYM_FUNC_START(__inval_icache_pou) alternative_if ARM64_HAS_CACHE_DIC isb ret @@ -111,10 +111,10 @@ alternative_else_nop_endif invalidate_icache_by_line x0, x1, x2, x3, 0, 0f ret -SYM_FUNC_END(invalidate_icache_range) +SYM_FUNC_END(__inval_icache_pou) /* - * __flush_dcache_area(start, end) + * __clean_inval_dcache_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned and invalidated to the PoC. @@ -122,13 +122,13 @@ SYM_FUNC_END(invalidate_icache_range) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START_PI(__flush_dcache_area) +SYM_FUNC_START_PI(__clean_inval_dcache_poc) dcache_by_line_op civac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__flush_dcache_area) +SYM_FUNC_END_PI(__clean_inval_dcache_poc) /* - * __clean_dcache_area_pou(start, end) + * __clean_dcache_pou(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoU. @@ -136,17 +136,17 @@ SYM_FUNC_END_PI(__flush_dcache_area) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__clean_dcache_area_pou) +SYM_FUNC_START(__clean_dcache_pou) alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret -SYM_FUNC_END(__clean_dcache_area_pou) +SYM_FUNC_END(__clean_dcache_pou) /* - * __inval_dcache_area(start, end) + * __inval_dcache_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are invalidated. Any partial lines at the ends of the interval are @@ -156,7 +156,7 @@ SYM_FUNC_END(__clean_dcache_area_pou) * - end - kernel end address of region */ SYM_FUNC_START_LOCAL(__dma_inv_area) -SYM_FUNC_START_PI(__inval_dcache_area) +SYM_FUNC_START_PI(__inval_dcache_poc) /* FALLTHROUGH */ /* @@ -181,11 +181,11 @@ SYM_FUNC_START_PI(__inval_dcache_area) b.lo 2b dsb sy ret -SYM_FUNC_END_PI(__inval_dcache_area) +SYM_FUNC_END_PI(__inval_dcache_poc) SYM_FUNC_END(__dma_inv_area) /* - * __clean_dcache_area_poc(start, end) + * __clean_dcache_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoC. @@ -194,7 +194,7 @@ SYM_FUNC_END(__dma_inv_area) * - end - virtual end address of region */ SYM_FUNC_START_LOCAL(__dma_clean_area) -SYM_FUNC_START_PI(__clean_dcache_area_poc) +SYM_FUNC_START_PI(__clean_dcache_poc) /* FALLTHROUGH */ /* @@ -204,11 +204,11 @@ SYM_FUNC_START_PI(__clean_dcache_area_poc) */ dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__clean_dcache_area_poc) +SYM_FUNC_END_PI(__clean_dcache_poc) SYM_FUNC_END(__dma_clean_area) /* - * __clean_dcache_area_pop(start, end) + * __clean_dcache_pop(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoP. @@ -216,13 +216,13 @@ SYM_FUNC_END(__dma_clean_area) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START_PI(__clean_dcache_area_pop) +SYM_FUNC_START_PI(__clean_dcache_pop) alternative_if_not ARM64_HAS_DCPOP - b __clean_dcache_area_poc + b __clean_dcache_poc alternative_else_nop_endif dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__clean_dcache_area_pop) +SYM_FUNC_END_PI(__clean_dcache_pop) /* * __dma_flush_area(start, size) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 143f625e7727..005b92148252 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -17,14 +17,14 @@ void sync_icache_aliases(unsigned long start, unsigned long end) { if (icache_is_aliasing()) { - __clean_dcache_area_pou(start, end); - __flush_icache_all(); + __clean_dcache_pou(start, end); + __clean_inval_all_icache_pou(); } else { /* * Don't issue kick_all_cpus_sync() after I-cache invalidation * for user mappings. */ - __flush_icache_range(start, end); + __clean_inval_cache_pou(start, end); } } @@ -76,20 +76,20 @@ EXPORT_SYMBOL(flush_dcache_page); /* * Additional functions defined in assembly. */ -EXPORT_SYMBOL(__flush_icache_range); +EXPORT_SYMBOL(__clean_inval_cache_pou); #ifdef CONFIG_ARCH_HAS_PMEM_API void arch_wb_cache_pmem(void *addr, size_t size) { /* Ensure order against any prior non-cacheable writes */ dmb(osh); - __clean_dcache_area_pop((unsigned long)addr, (unsigned long)addr + size); + __clean_dcache_pop((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); void arch_invalidate_pmem(void *addr, size_t size) { - __inval_dcache_area((unsigned long)addr, (unsigned long)addr + size); + __inval_dcache_poc((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif