From patchwork Mon May 24 08:29:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5556EC04FF3 for ; Mon, 24 May 2021 18:58:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1755C613F5 for ; Mon, 24 May 2021 18:58:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1755C613F5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=lKxNrAdwRxO/mdvOFBgWCjzB+ZkmmCIl3B9ouFAgR+0=; b=UEW6rl5boNkzSbybUsu+UKTkBF +2tjN1QupMhzDR/cvbbo8XVr/VNBLRgXhqmO/lQEGigtKutAbxxJVJrWQeWSl/cBjgI9tQJu/kLMW ZPyb+cFDEUjA6q5E4di02KvGPDzcOFj4hl1fU3dZuCgPHzALXJZxEKwLjFkl4b/wQuHM9cOHPPVJe p9P8fSg6FlfrIm/g2iOR1iZdQX+jl+L9ItwyD6EWV9qtU67HnW5iObn4zrdtYQIP12Orhj55V6OEE AQ2OOZkQN3/EQDvobZagNyCkMeEkEnW+B9MYK+ANg399d5oU1DOWOgbeqKdfTvM1K4fXDCSzarvwE uq/orUWA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llFkN-001Z3c-4X; Mon, 24 May 2021 18:56:15 +0000 Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yS-000lpK-Fk for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:10 +0000 Received: by mail-wm1-x349.google.com with SMTP id y129-20020a1c32870000b029016920cc7087so3843523wmy.4 for ; Mon, 24 May 2021 01:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fNkwKXSUFBBUoQvNQTJZeD58b/vaDnI+SYNs5VjqahQ=; b=dDJ422fpW93rxwBsiFP++pmczY1hD7Y4UWJl2pjpZjJX69KvqBSKN4YFN66oiEQGMk jdoXcFj5SuuZVRzXkt5auZP65rPgnYfrFBOY5d0uEC/D0f+QsBFdAOCYu0NncMlP5SPM +2rr96Oyb0smZ+TBD05P02nC+eBY50tVnurGJtg1CRFFnYbEuzwwIv/CUSbEVwY1rArR vQtFNX+FUWYbLKWwHXPqLEhcw5/NWfnr0LuCE+3lNGQe56moHjq0z7yVIcQpa/vUhCSI zVARzr3oAjCipnNj1o77K00uuJMB7uVUVsB6WujzcD+QPt0mcpZjo+Aav8I2G6eZj6WR MbbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fNkwKXSUFBBUoQvNQTJZeD58b/vaDnI+SYNs5VjqahQ=; b=CRXgVlr26hjOG6Q5RSuGADnHDTamNqjgKSExkvq1suO9XI2CJkwPxztEMOkVrjtGT/ DfPW6iGs7aBpMQSOZCMoe3KdioSQmlS7ABg86z4aibUK3Qz2882VIPosG58vghBg006a 9ZS0BWePWHWjWC8oXDXo0NGr7QeODbLcOvJ37XJKFWZxVej5CiX+G75LwpQuBoaDtSuI 9mB+0XKvB9VNFCvaCULBvnRPDh60mvI9I0H4YZoTY5dYqAhUR00ByWNV8HZKkIq2zF2c xxod2d4pScuAzuCx95AaSB58iu+QptPqWywG+QKsZlD2e73ohouQLCnTIilV4XaZ3peL QCEQ== X-Gm-Message-State: AOAM53080aJydNAFTPMoJ3+WTdqVt58c82fNir3pqfBMA1fEHpGF3ugh 8SK/8+rlkKp200e5B/vxGO94Gn8FN1KPMFSDS60//UE2zB+rWYITN9PNZWWpzMuDj4fQQgPv/mw R8Fupp7+8TxWrP30D6jIbTfIr2PtTeoC54MLk2aOWL+BmVP0qlUghQ3eA8Jv3s+1DEacmssJdEr mScQ== X-Google-Smtp-Source: ABdhPJyfKIG8pgxclz7CjVCk3bQ6lqU7r9XrUoRVXYuEc55ePvXVFYUwFSk7ZkiBTYiPhONMAZu0cXqsbw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:9dc6:: with SMTP id g189mr5328931wme.59.1621845005772; Mon, 24 May 2021 01:30:05 -0700 (PDT) Date: Mon, 24 May 2021 09:29:44 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-2-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 01/18] arm64: assembler: replace `kaddr` with `addr` From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013008_549885_BC800E9E X-CRM114-Status: GOOD ( 11.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland The `__dcache_op_workaround_clean_cache` and `dcache_by_line_op` macros are only expected to be usedc on kernel memory, without a user fault fixup, and so we named their address variables `kaddr` to make this clear. Subseuqent patches will modify these to also work on user memory with an (optional) user fault fixup, where `kaddr` won't make as much sense. To aid the legibility of patches, this patch (only) replaces `kaddr` with `addr` as a preparatory step. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Fuad Tabba Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Fuad Tabba Cc: Will Deacon --- arch/arm64/include/asm/assembler.h | 32 +++++++++++++++--------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8418c1bd8f04..6a0fbc599196 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -377,47 +377,47 @@ alternative_cb_end /* * Macro to perform a data cache maintenance for the interval - * [kaddr, kaddr + size) + * [addr, addr + size) * * op: operation passed to dc instruction * domain: domain used in dsb instruciton - * kaddr: starting virtual address of the region + * addr: starting virtual address of the region * size: size of the region - * Corrupts: kaddr, size, tmp1, tmp2 + * Corrupts: addr, size, tmp1, tmp2 */ - .macro __dcache_op_workaround_clean_cache, op, kaddr + .macro __dcache_op_workaround_clean_cache, op, addr alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE - dc \op, \kaddr + dc \op, \addr alternative_else - dc civac, \kaddr + dc civac, \addr alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2 dcache_line_size \tmp1, \tmp2 - add \size, \kaddr, \size + add \size, \addr, \size sub \tmp2, \tmp1, #1 - bic \kaddr, \kaddr, \tmp2 + bic \addr, \addr, \tmp2 9998: .ifc \op, cvau - __dcache_op_workaround_clean_cache \op, \kaddr + __dcache_op_workaround_clean_cache \op, \addr .else .ifc \op, cvac - __dcache_op_workaround_clean_cache \op, \kaddr + __dcache_op_workaround_clean_cache \op, \addr .else .ifc \op, cvap - sys 3, c7, c12, 1, \kaddr // dc cvap + sys 3, c7, c12, 1, \addr // dc cvap .else .ifc \op, cvadp - sys 3, c7, c13, 1, \kaddr // dc cvadp + sys 3, c7, c13, 1, \addr // dc cvadp .else - dc \op, \kaddr + dc \op, \addr .endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 - cmp \kaddr, \size + add \addr, \addr, \tmp1 + cmp \addr, \size b.lo 9998b dsb \domain .endm From patchwork Mon May 24 08:29:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D570C04FF3 for ; Mon, 24 May 2021 16:56:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3BAE6613F4 for ; Mon, 24 May 2021 16:56:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BAE6613F4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=y9xNshh3pFqboXr4VCitfiZdwyQTH8sTr2ybJhKyiFY=; b=g7mcp9gEvlW7Fb+vekHOry4hpD 74U4CnzJoSPudlp/deICxPPUTsiKW06ykEBMneQYCHSCQ2taFH9tC3M47rvtmpNpIFOajIRx7alBJ tUu9xo5MHXbQJp5U8KAr/EbeKsmpz6RPxOQgVFFZPMhmhm8SmNrfRlFzUcoCrqnykV7GeY/jxXDtf 2MfcilpnvrFJfub8sW0jlVaZNlVMebqVDCjnacDFJ8dtFUKOEiSViSB/2I9YfHq8vaJ3malylkdiL N7COeQk5Iah5NGJQCLvzaXi+9WWyciv7KNnZs2j841P1iNDFOdneeChKF8gsq55WvikiXFaEN+iCO h8iNqpfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llDqQ-001BER-8N; Mon, 24 May 2021 16:54:22 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yU-000lpN-5b for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:11 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id b24-20020a0cb3d80000b02901e78b82d74aso26655927qvf.20 for ; Mon, 24 May 2021 01:30:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ENW5yXANABtXqS4LSFiicgp+oOZVaFiuGmSutRHOKLg=; b=n35s2r8zg4kPPfXcRIrKGVlVttKF3YXWcVa0/3Fr+uDsfMQIWKI9pfOdyP03lK7krH Hkspcig5ZwiFqx4RSHP5KbXxwlPENa5tHSv4SCgXfKkqlmbpSckLQlBxgf0wnL4yc3wN UXTwgOQdgJBy2CSBEAp+oEDDQENKlRc2Z9RB9PbkjOrySE+tUKVSbczyi64bPpqedvqW WG2xWgLk4aaXxyPyCk4gKtjMlXzPiS13j+mwjZ/g7SWCmEkEDnFRXy5M3e4I3BLQ42tu TWCfsWLIFj2NHSkZkSm5w26zJYVPDWLqQvFMO+IsRLDcRB6Lx2MG3m/pWNrFu51bFz4t R1aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ENW5yXANABtXqS4LSFiicgp+oOZVaFiuGmSutRHOKLg=; b=ODPEVyFzkzIm/8U7TbqrDjeq2MziWe+RYF9NEKceUTUvspdP+b6IjNIgIYHaf8W4UU fJEeGKmoqja52uihqf3SMjmO/N3AsQ87xdSq4tIhQZmhPuCZMlMXtIt2JnAFMXRWjFdy M1U9hhpjwqA7PKTMVr4LObvMD2qcRmdxy8zaQV6brvCLPp8o+ZPPiI8MdOjo2h8F+4v0 D/PPuLrY++qfnLWMQbMlMyuFyPsxM6aB5eLjhZsOrt6wCgAWStQ9VELROLKtKOX8pGYU rx852Ilf+4Q6HFCWjUUnnhFi/Gw4DLreNHxb4souH+aA/5wYF64y32Xp9wS1C8vmun5I wM0w== X-Gm-Message-State: AOAM533BAn9cc9PjNIN3ih5x0lWRfGhe8eiCMzgMY/yuC8jNnrpKfpxz yxf2YqXCUgEBtMVEBSXc1NoNfesRd1u8fqJUfImN2NdAUjF+NsgQ6eaMsrlcEV+ltiHUVj3sxq5 dH7QpqGx72xMLa8hno3sUZxeFa7wCrGXjaJ4K/sbyMPNGwIgVcQa6lIdfwpLbET/hR5Rz3AtEG8 ZBqA== X-Google-Smtp-Source: ABdhPJyMiQfBxCyMCt6KVGAbGahKIyWaGpH/V7RzNif0KbBFzEOTWNWWX2OxUzXqiuCi5yt6uFWwh+SU5Q== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:eed4:: with SMTP id h20mr28681086qvs.40.1621845007757; Mon, 24 May 2021 01:30:07 -0700 (PDT) Date: Mon, 24 May 2021 09:29:45 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-3-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 02/18] arm64: assembler: add conditional cache fixups From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013010_234808_D8593895 X-CRM114-Status: GOOD ( 15.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland It would be helpful if we could use both `dcache_by_line_op` and `invalidate_icache_by_line` for user memory without accidentally fixing up unexpected faults when performing maintenance on kernel addresses. Let's make this possible by having both macros take an optional fixup label, and only generating an extable entry if a label is provided. At the same time, let's clean up the labels used to be globally unique using \@ as we do for other macros. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Fuad Tabba Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Fuad Tabba Cc: Will Deacon --- arch/arm64/include/asm/assembler.h | 39 +++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6a0fbc599196..0a276b46ef50 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -130,15 +130,27 @@ alternative_endif .endm /* - * Emit an entry into the exception table + * Create an exception table entry for `insn`, which will branch to `fixup` + * when an unhandled fault is taken. */ - .macro _asm_extable, from, to + .macro _asm_extable, insn, fixup .pushsection __ex_table, "a" .align 3 - .long (\from - .), (\to - .) + .long (\insn - .), (\fixup - .) .popsection .endm +/* + * Create an exception table entry for `insn` if `fixup` is provided. Otherwise + * do nothing. + */ + .macro _cond_extable, insn, fixup + .ifnc \fixup, + _asm_extable \insn, \fixup + .endif + .endm + + #define USER(l, x...) \ 9999: x; \ _asm_extable 9999b, l @@ -383,6 +395,7 @@ alternative_cb_end * domain: domain used in dsb instruciton * addr: starting virtual address of the region * size: size of the region + * fixup: optional label to branch to on user fault * Corrupts: addr, size, tmp1, tmp2 */ .macro __dcache_op_workaround_clean_cache, op, addr @@ -393,12 +406,12 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 add \size, \addr, \size sub \tmp2, \tmp1, #1 bic \addr, \addr, \tmp2 -9998: +.Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \addr .else @@ -418,8 +431,10 @@ alternative_endif .endif add \addr, \addr, \tmp1 cmp \addr, \size - b.lo 9998b + b.lo .Ldcache_op\@ dsb \domain + + _cond_extable .Ldcache_op\@, \fixup .endm /* @@ -427,20 +442,22 @@ alternative_endif * [start, end) * * start, end: virtual addresses describing the region - * label: A label to branch to on user fault. + * fixup: optional label to branch to on user fault * Corrupts: tmp1, tmp2 */ - .macro invalidate_icache_by_line start, end, tmp1, tmp2, label + .macro invalidate_icache_by_line start, end, tmp1, tmp2, fixup icache_line_size \tmp1, \tmp2 sub \tmp2, \tmp1, #1 bic \tmp2, \start, \tmp2 -9997: -USER(\label, ic ivau, \tmp2) // invalidate I line PoU +.Licache_op\@: + ic ivau, \tmp2 // invalidate I line PoU add \tmp2, \tmp2, \tmp1 cmp \tmp2, \end - b.lo 9997b + b.lo .Licache_op\@ dsb ish isb + + _cond_extable .Licache_op\@, \fixup .endm /* From patchwork Mon May 24 08:29:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1513AC04FF3 for ; Mon, 24 May 2021 16:32:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D57CB613D8 for ; Mon, 24 May 2021 16:32:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D57CB613D8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=orz/wos7Wp3uEeEwX8j9u7Ksc6exx/XZUeacFgkHU/I=; b=hnARBGKAOnX1ODNBQZEqCMpsQB jGE2Hvl40zpwWz4PZxFi/oQfuKTjWHaEA0FJCc6BFY9+BA5ikvOMQmTNBEt+GbQpKF4SReshY7HQU dijfZzecnyj46GYa64Y1toz/dz2s3/xR0Of6JXQKtbncR8bF6peQYdWxECQv/ehIBgw4ZVveNJn77 NQLorsaSum3QN+l3+HQp8hGZhj/Ys+1W/9l+LKf1kIVqA7YLJLl/hnQtlvcNEvFALhdrM93FTbp6y lxzRbRUFU3dlxqH3G/RiCHKSQKZe7U0AdNHxeAOloYJqe6X237MTGPJuC9z2SBXLd+4ljfaEDsOTa 3LtEwdgw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llDSb-0016zo-HW; Mon, 24 May 2021 16:29:46 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yW-000lpp-9v for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:13 +0000 Received: by mail-wr1-x449.google.com with SMTP id 7-20020adf95070000b02901104ad3ef04so12741414wrs.16 for ; Mon, 24 May 2021 01:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=YmlU9zjuFsIgzZ6SyP/pAHXFug+N0HjjRPVJmEU6amE=; b=AK6u8Xvq7En/tDAkHKoIGSEIgwsijk5UI5oTMMLQh0q7pqqiMLnNU3XFqWlQWTF2BD 6FdHVWaVhoneCmgCOZ0fjqgp35QSIerDZcHJiqhOnVpv6t4asiFuQpyHvG1UEcqh37gK K8Jd4iQl+t4zSMy58a/oQ8yP2INX4q3UmrRuidUtkeYX0yCgMXAUuoVye2FFFxKg2H8g cC5Qz+wxlrl31of2EhgmsnH0T0uZGm1XSkCrZZIAPmnt7ZRJ2bky+Mvzwk1Hmljl52/N 6lnfS0L1DRHBj12um0GM83Th7rGWChkTlX+IZ6VS42yfyYPRQeHDBR6EZExJW6jXsXe6 gDIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=YmlU9zjuFsIgzZ6SyP/pAHXFug+N0HjjRPVJmEU6amE=; b=mf9SLsSM0mB2ovmybg8T1vVKbxpXBLdN/xQKZKuNdNMdDwk1I1c7W86rQwUc3w1lfQ KtXK3v2KZk4Xkiz1Vw+pdwcVL/MBLPfwA7UobzJKwRO2pIBkXgMLbqQBQQs3lwGm3IfW 0kLOoFjX+EjN9UgH1l7C3mE9wRIxNzGE7AhX3pd8j+SRXbt1BLb3edBzisvqn0MWQk21 0Zs1sNCzTxGaKOw0oX4TuMRJ765WbNZggz7Lpra4ydHYf3lp4NZAyaBNw6n7P58c3sVQ 1WEXy0kaZ0AAVbAlBk1gTd3xxLWhSE+e+aWIGzYVJ0cLFth4WAA9L0LcyAHoNwHnv7UH OMZQ== X-Gm-Message-State: AOAM530g4xe1XLvWMBsPWWuiFwk8FE/I69htlG6usQchu8B6V6Z3D6m7 OQ3oR/weED4ljMeiZV0U7P5RV5dFcJSb3QzTy3KItCad35GiZ2Ah/b5lSfmBFVL1gnIPph43Zo7 6JS4FcwVyfKPw0NIfrgi4QyVNam1yNQEHsysFqzgXdZG6ImlK3v0zgDZw76HgzLnC6j6N7QIXBt l19g== X-Google-Smtp-Source: ABdhPJzE/pmeKdCZ4xsFJ7X5tXDpkNBCbOvatQtPsqs38KOLS4YtLK04oZXy7UTHVFEs6v9gDe4mvL35ww== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c92e:: with SMTP id h14mr18397486wml.179.1621845009758; Mon, 24 May 2021 01:30:09 -0700 (PDT) Date: Mon, 24 May 2021 09:29:46 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-4-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 03/18] arm64: Apply errata to swsusp_arch_suspend_exit From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013012_362284_D543870C X-CRM114-Status: UNSURE ( 9.04 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Arm errata covered by ARM64_WORKAROUND_CLEAN_CACHE require that "dc cvau" instructions get promoted to "dc civac". Reported-by: Mark Rutland Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/kernel/hibernate-asm.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..0ed2f72a6b94 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -91,7 +91,8 @@ SYM_CODE_START(swsusp_arch_suspend_exit) raw_dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x10, x3 -2: dc cvau, x4 /* clean D line / unified line */ +2: /* clean D line / unified line */ +alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE add x4, x4, x2 cmp x4, x1 b.lo 2b From patchwork Mon May 24 08:29:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BEE5C2B9F7 for ; Mon, 24 May 2021 23:33:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 58B7A6140B for ; Mon, 24 May 2021 23:33:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 58B7A6140B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=wVMtUvUYxLvKNC4Rj5DwEoLGgHJL0m1wHHMRP3mmfdo=; b=u8AqjH5fLx0eiB4SChTYH4wUrw ZdTIQh1aRcSRvdYzPIzrWsxIwCoAKhr5P1yjVjbSSPFLs8w7AzzTWjDKh8QyZ/41ppsR6u8NVMlkM /hzgnm4Y6hTLacpKwRsOYFIUytJ9O2pMhHdCiLR+TFSlXYS8G0YU+zNnvHX7mD4ur+f+ZYgXlvswa eCh6yu5rEr599U7MjA8yWqYAfadL1NIMnJbIJc56/kVIey/zvLx2z7CbpB9CvswPpoqFRf1uRtXCT ueQBF5jcbx3xU721C+LAy3dllDS1GMcP+KeO4m+OBrFe7AQ8ZYxFx3YHlO4xqiubueBzIvbJ/uxm0 6x3ExHlQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llK2d-002LnT-Rg; Mon, 24 May 2021 23:31:25 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yZ-000lq4-Jt for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:17 +0000 Received: by mail-wr1-x44a.google.com with SMTP id 2-20020adf94020000b0290110481f75ddso12736899wrq.21 for ; Mon, 24 May 2021 01:30:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9jxTZq434paFFRmnyHWZee+kMcfmIDCH9ZCbQKUPMzw=; b=aWtHnlLznXAwts7AVfLTQE816WjV27QJPc8pNbx7KS69ORaCSxEuWK9MgPNTulhhPS VvZHAdF8+Oy30FtyzyD74qV+NZBWxlDUtuLrt6ACOFJLUz9Im+tK0l3NjRANxtq9abxZ x8KVvSHe7yVPLJBqLnu0dtDYhw3rQfg2fS+bHnsMxU3JkdgldG+VdPQzLhDgUvh+imGO ccGOiTsruHrcyGMb/2IUGS+OUvuHBsM4iSE3V9fFgET4t18RnYtJ4ZrCOIAKq0lOMbXI 1VAhybo+l2VFyTtoA+UKdqK5twRQwiJrclpFDUU9fi1T1vHHFvBC7HlJSy1hPcgjYLYL D+rA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9jxTZq434paFFRmnyHWZee+kMcfmIDCH9ZCbQKUPMzw=; b=Q4J+Aal5WtEGtViXxVgfOlHcoGU/EiPT2ha48x2aoeWKlvxUD/3mlragpkqnGsmv1j uwgeCMqophJGutgLWfEPBsWnT9kPbP6QfuJT5K/yleiqQWHwLLbsybEI1XnozJGm0Jlh GLkaAdA+U55bqsXV8sWSbVc9298s+1Jzz+nP/sQWJGl3z0AiEAwHGDKDMDHdZLKldsOW RzYqif0e17jRaqd49V9nJlc4tUOS//jfauro3dAZj4SIYfu23waFLtS3m3c4JKtxGUgz MBvuwYLJUKYkmamq6+HBcGPXCskZ38DcDqdptOudWBeWCqvO6bp7rBHnP7JOc5b5Uq4T ZvUA== X-Gm-Message-State: AOAM530MlEzXB3sDaQtv/NfkT8l/P7mhXSQjCXDyIcRvOUVNDcGiMO5c 2EFdIhN2rf5zU2Ijk9vJIxtrhzb5+9FwWJeL8l0/uTRRoF6jUqhmWxE9B/4TnLbV5Y1wCaoyTKj GHdA70VIe+4Wft0uBSgfzFNmOu9yAs/J0z5AnYXaHsyRXOgZV+7uZNk9zZpS+TKFu/6Tx9AsckS 7kwg== X-Google-Smtp-Source: ABdhPJxYdIRO25TTljnN1hlc7ZL1Lp+EElnE5FHoKQqH7KXTPbzsE5oSbUbZhRJH27/NVwqws+Hrn5K6Yg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:7703:: with SMTP id t3mr19075782wmi.99.1621845011968; Mon, 24 May 2021 01:30:11 -0700 (PDT) Date: Mon, 24 May 2021 09:29:47 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-5-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 04/18] arm64: Do not enable uaccess for flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013015_674965_4BA93108 X-CRM114-Status: GOOD ( 13.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __flush_icache_range works on kernel addresses, and doesn't need uaccess. The existing code is a side-effect of its current implementation with __flush_cache_user_range fallthrough. Instead of fallthrough to share the code, use a common macro for the two where the caller specifies an optional fixup label if user access is needed. If provided, this label would be used to generate an extable entry. Simplify the code to use dcache_by_line_op, instead of replicating much of its functionality. No functional change intended. Possible performance impact due to the reduced number of instructions. Reported-by: Catalin Marinas Reported-by: Will Deacon Reported-by: Mark Rutland Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Link: https://lore.kernel.org/linux-arm-kernel/20210521121846.GB1040@C02TD0UTHF1T.local/ Signed-off-by: Fuad Tabba Acked-by: Catalin Marinas --- arch/arm64/mm/cache.S | 57 ++++++++++++++++++++++++++----------------- 1 file changed, 34 insertions(+), 23 deletions(-) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 2d881f34dd9d..7c54bcbf5a36 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -14,6 +14,34 @@ #include #include +/* + * __flush_cache_range(start,end) [fixup] + * + * Ensure that the I and D caches are coherent within specified region. + * This is typically used when code has been written to a memory region, + * and will be executed. + * + * - start - virtual start address of region + * - end - virtual end address of region + * - fixup - optional label to branch to on user fault + */ +.macro __flush_cache_range, fixup +alternative_if ARM64_HAS_CACHE_IDC + dsb ishst + b .Ldc_skip_\@ +alternative_else_nop_endif + mov x2, x0 + sub x3, x1, x0 + dcache_by_line_op cvau, ish, x2, x3, x4, x5, \fixup +.Ldc_skip_\@: +alternative_if ARM64_HAS_CACHE_DIC + isb + b .Lic_skip_\@ +alternative_else_nop_endif + invalidate_icache_by_line x0, x1, x2, x3, \fixup +.Lic_skip_\@: +.endm + /* * flush_icache_range(start,end) * @@ -25,7 +53,9 @@ * - end - virtual end address of region */ SYM_FUNC_START(__flush_icache_range) - /* FALLTHROUGH */ + __flush_cache_range + ret +SYM_FUNC_END(__flush_icache_range) /* * __flush_cache_user_range(start,end) @@ -39,34 +69,15 @@ SYM_FUNC_START(__flush_icache_range) */ SYM_FUNC_START(__flush_cache_user_range) uaccess_ttbr0_enable x2, x3, x4 -alternative_if ARM64_HAS_CACHE_IDC - dsb ishst - b 7f -alternative_else_nop_endif - dcache_line_size x2, x3 - sub x3, x2, #1 - bic x4, x0, x3 -1: -user_alt 9f, "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE - add x4, x4, x2 - cmp x4, x1 - b.lo 1b - dsb ish -7: -alternative_if ARM64_HAS_CACHE_DIC - isb - b 8f -alternative_else_nop_endif - invalidate_icache_by_line x0, x1, x2, x3, 9f -8: mov x0, #0 + __flush_cache_range 2f + mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 ret -9: +2: mov x0, #-EFAULT b 1b -SYM_FUNC_END(__flush_icache_range) SYM_FUNC_END(__flush_cache_user_range) /* From patchwork Mon May 24 08:29:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADAA7C04FF3 for ; Mon, 24 May 2021 22:02:54 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6EAB861404 for ; Mon, 24 May 2021 22:02:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6EAB861404 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ISw9ambF0SS+tsFlbsAVaLHvhw752v77HRGPpFZcwpk=; b=jLKoVH8R9fubX4i4QVcbhPFcbE RHrl4MXgx+2g2FeZr6OMcHZTY4oN/ZL1gqlbnh1JZYlIZsDpKyMmuGo6supKKOqjl9VkQT7CgUNt4 n/+l+gKHW5NPz+hMLBQHsxmDtd7RL1my9oJo2C7WRbT1HhF3S1ZYj+w0c+uLfonGmLTz3zcxLOVTP zHgQbyYMpUnrpYGYiuEmeQjFnW5d1BvTIY03F1lP6w7eZ6T4BTQBthxxUf9yaDraG5yGJsOhdBPe2 0RtXsyk6hwV8YhVIunrOiBrqJ7LLzASUpD1YtZDKFSSrM7e9F9kXFKW3mSRwOomaDZJfHU9EQSnlb w0eVryNg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llIcb-0021vl-WF; Mon, 24 May 2021 22:00:27 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5ya-000lq7-K0 for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:18 +0000 Received: by mail-qt1-x849.google.com with SMTP id w15-20020ac857cf0000b02901e11cd2e82fso23197263qta.12 for ; Mon, 24 May 2021 01:30:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=K4lK2k8fOKUYw6qudUOJCKllc+snFqyym1vk/00lmSU=; b=pP2KFeCT1+nd0oPuUNjJ6ZuJnjESHpWhrNQVa04PhTX2hM/q+SsXVUY0j4TSEjWjA/ KUA2jY4y89BsHvUz0HzM4/iaDV/7b7IXpS0pd/aT1zFrorZ19lPxjsTnSCDPXdhqvzUb 8oWKry1Z9ICtR+kr7fC/q5Lp3KWFdDszgD8BNew+yq3Y769Gl80xt8zvv6iFb8WLQ1b4 Z1tOQGBwgT8lQ/XsWN4erdDI/LUNshDZHOMS0TajBAMZT/wjUsvSfZCGFkNLXth5+x8N QLVOonTkKeix3AGw0c9mtUQaV2c79gcCscs5ix4XvPNeFScfNaO4SLmdHnJCY2xcEOmV LQlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=K4lK2k8fOKUYw6qudUOJCKllc+snFqyym1vk/00lmSU=; b=pJK08Xk5xNr48YDzBOzj7w7MARVjPxCPEjWeQqMig1uHvgGpXz7BHFB4sUwAQygwY0 XMLgLiW+FMxIkLMqXd52iuGY0e/ezF7lfQs4OSA6JhC+VlJINkXjGj5Xs8iAYJBDYv0F fNQo3j3aNCAa+M4rWa7udRVXcHhbDgdYxmBcX1kVimzK69HvECcVIrr82swfV9mZAJuY 8c8FhcFJZTtHTzKjH+hFe8N56ImafMBQNAMIqPmMWpiygWhevteA9hj0lYbWB/GGIETW KYIIQW3NYzsMJJRsgp+FkB1GbRLsNsEB92ToRpi2Am0rCjHSKOKyVI2QcH02erRj21oU F8xA== X-Gm-Message-State: AOAM530VgpGWHYCksOEFCzlKGJ0k3U6nK57LL4UG2W4a9L5Hd3DEncEW 5ml+dNWaKcHJiY8LnKbVW7IXom32f/Ue/ZzxFqJ4jvRlrgmwha9MclmlS1KtW789wHBiFVPFBfF eeSKUVYogIo0aCDDdC7Kp80ddBIpZHSmmuBPya6UpKUXcsj1JEGZGTW0mkDRGTBHHvdQj719yru 087Q== X-Google-Smtp-Source: ABdhPJzKminHfFCC/aAsn6U78ZWPLQfYyAe62+MfgpOZowD1qZDUcTaIncD7rLLJWLDSFHpWYNVyh7IdnA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:ad4:4c45:: with SMTP id cs5mr29324238qvb.6.1621845014228; Mon, 24 May 2021 01:30:14 -0700 (PDT) Date: Mon, 24 May 2021 09:29:48 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-6-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 05/18] arm64: Do not enable uaccess for invalidate_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013016_676833_7BA23077 X-CRM114-Status: GOOD ( 12.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org invalidate_icache_range() works on kernel addresses, and doesn't need uaccess. Remove the code that toggles uaccess_ttbr0_enable, as well as the code that emits an entry into the exception table (via the macro invalidate_icache_by_line). Changes return type of invalidate_icache_range() from int (which used to indicate a fault) to void, since it doesn't need uaccess and won't fault. Note that return value was never checked by any of the callers. No functional change intended. Possible performance impact due to the reduced number of instructions. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Acked-by: Mark Rutland Signed-off-by: Fuad Tabba Acked-by: Catalin Marinas --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/mm/cache.S | 11 +---------- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 52e5c1623224..a586afa84172 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -57,7 +57,7 @@ * - size - region size */ extern void __flush_icache_range(unsigned long start, unsigned long end); -extern int invalidate_icache_range(unsigned long start, unsigned long end); +extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); extern void __inval_dcache_area(void *addr, size_t len); extern void __clean_dcache_area_poc(void *addr, size_t len); diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 7c54bcbf5a36..14eac9d76d57 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -90,21 +90,12 @@ SYM_FUNC_END(__flush_cache_user_range) */ SYM_FUNC_START(invalidate_icache_range) alternative_if ARM64_HAS_CACHE_DIC - mov x0, xzr isb ret alternative_else_nop_endif - uaccess_ttbr0_enable x2, x3, x4 - - invalidate_icache_by_line x0, x1, x2, x3, 2f - mov x0, xzr -1: - uaccess_ttbr0_disable x1, x2 + invalidate_icache_by_line x0, x1, x2, x3 ret -2: - mov x0, #-EFAULT - b 1b SYM_FUNC_END(invalidate_icache_range) /* From patchwork Mon May 24 08:29:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A618C04FF3 for ; Mon, 24 May 2021 19:19:23 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9103610FA for ; Mon, 24 May 2021 19:19:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9103610FA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Lnj6fv+KUPOWqvQXDt2OoqAMm3BYX8l1O+MrAJJ+Efc=; b=cO9fhaKpTQ0gpYVP+hiiOxQMdZ 2kk8BqPEDp3RhrGTCFR35ADoBmqHQVB3nxEwvXdzB38AuP57nwl7BoOdSlG5wGwAizLhsSmVRUwCF vqTsq6MWAWnS4Unu/6NxW3WcC70m6ETHaKu+cQqAwZwC/7764usTtkSol6nU5hdLzVrmvXxzALiIn LygcOvoruKW7OJ8g76dMr/ltYu6gpmDX24AP3BkVRpOCn3qUToTXIC04jwwqsziXMaou+/2nHBJXE XkC/Mt/C6U9Kfy7XG4cZYBQxedDqFOj0Ss80ckiKoeLrmOjo9Qcw61OSdkbq04xDZsyDifUmD5C4d r8oEKbGg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llG4R-001dKH-WA; Mon, 24 May 2021 19:17:02 +0000 Received: from mail-ed1-x549.google.com ([2a00:1450:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yd-000lqY-Ep for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:20 +0000 Received: by mail-ed1-x549.google.com with SMTP id q18-20020a0564025192b02903888712212fso15174205edd.19 for ; Mon, 24 May 2021 01:30:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=sxIqC+Bgt7ix9ifKEzcWzK/fsKAMYfGl6MPufjOvHTU=; b=g3yn8ydEbXrayjIyu3oGitwQFK53KaumR/c6r9enkeJKEow3IW2BZreBb0zbP8swtB 9n6BLUayVHOnBt/EFtNS2WLI5nQelO7bQNMQSHvkAZIVhn/EKV+6HRiZlqiL2CIEGwMX yB64jTnwbd390xKY3TrGnRjDa5njjED5cXZJgJZ+PngGfyC0MdcP/a74iCo3K5q5q8QP Sy3YaWamd3k81j5dCXJmlFbIw+mO8HT/XH7YSMkcNYAXk8svAAhzR4KmFooAkp7MvriG 4YrDdpFvQxyTAW+VdhmuTYM0QN6V7JMFI6gbaGziyPAj+srYrDszQzshckggv60rd5YM elug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=sxIqC+Bgt7ix9ifKEzcWzK/fsKAMYfGl6MPufjOvHTU=; b=mHdops794C6z8Dh4S0XBkEbcq+DSt8f9akOQ70VZmb+yvBfWQPge04wDItdr5iH0SD Md0F1QLhIbTtiI3BK746yiuIrFLFANLVqNu71iQdxxrfU1mUlMJLXKzWSPalKqAxO9Ft wN8Tj/ugges6EVXglY2oHoaxIowZhmOc40DiEEP+pdpSViNxLylHz+Hn46BCA0QrlrVh zcQijcWpcXTMjccPsaMYIQCG7Qu/oFdpmh6tZde+GveqEXl8DteimtZl/sZuvg77P0zH ksS/ZFiJIbJ+Y7VGLnuCpnKq23VBCK/H2ooVW6elpeNB+AiqY4H69mELP/mmrq4gsq4b ISEA== X-Gm-Message-State: AOAM531VsXLp1Ev7H30J/Hvm7Kd6V7/X+FJdpkQm244Mn4lzwmpQgcyb 8zeGs7PLvKQUNNp3/ayIJZJovwZwgtLawoxXbZ7gVHVmf6QNQJJgup/MlxZuyazA3pnzJezmn7d ZRklIijCY659bgirs1MSeDR3o2CJwkd+FFIY/DVY8YrHgf+izq5vWFQXSBHg0bEQ2dzXeqbJt2+ b0VQ== X-Google-Smtp-Source: ABdhPJwObaHjF4surEbYc0CmgBq+TzWPJKJHV3dkjT5HSLRAI5otl5nWH6RNTHaY8G1LTiRUPLj8lybvTw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a17:906:e98:: with SMTP id p24mr22030150ejf.478.1621845016330; Mon, 24 May 2021 01:30:16 -0700 (PDT) Date: Mon, 24 May 2021 09:29:49 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-7-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 06/18] arm64: Downgrade flush_icache_range to invalidate From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013019_516114_E6A0692A X-CRM114-Status: GOOD ( 12.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since __flush_dcache_area is called right before, invalidate_icache_range is sufficient in this case. Rewrite the comment to better explain the rationale behind the cache maintenance operations used here. No functional change intended. Possible performance impact due to invalidating only the icache rather than invalidating and cleaning both caches. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Acked-by: Mark Rutland Signed-off-by: Fuad Tabba Acked-by: Catalin Marinas --- arch/arm64/kernel/machine_kexec.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..a03944fd0cd4 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -68,10 +68,14 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.kern_reloc = __pa(reloc_code); kexec_image_info(kimage); - /* Flush the reloc_code in preparation for its execution. */ + /* + * For execution with the MMU off, reloc_code needs to be cleaned to the + * PoC and invalidated from the I-cache. + */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + invalidate_icache_range((uintptr_t)reloc_code, + (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); return 0; } From patchwork Mon May 24 08:29:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5504AC04FF3 for ; Mon, 24 May 2021 19:33:35 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1772061401 for ; Mon, 24 May 2021 19:33:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1772061401 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=JLqLqrp4ciD2znpvict0UlpMnuqhi4o5bzWFTnAEOe0=; b=zzr2pBitxzchth1DLTAoqphIQA Iks7Hn3L5mZMTXAcDHTP87PJWadMBuTwRinktgz6KVjgysPp1gYvJwZckPZQsBTtteSKWsS7tNQHW 46npB+VfjMO8r8BA9zgJD+p8AasxQ8arN2MJUV/mn65DwPPdQ50ZpeQ9vSc+5jNwbpsSk8LCTNKC2 gzix9d+G0IafO1dk87KkvmgQUSuwjH5iDVNz4vCyXzz4KwTZ7qxRDU5adz0ahkkm55Nv7wKxRxQWG Bx+iL+7u0iD/Ny6zL7qfKTYqfjeynzow/wad12vdtHruZCyc4u4Z0t/5qXJCiMYLRhjbs7XslDdvV qBA35L7Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llGIW-001gJP-Pa; Mon, 24 May 2021 19:31:35 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5ye-000lqd-5N for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:21 +0000 Received: by mail-qt1-x84a.google.com with SMTP id r1-20020ac85c810000b02901fa9798cdb5so16873969qta.8 for ; Mon, 24 May 2021 01:30:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=M2VVX0ESisjFo1oSqpQg+Pv+TMb51d294anTB/SHs0Q=; b=NHPrUz0h7OVhzkeGMl6lzBHvKcwTgI75xytemu50nzCSf3cyXcfL6FhGq+iOJyUvuK glBLqb7dMC4E6cL8+S4Ct7D7FpljgBUSCgULbl/Nqz8H8bVIOeCrB4YaAXoT+nf4Uo12 UJ2SjotIHfBarLW4ZrTP9Zf0MvDAJxn++DtokgTiIqsCahaaSwKfWzTATJlifWgn4Co7 uY5vFss8tFAgc6E/CDIOWGddOAZ1eIEbU8Evz9TWv8pctoDZ0rxynNjoxZusHUvv1Y0V KdCQM6IwhIxVj+em08AoMgviJC6kCIuINdoZu8VRvXfC75g2xKszsI+2tAPtGBp7f9m6 TopA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=M2VVX0ESisjFo1oSqpQg+Pv+TMb51d294anTB/SHs0Q=; b=ZO8OZtF4ZGdWmuyq96Fmp95HsXSu6piJjzSpIRcwb3drIcZX/E/7OUXXCnsidmrrDb 1k0D4O3QutJrFnSsAEqpC8HYIE6HKj7eZeZrpJ5XDAQ+X7XQUGhMh3Pp/O+Ed7pXbGBt tqhgHtjHr22x0Di/gV/TBbfKXNYI0bhY62ugmxJrAAOKgDolWZ0W9CLneu89a8MgQrHp GaJDTG1sr6EdUC2M2zhWx+0qInh/CLzORdcLuDUyF2aDBTXaK9BaJ3W4rHLAGzDLUJqF Xlwm+gPh5pbHI6saFo8QxIps4pVfUSfptEGG7a56NBf2tpvldPieK3LQ54uhm8MWwJhN ndvw== X-Gm-Message-State: AOAM533RfmTfas7YRxyZhV0cVDwPDYqY2HZExc5t4vIajdzVLMmmU8Pa qsYGJlX+CkuR9HQPle+n7oncWfxO0AhtoRIWaUhbdaJ4/ZZQKNdKlzExFs8jj3bZyrIsxjq+1z0 DefP+7fNb8w9lOvuZtH7WmIsgP11zdZUa3Noxob32mGA+LHfZ9RET4JH3dUjbjVHssbBMuzty7u 2g8g== X-Google-Smtp-Source: ABdhPJz/AkXMuFd5sHrEBEr2Qz7uFw5vx+pVwoIWhbld399Va5sX9j+8hj5klKo5qZuyT96ozcYI+CDjwg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:6214:8f2:: with SMTP id dr18mr16977640qvb.25.1621845018383; Mon, 24 May 2021 01:30:18 -0700 (PDT) Date: Mon, 24 May 2021 09:29:50 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-8-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 07/18] arm64: assembler: remove user_alt From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013020_221657_B623A06B X-CRM114-Status: UNSURE ( 8.58 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org user_alt isn't being used anymore. It's also simpler and clearer to directly use alternative_insn and _cond_extable in-line when needed. Reported-by: Mark Rutland Link: https://lore.kernel.org/linux-arm-kernel/20210520125735.GF17233@C02TD0UTHF1T.local/ Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/alternative-macros.h | 5 ----- 1 file changed, 5 deletions(-) diff --git a/arch/arm64/include/asm/alternative-macros.h b/arch/arm64/include/asm/alternative-macros.h index 8a078fc662ac..477703578caa 100644 --- a/arch/arm64/include/asm/alternative-macros.h +++ b/arch/arm64/include/asm/alternative-macros.h @@ -197,11 +197,6 @@ alternative_endif #define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...) \ alternative_insn insn1, insn2, cap, IS_ENABLED(cfg) -.macro user_alt, label, oldinstr, newinstr, cond -9999: alternative_insn "\oldinstr", "\newinstr", \cond - _asm_extable 9999b, \label -.endm - #endif /* __ASSEMBLY__ */ /* From patchwork Mon May 24 08:29:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14DB7C04FF3 for ; Mon, 24 May 2021 17:24:40 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BD6E061260 for ; Mon, 24 May 2021 17:24:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BD6E061260 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=MTGVgSxq83YqZbijU6ZFqhplwI5nmKtW4l5vTXl54RE=; b=ECYIFyJJ6qKn/L+u9aoWC9oeAN 8g9ZIO2rD1AKvWoZQf1nShBY1/iSYmFNy8NTad0998J+CnR3AYRbGwWecp14/NNWNgv/b8sJ2Pcm7 VRPX0gWQy4ZLpzSYWUKSec85WuqZb0wQf2ThPlPcQDH+eLqRDXUcAO3BaLEYAkAOxvzOGMPsTTGfP feFqxKY5vlwP6cnurX+MshfoHiLsgem2g171azYNXZmgLA3+qa3aG5Y+i59MierLvHPKZLiTfCFoO V9yL0/+uvLzASmJ3y/UX/WbDRnABoy18T3Tl//oG8lBMs0znNEGW+duveLOmpYnPuNdGSk7154YxK ygwWK6qg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llEHQ-001G0i-98; Mon, 24 May 2021 17:22:18 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yg-000lr2-HG for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:23 +0000 Received: by mail-wr1-x449.google.com with SMTP id p11-20020adfcc8b0000b029011277d49b71so374699wrj.11 for ; Mon, 24 May 2021 01:30:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9LjdpidEJzf1ulFvn76tTBalozVLzOFzmjCu4klXNvE=; b=hPW2JDsPKJIRCsSZL4xLxc9VrSRIACHUiLAOcUGP/L5h8UW8kBA5DTk/uB2iJ+rYej pKzVD+Xweznqj7qPpVOnju13rf3h8xjXJCpavxIbZ2sYUJg5m+x3KQE5vxAMVLP3wF6U JrHPYIsd/9DSFKebr8EXvZtkHc8gEZAOladvuwdBBaQhamo2Sjh1h9pcdcqid75Grsff wThy6hA+WYYBGFz+w+0MluHK2UCZRYOIzRCrlHf44pBdNVmaxQX+hWFi7JTWcnX7gcwS WRBtKD4LvsBe2auRy6pURAMKe/Y77b4KB2tEvg+rSFvjzvEiMdSRy3gxd8JAXm7/MZdG pgog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9LjdpidEJzf1ulFvn76tTBalozVLzOFzmjCu4klXNvE=; b=BZcJRhuE1xIP2+v4JFvnWEH7PsJ3UwK5zrdQJts5tfYA5xBOzIInnyVsxMVuVuUjcn zmnm2WWYFOjPDJRXM4vNIEtOgfevsv1dBwn3lAOWiaq6Q4jtTkYgLVSDZ1aWVUJyQt6K lF0u9lXjDxCYxeUfZPr4q/qZWGLaH3jKhfF7pxQUebQ4nEK5W7yIuwVhxn+sLoYsqEOL I2trSHgtmdTMea032R1kZ5u4bzci6ITifj9MqxFeA2+Op+vstF3kNPdbe4thOO7jfD9C jHmSieDWGfA60KQi4EJReSZksM5wMDtgakLL3slwTaUtIfnXllHqygjTEeARM8uvYrt4 WXug== X-Gm-Message-State: AOAM5321ztbica1XHZGgod50qU4Jr2E7Tpr1Kdmkwj1XpCkmujBzvCEO LFWEhD1UJm8tRUufpm5TyV7beAHe6BXPgJTW+CXngC5pR67pBOCPp5QcP4HxMcEHh4fEf4YBnza Xrizy7mxGxVgYXi47QdJMa41mkjz6wLeKrj+2GPweMP4e8hmdktplq753iGnrQHOvDp1nLDp+pm 85ZA== X-Google-Smtp-Source: ABdhPJy55MT2AJTnB5EebLZ2pZDWkt07E+g6OPrM8pjAKoMcc3JBouEp3h+7SFgJxp/TpZz97HLSoSSPaQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:1b95:: with SMTP id b143mr3999902wmb.100.1621845020428; Mon, 24 May 2021 01:30:20 -0700 (PDT) Date: Mon, 24 May 2021 09:29:51 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-9-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 08/18] arm64: Move documentation of dcache_by_line_op From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013022_591237_F69B2A59 X-CRM114-Status: GOOD ( 11.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The comment describing the macro dcache_by_line_op is placed right before the previous macro of the one it describes, which is a bit confusing. Move it to the macro it describes (dcache_by_line_op). No functional change intended. Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/assembler.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0a276b46ef50..ced791124b28 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -387,6 +387,14 @@ alternative_cb_end bfi \tcr, \tmp0, \pos, #3 .endm + .macro __dcache_op_workaround_clean_cache, op, addr +alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE + dc \op, \addr +alternative_else + dc civac, \addr +alternative_endif + .endm + /* * Macro to perform a data cache maintenance for the interval * [addr, addr + size) @@ -398,14 +406,6 @@ alternative_cb_end * fixup: optional label to branch to on user fault * Corrupts: addr, size, tmp1, tmp2 */ - .macro __dcache_op_workaround_clean_cache, op, addr -alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE - dc \op, \addr -alternative_else - dc civac, \addr -alternative_endif - .endm - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 add \size, \addr, \size From patchwork Mon May 24 08:29:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276769 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55DDBC04FF3 for ; Mon, 24 May 2021 18:12:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2108B61406 for ; Mon, 24 May 2021 18:12:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2108B61406 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=BKOneyEzVkYWpvKVMHIIW+cj/osSeqZH6L5WrR2S+zI=; b=z4T1lF/mIeMn69/pvZ19K+zcVn jmVqDBped//7ftDmmNq8/lQFvkXpuULqor11ZkGPMRjAEDghza5r2ji1qL08rMt1p2yG9QOR8WAfi DdAgLhI42SGAuW4OxqOINETYGoGQcKnoRSyQIAqhNNtWztHCJIsXKA0iu8kWMIutBmCeG0ldYpQKs Ew8uORdlJBad1wKLpiRZvmfFJ6wykZ7opkmqJxMLB6waurxNxROjNboHhM2l9la520wgk6iIeJXm3 Ln1B+dq6ymEwBH8G0Xx9uipVjyKlTY0qzktBQ6gSSJk6Mw+1AWfKo5WgrVFwINbU9Od7+PEKtkZGD FiIV4irQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llF1g-001P1y-Mg; Mon, 24 May 2021 18:10:06 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yi-000lrK-Sg for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:26 +0000 Received: by mail-qk1-x749.google.com with SMTP id n3-20020a378b030000b02903a624ca95adso8862731qkd.17 for ; Mon, 24 May 2021 01:30:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=6CjxhVQT4fcLHx09RilT7W95JHz4NAcMGt19k9LWXko=; b=fTwom+vJznJs2oND7ApG9ekHPeHWQQpwMcdJj1s4TV8uKSmreVRTupDGGAn/1A0TNQ S+y+BabvsY0M8//44pB0sCV4m4a8rs7Jno7ZRMsFr1aMK1bV2Pbo5/ujeLKZRdcR0tZ5 EPFDG7bk1fv7CwDw0QdqgvZAc8Ud86tdHFRWEeOX8Q2+SMWxhc33/cuB07+ooN+rXBN2 0mfkkxdyBIiP4U1Fztr0Tz2UAiNKvCLnH9x+jQv+RHC4DN5K05paheoCpOPh2w/wqb1t 1acC9GoD47J2PuuU7YNF5XcQcAVDy4GcdmaacaxNDLCsn2IHiaOHZOsYDvbjSbP58gBP ESoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=6CjxhVQT4fcLHx09RilT7W95JHz4NAcMGt19k9LWXko=; b=ZE+jt02o3DwsVB2aH9bcO3CamX8moxGk/yI9YOWziij0vTe0c2Dll1wBsVOEup3xVb wPtIX0p5b9w5pVvc3U63EuZ2K55rYcMr1oCOkd9L5SfqAi27ZCJ7nklZLJm0P0vZtvmV xJE3d34QgBjRlgrKhuEaYSqoq2fdzckgPzGerVe6bxu5anDm0LbXdkH6ceKZXJT12otN +cAJ0hZEvfcXHj9TdUjNa/gROsf2xF965Yb6xfcf7/cSzyh3w8BkRPziKZ01yNI+EMYi Ni46Lrorjm1M2JmRW/DU2gwYVMX3pF/QkzBY5PQ35N/qSSLow8KFdHRIrikOLM76581z lXTg== X-Gm-Message-State: AOAM532AQ8cBpcK9lDpV2pzaoQw7QBLvc4ZwumKn13r04RwZeSQnlQRF ISIMNh3CHKV2mZPOizmJuriC6NhZPmE9sTOKFlNBWMP/Fpg77JTZ/NjGIg5H7kIyH65axdwj1Ul oBwEHq2WLJkny+qyV67hEsVujCRz0entlbZ98VfIN80j550vzGjrMCtmZkjP3guphZHdJsn+aH7 7W+Q== X-Google-Smtp-Source: ABdhPJyVjcX7zPqt/LPAbP6HslDlwhvGUdKwxdAjtF5O8ZmJtWzF4plVthdhOyICCAVQyc/lQd7a6wLHFg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:ecc8:: with SMTP id o8mr29012946qvq.23.1621845022467; Mon, 24 May 2021 01:30:22 -0700 (PDT) Date: Mon, 24 May 2021 09:29:52 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-10-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 09/18] arm64: Fix comments to refer to correct function __flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013024_946481_A308FDE7 X-CRM114-Status: GOOD ( 18.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Many comments refer to the function flush_icache_range, where the intent is in fact __flush_icache_range. Fix these comments to refer to the intended function. That's probably due to commit 3b8c9f1cdfc506e9 ("arm64: IPI each CPU after invalidating the I-cache for kernel mappings"), which renamed flush_icache_range() to __flush_icache_range() and added a wrapper. No functional change intended. Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/kernel/hibernate-asm.S | 4 ++-- arch/arm64/mm/cache.S | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 0ed2f72a6b94..ef2ab7caf815 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -45,7 +45,7 @@ * Because this code has to be copied to a 'safe' page, it can't call out to * other functions by PC-relative address. Also remember that it may be * mid-way through over-writing other functions. For this reason it contains - * code from flush_icache_range() and uses the copy_page() macro. + * code from __flush_icache_range() and uses the copy_page() macro. * * This 'safe' page is mapped via ttbr0, and executed from there. This function * switches to a copy of the linear map in ttbr1, performs the restore, then @@ -87,7 +87,7 @@ SYM_CODE_START(swsusp_arch_suspend_exit) copy_page x0, x1, x2, x3, x4, x5, x6, x7, x8, x9 add x1, x10, #PAGE_SIZE - /* Clean the copied page to PoU - based on flush_icache_range() */ + /* Clean the copied page to PoU - based on __flush_icache_range() */ raw_dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x10, x3 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 14eac9d76d57..910ae8f6a389 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -43,7 +43,7 @@ alternative_else_nop_endif .endm /* - * flush_icache_range(start,end) + * __flush_icache_range(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, From patchwork Mon May 24 08:29:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8457AC04FF3 for ; Mon, 24 May 2021 18:05:11 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 49BD061400 for ; Mon, 24 May 2021 18:05:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49BD061400 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=aqfIKuoJydfTAd5tbU0RAX4h77KSULaA9gexV7AcEfg=; b=49JBrf0jpR0eWyfpbKSe+8MzX3 W1Mu50VhVXY9cMdhiFbJj0X9NKntN2+uehTrbRZqet8KLp6vsstWyz/cljkU8mDNgW+u0P0BxAFwq 8JsPHgm+OZmPcB201wkxkc2VXdReJINoMFQYYbs5TobzTLgq8uAyS4JUr2i6SfsabNW747x1y99Ga rYtzFy43fvbGxdMjb9+5fRsKs8EqnXfuXmzL7eMQ+vJjDoiPVCBHWBYSS2h/yCDCc2mNR2xko9nMo W+pbwK6bqTYarZkLdBMB2FTWk2YZw0dF7UUbJyxGwskJf0Y2CVPfq62MaHr4NOSBZk3Bv5Qcbp/Ej vifhT/Gg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llEuv-001Nuo-Im; Mon, 24 May 2021 18:03:06 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yl-000lrc-Rt for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:29 +0000 Received: by mail-wr1-x449.google.com with SMTP id q15-20020adfc50f0000b0290111f48b865cso9240195wrf.4 for ; Mon, 24 May 2021 01:30:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=s0dBvpRNayjV+wOWasgTrQ5zk+FwxOmJXvIzi3zCJlo=; b=rxelJ/CrpVs6BpYZLT4xjWbeqfyjSz8pVf5T9mEOrguTBFOKdi++tg1NSA8EekdOGN 4LEBllhlm1xAO5Mm6UIOvfDrp/I52ZKwPmaQz0cRsbkezMqOBHuFqBQDnhbwbNZkTeTo xzow2NCK5PGw4Ra3kT79ohwb/A95SUSdrLzXLJzAgqIwTar+lGzU9vyt7+PXCUNyepB+ 9HtsXwcPW+NT6n2BduD1vOwwSve9E9WeJD6OvjMpnrnJXQWrEzOpIHPAPQ+AylYyEXEg plKOv+5yr/fLI9DZwLeUd9WOZyCdd2lLW7uGhjaL+FqwJccwSS9ElbRqV50dCE3SHFLx 0Opw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=s0dBvpRNayjV+wOWasgTrQ5zk+FwxOmJXvIzi3zCJlo=; b=kso4RhfZ/9AljsSsZ7kedSNe6dulXtz5zWb/qrYtJ778OtiwkCm0drc5Dn9y0vxq1S uZ4ePxHtmPcxUmdhTH0v1VzlVFi0EiEOijZ1RjT66xrmCWD7s6z4YMZ0KeZ7Mw6GUuNp BGSAYhHYhS6QyJ40baaCTFw2C8qtETa4NygXk8vmPoln9l8wLYJTdu1/LSd6JYEYkflu DYHYOQkC8md2X84f7/8c/TzmwzRBoUFKoakvikbwpgjXZe0KPt1EzgLlohmrs8xXTe55 oAOtsY6hNcprJMxZ0TAAmsFMRWYcR2QLFS5MVUttBqZJxUbc6UzZT+vwVckjqXQexV5s lHOQ== X-Gm-Message-State: AOAM530rwfNfHt+F+2TU+WN2VWwpQPsx+mi4hBu5u/kMko1MyKbW1dc6 cgqTafFAoGrXyj+ZEtVf/o2NHUCs6q2e8OP42r5bFh2B8G5sq7nXy1zu1IeDiDyPSfBjZQ2fwfP q8wpSW71YNyYiHTMzzEGuRepnzQHIcRJnct6Yq4nozdrRSV8C11cNg9zf2uKQemsl+s14ynmy9B VnCA== X-Google-Smtp-Source: ABdhPJyYljonKZHr2uruKokFEC+xtBKR/gWUxF8Tn16XbdIe4aMVs3SUgUts6kc5trjRSAI2R+ulOSESCQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a5d:4b04:: with SMTP id v4mr21138685wrq.92.1621845024647; Mon, 24 May 2021 01:30:24 -0700 (PDT) Date: Mon, 24 May 2021 09:29:53 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-11-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 10/18] arm64: __inval_dcache_area to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013027_923872_6EB17D35 X-CRM114-Status: GOOD ( 15.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. Because the code is shared with __dma_inv_area, it changes the parameters for that as well. However, __dma_inv_area is local to cache.S, so no other users are affected. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/head.S | 5 +---- arch/arm64/mm/cache.S | 16 +++++++++------- arch/arm64/mm/flush.c | 2 +- 4 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index a586afa84172..157234706817 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -59,7 +59,7 @@ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); -extern void __inval_dcache_area(void *addr, size_t len); +extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pop(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 96873dfa67fd..8df0ac8d9123 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -117,7 +117,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) dmb sy // needed before dc ivac with // MMU off - mov x1, #0x20 // 4 x 8 bytes + add x1, x0, #0x20 // 4 x 8 bytes b __inval_dcache_area // tail call SYM_CODE_END(preserve_boot_args) @@ -268,7 +268,6 @@ SYM_FUNC_START_LOCAL(__create_page_tables) */ adrp x0, init_pg_dir adrp x1, init_pg_end - sub x1, x1, x0 bl __inval_dcache_area /* @@ -382,12 +381,10 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x0, idmap_pg_dir adrp x1, idmap_pg_end - sub x1, x1, x0 bl __inval_dcache_area adrp x0, init_pg_dir adrp x1, init_pg_end - sub x1, x1, x0 bl __inval_dcache_area ret x28 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 910ae8f6a389..03c1a7659ffb 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -131,25 +131,24 @@ alternative_else_nop_endif SYM_FUNC_END(__clean_dcache_area_pou) /* - * __inval_dcache_area(kaddr, size) + * __inval_dcache_area(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are invalidated. Any partial lines at the ends of the interval are * also cleaned to PoC to prevent data loss. * - * - kaddr - kernel address - * - size - size in question + * - start - kernel start address of region + * - end - kernel end address of region */ SYM_FUNC_START_LOCAL(__dma_inv_area) SYM_FUNC_START_PI(__inval_dcache_area) /* FALLTHROUGH */ /* - * __dma_inv_area(start, size) + * __dma_inv_area(start, end) * - start - virtual start address of region - * - size - size in question + * - end - virtual end address of region */ - add x1, x1, x0 dcache_line_size x2, x3 sub x3, x2, #1 tst x1, x3 // end cache line aligned? @@ -230,8 +229,10 @@ SYM_FUNC_END_PI(__dma_flush_area) * - dir - DMA direction */ SYM_FUNC_START_PI(__dma_map_area) + add x1, x0, x1 cmp w2, #DMA_FROM_DEVICE b.eq __dma_inv_area + sub x1, x1, x0 b __dma_clean_area SYM_FUNC_END_PI(__dma_map_area) @@ -242,6 +243,7 @@ SYM_FUNC_END_PI(__dma_map_area) * - dir - DMA direction */ SYM_FUNC_START_PI(__dma_unmap_area) + add x1, x0, x1 cmp w2, #DMA_TO_DEVICE b.ne __dma_inv_area ret diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 6d44c028d1c9..be650b573b2a 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -90,7 +90,7 @@ EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); void arch_invalidate_pmem(void *addr, size_t size) { - __inval_dcache_area(addr, size); + __inval_dcache_area((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif From patchwork Mon May 24 08:29:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C4F6C04FF3 for ; Mon, 24 May 2021 21:26:44 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6535561003 for ; Mon, 24 May 2021 21:26:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6535561003 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=j/CBfZ25FQjvGcdG4yhpEXzo2O7Ml49trXwTXHy98ic=; b=YbGNU0FBEmvkXOeOG/BuWQslW2 ks9pAjmimpmXdxeTVoHKsU37JyNYfsYpE4HJdJWPGZ3/TO8BCzPhq8P8iTrk4LIJnlS+iDHihoc+P DDW/cw3hRhJt/vci7/8EhAbGHxPtjkdReLmxBciFTJGjGNlIi997lt5bHaL7lH6kBMtxfTZ1ZSxES P/s0VFNQfjS59DY0o1sRLI2XZPtHtamYMrg8obL9iA0xV2Ma10Gq79zNLfirZtWXe2jYo+g+avZv/ we8K+XMZ0cxcJ5OBxx9MnjnRtrkwgFRqIqyqYARiBjzwbLIVDgF1tJ8OEh6/jB07eeROQpOKuc+eW bmzuh1Ow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llI39-001vvv-Dm; Mon, 24 May 2021 21:23:50 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5ym-000lrf-Li for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:30 +0000 Received: by mail-wr1-x449.google.com with SMTP id i102-20020adf90ef0000b029010dfcfc46c0so12771696wri.1 for ; Mon, 24 May 2021 01:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=V4vmombU/a3h0epEXRze7QkCRtCALOwzbeaMNCJY1Eo=; b=ewluOoQuY73bDsUlVUvrF1dJE+zpU/9c9+rnRSPb37hU9HOjVWLZcYsoDep8XD2MZ8 V1LfltqsPmkPjq1EJ46ZErMbipUsdpJL+T0p2u/jor9G6PM+myRc272JT5sTyXtt8mSp QimHhhz7K815zpQ2RxPTq/ezdPdOmI8UIlt1AjHE3I28JVdycIFiOpqwxWv5U1YjXYQb AecwxLSvBHNoAdh8qUSR3oeH8zclQ4KMicNridP2LRE3C3FIlQ4RXqcV505WnfGNlLhQ 6/LSxnIMK1DuCm4qMphjtAzfRzqUAwS0a5ZK82H+NhVHfT8AdfLDm9dehG43L7tUydmM UI1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=V4vmombU/a3h0epEXRze7QkCRtCALOwzbeaMNCJY1Eo=; b=SPtY9pal69JpVUo1Yzn/lX36V54rTzRKYYCrJMMm2ulY69A4xH6OsLdUHymgbZRMWl SbSvoz7oRqwvJe7WQ6KrdLUOpXN4unn+Qs9vl0kfydS7uAhmnzRpX4YApQCLmvSr92sg mGm+yAvwKk/Ao7UQH84Z5YV0hSLMl0f7gvxN1Y1s97MD7IeDEwGEB8fLJBcWtOn9/WXp x6d5KMGKYt6DWa/NATFsYkSBnpYhkC5IZg4l2aHvW9EcVCZFqf6Xh1E4aFOnSOGwYG6Q Pm05RNOEn4jNep+kv7zmvmaLgRzEEapvxcvZDP9UzKR9NzIeneA03NqTnTt7jMbQwAYi 54Dw== X-Gm-Message-State: AOAM530+kURDyPDPKoUmSVTt0/Wry2Q6GQlLzvgBcGCeW739ZQ9lynP3 BmLyp4/R3oqAdhRM+A2xv4hvpS5FSCWe98t7xLCtCq94MxCT+BjMWg8dj6GEooEZR4Bxv7faUwa C0nTeDPQjiFrLvDLcgYj+Ybz4+9YDGCCdC+J7sayrprNvOOHPToogfv1zG8ylTLF6yse1XqCjXl r38g== X-Google-Smtp-Source: ABdhPJx4wpVCTweN/1S3k+pk6vcjLR2GbhR0M5B6Bdx98x+vj4DXYSHmofIbZOEWA1ZLSq602oMN/7ASCg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:cc83:: with SMTP id p3mr18994833wma.169.1621845026728; Mon, 24 May 2021 01:30:26 -0700 (PDT) Date: Mon, 24 May 2021 09:29:54 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-12-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 11/18] arm64: dcache_by_line_op to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013028_733418_F492E63F X-CRM114-Status: GOOD ( 12.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/assembler.h | 27 +++++++++++++-------------- arch/arm64/kvm/hyp/nvhe/cache.S | 1 + arch/arm64/mm/cache.S | 7 ++++++- 3 files changed, 20 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ced791124b28..c4cecf85dccf 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -397,40 +397,39 @@ alternative_endif /* * Macro to perform a data cache maintenance for the interval - * [addr, addr + size) + * [start, end) * * op: operation passed to dc instruction * domain: domain used in dsb instruciton - * addr: starting virtual address of the region - * size: size of the region + * start: starting virtual address of the region + * end: end virtual address of the region * fixup: optional label to branch to on user fault - * Corrupts: addr, size, tmp1, tmp2 + * Corrupts: start, end, tmp1, tmp2 */ - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup + .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 - add \size, \addr, \size sub \tmp2, \tmp1, #1 - bic \addr, \addr, \tmp2 + bic \start, \start, \tmp2 .Ldcache_op\@: .ifc \op, cvau - __dcache_op_workaround_clean_cache \op, \addr + __dcache_op_workaround_clean_cache \op, \start .else .ifc \op, cvac - __dcache_op_workaround_clean_cache \op, \addr + __dcache_op_workaround_clean_cache \op, \start .else .ifc \op, cvap - sys 3, c7, c12, 1, \addr // dc cvap + sys 3, c7, c12, 1, \start // dc cvap .else .ifc \op, cvadp - sys 3, c7, c13, 1, \addr // dc cvadp + sys 3, c7, c13, 1, \start // dc cvadp .else - dc \op, \addr + dc \op, \start .endif .endif .endif .endif - add \addr, \addr, \tmp1 - cmp \addr, \size + add \start, \start, \tmp1 + cmp \start, \end b.lo .Ldcache_op\@ dsb \domain diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 36cef6915428..3bcfa3cac46f 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -8,6 +8,7 @@ #include SYM_FUNC_START_PI(__flush_dcache_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 03c1a7659ffb..fff883f691f2 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -31,7 +31,7 @@ alternative_if ARM64_HAS_CACHE_IDC b .Ldc_skip_\@ alternative_else_nop_endif mov x2, x0 - sub x3, x1, x0 + mov x3, x1 dcache_by_line_op cvau, ish, x2, x3, x4, x5, \fixup .Ldc_skip_\@: alternative_if ARM64_HAS_CACHE_DIC @@ -108,6 +108,7 @@ SYM_FUNC_END(invalidate_icache_range) * - size - size in question */ SYM_FUNC_START_PI(__flush_dcache_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) @@ -126,6 +127,7 @@ alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif + add x1, x0, x1 dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret SYM_FUNC_END(__clean_dcache_area_pou) @@ -187,6 +189,7 @@ SYM_FUNC_START_PI(__clean_dcache_area_poc) * - start - virtual start address of region * - size - size in question */ + add x1, x0, x1 dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_poc) @@ -205,6 +208,7 @@ SYM_FUNC_START_PI(__clean_dcache_area_pop) alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif + add x1, x0, x1 dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_pop) @@ -218,6 +222,7 @@ SYM_FUNC_END_PI(__clean_dcache_area_pop) * - size - size in question */ SYM_FUNC_START_PI(__dma_flush_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__dma_flush_area) From patchwork Mon May 24 08:29:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 681E3C04FF3 for ; Mon, 24 May 2021 21:53:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2A66C61413 for ; Mon, 24 May 2021 21:53:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2A66C61413 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=cQSpICCiJJnsZR7NnvQjAJ98lAfnIDjafF+iwDbIV4I=; b=wijuyek55CcIhI0lY/4GxuxaxL Dwcrmj6ifT4DvsIrHs7bRuaQ968SYn5sC7H0KIx3x/mXgECCShpAOBEIqIcnjIwJIpnB578eMmYOx DqiQx+/bBLO9k3R3BHcdW/j7C2Sa4jadHQsxlpZ1qsMkQToEGQAkVRYIKMIukDSFSt+Hio4+oz1dL JVMTW6jG+9Km8TaE8/GLW8uaiAghl4jPZq8V25SN4S1o8VgJER2TWpf3+kpfUqkmAFs7EuPYqSvWU gIRFu9OkSqJs1Tk9DPj05GsVTAhhdBqibgMAsnoLAi8oT/KEF5prfpUIErVaR1IiwotNv+upW7VuL ZMwHSWcA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llITy-0020KI-3x; Mon, 24 May 2021 21:51:31 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yo-000ls6-RD for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:32 +0000 Received: by mail-wr1-x44a.google.com with SMTP id u20-20020a0560001614b02901115c8f2d89so12603579wrb.3 for ; Mon, 24 May 2021 01:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=rbd58z6lAN3UFEHYwZ48o3n0FNMG4hiJNxTBPIVr9GE=; b=oaclnG4pCs2+Z35W71Kny1zj2RMBOJcZ0zCQ3sNRW6+PmGqRSPhLoiwCCqllB2fGKq dCy2XlSIL8WRD74lN6TSguD5D8ITWRJf/5hntM4ds/yB2YhqP9vH4OfAaVzG40RTpyB7 ORJur+L2uvV7QLOk4v5LGoVxUZSteKXOKGiJfdNnhr/cULNSX7l9qReUvrCqi/xvbYR/ iVdLZ0YP7Wt8n9h7yRkWXMbByChS9z9L32eGUyirFzuc0al2RIwIct66f/2Gb5GVRl9c N4PrrQtxo2xRgofpzE2qSTvFq71fq6InuA8xgCOTWC7aEpxuwiD0izpjjTCvzjjAR7qh Jr0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=rbd58z6lAN3UFEHYwZ48o3n0FNMG4hiJNxTBPIVr9GE=; b=CzTHU3Il6Z0CxIRRA987vYLjA39biZHmZxfRChK5CgXXY3u9lTUkE731szsL0xcKJ4 NDsT444qgbO+7kvPYZO4ALQyoQukDgIXxaAHoAzGjtnHzQwmtwI4CRfP/X7eO8QOlZ4c cUnN8p0l9s0cB1Ez3U0i5C9oZvcKf2kLUhwFumqF26DHdIYZLd+Goy1C787ErmLBJyNI rw/n69qOMvyYR7m1myuEzmHp+rYVkEVMPGQvsRaQrDlix0/l0O1aQsN5MF9vYvKnCYEb HryRoVTkSKQarWddnhuD+gd4nJ00Kn+9e3xFIT+85lHmJ72hytqdyw1O5CX15zqyKWe8 iJLQ== X-Gm-Message-State: AOAM5308EPjj7Zd25JEDYGZZFllgwShsp45RhUQwiDTyxgYzgBrYcfN2 RjmV8H21NCZHPyrIx9Z0cv1me3/ySVOf0bjYin6dEUDWAFCRjxwsidE/KJUT0mo9BDIOm5YLTH3 OH3tFLpTccOau5zISkeluwOhCUIq0W20oGSGmuArwc/KQH+WuWJKtaZSk/iVioWh69JtIfojKYw rKlw== X-Google-Smtp-Source: ABdhPJzGrNL2yTTC7wfVBAQSRowXC3PPVIrXCsIUe2da5SkxKH5x2MbyQVxSs/q4p7MRbfa8PmpEqpc7VQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:f60f:: with SMTP id w15mr10479644wmc.5.1621845028770; Mon, 24 May 2021 01:30:28 -0700 (PDT) Date: Mon, 24 May 2021 09:29:55 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-13-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 12/18] arm64: __flush_dcache_area to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013030_913499_0EE769A0 X-CRM114-Status: GOOD ( 23.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/arch_gicv3.h | 3 ++- arch/arm64/include/asm/cacheflush.h | 8 ++++---- arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 3 ++- arch/arm64/kernel/hibernate.c | 18 +++++++++++------- arch/arm64/kernel/idreg-override.c | 3 ++- arch/arm64/kernel/kaslr.c | 12 +++++++++--- arch/arm64/kernel/machine_kexec.c | 20 +++++++++++++------- arch/arm64/kernel/smp.c | 8 ++++++-- arch/arm64/kernel/smp_spin_table.c | 7 ++++--- arch/arm64/kvm/hyp/nvhe/cache.S | 1 - arch/arm64/kvm/hyp/nvhe/setup.c | 3 ++- arch/arm64/kvm/hyp/pgtable.c | 13 ++++++++++--- arch/arm64/mm/cache.S | 9 ++++----- 14 files changed, 70 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 934b9be582d2..ed1cc9d8e6df 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -124,7 +124,8 @@ static inline u32 gic_read_rpr(void) #define gic_read_lpir(c) readq_relaxed(c) #define gic_write_lpir(v, c) writeq_relaxed(v, c) -#define gic_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l)) +#define gic_flush_dcache_to_poc(a,l) \ + __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) #define gits_read_baser(c) readq_relaxed(c) #define gits_write_baser(v, c) writeq_relaxed(v, c) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 157234706817..695f88864784 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -50,15 +50,15 @@ * - start - virtual start address * - end - virtual end address * - * __flush_dcache_area(kaddr, size) + * __flush_dcache_area(start, end) * * Ensure that the data held in page is written back. - * - kaddr - page address - * - size - region size + * - start - virtual start address + * - end - virtual end address */ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); -extern void __flush_dcache_area(void *addr, size_t len); +extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pop(void *addr, size_t len); diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 3578aba9c608..0ae2397076fd 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -137,7 +137,7 @@ void efi_virtmap_unload(void); static inline void efi_capsule_flush_cache_range(void *addr, int size) { - __flush_dcache_area(addr, size); + __flush_dcache_area((unsigned long)addr, (unsigned long)addr + size); } #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 25ed956f9af1..33293d5855af 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -180,7 +180,8 @@ static inline void *__kvm_vector_slot2addr(void *base, struct kvm; -#define kvm_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l)) +#define kvm_flush_dcache_to_poc(a,l) \ + __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..b40ddce71507 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -240,8 +240,6 @@ static int create_safe_exec_page(void *src_start, size_t length, return 0; } -#define dcache_clean_range(start, end) __flush_dcache_area(start, (end - start)) - #ifdef CONFIG_ARM64_MTE static DEFINE_XARRAY(mte_pages); @@ -383,13 +381,18 @@ int swsusp_arch_suspend(void) ret = swsusp_save(); } else { /* Clean kernel core startup/idle code to PoC*/ - dcache_clean_range(__mmuoff_data_start, __mmuoff_data_end); - dcache_clean_range(__idmap_text_start, __idmap_text_end); + __flush_dcache_area((unsigned long)__mmuoff_data_start, + (unsigned long)__mmuoff_data_end); + __flush_dcache_area((unsigned long)__idmap_text_start, + (unsigned long)__idmap_text_end); /* Clean kvm setup code to PoC? */ if (el2_reset_needed()) { - dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); - dcache_clean_range(__hyp_text_start, __hyp_text_end); + __flush_dcache_area( + (unsigned long)__hyp_idmap_text_start, + (unsigned long)__hyp_idmap_text_end); + __flush_dcache_area((unsigned long)__hyp_text_start, + (unsigned long)__hyp_text_end); } swsusp_mte_restore_tags(); @@ -474,7 +477,8 @@ int swsusp_arch_resume(void) * The hibernate exit text contains a set of el2 vectors, that will * be executed at el2 with the mmu off in order to reload hyp-stub. */ - __flush_dcache_area(hibernate_exit, exit_size); + __flush_dcache_area((unsigned long)hibernate_exit, + (unsigned long)hibernate_exit + exit_size); /* * KASLR will cause the el2 vectors to be in a different location in diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index e628c8ce1ffe..3dd515baf526 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -237,7 +237,8 @@ asmlinkage void __init init_feature_override(void) for (i = 0; i < ARRAY_SIZE(regs); i++) { if (regs[i]->override) - __flush_dcache_area(regs[i]->override, + __flush_dcache_area((unsigned long)regs[i]->override, + (unsigned long)regs[i]->override + sizeof(*regs[i]->override)); } } diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 341342b207f6..49cccd03cb37 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -72,7 +72,9 @@ u64 __init kaslr_early_init(void) * we end up running with module randomization disabled. */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - __flush_dcache_area(&module_alloc_base, sizeof(module_alloc_base)); + __flush_dcache_area((unsigned long)&module_alloc_base, + (unsigned long)&module_alloc_base + + sizeof(module_alloc_base)); /* * Try to map the FDT early. If this fails, we simply bail, @@ -170,8 +172,12 @@ u64 __init kaslr_early_init(void) module_alloc_base += (module_range * (seed & ((1 << 21) - 1))) >> 21; module_alloc_base &= PAGE_MASK; - __flush_dcache_area(&module_alloc_base, sizeof(module_alloc_base)); - __flush_dcache_area(&memstart_offset_seed, sizeof(memstart_offset_seed)); + __flush_dcache_area((unsigned long)&module_alloc_base, + (unsigned long)&module_alloc_base + + sizeof(module_alloc_base)); + __flush_dcache_area((unsigned long)&memstart_offset_seed, + (unsigned long)&memstart_offset_seed + + sizeof(memstart_offset_seed)); return offset; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a03944fd0cd4..3e79110c8f3a 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -72,7 +72,9 @@ int machine_kexec_post_load(struct kimage *kimage) * For execution with the MMU off, reloc_code needs to be cleaned to the * PoC and invalidated from the I-cache. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area((unsigned long)reloc_code, + (unsigned long)reloc_code + + arm64_relocate_new_kernel_size); invalidate_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); @@ -106,16 +108,18 @@ static void kexec_list_flush(struct kimage *kimage) for (entry = &kimage->head; ; entry++) { unsigned int flag; - void *addr; + unsigned long addr; /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); + __flush_dcache_area((unsigned long)entry, + (unsigned long)entry + + sizeof(kimage_entry_t)); flag = *entry & IND_FLAGS; if (flag == IND_DONE) break; - addr = phys_to_virt(*entry & PAGE_MASK); + addr = (unsigned long)phys_to_virt(*entry & PAGE_MASK); switch (flag) { case IND_INDIRECTION: @@ -124,7 +128,7 @@ static void kexec_list_flush(struct kimage *kimage) break; case IND_SOURCE: /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); + __flush_dcache_area(addr, addr + PAGE_SIZE); break; case IND_DESTINATION: break; @@ -151,8 +155,10 @@ static void kexec_segment_flush(const struct kimage *kimage) kimage->segment[i].memsz, kimage->segment[i].memsz / PAGE_SIZE); - __flush_dcache_area(phys_to_virt(kimage->segment[i].mem), - kimage->segment[i].memsz); + __flush_dcache_area( + (unsigned long)phys_to_virt(kimage->segment[i].mem), + (unsigned long)phys_to_virt(kimage->segment[i].mem) + + kimage->segment[i].memsz); } } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index dcd7041b2b07..5fcdee331087 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -122,7 +122,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) secondary_data.task = idle; secondary_data.stack = task_stack_page(idle) + THREAD_SIZE; update_cpu_boot_status(CPU_MMU_OFF); - __flush_dcache_area(&secondary_data, sizeof(secondary_data)); + __flush_dcache_area((unsigned long)&secondary_data, + (unsigned long)&secondary_data + + sizeof(secondary_data)); /* Now bring the CPU into our world */ ret = boot_secondary(cpu, idle); @@ -143,7 +145,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) pr_crit("CPU%u: failed to come online\n", cpu); secondary_data.task = NULL; secondary_data.stack = NULL; - __flush_dcache_area(&secondary_data, sizeof(secondary_data)); + __flush_dcache_area((unsigned long)&secondary_data, + (unsigned long)&secondary_data + + sizeof(secondary_data)); status = READ_ONCE(secondary_data.status); if (status == CPU_MMU_OFF) status = READ_ONCE(__early_cpu_boot_status); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index c45a83512805..58d804582a35 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -36,7 +36,7 @@ static void write_pen_release(u64 val) unsigned long size = sizeof(secondary_holding_pen_release); secondary_holding_pen_release = val; - __flush_dcache_area(start, size); + __flush_dcache_area((unsigned long)start, (unsigned long)start + size); } @@ -90,8 +90,9 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(pa_holding_pen, release_addr); - __flush_dcache_area((__force void *)release_addr, - sizeof(*release_addr)); + __flush_dcache_area((__force unsigned long)release_addr, + (__force unsigned long)release_addr + + sizeof(*release_addr)); /* * Send an event to wake up the secondary CPU. diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 3bcfa3cac46f..36cef6915428 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -8,7 +8,6 @@ #include SYM_FUNC_START_PI(__flush_dcache_area) - add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 7488f53b0aa2..5dffe928f256 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -134,7 +134,8 @@ static void update_nvhe_init_params(void) for (i = 0; i < hyp_nr_cpus; i++) { params = per_cpu_ptr(&kvm_init_params, i); params->pgd_pa = __hyp_pa(pkvm_pgtable.pgd); - __flush_dcache_area(params, sizeof(*params)); + __flush_dcache_area((unsigned long)params, + (unsigned long)params + sizeof(*params)); } } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index c37c1dc4feaf..10d2f04013d4 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -839,8 +839,11 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, stage2_put_pte(ptep, mmu, addr, level, mm_ops); if (need_flush) { - __flush_dcache_area(kvm_pte_follow(pte, mm_ops), - kvm_granule_size(level)); + kvm_pte_t *pte_follow = kvm_pte_follow(pte, mm_ops); + + __flush_dcache_area((unsigned long)pte_follow, + (unsigned long)pte_follow + + kvm_granule_size(level)); } if (childp) @@ -988,11 +991,15 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct kvm_pgtable *pgt = arg; struct kvm_pgtable_mm_ops *mm_ops = pgt->mm_ops; kvm_pte_t pte = *ptep; + kvm_pte_t *pte_follow; if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pgt, pte)) return 0; - __flush_dcache_area(kvm_pte_follow(pte, mm_ops), kvm_granule_size(level)); + pte_follow = kvm_pte_follow(pte, mm_ops); + __flush_dcache_area((unsigned long)pte_follow, + (unsigned long)pte_follow + + kvm_granule_size(level)); return 0; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index fff883f691f2..b2880aeba7ca 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -99,16 +99,15 @@ alternative_else_nop_endif SYM_FUNC_END(invalidate_icache_range) /* - * __flush_dcache_area(kaddr, size) + * __flush_dcache_area(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned and invalidated to the PoC. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_PI(__flush_dcache_area) - add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) From patchwork Mon May 24 08:29:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1171AC04FF3 for ; Mon, 24 May 2021 17:37:15 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D003D613EA for ; Mon, 24 May 2021 17:37:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D003D613EA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=FNpt37jykL2+UofIn3ul4hxykAdbAQLkkTuc1tfOolA=; b=10sw5hYpslVIjcTcimOT/+FSJY 5kZxRliIcVtZt8oV/U/1kbZeU9ac56UhERtxSsuyV6w0zg1mdk1M8L596TrbXSDLUHCTvVej/Xgac U2v3D06ucGStZhgzjMC9sHjT2mBqEY+TaiQ/lFuHXsVAFR4zjLMq9KBSoyUAL6P9s7T8EEqgXl8Os k6jUXwLScrRqgwMM/RqZEUwxScknq77tokApTk6E3e/KNSKo9jORqINBj1vPdoRbadE+VpwAqztrb rarH3ysNLlaBMxbJe3fy0h6r4cZTqs8TxEe0yScDU3Y4XXpjn71Jwtk2IW5U5DnvPaCOh3X2SGzAt mxMpHaNA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llESt-001IJp-7N; Mon, 24 May 2021 17:34:07 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yq-000lsO-QU for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:34 +0000 Received: by mail-wr1-x44a.google.com with SMTP id a5-20020a5d6ca50000b029011035a261adso12675733wra.17 for ; Mon, 24 May 2021 01:30:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=gMcjPTDSwJxNF67ObDKNLgquzHcYrBN6lFdOGjj3iks=; b=gN+z8Z4/HT64FQMguXBwyosv3++vDm/tu6tWYpyRdHu5DbI2x/lWQc0CAWrqzTLm0H OOP8aKwDmtBp6o3EE4eQFrR4ifIGMVjE0Uu1dtPuShYxru5yOW3Ukt4ftj8CbzD9VtDC jDgE54Tzv4Q1GBWn8XeOSUHsKdnSCvL/G5aobwrL78g2mvragX7VV7vpJfo6/KwDxg/p ccgW2GfO5K8ynd7nKPPCf98iYpuOZ20AWHcaDlMgtfb6ccKeJ9Y4TwTw+BsFmJqpZUH0 lDP8b5dvG0jij/YqsuuuoTPI3xb0Uj06o/VVfpIEYPQjf03RWb41L4SbaFKv6yYTuCdM SRxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=gMcjPTDSwJxNF67ObDKNLgquzHcYrBN6lFdOGjj3iks=; b=Ki0gZ6CAKJ2BY/Q/MEbklRe8n6MkTyiGmb50uCtNp/kYifyOWqZUlKy0aVvd9b2+G+ GUyea9ZpHediiZODVq48YbwPlbX0pSNf241fzqLuR5BVnjrVapqfYO1LmIMMMU8m0Fo9 Aymwu7VQ+V+VjLTAihMehuof9coIxNbrZg0KZvtOr7Wb8e2A6JNd8yaoiicP7Kma+FKq Jvo51G3boNh2TBF7G5sqpVp4Nnw5654sK2MK4gpXjNTE1XhO3X0wdTQwJk298d9i5nmc 02Be/ZqXzSVA0Q4X9sZTVizIQQOpk04q3NtwWqtBferujGdj4dAtHxb7zmyZeP91jHXl qVQg== X-Gm-Message-State: AOAM533/aRh6Kq7fquNUv9DzC/mRB/kZMxePX4TyaZDtL7bKmLX6E3dP WvzwVwCMvwg34LsZWFCnI4HYIqKklEW+kAL2KN8imN/1m4+LrKl0cxKNVBuOR2togYBUjDvzCyq aCr1OrEqXXs7222qr4vW9QuU1SyqYx+zeN2hflcVw6kQKV5wW8ARyrtIcfPmHbfYIIprLf++64Z ChMQ== X-Google-Smtp-Source: ABdhPJxLBeeRikE0S1XkHcb9+TlfIq6uhs+mn8IYDxrMo7K6d3TJu1lkmZ1BW/+V3rq53r0Vf01amAyoEw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c084:: with SMTP id r4mr18084864wmh.102.1621845030828; Mon, 24 May 2021 01:30:30 -0700 (PDT) Date: Mon, 24 May 2021 09:29:56 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-14-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 13/18] arm64: __clean_dcache_area_poc to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013032_879421_04FAA32B X-CRM114-Status: GOOD ( 15.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. Because the code is shared with __dma_clean_area, it changes the parameters for that as well. However, __dma_clean_area is local to cache.S, so no other users are affected. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/efi-entry.S | 5 +++-- arch/arm64/mm/cache.S | 16 +++++++--------- 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 695f88864784..3255878d6f30 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -60,7 +60,7 @@ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); -extern void __clean_dcache_area_poc(void *addr, size_t len); +extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 0073b24b5d25..b0f728fb61f0 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -28,6 +28,7 @@ SYM_CODE_START(efi_enter_kernel) * stale icache entries from before relocation. */ ldr w1, =kernel_size + add x1, x0, x1 bl __clean_dcache_area_poc ic ialluis @@ -36,7 +37,7 @@ SYM_CODE_START(efi_enter_kernel) * so that we can safely disable the MMU and caches. */ adr x0, 0f - ldr w1, 3f + adr x1, 3f bl __clean_dcache_area_poc 0: /* Turn off Dcache and MMU */ @@ -64,5 +65,5 @@ SYM_CODE_START(efi_enter_kernel) mov x2, xzr mov x3, xzr br x19 +3: SYM_CODE_END(efi_enter_kernel) -3: .long . - 0b diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index b2880aeba7ca..e2e2740c55ce 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -171,24 +171,23 @@ SYM_FUNC_END_PI(__inval_dcache_area) SYM_FUNC_END(__dma_inv_area) /* - * __clean_dcache_area_poc(kaddr, size) + * __clean_dcache_area_poc(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoC. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_LOCAL(__dma_clean_area) SYM_FUNC_START_PI(__clean_dcache_area_poc) /* FALLTHROUGH */ /* - * __dma_clean_area(start, size) + * __dma_clean_area(start, end) * - start - virtual start address of region - * - size - size in question + * - end - virtual end address of region */ - add x1, x0, x1 dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_poc) @@ -204,10 +203,10 @@ SYM_FUNC_END(__dma_clean_area) * - size - size in question */ SYM_FUNC_START_PI(__clean_dcache_area_pop) + add x1, x0, x1 alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif - add x1, x0, x1 dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_pop) @@ -236,7 +235,6 @@ SYM_FUNC_START_PI(__dma_map_area) add x1, x0, x1 cmp w2, #DMA_FROM_DEVICE b.eq __dma_inv_area - sub x1, x1, x0 b __dma_clean_area SYM_FUNC_END_PI(__dma_map_area) From patchwork Mon May 24 08:29:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6AC81C2B9F7 for ; Mon, 24 May 2021 23:52:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D1A8613E6 for ; Mon, 24 May 2021 23:52:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D1A8613E6 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=DR1J8iCaeAsDXT2XFeoPNMkf8WaJyBs2fnfP9yjaZCw=; b=ZwXDvSNA3yyoIjoM7iKsEDUNPV Vi8m+kfrzipzl6cFnuG+ciIo7y9VUPi0qenRrwbpb9zXhwzuW72TUl98TTJ2L/5Os6tRHThb/bW5Y cQ8cUUP9ARjuTf0acjbd63pBODJhAt33oJuVVFo7OXuvK5lft3g9DYjLYyM67W03q/xzmIPCOETfp I2Z9FV/vc3uzV36S/0hgXv3kOqakmO4ZvajHAKMSzdSA6hiwnB5EXj+p3B5zEEWmbBkWA3I42M1wX DJvWeoamrr0wetG7KO+KOPPTyKWNuYPpU/L+Hstwun5405sBD4CMo0QRrHeA2l3RZGTISKFf/LwMz yBXd2A7A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llKJO-002PPI-A0; Mon, 24 May 2021 23:48:43 +0000 Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yt-000lsk-3X for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:36 +0000 Received: by mail-wr1-x449.google.com with SMTP id u20-20020a0560001614b02901115c8f2d89so12603682wrb.3 for ; Mon, 24 May 2021 01:30:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PcjnW/fqCoXIV+w7Sz+O7O7ITwOtt0VA+eV7Tp0TNzU=; b=Kwccq5VpcLpcnotaXso3PRfJIrrNRgRVYUsG0n0NezLoTdcXiTOOYe1xFPu6SSrTAs B4rcfIVCpZrsh3yxBJNgb7QqzUjBlb2Mgfvz6OjhYWor2uhxsmWa3L0cTfCTzM6xMSyz aDb4PBm3O+q2aLai3341egqyJHiSY1US8pEedexUOP0/tEvgd3SMZxdQL/wiJy5punQX HD+JFlsrSpuBF1520GbEqQ6LZKcTa1kGdCu/QcY00tAIrxpUokneINXQflB7OaP3TF5E GJdTD5KLdt63LSJwR2K78mY6aV6bwNy4xwxYVxbTMIb8tuiDfmINBIOmIUgC3Wylm94y Y3Qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PcjnW/fqCoXIV+w7Sz+O7O7ITwOtt0VA+eV7Tp0TNzU=; b=F55kiNMfMG1jyuxkfqyvngTaKIqfI1f65HnVOOjila/GD8JBvNI1EmW2VCTX24g8p0 yZhLhag6Vw4kbw9y8uWkgiWpR6JFUwD5bIT7oFvfA6K4qJETd1v9DgsfWTjFDB3a0eVH 45/to1krd5mNCmrzDJ51Tdaa6lbvdG4G/9EDji/vDarEhWxRr3EvHFZGDTihRH9PldrA OYCMpTPSaUqZI43WgLIBv+pcyrbsM3xkv0AmRuJWRu9ujol10YxsrgJ7JqaJah7DQIdq vo95juDyBHciqAzda6TuORKqgrl9a6LY44XeJw9UUVhEJoc4Q9fbfcAAb6BTktPicXC1 tCDA== X-Gm-Message-State: AOAM531+1H3lxIQPGO91wy9IOCKQH0SNKVR+y5HUL0DcfbHrCHVWFE3z MY8gt9niKAZ62foufyPsMZ4URUMnOykTtzBfe4KFd71p8alWITvsle35nL2SVpd0H3nHemBSiMR q9+nsJh8DyvK6mKMTjclxMNflFok0UthrSRO91XFis8439ybw3FAIg5qbcddR3cOmz0cH8qLP3C GZSQ== X-Google-Smtp-Source: ABdhPJyiJFHwvcoelZAsbRCZ5mRK3oDcPgriYNUO90yNXriCgsqJRWjdKW4CKrcwl4M+VLC03tdcy3yZSw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a5d:534f:: with SMTP id t15mr20252107wrv.206.1621845033001; Mon, 24 May 2021 01:30:33 -0700 (PDT) Date: Mon, 24 May 2021 09:29:57 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-15-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 14/18] arm64: __clean_dcache_area_pop to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013035_171088_0A14A2C0 X-CRM114-Status: GOOD ( 14.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/lib/uaccess_flushcache.c | 4 ++-- arch/arm64/mm/cache.S | 9 ++++----- arch/arm64/mm/flush.c | 2 +- 4 files changed, 8 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 3255878d6f30..fa5641868d65 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -61,7 +61,7 @@ extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pop(void *addr, size_t len); +extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); extern void sync_icache_aliases(void *kaddr, unsigned long len); diff --git a/arch/arm64/lib/uaccess_flushcache.c b/arch/arm64/lib/uaccess_flushcache.c index c83bb5a4aad2..62ea989effe8 100644 --- a/arch/arm64/lib/uaccess_flushcache.c +++ b/arch/arm64/lib/uaccess_flushcache.c @@ -15,7 +15,7 @@ void memcpy_flushcache(void *dst, const void *src, size_t cnt) * barrier to order the cache maintenance against the memcpy. */ memcpy(dst, src, cnt); - __clean_dcache_area_pop(dst, cnt); + __clean_dcache_area_pop((unsigned long)dst, (unsigned long)dst + cnt); } EXPORT_SYMBOL_GPL(memcpy_flushcache); @@ -33,6 +33,6 @@ unsigned long __copy_user_flushcache(void *to, const void __user *from, rc = raw_copy_from_user(to, from, n); /* See above */ - __clean_dcache_area_pop(to, n - rc); + __clean_dcache_area_pop((unsigned long)to, (unsigned long)to + n - rc); return rc; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index e2e2740c55ce..b71fcf56516b 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -194,16 +194,15 @@ SYM_FUNC_END_PI(__clean_dcache_area_poc) SYM_FUNC_END(__dma_clean_area) /* - * __clean_dcache_area_pop(kaddr, size) + * __clean_dcache_area_pop(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoP. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_PI(__clean_dcache_area_pop) - add x1, x0, x1 alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index be650b573b2a..b2c226d93ca5 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -84,7 +84,7 @@ void arch_wb_cache_pmem(void *addr, size_t size) { /* Ensure order against any prior non-cacheable writes */ dmb(osh); - __clean_dcache_area_pop(addr, size); + __clean_dcache_area_pop((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); From patchwork Mon May 24 08:29:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33E24C2B9F7 for ; Mon, 24 May 2021 23:03:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DBD046138C for ; Mon, 24 May 2021 23:03:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DBD046138C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PHB+XveAMemT6P0Yrm2gmVrLTONUFArWScE4d8htGag=; b=AjhEv8QDNchMpO10Md5TxwdstE vq1Yj8/a6O6EZwr4fXIQkXjeXF8cJkFZ3IIdXzmBltzOQ1xR6Vgawn4pz0RynXaA8HutEe/V1olVJ xMPj4U7bpKcyeXDrTJmBF0mMvcNRvV4scc7O5WtUOrcwjp//ivyc6nRox+WFNM45DOSRjQRkOZWwa EuVpc6yE5yRIlHyeadHhyDpXDXQNo6TXOaHaZ9W05Xqizrc4/XAo/JnGj2oxkt/fgt2A7uG6Ee6Lp zXVbiDS6DRNDG+xGWB6S7ieD/RJlKq4QW2ytb+vPf5vd4AjB19fdwn9WP+rnbG7rrUJYrA53T/49g V5ayFqNQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llJZ6-002Fk4-Qj; Mon, 24 May 2021 23:00:55 +0000 Received: from mail-ej1-x64a.google.com ([2a00:1450:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yw-000lt3-FM for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:39 +0000 Received: by mail-ej1-x64a.google.com with SMTP id p18-20020a1709067852b02903dab2a3e1easo4324048ejm.17 for ; Mon, 24 May 2021 01:30:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=OUgnP7QwwjnFyJeMkmUD5OgbsQQkr0a4dVJNwaNnXqo=; b=rC5IO/i5qgfWXKnNsmTO3yPg3khXDMnrh27ppMYG+ncCNWUngHr9jnzf26Mbb0Fk1Y spUA/+RJvL8LYl0OGwvPLODP6PNbcNfPm5LxrvLWUMyFajM/+vMmRliqcYMv1iKx91D7 V2bTbRDUuS/mWMW+1xlsHcAX9dW99uTlV0RSa+mgvmZGwlXBPTbet1J8Ph1faNdNY5c7 MqrWciXrTqojfVK7MENJ0SI+Ahe9kflz2v4zhU3z/1KzZl+xB4F3Ara5sqVgskrNI3Pq KBau7jJEKR7Qbu1zXw8q2zHjGqlGLq1mD0PhDZxtKWwMDQQl4aD9Ys4c3uHEhgrNldkO 2hkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=OUgnP7QwwjnFyJeMkmUD5OgbsQQkr0a4dVJNwaNnXqo=; b=DgtFcvxJwfz1olLip49fOWRdaklg71Fen+smwyQGiRphypGobScDR+VvKX7FEkcx57 g0I1CKLHolBX+sKSmaqepRaq7xlqiGZFGQHjiSvpnskW7+Pcte7Xl4/Z/RAUGK1RZy9i lKksIGR2IaQ1euIpeC2yaii53tAssi1nhrYZIllPMzkFYxioX1gXydkHk27sd6gQ2x2K 3OzAV7TKi+rGxbQhALRL4k/qtWJmvLUqi3GnlMSVYwbcM2JpxXh9xIPAOHUK5luDUX62 jxZ9a+yt6lKuwGZa0dGQcr6yOG70BBgrSSuAnbZ0M/lcAC7o+xfRwhv83MeDIOj1Ml8D Tnxw== X-Gm-Message-State: AOAM533z4BYE/MZvfNdRUcwWLyTs0dnb8RhERhBPznsZ821IM8d4RmLL kBHq6lNqkTmUKErz3sSlg838tHd9gZCm1UFwPOIphZqSUjqCIDhUequXTiQ1TeqL/YjMcqwzhFl OgqA7+cV0fpw5YcMEODXG1zhmeNWhyU9NZSLhzFBA7iRfHnUDmEkgoJTE7tBInaQnZRrWCYFq7h b0pg== X-Google-Smtp-Source: ABdhPJzatlczQl2VKZq52n8QP167mIDcmSGYegqWr9k+5mx0JvklQ1c4S1ZX6jGqHv/PUoXHNf1Hdo/UTg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:aa7:cd55:: with SMTP id v21mr25136643edw.344.1621845034917; Mon, 24 May 2021 01:30:34 -0700 (PDT) Date: Mon, 24 May 2021 09:29:58 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-16-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 15/18] arm64: __clean_dcache_area_pou to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013038_537499_8EA11024 X-CRM114-Status: GOOD ( 13.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/mm/cache.S | 9 ++++----- arch/arm64/mm/flush.c | 2 +- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index fa5641868d65..f86723047315 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -62,7 +62,7 @@ extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pou(void *addr, size_t len); +extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); extern long __flush_cache_user_range(unsigned long start, unsigned long end); extern void sync_icache_aliases(void *kaddr, unsigned long len); diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index b71fcf56516b..ea605d94182f 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -113,20 +113,19 @@ SYM_FUNC_START_PI(__flush_dcache_area) SYM_FUNC_END_PI(__flush_dcache_area) /* - * __clean_dcache_area_pou(kaddr, size) + * __clean_dcache_area_pou(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoU. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START(__clean_dcache_area_pou) alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif - add x1, x0, x1 dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret SYM_FUNC_END(__clean_dcache_area_pou) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index b2c226d93ca5..0341bcc6fdf3 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -19,7 +19,7 @@ void sync_icache_aliases(void *kaddr, unsigned long len) unsigned long addr = (unsigned long)kaddr; if (icache_is_aliasing()) { - __clean_dcache_area_pou(kaddr, len); + __clean_dcache_area_pou(kaddr, kaddr + len); __flush_icache_all(); } else { /* From patchwork Mon May 24 08:29:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0303C04FF3 for ; Mon, 24 May 2021 19:25:27 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 926FA613F6 for ; Mon, 24 May 2021 19:25:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 926FA613F6 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=+q6pU83EKdC/djnI4CBoW/wljMUPFesbaQugQW/YnAs=; b=3oWAcpoHCNy4vOIyg78CV+iWI7 IuSHr4cGU8PdzbJHKPimBC2MKZKkJ6ZmHrL9Ja8/dGGWfLvBFbh3b2J33DCO33TgnR6Xt5pXkimQw Z/eIEbonptFHgApXdQINhBfSHzMmZDHCVmXJZKFI4UxtyD7FyaH3T2H4l21Pb/FO5e6rOpVQ1LtlQ D77GEK9Lec79ry0oLgrmta9o1pEiwvbsxQhTV+0XR6SzU4B/N2VywOIjsufbHWZcuqg3vBZx+RLpy qpFW323QDwEPP8c+2GBWzi2+tTXety5DI6IiN35ZN34pf2pMCHG+2f2ukW1TSw+TQZF6lFZqvXoxX mlXBYKhw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llGAE-001eUl-Sd; Mon, 24 May 2021 19:23:01 +0000 Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yx-000ltO-Ds for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:40 +0000 Received: by mail-wr1-x44a.google.com with SMTP id 1-20020adf93810000b029010fd5ac4ed7so12740726wrp.15 for ; Mon, 24 May 2021 01:30:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=B2oqCjWGiDlRsgi1uC95SoV3X4XsCQk4Vg0zao3opro=; b=TW8xck4vBuEkcHGvsJEsB7jRxU8mTLGTuSf5Y0zDhvBzlbNKuyDQM5hR20pfL/gGRV Any11WfSaUcqU3dnuwr/1nh6KlZ0A/qOpduuLcVDrWhGQCTsiKklPm8VOQxtvxwwdKor 6fOpWPkzYwp6UZ4x+tIMc28DHd4s5rTjePfcm+5lBxEwoYxLRoo9B5SkPTdlOwpUEotZ +je0wAxa3E+IIo0ETl3uLehpseBQqPzr5q0FJM/CR0P+ZwjSYvX6VmcP3PbXWwLioXuC Pl25tI6UKP6tLKcN4gpszLr/sx/D42zTZzlIwqtmGxIqx7UTIiuWAgYU0ERj/UEPfvdN 2xQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=B2oqCjWGiDlRsgi1uC95SoV3X4XsCQk4Vg0zao3opro=; b=t6ofValbuMYq3Jb797s0dwRuVuzt8QCvKIXRtnaVCa2+B5rX3ppo1feNgqooRFbo0s qfvndlU3nfcaWkc/kIzMP2GUZQH8bwp+P7+wdsHDX638S1ctWnWD6+PGYST4+wKeG+eR b0VtkRGD+ZeqZxGbiMLaGvglLqFmnsqWmal6VQh4uKfc8A5RxkcFwWNoY8couHPSROFR MLYKaLs+8ULbQTqS4Q4fxSnCJv+WyEfaEx6YqjNUR15wDK3j223e9DoIooYisGKy2n8p um+xPfiX4ptqr1lu84wV2LfEBqlty3Zi/1JxbQJtxaCv/BDEdkl6RNrJnRVfFznC5+w4 HLEQ== X-Gm-Message-State: AOAM530I4KmVBZAt0pcW0W2Q3o6/cy2apgQ7P95yjY50wXCzO9cHeRCS BsSUSEfGAU7g3RrVHvothKiTzT21SnI0b2cTM7bj0MQEga9i/8lCn/rpYykpZkfC5z0/PPonMk4 YY/JIHgXkEiL9sJfWnE8IGOma+KPZerRb4p6nyzDEFEZ6lJvyLSXXp+XNX802tgaIHUQuYD3xuy W9JA== X-Google-Smtp-Source: ABdhPJxO0fMchjgrm8S/EOQSSYYuztAK0OfBME10JPKxZA4eRypRvUKSJHh79hfXMG6vS3K1SjQ+7Qps/Q== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:35cc:: with SMTP id r12mr2767070wmq.1.1621845036737; Mon, 24 May 2021 01:30:36 -0700 (PDT) Date: Mon, 24 May 2021 09:29:59 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-17-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 16/18] arm64: sync_icache_aliases to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013039_488866_5C0773C4 X-CRM114-Status: GOOD ( 14.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/probes/uprobes.c | 2 +- arch/arm64/mm/flush.c | 21 ++++++++++----------- 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index f86723047315..70b389a8dea5 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -64,7 +64,7 @@ extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); extern long __flush_cache_user_range(unsigned long start, unsigned long end); -extern void sync_icache_aliases(void *kaddr, unsigned long len); +extern void sync_icache_aliases(unsigned long start, unsigned long end); static inline void flush_icache_range(unsigned long start, unsigned long end) { diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c index 2c247634552b..9be668f3f034 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -21,7 +21,7 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, memcpy(dst, src, len); /* flush caches (dcache/icache) */ - sync_icache_aliases(dst, len); + sync_icache_aliases((unsigned long)dst, (unsigned long)dst + len); kunmap_atomic(xol_page_kaddr); } diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 0341bcc6fdf3..c4ca7e05fdb8 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -14,28 +14,25 @@ #include #include -void sync_icache_aliases(void *kaddr, unsigned long len) +void sync_icache_aliases(unsigned long start, unsigned long end) { - unsigned long addr = (unsigned long)kaddr; - if (icache_is_aliasing()) { - __clean_dcache_area_pou(kaddr, kaddr + len); + __clean_dcache_area_pou(start, end); __flush_icache_all(); } else { /* * Don't issue kick_all_cpus_sync() after I-cache invalidation * for user mappings. */ - __flush_icache_range(addr, addr + len); + __flush_icache_range(start, end); } } -static void flush_ptrace_access(struct vm_area_struct *vma, struct page *page, - unsigned long uaddr, void *kaddr, - unsigned long len) +static void flush_ptrace_access(struct vm_area_struct *vma, unsigned long start, + unsigned long end) { if (vma->vm_flags & VM_EXEC) - sync_icache_aliases(kaddr, len); + sync_icache_aliases(start, end); } /* @@ -48,7 +45,7 @@ void copy_to_user_page(struct vm_area_struct *vma, struct page *page, unsigned long len) { memcpy(dst, src, len); - flush_ptrace_access(vma, page, uaddr, dst, len); + flush_ptrace_access(vma, (unsigned long)dst, (unsigned long)dst + len); } void __sync_icache_dcache(pte_t pte) @@ -56,7 +53,9 @@ void __sync_icache_dcache(pte_t pte) struct page *page = pte_page(pte); if (!test_bit(PG_dcache_clean, &page->flags)) { - sync_icache_aliases(page_address(page), page_size(page)); + sync_icache_aliases((unsigned long)page_address(page), + (unsigned long)page_address(page) + + page_size(page)); set_bit(PG_dcache_clean, &page->flags); } } From patchwork Mon May 24 08:30:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12277387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA35EC2B9F7 for ; Tue, 25 May 2021 00:14:38 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A7DD6613F5 for ; Tue, 25 May 2021 00:14:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7DD6613F5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=IvefBwCXUxUoi8JQAzXZEkW2mU3xMhWTSOd+m/txpKw=; b=g6q4gIKG/PFSYsnxBww9i00cFN Q16D5aNnSlZ6F0KBgOAvShrf7jdkA+JTIrTaPXVNHM8v0CXs91glr0qkcUu0agB57OWxSaFRJLEMX KbhjP115Upg4K0nHCxJ776bjYfptQJOwph1J1mua2e6/6zT+akUTobgd84UYoizfsPEDkqJzBsNr4 GObfkjIhQkpvo9pxjaOgeYQETycByByEoIOUYwwjtUcTYeaovxCHwUu7oE4IKGkLf3RbsjtlgJto1 d/JwIw10+ZQRj45JPr4V1baA5TPyABF/qSInvzQXeHc9XPc5DK5V+hpzuyBVAWapST/+GqvggAWpn bD/hWjCQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llKf3-002U9j-Ux; Tue, 25 May 2021 00:11:07 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5yy-000ltc-OM for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:42 +0000 Received: by mail-qt1-x84a.google.com with SMTP id 1-20020aed31010000b029019d1c685840so23189625qtg.3 for ; Mon, 24 May 2021 01:30:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=LqapZkW6bGcuDXoTSxJIrLt2XyVt6VhDgfCDGVD2pNA=; b=ngyap7HPhZiBl+Iwho0v5Hfq7o+GPuU5TByaDUwHa1r+F3viDb6lvGf9Hpkyi3XUgN nIhpHj1HznVjEXt3zWMwS14eaYiTYBumvvURkubyxSulbjD2RFMfz0HQMLUdTxVJag6Y wrVhVSvltQQVTL+EGLR/Cl6lkFkuHemzttDDc2XeELhA77FTSlvN/VF2OUGJ8Vy3ImSo q5rAXs7+JBAN3xyvFsbZdS00pnh982nZSz9iF+iQkDtOx6qhtbDQZLR5NP5LkWHF9IRh lkwQRZ0AdQsXh9BnQ8Q0ECzMDNpYtmBQQRP+rVTvp3m73KO/UARn/z9UHiPKd1JkVSyF zrLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=LqapZkW6bGcuDXoTSxJIrLt2XyVt6VhDgfCDGVD2pNA=; b=CATrnt9r6aj5PIu3UpbUqxZaLvmt9KZ4V3wawu8RAjvAkCiqw5wA5skXVVgdo7eNrj 5OFsixm4wfu+jMJZwnHweQnAxPaD88KKC3xmRB5dEgQOFNoECnAvwjHpRiHlXqTSYQrk cGEQhxCCiVLdM9afV34SiDrsnfbFkdBvWqi5DDZF+3D8Z0za8nbj0bOZUIeupBPk06GE 6cYPf4KEEETNQhzcZzRuwHro0jSkhONTlLEXy3gaa0RVNN/UNhffN8Lhv3TfL9WwbgzS U6UrHVZwBbPFr0uecDUtnHbojxmhrwFd2V4vgufRLJ1ytn53CT7G5U1hgbApoSTGSfpr jZVQ== X-Gm-Message-State: AOAM532MDfiS4ImvPSQMfKgEbxd5NuRJuw6/0HL6fm5G4l6MEoE8G48u tAMI6AZYclPcVh5TFrUWeaUdiLj7bMjazbeDMM67MZaY5jgM2wO6LADi/rfY+ZcXafJolvkkAtQ 2OXuGL6TFO1h+DD3FDW2SAkiRdkOYwCG1jgDXInpLQsfcugdGrIMxAKKFk0b2mpBK8gJhQqfHDA HN+A== X-Google-Smtp-Source: ABdhPJw71FDcNI9iE/462msbPN1Dx/ivBTVA7Y45yqmKb+TF9fUdlj7g1wEvS+1Rt8QP/N6NFpHMdrxeNg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:6214:21ef:: with SMTP id p15mr28791700qvj.20.1621845038984; Mon, 24 May 2021 01:30:38 -0700 (PDT) Date: Mon, 24 May 2021 09:30:00 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-18-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 17/18] arm64: Fix cache maintenance function comments From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013040_812876_CBF207EF X-CRM114-Status: GOOD ( 11.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Fix and expand comments for the cache maintenance functions in cacheflush.h. Adds comments to functions that weren't described before. Explains what the functions do using Arm Architecture Reference Manual terminology. No functional change intended. Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/cacheflush.h | 47 ++++++++++++++++++----------- 1 file changed, 30 insertions(+), 17 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 70b389a8dea5..26617df1fa45 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -30,31 +30,44 @@ * the implementation assumes non-aliasing VIPT D-cache and (aliasing) * VIPT I-cache. * - * flush_icache_range(start, end) + * All functions below apply to the interval [start, end) + * - start - virtual start address (inclusive) + * - end - virtual end address (exclusive) * - * Ensure coherency between the I-cache and the D-cache in the - * region described by start, end. - * - start - virtual start address - * - end - virtual end address + * __flush_icache_range(start, end) * - * invalidate_icache_range(start, end) - * - * Invalidate the I-cache in the region described by start, end. - * - start - virtual start address - * - end - virtual end address + * Ensure coherency between the I-cache and the D-cache region to + * the Point of Unification. * * __flush_cache_user_range(start, end) * - * Ensure coherency between the I-cache and the D-cache in the - * region described by start, end. - * - start - virtual start address - * - end - virtual end address + * Ensure coherency between the I-cache and the D-cache region to + * the Point of Unification. + * Use only if the region might access user memory. + * + * invalidate_icache_range(start, end) + * + * Invalidate I-cache region to the Point of Unification. * * __flush_dcache_area(start, end) * - * Ensure that the data held in page is written back. - * - start - virtual start address - * - end - virtual end address + * Clean and invalidate D-cache region to the Point of Coherency. + * + * __inval_dcache_area(start, end) + * + * Invalidate D-cache region to the Point of Coherency. + * + * __clean_dcache_area_poc(start, end) + * + * Clean D-cache region to the Point of Coherency. + * + * __clean_dcache_area_pop(start, end) + * + * Clean D-cache region to the Point of Persistence. + * + * __clean_dcache_area_pou(start, end) + * + * Clean D-cache region to the Point of Unification. */ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); From patchwork Mon May 24 08:30:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12276753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE366C04FF3 for ; Mon, 24 May 2021 17:58:10 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8958D61002 for ; Mon, 24 May 2021 17:58:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8958D61002 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ewxK7XN3TmDL0ZfGwf3RbLzNPUPel+rnBFFA5JDKWNU=; b=xjSBfPfcHQfBzqv9PwkqTFeS9c l/+3visbA18pgAutrBI8AkOvlcwqPfaIkpQ3YnAZDkJezJfoaCJT0M4vAmRAQzPMT5W/ZqkDlhLe5 gvlE0ZCjvemsfPuh1LZ1y69JLOEti/N0qg4/SkpOLl3LhrZiv+jywAeVHxWDCnhNnaNYsiu433IWm YbY+cy/hStW3b4UnQyeNvPuW8kK74y4H2URWt/4OlJFkuv59GejBy0TGKbyN6cZZkuiUoovUvELoQ n3OV5+S8S67tIbgh3nc1LyiGVRWVE0shGMXMT/z6PbPc14UkulLPHdLklkaP7bsouhYgfP/v6A0qp H7O9Xpuw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1llEnC-001MOt-BW; Mon, 24 May 2021 17:55:07 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1ll5z0-000lu7-KQ for linux-arm-kernel@lists.infradead.org; Mon, 24 May 2021 08:30:45 +0000 Received: by mail-qt1-x84a.google.com with SMTP id w15-20020ac857cf0000b02901e11cd2e82fso23198398qta.12 for ; Mon, 24 May 2021 01:30:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=hh6pEls5IFAjhauk+sr6KpH2mPwytfnPqkQ0GNiIp8U=; b=ur4quaDX3Dkr65zL33O2/b9WePHUu6AUtY+rbiM6aa5go+H7i2cHYHn6uRXpvJAy+r 8ZUTdXWMz1aGMDoV7BK0e+Taep26+r7wUmItrfBJvyd57ARwlEJG2Zv0+th98FE3pxtb Fvwb2eUUGCr0NyT1ibNKRZhcWENvZFC5QE2QvDyN2+tnvLOixfF8JsZd+2/14lqymC7w 2VA4rk3rK7cZ/nbp6q7aSfNhB8lYnwfCQwQ1rbejZxV8JU+j/48x+zoeY5rVx16cuuba g+1SIv3IBCCBPrvUQlL/3wlKFqlGlWc0Jfr9HNC8F7K3qVXSXI2PyY3bzlLEGMKYwOhV 0KSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hh6pEls5IFAjhauk+sr6KpH2mPwytfnPqkQ0GNiIp8U=; b=mfkgM+KXpNBQ3OgO1sKspAdI/uHiIjaAZZO38FqMOYadkXK6X149wWvhY7yMLMpCF+ qc5CpxmaFLBhffr+8G3dI7CMth0+dZwDRvpZqd4Aul3FkYpB8dO7ZmR9lKQF8ULE1e2h eSJDnqZvjimhylye+iVy89Yqf0Fw5pJpYSBwGoRdT7qYImfZOzM0nccZEKjmLKor8uDE dsPjl6ru6fOIBy5z4/pwB+PWanAjI5zDhMN+eeaTjwl/6py/h6MG5bN4KNPEF8rL1xMx 1beqKunRFPuhTr2HYzRsTOWLm2mJqGx185OSziqsA6+ah1HS9c+/kCshXsyLSx6LYOrh yEQg== X-Gm-Message-State: AOAM532l5O3UsNOLVdr52C0EHX7qgu/7c8ZPFLGvIjo8Q3J35hKpTxgE BCMQocu0viIIpiOeKHlvwsEHien6gerpIpLET1p8AlfMszOAkKYcDUM5cCzhywZ2YQHqt963sG+ EuFaP2jBw7d82YF1Qm6tutZEz4G4kduFmCsMl/4thmf4sw4RU5WeA4nbGdhGqbGJ5+S8n//+ZtP lTdQ== X-Google-Smtp-Source: ABdhPJyAqREvhKS/LY+HmCUr71rpPdFAzhN4t71EAu2qpfDaaxwh1LZmCD79zm11k+JYVtVMc1JK0BJ5+A== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:6214:32a:: with SMTP id j10mr28998494qvu.30.1621845041180; Mon, 24 May 2021 01:30:41 -0700 (PDT) Date: Mon, 24 May 2021 09:30:01 +0100 In-Reply-To: <20210524083001.2586635-1-tabba@google.com> Message-Id: <20210524083001.2586635-19-tabba@google.com> Mime-Version: 1.0 References: <20210524083001.2586635-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.818.g46aad6cb9e-goog Subject: [PATCH v4 18/18] arm64: Rename arm64-internal cache maintenance functions From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210524_013042_721350_D129CB21 X-CRM114-Status: GOOD ( 25.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Although naming across the codebase isn't that consistent, it tends to follow certain patterns. Moreover, the term "flush" isn't defined in the Arm Architecture reference manual, and might be interpreted to mean clean, invalidate, or both for a cache. Rename arm64-internal functions to make the naming internally consistent, as well as making it consistent with the Arm ARM, by specifying whether it applies to the instruction, data, or both caches, whether the operation is a clean, invalidate, or both. Also specify which point the operation applies to, i.e., to the point of unification (PoU), coherency (PoC), or persistence (PoP). This commit applies the following sed transformation to all files under arch/arm64: "s/\b__flush_cache_range\b/caches_clean_inval_pou_macro/g;"\ "s/\b__flush_icache_range\b/caches_clean_inval_pou/g;"\ "s/\binvalidate_icache_range\b/icache_inval_pou/g;"\ "s/\b__flush_dcache_area\b/dcache_clean_inval_poc/g;"\ "s/\b__inval_dcache_area\b/dcache_inval_poc/g;"\ "s/__clean_dcache_area_poc\b/dcache_clean_poc/g;"\ "s/\b__clean_dcache_area_pop\b/dcache_clean_pop/g;"\ "s/\b__clean_dcache_area_pou\b/dcache_clean_pou/g;"\ "s/\b__flush_cache_user_range\b/caches_clean_inval_user_pou/g;"\ "s/\b__flush_icache_all\b/icache_inval_all_pou/g;" Note that __clean_dcache_area_poc is deliberately missing a word boundary check at the beginning in order to match the efistub symbols in image-vars.h. Also note that, despite its name, __flush_icache_range operates on both instruction and data caches. The name change here reflects that. No functional change intended. Acked-by: Mark Rutland Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/arch_gicv3.h | 2 +- arch/arm64/include/asm/cacheflush.h | 36 +++++++++--------- arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 6 +-- arch/arm64/kernel/alternative.c | 2 +- arch/arm64/kernel/efi-entry.S | 4 +- arch/arm64/kernel/head.S | 8 ++-- arch/arm64/kernel/hibernate-asm.S | 4 +- arch/arm64/kernel/hibernate.c | 12 +++--- arch/arm64/kernel/idreg-override.c | 2 +- arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kernel/insn.c | 2 +- arch/arm64/kernel/kaslr.c | 6 +-- arch/arm64/kernel/machine_kexec.c | 10 ++--- arch/arm64/kernel/smp.c | 4 +- arch/arm64/kernel/smp_spin_table.c | 4 +- arch/arm64/kernel/sys_compat.c | 2 +- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/cache.S | 4 +- arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 2 +- arch/arm64/kvm/hyp/pgtable.c | 4 +- arch/arm64/lib/uaccess_flushcache.c | 4 +- arch/arm64/mm/cache.S | 58 ++++++++++++++--------------- arch/arm64/mm/flush.c | 12 +++--- 25 files changed, 98 insertions(+), 98 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index ed1cc9d8e6df..4ad22c3135db 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -125,7 +125,7 @@ static inline u32 gic_read_rpr(void) #define gic_write_lpir(v, c) writeq_relaxed(v, c) #define gic_flush_dcache_to_poc(a,l) \ - __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) + dcache_clean_inval_poc((unsigned long)(a), (unsigned long)(a)+(l)) #define gits_read_baser(c) readq_relaxed(c) #define gits_write_baser(v, c) writeq_relaxed(v, c) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 26617df1fa45..543c997eb3b7 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -34,54 +34,54 @@ * - start - virtual start address (inclusive) * - end - virtual end address (exclusive) * - * __flush_icache_range(start, end) + * caches_clean_inval_pou(start, end) * * Ensure coherency between the I-cache and the D-cache region to * the Point of Unification. * - * __flush_cache_user_range(start, end) + * caches_clean_inval_user_pou(start, end) * * Ensure coherency between the I-cache and the D-cache region to * the Point of Unification. * Use only if the region might access user memory. * - * invalidate_icache_range(start, end) + * icache_inval_pou(start, end) * * Invalidate I-cache region to the Point of Unification. * - * __flush_dcache_area(start, end) + * dcache_clean_inval_poc(start, end) * * Clean and invalidate D-cache region to the Point of Coherency. * - * __inval_dcache_area(start, end) + * dcache_inval_poc(start, end) * * Invalidate D-cache region to the Point of Coherency. * - * __clean_dcache_area_poc(start, end) + * dcache_clean_poc(start, end) * * Clean D-cache region to the Point of Coherency. * - * __clean_dcache_area_pop(start, end) + * dcache_clean_pop(start, end) * * Clean D-cache region to the Point of Persistence. * - * __clean_dcache_area_pou(start, end) + * dcache_clean_pou(start, end) * * Clean D-cache region to the Point of Unification. */ -extern void __flush_icache_range(unsigned long start, unsigned long end); -extern void invalidate_icache_range(unsigned long start, unsigned long end); -extern void __flush_dcache_area(unsigned long start, unsigned long end); -extern void __inval_dcache_area(unsigned long start, unsigned long end); -extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); -extern long __flush_cache_user_range(unsigned long start, unsigned long end); +extern void caches_clean_inval_pou(unsigned long start, unsigned long end); +extern void icache_inval_pou(unsigned long start, unsigned long end); +extern void dcache_clean_inval_poc(unsigned long start, unsigned long end); +extern void dcache_inval_poc(unsigned long start, unsigned long end); +extern void dcache_clean_poc(unsigned long start, unsigned long end); +extern void dcache_clean_pop(unsigned long start, unsigned long end); +extern void dcache_clean_pou(unsigned long start, unsigned long end); +extern long caches_clean_inval_user_pou(unsigned long start, unsigned long end); extern void sync_icache_aliases(unsigned long start, unsigned long end); static inline void flush_icache_range(unsigned long start, unsigned long end) { - __flush_icache_range(start, end); + caches_clean_inval_pou(start, end); /* * IPI all online CPUs so that they undergo a context synchronization @@ -135,7 +135,7 @@ extern void copy_to_user_page(struct vm_area_struct *, struct page *, #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1 extern void flush_dcache_page(struct page *); -static __always_inline void __flush_icache_all(void) +static __always_inline void icache_inval_all_pou(void) { if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC)) return; diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 0ae2397076fd..1bed37eb013a 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -137,7 +137,7 @@ void efi_virtmap_unload(void); static inline void efi_capsule_flush_cache_range(void *addr, int size) { - __flush_dcache_area((unsigned long)addr, (unsigned long)addr + size); + dcache_clean_inval_poc((unsigned long)addr, (unsigned long)addr + size); } #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 33293d5855af..f4cbfa9025a8 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -181,7 +181,7 @@ static inline void *__kvm_vector_slot2addr(void *base, struct kvm; #define kvm_flush_dcache_to_poc(a,l) \ - __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) + dcache_clean_inval_poc((unsigned long)(a), (unsigned long)(a)+(l)) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { @@ -209,12 +209,12 @@ static inline void __invalidate_icache_guest_page(kvm_pfn_t pfn, { if (icache_is_aliasing()) { /* any kind of VIPT cache */ - __flush_icache_all(); + icache_inval_all_pou(); } else if (is_kernel_in_hyp_mode() || !icache_is_vpipt()) { /* PIPT or VPIPT at EL2 (see comment in __kvm_tlb_flush_vmid_ipa) */ void *va = page_address(pfn_to_page(pfn)); - invalidate_icache_range((unsigned long)va, + icache_inval_pou((unsigned long)va, (unsigned long)va + size); } } diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index c906d20c7b52..3fb79b76e9d9 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -181,7 +181,7 @@ static void __nocfi __apply_alternatives(struct alt_region *region, bool is_modu */ if (!is_module) { dsb(ish); - __flush_icache_all(); + icache_inval_all_pou(); isb(); /* Ignore ARM64_CB bit from feature mask */ diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index b0f728fb61f0..61a87fa1c305 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -29,7 +29,7 @@ SYM_CODE_START(efi_enter_kernel) */ ldr w1, =kernel_size add x1, x0, x1 - bl __clean_dcache_area_poc + bl dcache_clean_poc ic ialluis /* @@ -38,7 +38,7 @@ SYM_CODE_START(efi_enter_kernel) */ adr x0, 0f adr x1, 3f - bl __clean_dcache_area_poc + bl dcache_clean_poc 0: /* Turn off Dcache and MMU */ mrs x0, CurrentEL diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 8df0ac8d9123..6928cb67d3a0 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -118,7 +118,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) // MMU off add x1, x0, #0x20 // 4 x 8 bytes - b __inval_dcache_area // tail call + b dcache_inval_poc // tail call SYM_CODE_END(preserve_boot_args) /* @@ -268,7 +268,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) */ adrp x0, init_pg_dir adrp x1, init_pg_end - bl __inval_dcache_area + bl dcache_inval_poc /* * Clear the init page tables. @@ -381,11 +381,11 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x0, idmap_pg_dir adrp x1, idmap_pg_end - bl __inval_dcache_area + bl dcache_inval_poc adrp x0, init_pg_dir adrp x1, init_pg_end - bl __inval_dcache_area + bl dcache_inval_poc ret x28 SYM_FUNC_END(__create_page_tables) diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index ef2ab7caf815..81c0186a5e32 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -45,7 +45,7 @@ * Because this code has to be copied to a 'safe' page, it can't call out to * other functions by PC-relative address. Also remember that it may be * mid-way through over-writing other functions. For this reason it contains - * code from __flush_icache_range() and uses the copy_page() macro. + * code from caches_clean_inval_pou() and uses the copy_page() macro. * * This 'safe' page is mapped via ttbr0, and executed from there. This function * switches to a copy of the linear map in ttbr1, performs the restore, then @@ -87,7 +87,7 @@ SYM_CODE_START(swsusp_arch_suspend_exit) copy_page x0, x1, x2, x3, x4, x5, x6, x7, x8, x9 add x1, x10, #PAGE_SIZE - /* Clean the copied page to PoU - based on __flush_icache_range() */ + /* Clean the copied page to PoU - based on caches_clean_inval_pou() */ raw_dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x10, x3 diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b40ddce71507..46a0b4d6e251 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -210,7 +210,7 @@ static int create_safe_exec_page(void *src_start, size_t length, return -ENOMEM; memcpy(page, src_start, length); - __flush_icache_range((unsigned long)page, (unsigned long)page + length); + caches_clean_inval_pou((unsigned long)page, (unsigned long)page + length); rc = trans_pgd_idmap_page(&trans_info, &trans_ttbr0, &t0sz, page); if (rc) return rc; @@ -381,17 +381,17 @@ int swsusp_arch_suspend(void) ret = swsusp_save(); } else { /* Clean kernel core startup/idle code to PoC*/ - __flush_dcache_area((unsigned long)__mmuoff_data_start, + dcache_clean_inval_poc((unsigned long)__mmuoff_data_start, (unsigned long)__mmuoff_data_end); - __flush_dcache_area((unsigned long)__idmap_text_start, + dcache_clean_inval_poc((unsigned long)__idmap_text_start, (unsigned long)__idmap_text_end); /* Clean kvm setup code to PoC? */ if (el2_reset_needed()) { - __flush_dcache_area( + dcache_clean_inval_poc( (unsigned long)__hyp_idmap_text_start, (unsigned long)__hyp_idmap_text_end); - __flush_dcache_area((unsigned long)__hyp_text_start, + dcache_clean_inval_poc((unsigned long)__hyp_text_start, (unsigned long)__hyp_text_end); } @@ -477,7 +477,7 @@ int swsusp_arch_resume(void) * The hibernate exit text contains a set of el2 vectors, that will * be executed at el2 with the mmu off in order to reload hyp-stub. */ - __flush_dcache_area((unsigned long)hibernate_exit, + dcache_clean_inval_poc((unsigned long)hibernate_exit, (unsigned long)hibernate_exit + exit_size); /* diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index 3dd515baf526..53a381a7f65d 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -237,7 +237,7 @@ asmlinkage void __init init_feature_override(void) for (i = 0; i < ARRAY_SIZE(regs); i++) { if (regs[i]->override) - __flush_dcache_area((unsigned long)regs[i]->override, + dcache_clean_inval_poc((unsigned long)regs[i]->override, (unsigned long)regs[i]->override + sizeof(*regs[i]->override)); } diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index bcf3c2755370..c96a9a0043bf 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -35,7 +35,7 @@ __efistub_strnlen = __pi_strnlen; __efistub_strcmp = __pi_strcmp; __efistub_strncmp = __pi_strncmp; __efistub_strrchr = __pi_strrchr; -__efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc; +__efistub_dcache_clean_poc = __pi_dcache_clean_poc; #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) __efistub___memcpy = __pi_memcpy; diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c index 6c0de2f60ea9..51cb8dc98d00 100644 --- a/arch/arm64/kernel/insn.c +++ b/arch/arm64/kernel/insn.c @@ -198,7 +198,7 @@ int __kprobes aarch64_insn_patch_text_nosync(void *addr, u32 insn) ret = aarch64_insn_write(tp, insn); if (ret == 0) - __flush_icache_range((uintptr_t)tp, + caches_clean_inval_pou((uintptr_t)tp, (uintptr_t)tp + AARCH64_INSN_SIZE); return ret; diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 49cccd03cb37..cfa2cfde3019 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -72,7 +72,7 @@ u64 __init kaslr_early_init(void) * we end up running with module randomization disabled. */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - __flush_dcache_area((unsigned long)&module_alloc_base, + dcache_clean_inval_poc((unsigned long)&module_alloc_base, (unsigned long)&module_alloc_base + sizeof(module_alloc_base)); @@ -172,10 +172,10 @@ u64 __init kaslr_early_init(void) module_alloc_base += (module_range * (seed & ((1 << 21) - 1))) >> 21; module_alloc_base &= PAGE_MASK; - __flush_dcache_area((unsigned long)&module_alloc_base, + dcache_clean_inval_poc((unsigned long)&module_alloc_base, (unsigned long)&module_alloc_base + sizeof(module_alloc_base)); - __flush_dcache_area((unsigned long)&memstart_offset_seed, + dcache_clean_inval_poc((unsigned long)&memstart_offset_seed, (unsigned long)&memstart_offset_seed + sizeof(memstart_offset_seed)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3e79110c8f3a..03ceabe4d912 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -72,10 +72,10 @@ int machine_kexec_post_load(struct kimage *kimage) * For execution with the MMU off, reloc_code needs to be cleaned to the * PoC and invalidated from the I-cache. */ - __flush_dcache_area((unsigned long)reloc_code, + dcache_clean_inval_poc((unsigned long)reloc_code, (unsigned long)reloc_code + arm64_relocate_new_kernel_size); - invalidate_icache_range((uintptr_t)reloc_code, + icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); @@ -111,7 +111,7 @@ static void kexec_list_flush(struct kimage *kimage) unsigned long addr; /* flush the list entries. */ - __flush_dcache_area((unsigned long)entry, + dcache_clean_inval_poc((unsigned long)entry, (unsigned long)entry + sizeof(kimage_entry_t)); @@ -128,7 +128,7 @@ static void kexec_list_flush(struct kimage *kimage) break; case IND_SOURCE: /* flush the source pages. */ - __flush_dcache_area(addr, addr + PAGE_SIZE); + dcache_clean_inval_poc(addr, addr + PAGE_SIZE); break; case IND_DESTINATION: break; @@ -155,7 +155,7 @@ static void kexec_segment_flush(const struct kimage *kimage) kimage->segment[i].memsz, kimage->segment[i].memsz / PAGE_SIZE); - __flush_dcache_area( + dcache_clean_inval_poc( (unsigned long)phys_to_virt(kimage->segment[i].mem), (unsigned long)phys_to_virt(kimage->segment[i].mem) + kimage->segment[i].memsz); diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 5fcdee331087..9b4c1118194d 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -122,7 +122,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) secondary_data.task = idle; secondary_data.stack = task_stack_page(idle) + THREAD_SIZE; update_cpu_boot_status(CPU_MMU_OFF); - __flush_dcache_area((unsigned long)&secondary_data, + dcache_clean_inval_poc((unsigned long)&secondary_data, (unsigned long)&secondary_data + sizeof(secondary_data)); @@ -145,7 +145,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) pr_crit("CPU%u: failed to come online\n", cpu); secondary_data.task = NULL; secondary_data.stack = NULL; - __flush_dcache_area((unsigned long)&secondary_data, + dcache_clean_inval_poc((unsigned long)&secondary_data, (unsigned long)&secondary_data + sizeof(secondary_data)); status = READ_ONCE(secondary_data.status); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index 58d804582a35..7e1624ecab3c 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -36,7 +36,7 @@ static void write_pen_release(u64 val) unsigned long size = sizeof(secondary_holding_pen_release); secondary_holding_pen_release = val; - __flush_dcache_area((unsigned long)start, (unsigned long)start + size); + dcache_clean_inval_poc((unsigned long)start, (unsigned long)start + size); } @@ -90,7 +90,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(pa_holding_pen, release_addr); - __flush_dcache_area((__force unsigned long)release_addr, + dcache_clean_inval_poc((__force unsigned long)release_addr, (__force unsigned long)release_addr + sizeof(*release_addr)); diff --git a/arch/arm64/kernel/sys_compat.c b/arch/arm64/kernel/sys_compat.c index 265fe3eb1069..db5159a3055f 100644 --- a/arch/arm64/kernel/sys_compat.c +++ b/arch/arm64/kernel/sys_compat.c @@ -41,7 +41,7 @@ __do_compat_cache_op(unsigned long start, unsigned long end) dsb(ish); } - ret = __flush_cache_user_range(start, start + chunk); + ret = caches_clean_inval_user_pou(start, start + chunk); if (ret) return ret; diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1cb39c0803a4..c1953f65ca0e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1064,7 +1064,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) stage2_unmap_vm(vcpu->kvm); else - __flush_icache_all(); + icache_inval_all_pou(); } vcpu_reset_hcr(vcpu); diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 36cef6915428..958734f4d6b0 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -7,7 +7,7 @@ #include #include -SYM_FUNC_START_PI(__flush_dcache_area) +SYM_FUNC_START_PI(dcache_clean_inval_poc) dcache_by_line_op civac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__flush_dcache_area) +SYM_FUNC_END_PI(dcache_clean_inval_poc) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 5dffe928f256..8143ebd4fb72 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -134,7 +134,7 @@ static void update_nvhe_init_params(void) for (i = 0; i < hyp_nr_cpus; i++) { params = per_cpu_ptr(&kvm_init_params, i); params->pgd_pa = __hyp_pa(pkvm_pgtable.pgd); - __flush_dcache_area((unsigned long)params, + dcache_clean_inval_poc((unsigned long)params, (unsigned long)params + sizeof(*params)); } } diff --git a/arch/arm64/kvm/hyp/nvhe/tlb.c b/arch/arm64/kvm/hyp/nvhe/tlb.c index 83dc3b271bc5..38ed0f6f2703 100644 --- a/arch/arm64/kvm/hyp/nvhe/tlb.c +++ b/arch/arm64/kvm/hyp/nvhe/tlb.c @@ -104,7 +104,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, * you should be running with VHE enabled. */ if (icache_is_vpipt()) - __flush_icache_all(); + icache_inval_all_pou(); __tlb_switch_to_host(&cxt); } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 10d2f04013d4..e9ad7fb28ee3 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -841,7 +841,7 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (need_flush) { kvm_pte_t *pte_follow = kvm_pte_follow(pte, mm_ops); - __flush_dcache_area((unsigned long)pte_follow, + dcache_clean_inval_poc((unsigned long)pte_follow, (unsigned long)pte_follow + kvm_granule_size(level)); } @@ -997,7 +997,7 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, return 0; pte_follow = kvm_pte_follow(pte, mm_ops); - __flush_dcache_area((unsigned long)pte_follow, + dcache_clean_inval_poc((unsigned long)pte_follow, (unsigned long)pte_follow + kvm_granule_size(level)); return 0; diff --git a/arch/arm64/lib/uaccess_flushcache.c b/arch/arm64/lib/uaccess_flushcache.c index 62ea989effe8..baee22961bdb 100644 --- a/arch/arm64/lib/uaccess_flushcache.c +++ b/arch/arm64/lib/uaccess_flushcache.c @@ -15,7 +15,7 @@ void memcpy_flushcache(void *dst, const void *src, size_t cnt) * barrier to order the cache maintenance against the memcpy. */ memcpy(dst, src, cnt); - __clean_dcache_area_pop((unsigned long)dst, (unsigned long)dst + cnt); + dcache_clean_pop((unsigned long)dst, (unsigned long)dst + cnt); } EXPORT_SYMBOL_GPL(memcpy_flushcache); @@ -33,6 +33,6 @@ unsigned long __copy_user_flushcache(void *to, const void __user *from, rc = raw_copy_from_user(to, from, n); /* See above */ - __clean_dcache_area_pop((unsigned long)to, (unsigned long)to + n - rc); + dcache_clean_pop((unsigned long)to, (unsigned long)to + n - rc); return rc; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index ea605d94182f..5051b3c1a4f1 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -15,7 +15,7 @@ #include /* - * __flush_cache_range(start,end) [fixup] + * caches_clean_inval_pou_macro(start,end) [fixup] * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -25,7 +25,7 @@ * - end - virtual end address of region * - fixup - optional label to branch to on user fault */ -.macro __flush_cache_range, fixup +.macro caches_clean_inval_pou_macro, fixup alternative_if ARM64_HAS_CACHE_IDC dsb ishst b .Ldc_skip_\@ @@ -43,7 +43,7 @@ alternative_else_nop_endif .endm /* - * __flush_icache_range(start,end) + * caches_clean_inval_pou(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -52,13 +52,13 @@ alternative_else_nop_endif * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__flush_icache_range) - __flush_cache_range +SYM_FUNC_START(caches_clean_inval_pou) + caches_clean_inval_pou_macro ret -SYM_FUNC_END(__flush_icache_range) +SYM_FUNC_END(caches_clean_inval_pou) /* - * __flush_cache_user_range(start,end) + * caches_clean_inval_user_pou(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -67,10 +67,10 @@ SYM_FUNC_END(__flush_icache_range) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__flush_cache_user_range) +SYM_FUNC_START(caches_clean_inval_user_pou) uaccess_ttbr0_enable x2, x3, x4 - __flush_cache_range 2f + caches_clean_inval_pou_macro 2f mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 @@ -78,17 +78,17 @@ SYM_FUNC_START(__flush_cache_user_range) 2: mov x0, #-EFAULT b 1b -SYM_FUNC_END(__flush_cache_user_range) +SYM_FUNC_END(caches_clean_inval_user_pou) /* - * invalidate_icache_range(start,end) + * icache_inval_pou(start,end) * * Ensure that the I cache is invalid within specified region. * * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(invalidate_icache_range) +SYM_FUNC_START(icache_inval_pou) alternative_if ARM64_HAS_CACHE_DIC isb ret @@ -96,10 +96,10 @@ alternative_else_nop_endif invalidate_icache_by_line x0, x1, x2, x3 ret -SYM_FUNC_END(invalidate_icache_range) +SYM_FUNC_END(icache_inval_pou) /* - * __flush_dcache_area(start, end) + * dcache_clean_inval_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned and invalidated to the PoC. @@ -107,13 +107,13 @@ SYM_FUNC_END(invalidate_icache_range) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START_PI(__flush_dcache_area) +SYM_FUNC_START_PI(dcache_clean_inval_poc) dcache_by_line_op civac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__flush_dcache_area) +SYM_FUNC_END_PI(dcache_clean_inval_poc) /* - * __clean_dcache_area_pou(start, end) + * dcache_clean_pou(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoU. @@ -121,17 +121,17 @@ SYM_FUNC_END_PI(__flush_dcache_area) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__clean_dcache_area_pou) +SYM_FUNC_START(dcache_clean_pou) alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret -SYM_FUNC_END(__clean_dcache_area_pou) +SYM_FUNC_END(dcache_clean_pou) /* - * __inval_dcache_area(start, end) + * dcache_inval_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are invalidated. Any partial lines at the ends of the interval are @@ -141,7 +141,7 @@ SYM_FUNC_END(__clean_dcache_area_pou) * - end - kernel end address of region */ SYM_FUNC_START_LOCAL(__dma_inv_area) -SYM_FUNC_START_PI(__inval_dcache_area) +SYM_FUNC_START_PI(dcache_inval_poc) /* FALLTHROUGH */ /* @@ -166,11 +166,11 @@ SYM_FUNC_START_PI(__inval_dcache_area) b.lo 2b dsb sy ret -SYM_FUNC_END_PI(__inval_dcache_area) +SYM_FUNC_END_PI(dcache_inval_poc) SYM_FUNC_END(__dma_inv_area) /* - * __clean_dcache_area_poc(start, end) + * dcache_clean_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoC. @@ -179,7 +179,7 @@ SYM_FUNC_END(__dma_inv_area) * - end - virtual end address of region */ SYM_FUNC_START_LOCAL(__dma_clean_area) -SYM_FUNC_START_PI(__clean_dcache_area_poc) +SYM_FUNC_START_PI(dcache_clean_poc) /* FALLTHROUGH */ /* @@ -189,11 +189,11 @@ SYM_FUNC_START_PI(__clean_dcache_area_poc) */ dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__clean_dcache_area_poc) +SYM_FUNC_END_PI(dcache_clean_poc) SYM_FUNC_END(__dma_clean_area) /* - * __clean_dcache_area_pop(start, end) + * dcache_clean_pop(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoP. @@ -201,13 +201,13 @@ SYM_FUNC_END(__dma_clean_area) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START_PI(__clean_dcache_area_pop) +SYM_FUNC_START_PI(dcache_clean_pop) alternative_if_not ARM64_HAS_DCPOP - b __clean_dcache_area_poc + b dcache_clean_poc alternative_else_nop_endif dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__clean_dcache_area_pop) +SYM_FUNC_END_PI(dcache_clean_pop) /* * __dma_flush_area(start, size) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index c4ca7e05fdb8..2aaf950b906c 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -17,14 +17,14 @@ void sync_icache_aliases(unsigned long start, unsigned long end) { if (icache_is_aliasing()) { - __clean_dcache_area_pou(start, end); - __flush_icache_all(); + dcache_clean_pou(start, end); + icache_inval_all_pou(); } else { /* * Don't issue kick_all_cpus_sync() after I-cache invalidation * for user mappings. */ - __flush_icache_range(start, end); + caches_clean_inval_pou(start, end); } } @@ -76,20 +76,20 @@ EXPORT_SYMBOL(flush_dcache_page); /* * Additional functions defined in assembly. */ -EXPORT_SYMBOL(__flush_icache_range); +EXPORT_SYMBOL(caches_clean_inval_pou); #ifdef CONFIG_ARCH_HAS_PMEM_API void arch_wb_cache_pmem(void *addr, size_t size) { /* Ensure order against any prior non-cacheable writes */ dmb(osh); - __clean_dcache_area_pop((unsigned long)addr, (unsigned long)addr + size); + dcache_clean_pop((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); void arch_invalidate_pmem(void *addr, size_t size) { - __inval_dcache_area((unsigned long)addr, (unsigned long)addr + size); + dcache_inval_poc((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif