From patchwork Thu May 20 12:43:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39BE7C433ED for ; Thu, 20 May 2021 12:46:13 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B951261001 for ; Thu, 20 May 2021 12:46:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B951261001 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8n53kGhV0S/7F7Y60B+jZP9a/1RAGsjpg+u/SAkFPqA=; b=FLEoWcScG+wsvG UvCHYfpiwALJfthi9duJTS+0vakAappPG+pUy5NtZIS1SjqmsK/dDwMt2FSZcwwQgVWGMZOTRgeea vdBRqT4DqwFtVX7ScPCuDosQCmbQP8SZi+srUmUCLy9SBJ07hivMoNXPFMsZgm/HFB8LhmqCdW8kD CN0pianLiN8U3PmHyDM8BlyUujNAXn5mamEpDdxLqq9zJ9gEpHeC3JmnBQ/eNat6895I5hgcxfgSe IcbNCcg8JnUOh7x2yGyOf/FlXPLbEBvL3iojAYbm7pm8yAFilhZBvhsf8Xe8pOmH49134Llp5bUMQ x1vGw8pZLgFivjuFWIFw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2f-000tyJ-32; Thu, 20 May 2021 12:44:45 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2B-000txw-1j for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=e6dSWW+QCaWstEpXxNByulXSeGIX79n+Seg/Gr9y1Yc=; b=XYfCvEODzrEbGHLjkof2NtRwYZ +jn8guEebIr5cVKli1S4DLL491lDKResfWU0kIqlMraYFr7tkHTC6Fn9TRmjLo85Rbmeph4jLHCOt D8WMgdU5KrstTtY4GEXscY0AkN1SX2Pq+CCRKp6+lktkJ90DvXemU+YEvXWZATrPTu0phghbMmWK2 B2RFUO5C7ZAmI39jmy8jJoS855nyqrLkKvj/tqaOSoxdJ63ioqpyFcpLGGLXT1BOfZ6TLSQ4ZLUW1 uPw61+MTMiUWtM8RkHRex/UKYcHQHmFeYmbGI4C9pYgHX2iXusiV6xOY4P2pnOXOyQsAs3Q+4oUEQ A8QSzOlA==; Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji28-00GKbh-6N for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:13 +0000 Received: by mail-qt1-x84a.google.com with SMTP id x19-20020ac87a930000b02901f6125bcda0so34656qtr.19 for ; Thu, 20 May 2021 05:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=e6dSWW+QCaWstEpXxNByulXSeGIX79n+Seg/Gr9y1Yc=; b=XA1zYWYn479Vpo+A7RiqVOi07luESOjxAp9BBQnPxIcB6P0XIFu2IeVpz4lcCG+7ad R23ACmBjdFNXynotmCG5iZZva98xKwQopBVdw/uAPJRwqNREI67SYObEUvJ6B9IRTPrV LJ9ORVgpyYADZpz+oJRKBM+AWCy6qUsxNdJBhyPvs4l7S90RxQUd7MSu89MKbfbq3Ocf r9tN1dq0xzoDsb2gTxn+9ogj4K0BDEw0gasnITDTz6OGxSgdZXK3KAaqW4kipjLjKzKU BYuoDDKOzI7B0thnOuqWQFSdIX2n6MgRAw6GIKWdZUT3uCfUincoUrKeq51sw+FcCsz/ chEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=e6dSWW+QCaWstEpXxNByulXSeGIX79n+Seg/Gr9y1Yc=; b=CY+XBnwwdjGAchlkoOVi3qCmjWNXju4QyeCi3PpPz69xhqPAvqIDI5xme8Tg1wEm3S FcAFsK5a+sZQ0e/l5zZpEZ8AuZGbjTnzTAGYqFnrYyZAphlFLckqYm4pVnCak3+JTMEo +5ZMGHtZ85xfeDvC2HS9f4GPAhqCqbYadufEH8kMVJXhn44YzfXxCTosnp/wLDg9T7yr 4vuBeJagxDKJxhROxuJ6+eh39mZZq3cTsy3nwEGJFj9kvqIwTjWEbep+nUGphO94aOMH ZD0hdi0SMrKohtSj+L03RdH+VHDSdjC5vpBAAT8U2/1z6GU5gdk/gjxHKTsBsJNRrPUk 48fQ== X-Gm-Message-State: AOAM531gdp5uM+rsKcpc46Cc+kpYlwiy1FrWcDuyMwkw2lPSvt8AUY2J kX+6iAnr1T6Y1754q/YI8FAaFLNP23RLwNU8nGyvju7S141MEpabruMKIpJeynj1aynKEKd5H1n fP/hzPv/sWvRaLwOOx0w2U2H2tUx0Nyq5uUV689laW9i0cHRLNHC9GAWA1OSFA5KdPwJ5kJ8FzX BGWA== X-Google-Smtp-Source: ABdhPJxB22ynKDDDc5swvPA8EQJxJBVMrwxPlGkexrKfqfFIxv3TNHTSBNgZh1LRmbBuWi6pi2plZLO+Lw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:f883:: with SMTP id u3mr5218102qvn.44.1621514650269; Thu, 20 May 2021 05:44:10 -0700 (PDT) Date: Thu, 20 May 2021 13:43:49 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-2-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 01/18] arm64: assembler: replace `kaddr` with `addr` From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054412_273065_689966DC X-CRM114-Status: GOOD ( 11.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland The `__dcache_op_workaround_clean_cache` and `dcache_by_line_op` macros are only expected to be usedc on kernel memory, without a user fault fixup, and so we named their address variables `kaddr` to make this clear. Subseuqent patches will modify these to also work on user memory with an (optional) user fault fixup, where `kaddr` won't make as much sense. To aid the legibility of patches, this patch (only) replaces `kaddr` with `addr` as a preparatory step. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Fuad Tabba Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Fuad Tabba Cc: Will Deacon --- arch/arm64/include/asm/assembler.h | 32 +++++++++++++++--------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8418c1bd8f04..6a0fbc599196 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -377,47 +377,47 @@ alternative_cb_end /* * Macro to perform a data cache maintenance for the interval - * [kaddr, kaddr + size) + * [addr, addr + size) * * op: operation passed to dc instruction * domain: domain used in dsb instruciton - * kaddr: starting virtual address of the region + * addr: starting virtual address of the region * size: size of the region - * Corrupts: kaddr, size, tmp1, tmp2 + * Corrupts: addr, size, tmp1, tmp2 */ - .macro __dcache_op_workaround_clean_cache, op, kaddr + .macro __dcache_op_workaround_clean_cache, op, addr alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE - dc \op, \kaddr + dc \op, \addr alternative_else - dc civac, \kaddr + dc civac, \addr alternative_endif .endm - .macro dcache_by_line_op op, domain, kaddr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2 dcache_line_size \tmp1, \tmp2 - add \size, \kaddr, \size + add \size, \addr, \size sub \tmp2, \tmp1, #1 - bic \kaddr, \kaddr, \tmp2 + bic \addr, \addr, \tmp2 9998: .ifc \op, cvau - __dcache_op_workaround_clean_cache \op, \kaddr + __dcache_op_workaround_clean_cache \op, \addr .else .ifc \op, cvac - __dcache_op_workaround_clean_cache \op, \kaddr + __dcache_op_workaround_clean_cache \op, \addr .else .ifc \op, cvap - sys 3, c7, c12, 1, \kaddr // dc cvap + sys 3, c7, c12, 1, \addr // dc cvap .else .ifc \op, cvadp - sys 3, c7, c13, 1, \kaddr // dc cvadp + sys 3, c7, c13, 1, \addr // dc cvadp .else - dc \op, \kaddr + dc \op, \addr .endif .endif .endif .endif - add \kaddr, \kaddr, \tmp1 - cmp \kaddr, \size + add \addr, \addr, \tmp1 + cmp \addr, \size b.lo 9998b dsb \domain .endm From patchwork Thu May 20 12:43:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04CB4C433ED for ; Thu, 20 May 2021 12:46:37 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8809261184 for ; Thu, 20 May 2021 12:46:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8809261184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nnM6OPrjc++p9jTFIaoEx0we77dJM0ucyFc9lYdrAXc=; b=pmDOQ907BzuUPp ifpJCuACi93XQV0Jpv9HAEgjPFwa9zYgqyN051ad44f4fWljXsE8YIcRFXrAWxOT52s4Ne+CdVzkx 4IyRj8XIYn6q1gVW17SkP12wzYu//bqP5mQME1XYy3ErVa42fgHmamTrOeFOApuT5KFlrLTQs8K4p VtuhXb2ajRF06qTsgmSXUPlegi+omfF9da9RoOuOrKWzxLIcHhBZvXwulu3oTeGSZp4syPv3myPQ0 uRTe8/9o0mVf33MzkyOTLp4Fb9K//I0AhQrY9en/q+ZoGeDY/LEVzkePU3o0QqesTT1ajfN9V3Nt9 +FWOl7mUhQ9XAnSquZQw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji37-000uKE-Ht; Thu, 20 May 2021 12:45:13 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2D-000tyK-Nt for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=KDPGYyPzMnkWgOQgmBvxX+NxkJicVg2L+NLCrWpY0nE=; b=gMO8AK1AWoiucJaJpVETY7ckHU Hopymvay5rknzvgqhuMCcJtebdhFj5U/LBkt9yv9xW0aMpWFArsZwAs70PvOkLpi/gQQZd81rQJPN gC9AZHX19LT35FVw/NrIROykVcHdp11fS//HuhyeBZMrVcELpla1EktBrj9wRYRwXyDW6o80XeEN2 3+dV/EdlW7MYPXJj8n7CBu9orugEGJJOo8GfXN4F7gkOxh/2ZL39H61YIIcwABienZgy/z6T61//W lydAlR/NF7VueUgb67bRqeZRbi+kzTnb+pYBaMJUZ27F9KqL4Z+pixmP2ZLFzsjCeEeKJb+UFAaYr DU5Y4R1A==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2B-00GKc9-4c for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:16 +0000 Received: by mail-wr1-x44a.google.com with SMTP id u20-20020a0560001614b02901115c8f2d89so8501160wrb.3 for ; Thu, 20 May 2021 05:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KDPGYyPzMnkWgOQgmBvxX+NxkJicVg2L+NLCrWpY0nE=; b=mkEtYX5k5nSIEoECJzSwKj0OD0FbogkJ8FQPnY/DUeNP6hiO620sfX74a+mOZDUhRu T6E67WjKhdyHZBxcYEa2iznAfgc+XKWUSs+behUn3G85NUdzwqJEs8yN7MzpHHq3evtc 86vjfL2Rxt+njmoCfiOmg1faSP8Xp1MsAFzsnTl+zIPgJ8QUsXjXvNFpC1ppOST+Q0+L tSEUxJ8WjfPhcia3WEqJN7TwlBkVbHZ1Xzct3LxOcx009jHSGgOrDe/ikMpuo8kq4Gp8 Fvb3o51h0EvpnKsLeDq3tapFiY0fOwwn1GzoCzHPCj8b0z7aVU+v8oFAPo30S5HwlMu7 vWqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KDPGYyPzMnkWgOQgmBvxX+NxkJicVg2L+NLCrWpY0nE=; b=HD7+CRTakjFZYYGav3m224ZtvdFAjLubtnpCtZXYy+WCH+TMBEZaDNtFE0Cr8JuZeb qYqTnFVnoZR87/dBwdku4DZh27PlOei5wOVKSKxGNqtJjd8O+IAVMFiqi1GX14rhgVI/ j3a5ynj+BrcLTwqcZsO3FHDbzWTn8AoIZPT/S4W1E1fPqGyXc3+6t6h964n8Sx4Nwmiy DJyEuqoVQbrjdy6cdKWTYv5U8f7wbTHsU52TlyaQo09GzkIAxaJ2d1CdOvTvn8YDSPLj jhi0uUH7StAwwEpsz6Qv6UvY1oezgQbdl5fsNiGkVfB31mHECok67bO2nZmQ1b/O/EFc zEmA== X-Gm-Message-State: AOAM531EKv7Cgb86Ehhmjm57rCh0z7WcxtYsmy0dSksF7ehy8IQfLCs0 qv2J6rxBXwzD94+KTHLpIx5oBlM43OD3vVcGy21NIdxARvj/9UXzyC0e+cBjs7X5MAhRzglRn9e vD8rD2fZBJ5fg9y1g/QOIEmLFY7/9KWfpEsW0NTVLreDIaULuCQ9zDZiSMxagUt0W7BKcqLr9ms jRpw== X-Google-Smtp-Source: ABdhPJz4PdNI9eZ8Mcbd15lAKmd0F27Unm31NjTDRF0bj180zMMuZ/D9TJzzeC0azjfnSA5tT+lI5jGszQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:adf:f212:: with SMTP id p18mr4026328wro.318.1621514652435; Thu, 20 May 2021 05:44:12 -0700 (PDT) Date: Thu, 20 May 2021 13:43:50 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-3-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 02/18] arm64: assembler: add conditional cache fixups From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054415_204386_4EE7DC60 X-CRM114-Status: GOOD ( 15.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland It would be helpful if we could use both `dcache_by_line_op` and `invalidate_icache_by_line` for user memory without accidentally fixing up unexpected faults when performing maintenance on kernel addresses. Let's make this possible by having both macros take an optional fixup label, and only generating an extable entry if a label is provided. At the same time, let's clean up the labels used to be globally unique using \@ as we do for other macros. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Fuad Tabba Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Fuad Tabba Cc: Will Deacon --- arch/arm64/include/asm/assembler.h | 39 +++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6a0fbc599196..0a276b46ef50 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -130,15 +130,27 @@ alternative_endif .endm /* - * Emit an entry into the exception table + * Create an exception table entry for `insn`, which will branch to `fixup` + * when an unhandled fault is taken. */ - .macro _asm_extable, from, to + .macro _asm_extable, insn, fixup .pushsection __ex_table, "a" .align 3 - .long (\from - .), (\to - .) + .long (\insn - .), (\fixup - .) .popsection .endm +/* + * Create an exception table entry for `insn` if `fixup` is provided. Otherwise + * do nothing. + */ + .macro _cond_extable, insn, fixup + .ifnc \fixup, + _asm_extable \insn, \fixup + .endif + .endm + + #define USER(l, x...) \ 9999: x; \ _asm_extable 9999b, l @@ -383,6 +395,7 @@ alternative_cb_end * domain: domain used in dsb instruciton * addr: starting virtual address of the region * size: size of the region + * fixup: optional label to branch to on user fault * Corrupts: addr, size, tmp1, tmp2 */ .macro __dcache_op_workaround_clean_cache, op, addr @@ -393,12 +406,12 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 add \size, \addr, \size sub \tmp2, \tmp1, #1 bic \addr, \addr, \tmp2 -9998: +.Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \addr .else @@ -418,8 +431,10 @@ alternative_endif .endif add \addr, \addr, \tmp1 cmp \addr, \size - b.lo 9998b + b.lo .Ldcache_op\@ dsb \domain + + _cond_extable .Ldcache_op\@, \fixup .endm /* @@ -427,20 +442,22 @@ alternative_endif * [start, end) * * start, end: virtual addresses describing the region - * label: A label to branch to on user fault. + * fixup: optional label to branch to on user fault * Corrupts: tmp1, tmp2 */ - .macro invalidate_icache_by_line start, end, tmp1, tmp2, label + .macro invalidate_icache_by_line start, end, tmp1, tmp2, fixup icache_line_size \tmp1, \tmp2 sub \tmp2, \tmp1, #1 bic \tmp2, \start, \tmp2 -9997: -USER(\label, ic ivau, \tmp2) // invalidate I line PoU +.Licache_op\@: + ic ivau, \tmp2 // invalidate I line PoU add \tmp2, \tmp2, \tmp1 cmp \tmp2, \end - b.lo 9997b + b.lo .Licache_op\@ dsb ish isb + + _cond_extable .Licache_op\@, \fixup .endm /* From patchwork Thu May 20 12:43:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71BDDC433ED for ; Thu, 20 May 2021 12:46:54 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0209461001 for ; Thu, 20 May 2021 12:46:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0209461001 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jkku/NH5WxgSRSbFk9Q8st7tj/1tkeBJwjoO8i0mmr4=; b=a1F3XI2x6BEDfw c6HCaNKQWNpkfm0LKI2n19pUbuIjxPjRqhNPQ8nRveQg9IKGU3euxGZKHV1dUs8bskfKLTJlvVTaW 6Gg+eIX+8ftmiYP5CGbxQcgUHQdgimidK6HWbjk34cuQSwKfyZNHLAEoPdTNFELtx9p1RSIU7Z2nY T/3bEbMc05p09f5pJM8jOIJNg6Qu3CLqPL9m76V6MSvQluEp1vxN9C3+6gm7SvT6oNr9PNfC/I3wY VVKvXybPJ+RSbHgxDheY6HGSGlJ9Dyvd37E0uoQPy1gMoR7JRkVPNjrfaD/OY2ZbkDsSu74/0gOBf QSTyurEiBMrssKovyjcQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji3M-000uTz-0B; Thu, 20 May 2021 12:45:28 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2H-000tym-0H for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=ahqZ8V/+zcLNEI2Yj0kSxAMlF/YVwDMa5CLwXEXBovw=; b=30p5HmOiRXYpEHC/bdaQhQ0zlm mGre2nsNp2im8re4pO2wqLCBOrKRnNWgrkzzZNcLDhmFHnBrp2OxsAPoAbhHNErtE9NwCX1VRia9V Lxz9obgyy2Wtf4rBlEP/kl6I+wLr1SzG9AT+lyhevXmw99isC/an9EV/lq4t0McnaYtwTAzGV7kCb LpTuzsKLYEohUiwXmOrsIXEC4VTIdP5PtHZiUgvCBbPGZXP0dUKvBm2H3Y67rvFi7bTg09AWCkOGf BitG5GzBcYnkcGgzILRNBd7IOy56iFW+i4bmCZ1j8rnv5IimYM63qKCDUd4WbYgHyY+i0l/GxMz/X k2X4pemQ==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2E-00GKca-K3 for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:19 +0000 Received: by mail-wm1-x34a.google.com with SMTP id l15-20020a1ced0f0000b029017140e7436dso1388706wmh.2 for ; Thu, 20 May 2021 05:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=ahqZ8V/+zcLNEI2Yj0kSxAMlF/YVwDMa5CLwXEXBovw=; b=jaFQlCkHRzqz5XueSEw94oNsFY6IAfkC166o5KByFtwdmIk8KAtjck+ObiAt7Hbhqo dSyXeA9x30RhG/P+HQK+V+uMKucnC3qFBx+7vPeL4a0zpLGvTvDgTK1M4XNlEqXncPYw qwNCy2rNeILSbbu4hiWs2uKIMFbIcmcKBa2hF0T3dE7MFhZwHmq0R8YpmF7aBxM+3EdG vYz4puQTsLl8Yg9pBgeR9PuPmSq79EcomMAs2wkxQE6NKiCpO2/uRityOjImEXE7x5H8 FZzVS5zlRnFATRvvRZznaYdNZUTRr3tdse4IGgnEfJnXgRUewEZLxCuum/8dNtYiNouz QA5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=ahqZ8V/+zcLNEI2Yj0kSxAMlF/YVwDMa5CLwXEXBovw=; b=oisks2lVb0iz76pQYqxv4p8EApovWZy/XD9wn/9veakUhQaRgUfvowUfQyAM9UqHGU YN+7eXLgnNMJ/uHDczej+fNp+u6mQ81O6FS2x5S9FH0Yrx/ZWKXAd9o3m4DOwyBYZ/o5 NnukgJJ/1RIpJc9pR5XJbkWt0gdcFTBYlaHvYbjeD/RmgTrnXSPVFjnTO+WeUrMTKDOz 5eRlDkh+kRXVvm8X8MVQzrSv52f0NhehTqSJsgFXw7wAwGyFIjw8tViZ0ExVKnEOyTVz hcAFeRlqWE9RgIEllZ+Pbp2V3MmkdivLPE6bnNrnmLkU3MPu5PsBTssSrwSoaLQn70y3 k5ew== X-Gm-Message-State: AOAM530g+S2m5gM5qRbRuejc8/XJbv1DUuRTMJdS7qygefIG1Q8RI4YS Rb5PeVrEF3+xcXUF4KFbHlF0bnmhQiWt+2lzNwAqfW05VxGekJBT1j5YWWFksH4bsVa3yxzsJRC MwdpUkAeZFIbEfUdVTU1XYTDKcaq4k93w4HSFGBVNmDMCfCm10s2YWD/szAzP2NI7Q+CcthnJ17 MVlQ== X-Google-Smtp-Source: ABdhPJzevCR+H4S3Z4/LFs3rWnm++FOYggZQ/c+D/IRSwlOvL9U1T/mLPL5s6pq25S6i996P3fiuRBArVA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:35cc:: with SMTP id r12mr1505600wmq.1.1621514654469; Thu, 20 May 2021 05:44:14 -0700 (PDT) Date: Thu, 20 May 2021 13:43:51 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-4-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 03/18] arm64: Apply errata to swsusp_arch_suspend_exit From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054418_676381_548D4B1C X-CRM114-Status: UNSURE ( 9.58 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The Arm errata covered by ARM64_WORKAROUND_CLEAN_CACHE require that "dc cvau" instructions get promoted to "dc civac". Reported-by: Mark Rutland Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/kernel/hibernate-asm.S | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 8ccca660034e..0ed2f72a6b94 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -91,7 +91,8 @@ SYM_CODE_START(swsusp_arch_suspend_exit) raw_dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x10, x3 -2: dc cvau, x4 /* clean D line / unified line */ +2: /* clean D line / unified line */ +alternative_insn "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE add x4, x4, x2 cmp x4, x1 b.lo 2b From patchwork Thu May 20 12:43:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAF7FC433B4 for ; Thu, 20 May 2021 12:47:05 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7266561001 for ; Thu, 20 May 2021 12:47:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7266561001 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=x3QzVR8GIfV1xtDOQVY1ZxRzts9GV9ko2VUtCfa5dFg=; b=kUsszlJ5mPYahi Nj3mknxNV1HVWWo4zSK3SjaAj37a9sFRFihqNVR3F+kW1+8/br0oTLMSigNTekkAReWEwnm6rgg0Y McpML4UtUazQorTuvCbMpG343M21ADfWQZD44QJvvr6VHvJ5WXa5s+84TXFHJ4fvSjNSZ2/eN5ugF 5OwbRziN6lQw0VINGP0D7AvxlTiWMM6slXa4HnrUbD+rJW6BBeiUSHrwif+cFNizaZcHRsXd6R3pD N4158HGjKDQktNKVgWkSRL3rms7L+q+YkFTzo+szH/SslwojArACecxhQllGBxezSmruJf9Rlwiib tEitIGw+J1TYdU5+8GDQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji3a-000ucB-3Z; Thu, 20 May 2021 12:45:43 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2H-000tyo-FL for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:22 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=fiOfESK8uNKrmxRFnQd8OuF11vrmOMEdK04yXlOW+Vk=; b=H619mAyBPZda2yMhJufqhLbCXR DQowsziYA5yfHsWwngj729Lsmzz6AsqRPQUantGwxAExXzRsMeQGRpg2fFSW7b7xI/3dGOsVePl5l XqAbDc6mxE7VUbCd2OFX1QpejbAsv9sBnOXkj771xaF68NWt1dpoxC/hlo4Ww+hiiSs3nU+mnYx9D xGd3s6L3Sk8DDNvHMEEdLoBKP6VZ2rwk13CNETt/AfivLhuhp3hRk+CDqSNw2Ers7OgztX/TCIFda F6nUP3JxbC6yGIRr6rZ6nNtvPMu1UlYmHA1gDO54f+9JfgF1VAzmV/CZI8Ma4bLnLsUtJ6wUMrpro etlFYRNQ==; Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2F-00GKco-0i for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:20 +0000 Received: by mail-qt1-x849.google.com with SMTP id d7-20020ac811870000b02901e65f85117bso12160535qtj.18 for ; Thu, 20 May 2021 05:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fiOfESK8uNKrmxRFnQd8OuF11vrmOMEdK04yXlOW+Vk=; b=h/DugLhKCzSHMxiupT9gr/VKpN7SaM+S75lj4uvD5X7aH4rYMzKeJYIO5by4989Xlq fpqhjIbayGnCUVIbbWPfXOApMJF+2V+C+YTDsqjlZ6UOp9K7pu9iBLXJKzKaHphDEDUt E9Vmp+Glwh5It54akach7EAs1645zPcDof4Iq1QhmVrK44udTdKSXkWsad1204Nwu++W RnO/0LHdbeOb18QQHZQ8Qwk+VvfoTAFlMH0V8A1pKgD/rCVZuCoqW6D782Lmqo8W9hsL 7v/tjpOJzPjA1eDMR+lkm+dF7NrSWM/OBoUyh4n757nnkgvTZtGO19AUQ0H7xIdPlXYI QD5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fiOfESK8uNKrmxRFnQd8OuF11vrmOMEdK04yXlOW+Vk=; b=TsOEowQ4gPhSvCG6FJbAKvaGRSYnIKOQWMWkPBQ0NlcLIpMvrMfy4wgbGMQkyF20Ei SipzJcXA+1BY0tbLBPHMm2zqYx9s8fbftOQmvx+SPycVA1dIjiSxnV1i4zgDG2lgHPNE 1kKsZQagdGNvj/lHwlB68i/mF59U89nTYhI7j8tBSKPbqWLkb6Dq7XcDCNpEf35KSqua z3fPpym/o+pLocHJmgoBlq5RYHUK8NofKT9ylocCpFuO5xOn6IjvXiBLjdgzoE1B02pH 9crQ2S7JWlArqEX2Aa7FL3sVGI8ZkQI1UAJY3gewawMgjN0SbJZyGlqHw0pyaGDf0VRl GBrA== X-Gm-Message-State: AOAM532CMT1ohMMqtAvSPfOgRRUBp10u4Y/QatmcBkSpPy4icJh2DrVY xbZ+Lwy7wSRXrv4OS5AsTBKanle4bmeSDWtXoxckAHZ1x0elsyooIS6G9m0X6JxaJGJjL9UQFCq gMhDyrqlgNx61kWxK+cJp6yqKjteZnWDS64gmnyVTEvg7hKsNhXwZy9emJ3h67sO2rfsmzkbuHj RiuA== X-Google-Smtp-Source: ABdhPJxT7DWTU1WS3W3nRaXIE1BrP9QhC7u2TSoQ5EX3+7XV1khSYGcAh7f/usNLopG3IGCvemFTw3Y7Fw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:ad4:5ca2:: with SMTP id q2mr5421089qvh.49.1621514656695; Thu, 20 May 2021 05:44:16 -0700 (PDT) Date: Thu, 20 May 2021 13:43:52 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-5-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 04/18] arm64: assembler: user_alt label optional From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054419_077978_AF6C9D7B X-CRM114-Status: GOOD ( 12.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Make the label for the extable entry in user_alt optional, only generating an extable entry if provided. This is needed later in the series, to avoid instruction duplication in the assembly code. While at it, clean up the label used to be globally unique using \@ as for other macros. Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/alternative-macros.h | 9 ++++++--- arch/arm64/mm/cache.S | 2 +- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/alternative-macros.h b/arch/arm64/include/asm/alternative-macros.h index 8a078fc662ac..01ef954c9b2d 100644 --- a/arch/arm64/include/asm/alternative-macros.h +++ b/arch/arm64/include/asm/alternative-macros.h @@ -197,9 +197,12 @@ alternative_endif #define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...) \ alternative_insn insn1, insn2, cap, IS_ENABLED(cfg) -.macro user_alt, label, oldinstr, newinstr, cond -9999: alternative_insn "\oldinstr", "\newinstr", \cond - _asm_extable 9999b, \label +.macro user_alt, oldinstr, newinstr, cond, label +.Lextable_\@: + alternative_insn "\oldinstr", "\newinstr", \cond + .ifnc \label, + _asm_extable .Lextable_\@, \label + .endif .endm #endif /* __ASSEMBLY__ */ diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 2d881f34dd9d..5ff8dfa86975 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -47,7 +47,7 @@ alternative_else_nop_endif sub x3, x2, #1 bic x4, x0, x3 1: -user_alt 9f, "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE +user_alt "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE, 9f add x4, x4, x2 cmp x4, x1 b.lo 1b From patchwork Thu May 20 12:43:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 444CBC433ED for ; Thu, 20 May 2021 12:47:51 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BB85061184 for ; Thu, 20 May 2021 12:47:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB85061184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZPBAThwdR9OpwBG/KIwqgCMNtigGoiEHgJmy4cUUE6E=; b=FmMYATssdKFf3e +YajSq2JfDrB2I31sOrHPqzX0HKQG5drmaHhjXsokT5Cgi9cjM+jvxepM8D8JXD7eVqqUvYZcnypx WbAicBFbe8KlF9V0EYMDIRtPWrXUTY12lef5STf++4s+zFDtD4KXNXb+A+ohljA78P4NleQ07xG31 99I79JH4WS2xovHEnn8NaeGazPZAuV4SlnA0rAlLFx1BQaTtVWAU2/536YcNzjm5S48Ri3S2MLj+I bOcgMQTq6Ex+0KbXJq9smCyf8b87/8cytMV0G122ieQlVojBFm+tQK6VLcIvdNBPnKdRLs/PUBYJS QfxJHZ20DtNr65o1CCRw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji44-000ut0-CZ; Thu, 20 May 2021 12:46:12 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2K-000tzE-Dl for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=y9sZY7i6M6Si+BnsNu0zYtWboK89Av3bx/Kh9fRDw9I=; b=HexGiq/T6m0I/rNKxADpUjUDZE 0e88IEUM33gUweIoEN+o4MbXRdE45ZI42TvSctEFLdT7PufA8PrCYhxmDzenwJpWc/CUUf/xbzUVA Kbbvnkc/lE64MRGwHKFvzx/xYP5gBzABwB5241rSnYv6yjJIviKBgiDmdbpgrWA2xpuZh+pPTQo4S +ruMgZ+8zXmOxFtAtQNNhUo4G0fJFpy/ZePzVl/ECpMTkv7Vsbe3tszmccaxUTV779xe4aC4gmlEn w3zJudHdilQrGUGHqYvV3xBd6YqPy6iI1J4nzJj7J8fwcfABlLPC48ebG+8ISsRfI9wQEjijiKUQh 8EoUqs7A==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2H-00GKdF-MT for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:23 +0000 Received: by mail-wr1-x449.google.com with SMTP id f19-20020adfb6130000b02901121afc9a31so1899196wre.10 for ; Thu, 20 May 2021 05:44:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=y9sZY7i6M6Si+BnsNu0zYtWboK89Av3bx/Kh9fRDw9I=; b=IKYB7v0eEan/h1zlFMckoa1jPIUgQ1zeQ0W5irTPL4a6Q+Xayl+DPb+tu0DqgFA1Uv DDqscsoasR8wB7Mazx/L/4auUXiJAAi62twHJjc5zlpu2eRO7bJg+QZ3RqSFad+dIjkE D5PETTUINRRZZETyrEtYiNArJzIivHMg/lmGuysQBTootdqR/jovLpyk4GgFCvMWsd/a J1UnFJJk5AZOfpO0xnf/ljOGo6u9ecmCK3PhqZneJtXkx3WY4RBM8OH3mYDSd6gNvRR7 rZqywabHBBRcR+oti48ANgQoa08eL2LB/IfhpqstIqoE+sKXv1drHe2axLidMvUIf+p6 dBuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=y9sZY7i6M6Si+BnsNu0zYtWboK89Av3bx/Kh9fRDw9I=; b=eDXmiM3GW+dlPwGw4iKzDnk7OPazhFsOMInFbxc3wUqi1oSNGRgMrbi/jo/YzvhkCO DtfU4//Ha1DoO+0AhXFaq7jWuummBaBQZSfJVui0W02+o4rvXQvI9BUs8XgTda0XoYTd BsmtkogDqSEITAijzyocVxVOI96swBlPWNu93lJrQOfg+euEc0fJhO7vc1tpPWTZC8UL l+hONskDrYb741gvqOLTbigdj2g39f2Gh/aNoQ0/BCkwWIAwPM/AKSZmXN1s/mahXdoy FqOY/luXjIOrSMegaawkqvASCX8cTa95ANYj8WKdfa7Od/c3TpQdtIXYPwJSRUEggDOb fPFg== X-Gm-Message-State: AOAM532Nm0Rld+72lM92cCfJQFVJ+6EcYBYcUQB/3zWUHi5JtrFPeps+ Qm0NzapZW590kFeL4UlwNOsEOQdGwJsRAo1t+Ek+1Lbct65cjNyabSaXNe+9LiXCUBYba+xTc+u PzxMAsGVio1qlJlSwnt4LGWoedTfIj6MSlskhVJTvQ+pz3Y9FxJ1ufWXi8AXrs/SgdiMrXtBeMF DfCQ== X-Google-Smtp-Source: ABdhPJxNt3K2lx5C1BC/WQzLQjDOTOzqw/XidTTgo9R7BLM6RsgqX17EsRrudBRVZc9ySO5jhZhXgWFOXw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a05:600c:4b92:: with SMTP id e18mr3472289wmp.71.1621514658992; Thu, 20 May 2021 05:44:18 -0700 (PDT) Date: Thu, 20 May 2021 13:43:53 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-6-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 05/18] arm64: Do not enable uaccess for flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054421_756426_DF8A5B5D X-CRM114-Status: GOOD ( 13.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __flush_icache_range works on the kernel linear map, and doesn't need uaccess. The existing code is a side-effect of its current implementation with __flush_cache_user_range fallthrough. Instead of fallthrough to share the code, use a common macro for the two where the caller specifies an optional fixup label if user access is needed. If provided, this label would be used to generate an extable entry. No functional change intended. Possible performance impact due to the reduced number of instructions. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba Acked-by: Mark Rutland Acked-by: Catalin Marinas --- arch/arm64/mm/cache.S | 64 +++++++++++++++++++++++++++---------------- 1 file changed, 41 insertions(+), 23 deletions(-) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 5ff8dfa86975..c6bc3b8138e1 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -14,6 +14,41 @@ #include #include +/* + * __flush_cache_range(start,end) [fixup] + * + * Ensure that the I and D caches are coherent within specified region. + * This is typically used when code has been written to a memory region, + * and will be executed. + * + * - start - virtual start address of region + * - end - virtual end address of region + * - fixup - optional label to branch to on user fault + */ +.macro __flush_cache_range, fixup +alternative_if ARM64_HAS_CACHE_IDC + dsb ishst + b .Ldc_skip_\@ +alternative_else_nop_endif + dcache_line_size x2, x3 + sub x3, x2, #1 + bic x4, x0, x3 +.Ldc_loop_\@: +user_alt "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE, \fixup + add x4, x4, x2 + cmp x4, x1 + b.lo .Ldc_loop_\@ + dsb ish + +.Ldc_skip_\@: +alternative_if ARM64_HAS_CACHE_DIC + isb + b .Lic_skip_\@ +alternative_else_nop_endif + invalidate_icache_by_line x0, x1, x2, x3, \fixup +.Lic_skip_\@: +.endm + /* * flush_icache_range(start,end) * @@ -25,7 +60,9 @@ * - end - virtual end address of region */ SYM_FUNC_START(__flush_icache_range) - /* FALLTHROUGH */ + __flush_cache_range + ret +SYM_FUNC_END(__flush_icache_range) /* * __flush_cache_user_range(start,end) @@ -39,34 +76,15 @@ SYM_FUNC_START(__flush_icache_range) */ SYM_FUNC_START(__flush_cache_user_range) uaccess_ttbr0_enable x2, x3, x4 -alternative_if ARM64_HAS_CACHE_IDC - dsb ishst - b 7f -alternative_else_nop_endif - dcache_line_size x2, x3 - sub x3, x2, #1 - bic x4, x0, x3 -1: -user_alt "dc cvau, x4", "dc civac, x4", ARM64_WORKAROUND_CLEAN_CACHE, 9f - add x4, x4, x2 - cmp x4, x1 - b.lo 1b - dsb ish -7: -alternative_if ARM64_HAS_CACHE_DIC - isb - b 8f -alternative_else_nop_endif - invalidate_icache_by_line x0, x1, x2, x3, 9f -8: mov x0, #0 + __flush_cache_range 2f + mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 ret -9: +2: mov x0, #-EFAULT b 1b -SYM_FUNC_END(__flush_icache_range) SYM_FUNC_END(__flush_cache_user_range) /* From patchwork Thu May 20 12:43:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4B3DC433ED for ; Thu, 20 May 2021 12:48:03 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F8A761163 for ; Thu, 20 May 2021 12:48:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F8A761163 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qTeV4BBzS4MLlIf9vq8ghBna09U+3Snsz7mCQCkY/pc=; b=O6KDomjvjItetY UslE5aV67mZR6hIpTZCkpEmuHlxnhzQ9EcrRXkBFht+H6vt9L/Zl+kcGJByF0jDS6yf+EVIEHzi50 Xxei0+8HO51ps6OxisWWQedgwE+IhnACfUXWAvTtMAyHtAAGXis97XMwouPSfw5rxYcVe6bN3LUx4 oz8iVEE3mam9orQ4kFqk+kIEF20irN184E1QeL1n3tkIEr1kgNPx+YFE4O4FFfW3RfIdnZPgqwuV+ pglsnwu5sj8vHCNS04AefIBMnYJViMjWqLgN/eI2xV5kXgkXIEC39UZ8wbNnXAHLdk903LQtpJTGV C5HzcAeX32itmHEBENgQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji4P-000vEI-A6; Thu, 20 May 2021 12:46:33 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2M-000u0n-D7 for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=dmnjCj8KjLvWl3PuSvNM5eR1F4UYatdLENE/+5L9i1g=; b=MR20rK2xmoD+bpAQrm0UzMG7wJ RMcsqlpfhZ6mfUHL8335Q+vDNiL95rNTkNj7wz1BDnw9AAOex5r9EIjwgaUuImftF7jhU/k6JalUk Cuyy2bhyTLgNoX8jeZXjCU8wDnsb2v8GneSgHLiIbN5vxBIM8YDGBBrqZ5fzvOBezwgWA8XfJ6m58 iovYP8h6sRJi15IL/SSxF9perM/Pbw/qaeNwW/wvF6mapwR9+ELfT5B9WLZ7AdXY3ozw1Fnpcs+pd JA5Rt40sBcW3T47d6KoJlO6QS+4+Ws09DD8NoWcwBfhd/ek2xG76qgNHaZCclVQ7mVI+NGJo7OxiJ GN7kMraw==; Received: from mail-wm1-x349.google.com ([2a00:1450:4864:20::349]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2J-00GKdM-PJ for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:25 +0000 Received: by mail-wm1-x349.google.com with SMTP id l15-20020a1ced0f0000b029017140e7436dso1388780wmh.2 for ; Thu, 20 May 2021 05:44:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=dmnjCj8KjLvWl3PuSvNM5eR1F4UYatdLENE/+5L9i1g=; b=LqVaOHs5PM+OF9/4NmPyjGIgH16yvLQX2e2n2qJI4WgdOsl/L3kd8QRZDepk1hEoNI 29B5kDSnzZnmJ5mGVm70RBJFCRHVlqOiJXZk/zCSSoqxFgNxBwIWEyiT8x7UZm03nERo aczum/wzF10XO0BOSJXz+YRwXKiTCzjVj+4A0w5SsmPUHrM5ypdtdW7motCEaQ8tKLqf IbaOXaVGkmMXsJhGkhwA73oxqym1tXqUk0oCPEoakB3cVbYjyhWesL/4dOh7V0j5ViHT v9kQXnekQ1edBpkk7DRKFqhShrEmSyX+/DFj5ubYK3Vhu/MwdX9e5KVFpcA4tWSG64vb KSPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=dmnjCj8KjLvWl3PuSvNM5eR1F4UYatdLENE/+5L9i1g=; b=pZJ6jYJvnRKj3kaNaYKg3wsMXSFvuCUeeYQgqtutRINceF8gyrZhRlpnPRQxL7zVUa etiwoie4ijZPbgm6yVAxrbR3BSXmvjx22OYpEzex/UanIytj7/Y8HZhntMTz7jTbzw9J 5ABJX6DTW3/3ikuSkiQDUKNID8JbAfg8rCx1sbBCcbZPuFU4H7tLX8VrVOFHyf9HMVdp D0YZ5aNBw+RPYgataWs/1THWjNRdqH4Ks1MvTMDw/Rigz+k9cHgAntVe+xxYlT/jy70L PV0xPIsFyYxpweDBnXaoEujwsVOvmfIdrRVhOxXkeGPgXTbigA2g0+bwacK3ejjSbae7 H95Q== X-Gm-Message-State: AOAM532Fe4/u+yd0NfMdwZ0EUOl9//iD3D8GCgJBvfLNvFohOXxoRUP+ jLjExDEghtPEXswv3uJHRAy+LyaEJK3Yq/enZumC1AM6E0CD3ajGS84kn10pGncaGKHpXFReOsY JwGi85VsY5UBPc9htlI3tM/zWCBZZpWkHqYeNzcIl7JdYbsHTQTzWZJAk1Nrgh0UhPPZpvNLcuk 2L6g== X-Google-Smtp-Source: ABdhPJwXDKKnFk3OZPOJeddGyVke7DK0c6lcKWRDPsqy3mF1jWuNDulpUJMVQ3+gGyVJ+/wnNQHRyG1zrQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:4e0b:: with SMTP id g11mr3452331wmh.3.1621514661208; Thu, 20 May 2021 05:44:21 -0700 (PDT) Date: Thu, 20 May 2021 13:43:54 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-7-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 06/18] arm64: Do not enable uaccess for invalidate_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054423_841501_42BD2A96 X-CRM114-Status: GOOD ( 12.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org invalidate_icache_range() works on the kernel linear map, and doesn't need uaccess. Remove the code that toggles uaccess_ttbr0_enable, as well as the code that emits an entry into the exception table (via the macro invalidate_icache_by_line). Changes return type of invalidate_icache_range() from int (which used to indicate a fault) to void, since it doesn't need uaccess and won't fault. Note that return value was never checked by any of the callers. No functional change intended. Possible performance impact due to the reduced number of instructions. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba Acked-by: Mark Rutland Acked-by: Catalin Marinas --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/mm/cache.S | 11 +---------- 2 files changed, 2 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 52e5c1623224..a586afa84172 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -57,7 +57,7 @@ * - size - region size */ extern void __flush_icache_range(unsigned long start, unsigned long end); -extern int invalidate_icache_range(unsigned long start, unsigned long end); +extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); extern void __inval_dcache_area(void *addr, size_t len); extern void __clean_dcache_area_poc(void *addr, size_t len); diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index c6bc3b8138e1..7318a40dd6ca 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -97,21 +97,12 @@ SYM_FUNC_END(__flush_cache_user_range) */ SYM_FUNC_START(invalidate_icache_range) alternative_if ARM64_HAS_CACHE_DIC - mov x0, xzr isb ret alternative_else_nop_endif - uaccess_ttbr0_enable x2, x3, x4 - - invalidate_icache_by_line x0, x1, x2, x3, 2f - mov x0, xzr -1: - uaccess_ttbr0_disable x1, x2 + invalidate_icache_by_line x0, x1, x2, x3 ret -2: - mov x0, #-EFAULT - b 1b SYM_FUNC_END(invalidate_icache_range) /* From patchwork Thu May 20 12:43:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09390C433B4 for ; Thu, 20 May 2021 12:48:27 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7D13761163 for ; Thu, 20 May 2021 12:48:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7D13761163 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6nh7EmmHRA/bRjOnGrwZXk2ASBd+OlyDXAuphL6K57E=; b=aWANCnXpZ8MDZu lAQAh+z+X5TYvrnOIj5ac8z++0fvI5kVL0uGyEmi1WnjclqkPMNxDszmXzIOhwJ48uJjYK/Aui3BY oq/rMYRjSz7Z1lmjlAwh4pIcDBHlnWgpphVRtzCayGTU9jeFmqMdFynvAmUNxlGBw4ullzcOr0QYy YIy+QAckB645KP6gWdKbmtouOIXpRP6hgSbsxCSrJsBBsGUVKHmsgTySi/af4jXF80jXozOEJemOC fVRxygyjzc4rQ464h10k5nCFRKn+feQ7lHOQAYdR05z39i5joS1gtZDhURKD3Ewnj3oDZnlb1NyCI UsU/vXgVSoQfJ5WC5voQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji4k-000vca-LB; Thu, 20 May 2021 12:46:55 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2N-000u12-Tb for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=EAsOuqOYSQDkzD7u/dqW8zn+FKdKohuXnegMwBwSH8Y=; b=jp8OT6ypF2kn+KjmwAaziDRRqd qxWUVsul3wekq39kShZyYQ+x+q64geuGT3csHtNUmqRNh3Cz9U2CTUWAplNGbmM7ouu3nsmY+GD0g VVX4GcQI5t1TpSk2Hvp9YXZ3ED+rojYjQlTOzCghuzHeAl3BBR2KOhb+/fuKbqASE/2HgXIx1bhiD +ZGQ6O3VR7n8ajBvWOxAa3A2+hNrCdUFjcbTave6jlOoyvorq06CuQS7vuMf3a4A8V/bim4JC6B89 XhrnB9cr4TVUYK17gpIUi1UYkDvAyApBT0zVOrDOXsrPCYhUuJOfxIlY3TL2cMwm1anUTZRgg+AEx kvG/aXqw==; Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2L-00GKdc-Bt for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:26 +0000 Received: by mail-qv1-xf49.google.com with SMTP id w14-20020a056214012eb02901f3a4388530so4717961qvs.17 for ; Thu, 20 May 2021 05:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EAsOuqOYSQDkzD7u/dqW8zn+FKdKohuXnegMwBwSH8Y=; b=iDPfjFAHFYMLg52cFCEGK7Ku4d34vpGZSxgF3DW/Ej7HrleVY/6JQGX3lHWVcP0OJu odfhweufwDfSrocz+hw0fcE3v4p0QYnuEPclROGUXNaeMSE8FoeA3nHCzsgJTCp+oeK8 1FRvUzlDrPW1jwDIAR5ng/J+EHYBXi9hxZcWy9EHeEOiv3oLVXmX+X14F0UAojA8YcHm wKdkNhG7OC1Fy1MFTUylarcqXjggMe2Mk7ITso4gmwZNzq0nQD2Lp4Fo4XiEYNUskByZ 3m+xivaDYj+4sNaXB3A0iVF2EtnYbG1hxlxT67lHB/+0pfNK1HMZ6pjBDQua03tR1bcD 3Jcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EAsOuqOYSQDkzD7u/dqW8zn+FKdKohuXnegMwBwSH8Y=; b=YsWMkpq3i55wB3W8pRUqMDm6WLnOuqbvWZ91WNxQEO4ESO2/wdO5rT+tjuGkGQ4bq2 c/CV1M+rj/hgUmUiQzuWwkeVmvQVkBCJqNxxoSYv7wnHK0Vfl2GA7kHMgUU9bpY3Okz4 4jn91tFAmwOE5BaA8zXUSPuLtQRmcoDAjC4bGrX12D0w1Odw05p2/+/VvW8Bp8+Reu3B uCLTGPllrt4ku4w08TvriYVsQ3GgkFTxxUaU9POGJJL9bnfdYrVtcM8TlTQXzbfUnJHu F00fUCmZPnNhklnnF+aqV3dPaFERGUMmC/mVoEUofDeQ2m388O0C4juaFxtyjtk5EYW1 o7vw== X-Gm-Message-State: AOAM532+cHNxQDwmngzQYrT+0RkVmelyroRlGyFHJkYKSByhkAzwOL2o UXcHm4LLFf0cMDbNHxCmU1XzrZZkvFSZSqFXtFCG21gFEPLzFPnYL+s1PmRLNizsckoHZkXZq8K xMPbLXcokJG89OdKlXn8uKIs/ccJDmfG14xmRQ8r1cxL2QfpXNfGCqvq8CiZNQGeG2GCvjnAMIo twXQ== X-Google-Smtp-Source: ABdhPJyJGqokOrPGq2sqsm9kMP+ARGRWJQff+XzIZn1vJSBjr63p00h3N9nClCO1ieImaiFTjpIqUW4orQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:ea4a:: with SMTP id u10mr5623325qvp.8.1621514663097; Thu, 20 May 2021 05:44:23 -0700 (PDT) Date: Thu, 20 May 2021 13:43:55 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-8-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 07/18] arm64: Downgrade flush_icache_range to invalidate From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054425_425023_3A0CC27F X-CRM114-Status: GOOD ( 12.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Since __flush_dcache_area is called right before, invalidate_icache_range is sufficient in this case. Rewrite the comment to better explain the rationale behind the cache maintenance operations used here. No functional change intended. Possible performance impact due to invalidating only the icache rather than invalidating and cleaning both caches. Reported-by: Catalin Marinas Reported-by: Will Deacon Link: https://lore.kernel.org/linux-arch/20200511110014.lb9PEahJ4hVOYrbwIb_qUHXyNy9KQzNFdb_I3YlzY6A@z/ Signed-off-by: Fuad Tabba Acked-by: Mark Rutland Acked-by: Catalin Marinas --- arch/arm64/kernel/machine_kexec.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 90a335c74442..a03944fd0cd4 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -68,10 +68,14 @@ int machine_kexec_post_load(struct kimage *kimage) kimage->arch.kern_reloc = __pa(reloc_code); kexec_image_info(kimage); - /* Flush the reloc_code in preparation for its execution. */ + /* + * For execution with the MMU off, reloc_code needs to be cleaned to the + * PoC and invalidated from the I-cache. + */ __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); - flush_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + - arm64_relocate_new_kernel_size); + invalidate_icache_range((uintptr_t)reloc_code, + (uintptr_t)reloc_code + + arm64_relocate_new_kernel_size); return 0; } From patchwork Thu May 20 12:43:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 896B2C433B4 for ; Thu, 20 May 2021 12:48:47 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 10A7561163 for ; Thu, 20 May 2021 12:48:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 10A7561163 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NqcVqrUZ4LvRKuPbHafDXdp+xBdkpiWaPWqAim8I4DQ=; b=HQDZwy9bxXVPJk hU4zY/+VRiFcLjzOjdPluFrVCkZx9ye7/uCg42JnFIudK40WzYhyOVw2REOpFF/TZ/1gqb0AIqodP wkaqsibh10hOoyqL2Zd0LooxT+mRSLvi0FpPdufZA4R9AZBGQjuQgNJR72MDMZuW3H3kTkkOckztu /keSRYYtG5xLLY4Zmn3WIOqbQnjY8ffnQ1vs8tEtIGwWo57UfnIW4l7ErtBtDPAg/Nb0B1U2R1ysb 5U9hg8dp7D9WJv1cweIC/1YULZzjuarZHeO+6cWIAR5Nxi4dG8azvTX0Imd7XvpkoSs/p8loydZY7 jVUj9jZaN+373ie1vpiA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji52-000vnb-65; Thu, 20 May 2021 12:47:14 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2P-000u1L-LG for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:29 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=M2Sla/jf/pjrwPm+RYCmA/9jHt4HgBxeLsb7dVByTVM=; b=y+epkNCtQ6mrhE65cpZF1BafeU b39Ocrx9mNz+pNlgNHL93DiinfmgP6KxhxMi4Nwk6x0+CCjOkjOAJroD/qMGcGdjucFFelvIAmFH7 phMLrlp8nOy0mwm4qhgkmE8FK+0n4OplXfDOrh7lfZ+p3Zj+nEN/p1pnUbiy6YVK3cPpj5r5hq+J8 J3hJtCWde3a7911G9WREf/9Y+mKJh2/ZPsJXxhBeoeAyz2SZ+BLBF9uzW2Dhn3NUSQM8eePIdyTw4 P9/aw1SVEffVfUucTXGSWklIr+2XLTXNVviIaGQhAPOYXKPpYf+awYh8emQ7uloSUj9DQf0/hcBti oqK9151A==; Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2N-00GKeG-6x for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:28 +0000 Received: by mail-qv1-xf49.google.com with SMTP id v3-20020a0cdd830000b02901efe0c3571eso10208057qvk.5 for ; Thu, 20 May 2021 05:44:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=M2Sla/jf/pjrwPm+RYCmA/9jHt4HgBxeLsb7dVByTVM=; b=v1XCFr3qwxS0yy6U3EwwrxhEaKxYk2fl3rkeuk8jONApryuqjt1b/3uk8ynGx/E80X Um/kWXH1m5kg17CxYoU6k4Z4Aolvgkimrn5zOLSKaDGGX4J1aHf2WcXfoLtW4H1OMEpv DTZ57PwSDZ7ct64b033++p2zvUmAmIw9xEQfUItMQfHvCbVW2gUebSKq+fpi/pHkBEh2 UjDydCePksWSdsLT4GB1yFAIuoCRlWoOlzUt8uH2Y2l2zI53w1f2V9dv2QgpdhTzTqK+ VWdAxVS2cXeJOqtkQrlb6GxN7TweZj7PIPzfi1iLVFRat7ZfliMss3NuQkeo9MA3NJyp lawA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=M2Sla/jf/pjrwPm+RYCmA/9jHt4HgBxeLsb7dVByTVM=; b=fxGRN0XWsfYXLrFETdoS7qk4ZABwNezWz7X3EFQ344VMA3rYXVTrGNsuZ1d8KDK8Uv 8lqjg1J0VW3fJouWmIlafvskpvGKk0qt3+y2XedT8ekgyY6STHJxG84fwORDf9RnTeWn x9UbmrezZmY+V53D4RF0hbl1BO6B8Zr5Cf1N8cjR1o90vO6S1B40sQOjiTrs2M8tzKKI Jlzp7ttOXijuOFKECDhdBWWY8KwplA/FW0t4wClrkYX9D3YGLDSpuHMCw30kcVzzTSt5 EtZgHiVcajsOMIqpjwLLkabRwW4Vu9LaDlpke0YuylVWSxS3N2tXzvYQBK8HnrPvJXcF nM+A== X-Gm-Message-State: AOAM530poowSypRMQK6UJlnM/pBJl4JlIZzi61x/AZ/lkE/DepvcJBOS jLmO99HZ4F3isB1B4w5IiFYUpdz80OrzC2Y46y153XCmgacBiTYKzSnDSxdXRMhpaCttgotRsct sDZ8n0VZlVGKhuQCa7Kptx3xzphBaVoebbIwdqO/PUYLrZ78jbx+gEgpsIL1DdsW/aN/pOLGNzy fRWQ== X-Google-Smtp-Source: ABdhPJxPajU8cRrRpvil8MuEM16kRitkfEByRzrCKJZGiCgn2d9XIT0wRyDJzCumMhqcrJzJ5k2tSSErzQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:e34b:: with SMTP id a11mr5466553qvm.24.1621514665219; Thu, 20 May 2021 05:44:25 -0700 (PDT) Date: Thu, 20 May 2021 13:43:56 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-9-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 08/18] arm64: Move documentation of dcache_by_line_op From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054427_272111_EDE81A84 X-CRM114-Status: GOOD ( 11.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The comment describing the macro dcache_by_line_op is placed right before the previous macro of the one it describes, which is a bit confusing. Move it to the macro it describes (dcache_by_line_op). No functional change intended. Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/assembler.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 0a276b46ef50..ced791124b28 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -387,6 +387,14 @@ alternative_cb_end bfi \tcr, \tmp0, \pos, #3 .endm + .macro __dcache_op_workaround_clean_cache, op, addr +alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE + dc \op, \addr +alternative_else + dc civac, \addr +alternative_endif + .endm + /* * Macro to perform a data cache maintenance for the interval * [addr, addr + size) @@ -398,14 +406,6 @@ alternative_cb_end * fixup: optional label to branch to on user fault * Corrupts: addr, size, tmp1, tmp2 */ - .macro __dcache_op_workaround_clean_cache, op, addr -alternative_if_not ARM64_WORKAROUND_CLEAN_CACHE - dc \op, \addr -alternative_else - dc civac, \addr -alternative_endif - .endm - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 add \size, \addr, \size From patchwork Thu May 20 12:43:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8837DC433ED for ; Thu, 20 May 2021 12:49:13 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C28961163 for ; Thu, 20 May 2021 12:49:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C28961163 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ynVYVKTg0UV97aS4ty1ynSREQvcX2qj/vNa28o+iFB8=; b=cRhD6PJJMllz8X PQrxz3rXUgjCr2mfX4mNjm++Ah1JKow8PpOS/zmiGBoZvWEmNKjd582M8uH3AgvzauJJWnhpQ2bh/ RleNepZYHXT/s0LvOPzLp7PLpPLgwhQe2VjcB3xiToIsE783j2JI7y2UkBhrKrHPk+KEUUrMSrddZ T/+s58wyD1I7LkMV+M+dgxgwI9QPXvWHDXYF2ie0RLp3mUdExroZzwuAI18Zs+MJZj44p1VfRzbah JZW8fFgSc4g4c05K1dxnhvQe5pTloKuz1jQt/7ZzFfHV+SLHp0EoIYUnRVhNhtuNRiK96CH+oy7Xt R0TFrYhHFM4rs3BV+YKw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji5Q-000w7v-Io; Thu, 20 May 2021 12:47:36 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2S-000u1c-5g for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:32 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=7BZZdjgKJC74gmsJNzueWJli3QPBatE8jTGovmrrUAs=; b=P+Uy7diUFtnZy93o+aRjcJL3Hx RBr7Me6ZS8NdX5Q5CB1SITULYbwK+zp7RmLYJGJuXsuef0SMPMLzhR9oNtrNWPWzr0b4H4gjiy+WJ mGnnxO3WCZN59Zy+XoItY13R4JJCTO1e25Mm578PKyLtKnU037qd/RYhgjnJb/sWn3XR/9ioxl2mC 9eBESD2zyV0feVmt5AiiV7+Tx3v4+x0HzS1qxQrCz//3zJ4AvLUBrPCX8M+t2Ctf7JDvCqbYUkoZs B39M2hiP/7oKj9wgrDcKxdE6DuyfrdTJRVu7YNEiJdo5vyOLKRCOnp43KTdaDqkJe4RW81b/P/rb3 InBFZpgA==; Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2P-00GKen-LB for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:30 +0000 Received: by mail-qk1-x749.google.com with SMTP id u126-20020a3792840000b02902e769005fe1so12373784qkd.2 for ; Thu, 20 May 2021 05:44:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=7BZZdjgKJC74gmsJNzueWJli3QPBatE8jTGovmrrUAs=; b=I6u/tY6xm6POIbz2zVrsnpPDbqOgjLtwd8hqC1FyaI12DEHWLRae3xwycFSHcN+O6K cL2I0uIGNZvSvMyejjwh7aZA5nRqAYVK54P4IfL3gdrrPW/99Oqp0D75ejxjGtjGqig5 41hvo2AJFm3UoCixLlRtjj73PNpGGfaz2N5uZJSK4q6LsHQh8e00riCRtNkNRk0y4SUs K64SgqNy1Nwq5qoVsxynBt8+0ct4bdoPW/mBq1R661hMVHVJNCYL9NH1oId6mMgdSszX 8gRMex8oUQF4opmFupl1GTTGtJiKdTG76fXOMSLHzr/4Y2UlTU3lN3Md1EJrIAgPw731 joEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=7BZZdjgKJC74gmsJNzueWJli3QPBatE8jTGovmrrUAs=; b=JK1VLqD2xgvtkPj/vqT+79Qr0mV8bL3KkPBg1AIbhvOR6IG4YGxyWpjYQteubtblCx Jypq7nnhhhAPjAGZsK6LCm/1tUECyyM311YCw2oTbFgvMO9Z6cwD3fLDLH6zwyI5sy2i R7V5KQPvqPigtZ0GWyzuTlXk8IUolHytLssKZh89Wk08cVsa9ABvX18y/0BYJAC13MMq 3JdTtQ+1DNjkyecCLogVVD1FYqCs21R/AegG3DUrYFrwzEw6keMpIMcUJn60ep6el3LQ zuWOc1QvHbhNGPBfjhBM8m/krWK3oYHzUY4IkvsDgqwhCnEsVrTjSFsVkYd6xljwWZdQ U++Q== X-Gm-Message-State: AOAM5303kxWMUhs//ED/2nm2dqmqpZ66QakBoywiEM3iSZNWpZYYxqal snAJ5eqgMLym0xhWgH0A0IVac8EX1RIKHkIRxEcobf1bR6ctkBQrkT+6eON6JhKd6kIGDXWDjdU PxoUnGoycg/NfuayA7kuWc3lKGokGqBXfAAR3JdSIO09qYwt5QoTD0px9ROesTT2+cDuCSo+NLm VuTQ== X-Google-Smtp-Source: ABdhPJxJjaFlPkXGH99ACDYlC204HuHXeyuL7Til9z/qiZBe9kR7XPH6Pzn3O4AhvG6vuKFVN1skAlB+KA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:e98a:: with SMTP id z10mr5304774qvn.36.1621514667193; Thu, 20 May 2021 05:44:27 -0700 (PDT) Date: Thu, 20 May 2021 13:43:57 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-10-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 09/18] arm64: Fix comments to refer to correct function __flush_icache_range From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054429_718152_7AED53B9 X-CRM114-Status: GOOD ( 18.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Many comments refer to the function flush_icache_range, where the intent is in fact __flush_icache_range. Fix these comments to refer to the intended function. That's probably due to commit 3b8c9f1cdfc506e9 ("arm64: IPI each CPU after invalidating the I-cache for kernel mappings"), which renamed flush_icache_range() to __flush_icache_range() and added a wrapper. No functional change intended. Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/kernel/hibernate-asm.S | 4 ++-- arch/arm64/mm/cache.S | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index 0ed2f72a6b94..ef2ab7caf815 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -45,7 +45,7 @@ * Because this code has to be copied to a 'safe' page, it can't call out to * other functions by PC-relative address. Also remember that it may be * mid-way through over-writing other functions. For this reason it contains - * code from flush_icache_range() and uses the copy_page() macro. + * code from __flush_icache_range() and uses the copy_page() macro. * * This 'safe' page is mapped via ttbr0, and executed from there. This function * switches to a copy of the linear map in ttbr1, performs the restore, then @@ -87,7 +87,7 @@ SYM_CODE_START(swsusp_arch_suspend_exit) copy_page x0, x1, x2, x3, x4, x5, x6, x7, x8, x9 add x1, x10, #PAGE_SIZE - /* Clean the copied page to PoU - based on flush_icache_range() */ + /* Clean the copied page to PoU - based on __flush_icache_range() */ raw_dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x10, x3 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 7318a40dd6ca..80da4b8718b6 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -50,7 +50,7 @@ alternative_else_nop_endif .endm /* - * flush_icache_range(start,end) + * __flush_icache_range(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, From patchwork Thu May 20 12:43:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0F9FC4361A for ; Thu, 20 May 2021 12:49:33 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 56E1861001 for ; Thu, 20 May 2021 12:49:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 56E1861001 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LcsFymNCXUySFt76tFHQkKWYPJv99YL9m/jvK4/zxT8=; b=VCirSfcDiAUXIV s4sTQoZMV6yWL+nif4jAOaDOY1Q8qSU0gcpKWA86bscdgBz2hvemCfp70PWO1hXdUebe5ARTlBbSX TDdcLWneTn+72n5gtoWKWLiXKGLVZWJDZm2T48BiNwUhizrzpz48lmAHsxr1m3NvZ1MNg+fh5RkRX 5L3K4dNcLA7fSQBxpi/0ZXJVc2/a4hdbLOrQS0gBxKGgFUSFj1UFYNkEUJNHM3OK8TdDth+M2X+Mj 9cTh11219Fk8vDKBrIKWU53xjVCzn7uTdBLyye3Btvq1ViswqELwZcIOvoLU2Azs1ZL2526tqoZ2B NDWCaM9L+7O/u/Vci91g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji5k-000wN3-3Z; Thu, 20 May 2021 12:47:56 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2U-000u1s-CZ for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=E2iNWjxbuYP1ToNdaOW918Q7826xkkT2mW79gH162Vc=; b=r6EBJAACwy+WrgSe9wA23iY8/w nVIeb0m9hWooZfNKmpYIQvWe7wZ3Jc+3v6amIvKtfi58q4+Di8PfRT4ixPfaMWcW/ENkweXD4P2Ph kDZKrqFxqzYTbSkhhvl0cMBv5VCy7cRvQhPub/zMm1WdZjUY8KSVlsamFCCR2hTMFSyKB+Tt6cm98 FI8hRB/y966ROPmM5kuhEdbK5R9ipXiGGMIuuUHJXBLt8WtpsOH+pUIGBRmWL9+yTdrc4RHS3bDq4 oJmqb5jsQK3DH1KIXynWUEqUZM3fBGgyU9GrZlIdPRWTNC58jNbJkC1eBtpSTI8NUWg8IcmmdZMhM Bl9T4v2g==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2R-00GKfD-Mt for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:33 +0000 Received: by mail-wm1-x34a.google.com with SMTP id n127-20020a1c27850000b02901717a27c785so2289399wmn.9 for ; Thu, 20 May 2021 05:44:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=E2iNWjxbuYP1ToNdaOW918Q7826xkkT2mW79gH162Vc=; b=HXWJc/n3DAhHhZa7egbmncaGnAWr5CyALt1mThq5yQFO6xwwasJnao5d3ysK2lljgE d7M6ZriD3l3UG+3Jqyg0iruvxNZt9N6PCMU+KzzTS/Uh6R46g0yxAYRbvOE/ZhSDaDBw mQZS7maRHE1VPhd4g2vLl9zJj4iYX+iLGdn4IIiltaI3RVRqpbuILJXVnjk4/OWe/Upi BZKWTgVzYjSHZCUdzRnBlaDX/jCEERFrJ4Ah5uCkn2/tYhF5N/bGK+MQqPgqs17BWjVy qPkA8/J7rANl9+aWwiSlG4u/9r4GNwST559yPP6sLJ2cC9X85vhU0RqXuJWhBuHiVATP qAJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=E2iNWjxbuYP1ToNdaOW918Q7826xkkT2mW79gH162Vc=; b=MUAuYrARjmPrWnfP7Rc3ivoNU9V4KgMCVC+O7z+q737SwkxbRjSMCEIkz5E5HAc4Ed /S+Cue3vd7pHrOlbBtpRXK6PT3IdFSFkRWDAD3J+5YRRDfwvAcdKCyEr3ina3HWy6fVH PtGgus8OVlryNUsa2fWZyznBaL7oxRZI7WK2cOQncmt1/PQcYVmp2eNyfc4R1F4TJyTH CX5gWMNyq4+jqGKRfgqtROMZ9CSEM1STWrk3Q3zj7qX7dY6Gf4OQf2JSCZew7iSvB0aL JjpYavAPQgA8d+f998w10CbwO9mlPge9Uaxfxad5clWd5z9XD46I695+41qR1Q8Gy7ym Jvgg== X-Gm-Message-State: AOAM530PzNcNa4N0JlhxQ7it5ZShKWghcQT9FzspYMtqTroKFV879Cg/ i5kADdVlR4RW+ZGbIq4gdmpl+Tm+roveVo0gVhjLOIUPlSLT9WrKCyxfo6DWIWJzsppS+UYcwHc x9mLJ25Bp8YPxm0wP03FVD3vWGEK7eHwj+L6JDPJIPp8F168wKj+oiW8fWam5h7wLZs3ZZgmUBK DGug== X-Google-Smtp-Source: ABdhPJzofVO53IUQ4nGzc798SqPFGMEcnD7jqmbI3jSRqVrdINY3ZONVgBDkTG3l6IvCwtlQTG5CxaI6MA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c344:: with SMTP id l4mr3861265wmj.120.1621514669568; Thu, 20 May 2021 05:44:29 -0700 (PDT) Date: Thu, 20 May 2021 13:43:58 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-11-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 10/18] arm64: __inval_dcache_area to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054431_770583_83227F84 X-CRM114-Status: GOOD ( 15.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. Because the code is shared with __dma_inv_area, it changes the parameters for that as well. However, __dma_inv_area is local to cache.S, so no other users are affected. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/head.S | 5 +---- arch/arm64/mm/cache.S | 16 +++++++++------- arch/arm64/mm/flush.c | 2 +- 4 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index a586afa84172..157234706817 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -59,7 +59,7 @@ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); -extern void __inval_dcache_area(void *addr, size_t len); +extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pop(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 96873dfa67fd..8df0ac8d9123 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -117,7 +117,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) dmb sy // needed before dc ivac with // MMU off - mov x1, #0x20 // 4 x 8 bytes + add x1, x0, #0x20 // 4 x 8 bytes b __inval_dcache_area // tail call SYM_CODE_END(preserve_boot_args) @@ -268,7 +268,6 @@ SYM_FUNC_START_LOCAL(__create_page_tables) */ adrp x0, init_pg_dir adrp x1, init_pg_end - sub x1, x1, x0 bl __inval_dcache_area /* @@ -382,12 +381,10 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x0, idmap_pg_dir adrp x1, idmap_pg_end - sub x1, x1, x0 bl __inval_dcache_area adrp x0, init_pg_dir adrp x1, init_pg_end - sub x1, x1, x0 bl __inval_dcache_area ret x28 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 80da4b8718b6..5170d9ab450a 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -138,25 +138,24 @@ alternative_else_nop_endif SYM_FUNC_END(__clean_dcache_area_pou) /* - * __inval_dcache_area(kaddr, size) + * __inval_dcache_area(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are invalidated. Any partial lines at the ends of the interval are * also cleaned to PoC to prevent data loss. * - * - kaddr - kernel address - * - size - size in question + * - start - kernel start address of region + * - end - kernel end address of region */ SYM_FUNC_START_LOCAL(__dma_inv_area) SYM_FUNC_START_PI(__inval_dcache_area) /* FALLTHROUGH */ /* - * __dma_inv_area(start, size) + * __dma_inv_area(start, end) * - start - virtual start address of region - * - size - size in question + * - end - virtual end address of region */ - add x1, x1, x0 dcache_line_size x2, x3 sub x3, x2, #1 tst x1, x3 // end cache line aligned? @@ -237,8 +236,10 @@ SYM_FUNC_END_PI(__dma_flush_area) * - dir - DMA direction */ SYM_FUNC_START_PI(__dma_map_area) + add x1, x0, x1 cmp w2, #DMA_FROM_DEVICE b.eq __dma_inv_area + sub x1, x1, x0 b __dma_clean_area SYM_FUNC_END_PI(__dma_map_area) @@ -249,6 +250,7 @@ SYM_FUNC_END_PI(__dma_map_area) * - dir - DMA direction */ SYM_FUNC_START_PI(__dma_unmap_area) + add x1, x0, x1 cmp w2, #DMA_TO_DEVICE b.ne __dma_inv_area ret diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index ac485163a4a7..4e3505c2bea6 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -88,7 +88,7 @@ EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); void arch_invalidate_pmem(void *addr, size_t size) { - __inval_dcache_area(addr, size); + __inval_dcache_area((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif From patchwork Thu May 20 12:43:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3E5EC433ED for ; Thu, 20 May 2021 12:50:10 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 831BC61184 for ; Thu, 20 May 2021 12:50:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 831BC61184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QwfZd9zlAYxs1wAMFY6BH3czrDAg1bRXf1oNO3/inns=; b=nqBlCIZm5hWKqW d0TnCIYqda3ZUJGzEsYEtABaZKW27Ec2ca7CBKicteThVOfr+BOOI1/IpnaR7FKdE6+8NISva4JLY FWZFOUc3AY4qqBbqPzAdZP0xIcWehJ13yCBb47DWUHvczkoXqbi/oGY+AUYkdPh4J7IMcbQRWo8Dv Sv2LTV7MkxdGd60iJbRKOeafZNZe2jMWWsXL1Z9TboXafFqijo63f6YRVZyLBA8P1jYwwfNDteBLZ 3YKtkzcP9KfJpQR7xymo+leZ19YDi5Jyi37vxAwLlF4IrQnI38Q4zhoSEU7PcpFbpWfQWPCQje0bO 2aEfujbo43QeoCo9/1yQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji6H-000wmZ-5u; Thu, 20 May 2021 12:48:29 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2W-000u2L-PS for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=HdRq0+Koen2r2SBcAtNy4/l729eFX5JygRHHNxjltB4=; b=SJCgzoGjhkyXrjvVm2bxs4+D22 t1/rb6te93i4VKQ8/7QlyR05UzvcbvXZTUdYS16HnJYWECyjuhiDcQUk3DDxGZAJVXFKThbXm+QB7 IbTn5FSHVtTnkD1hrsxc9uWeRGV1VlCe88AXKRRL06pK5aqdir9693T4Sjh8opjrs5en4855NnbPE I0atPvR648MBd877Y6pagqTsglMe4KGZWeLIf/Azo4DsSdTUsUlXKSoxBq9VNBxFapokpsCs9kgao lCdrDYILTnl3ZwI1Yjq/dZuzOpdloJ8eAMGVN8oyoIDBN9mgUfot50wi5QVRkBu4svKvs5cmOfmgw FBzrRJxA==; Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2U-00GKfm-5r for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:35 +0000 Received: by mail-qt1-x84a.google.com with SMTP id f17-20020ac87f110000b02901e117339ea7so12143386qtk.16 for ; Thu, 20 May 2021 05:44:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=HdRq0+Koen2r2SBcAtNy4/l729eFX5JygRHHNxjltB4=; b=VLI+9tIRb4ANmxfC7nRnMfk3OayhlBKmYzGuOSw+ps1MDk4cyuGBeg+2DL+zCTM0Rk jC8xFZY6326rYZQQxfbJpfl5ZIz9Sye4viO26kB1imJ3rjWCgMPAPurwkcNiBVe5eh9A QKfF0K9mcl17xz933lYGok0XcQyI6NWwWztQTtGnMyH6aVXHrwM8D8b2GcUsMhvY3O4A dG+kujthMNG3dHPk7X25x7ZVwMSAfdO9ahuIo316Ennx6NnPc9L/WQJxDJ9Wz2+YwSEv 39hcgdXQn6/M6cqQLMqt0ci0M4rDz/+zrp010vioBxEBvkZTJKHTsRDRrw27RfsXaGoP c+Xw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=HdRq0+Koen2r2SBcAtNy4/l729eFX5JygRHHNxjltB4=; b=qGttxKa+ySIUuPkHgN3FtU2EG/ymGabI2pQzkFlNK5SFy2vGoRF8yY9ijxMkeTucKR lq6otSIWUNllsU6506GMqcHQ4Y58vC/RxCva1cnabYHeioRzTdLFRIlxa1HxrlB85KY1 HSZ7Jq1relvOOpCIf9h5KM2IvctngY+crEKyyY/3jL1W8BRvr4Kj7XkBwvIRopJGoGGx 6Rnl1Ri582ft6s9zDJZ2/73Uzz91kbGXnXwsWZRuCLHYlPgaoMkWinVW2IpcO+ZK96/2 NGEye/dmb7FNbgBBc5fIy8+VWWmL82doKeFElQUOT9cEe3SZNr9/TVnnYkRAbBL/tN5n TN0g== X-Gm-Message-State: AOAM5336lbVItZC/lCB12P1iyks881bpOCIEpCskkst9QULrSVQ2Xbn0 y0fqp3HzI5nGHPWGLzlPDR5qOp9qj9QbOOBfhB366jeSpQS0aKoNp1ni9cBiNz7wHlcgRZ8uYx2 pjxU4ShFi5DXhbH0srrEQrspZPu8kh3uT8Iv7uCQoe0Jj2rrlQK/L2+vOe27p+0RJq7/UuFIe2S wEuw== X-Google-Smtp-Source: ABdhPJxDAauvjmH4N4dCsdfn0/reqlRL/1mb6t7+vpXZbo6/dbwMzyeJFYDOTVteDevUYwfh0ClnQPPIHA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:ad4:4f21:: with SMTP id fc1mr5229991qvb.16.1621514671611; Thu, 20 May 2021 05:44:31 -0700 (PDT) Date: Thu, 20 May 2021 13:43:59 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-12-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 11/18] arm64: dcache_by_line_op to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054434_240274_941562C8 X-CRM114-Status: GOOD ( 13.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/assembler.h | 27 +++++++++++++-------------- arch/arm64/kvm/hyp/nvhe/cache.S | 1 + arch/arm64/mm/cache.S | 5 +++++ 3 files changed, 19 insertions(+), 14 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ced791124b28..c4cecf85dccf 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -397,40 +397,39 @@ alternative_endif /* * Macro to perform a data cache maintenance for the interval - * [addr, addr + size) + * [start, end) * * op: operation passed to dc instruction * domain: domain used in dsb instruciton - * addr: starting virtual address of the region - * size: size of the region + * start: starting virtual address of the region + * end: end virtual address of the region * fixup: optional label to branch to on user fault - * Corrupts: addr, size, tmp1, tmp2 + * Corrupts: start, end, tmp1, tmp2 */ - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup + .macro dcache_by_line_op op, domain, start, end, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 - add \size, \addr, \size sub \tmp2, \tmp1, #1 - bic \addr, \addr, \tmp2 + bic \start, \start, \tmp2 .Ldcache_op\@: .ifc \op, cvau - __dcache_op_workaround_clean_cache \op, \addr + __dcache_op_workaround_clean_cache \op, \start .else .ifc \op, cvac - __dcache_op_workaround_clean_cache \op, \addr + __dcache_op_workaround_clean_cache \op, \start .else .ifc \op, cvap - sys 3, c7, c12, 1, \addr // dc cvap + sys 3, c7, c12, 1, \start // dc cvap .else .ifc \op, cvadp - sys 3, c7, c13, 1, \addr // dc cvadp + sys 3, c7, c13, 1, \start // dc cvadp .else - dc \op, \addr + dc \op, \start .endif .endif .endif .endif - add \addr, \addr, \tmp1 - cmp \addr, \size + add \start, \start, \tmp1 + cmp \start, \end b.lo .Ldcache_op\@ dsb \domain diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 36cef6915428..3bcfa3cac46f 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -8,6 +8,7 @@ #include SYM_FUNC_START_PI(__flush_dcache_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 5170d9ab450a..3b5461a32b85 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -115,6 +115,7 @@ SYM_FUNC_END(invalidate_icache_range) * - size - size in question */ SYM_FUNC_START_PI(__flush_dcache_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) @@ -133,6 +134,7 @@ alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif + add x1, x0, x1 dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret SYM_FUNC_END(__clean_dcache_area_pou) @@ -194,6 +196,7 @@ SYM_FUNC_START_PI(__clean_dcache_area_poc) * - start - virtual start address of region * - size - size in question */ + add x1, x0, x1 dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_poc) @@ -212,6 +215,7 @@ SYM_FUNC_START_PI(__clean_dcache_area_pop) alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif + add x1, x0, x1 dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_pop) @@ -225,6 +229,7 @@ SYM_FUNC_END_PI(__clean_dcache_area_pop) * - size - size in question */ SYM_FUNC_START_PI(__dma_flush_area) + add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__dma_flush_area) From patchwork Thu May 20 12:44:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4643EC433B4 for ; Thu, 20 May 2021 12:50:46 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C835961163 for ; Thu, 20 May 2021 12:50:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C835961163 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=h6Bm1PqGnfJKveIOsOwxUXVMLh1Bjqr43g2Cy+ue89o=; b=CsjI9lztoilqvF fcNfgTcaARoYFrTGcdx6QjMbu6O6RK50Cv2pX3xDrYT/9zIHEjek9/Tdrj6V+dB/d970jCxmf52IR u9zzienwo8+NYWsK4cKzUE3LgcwoMVw5IrWD1LylGyD6aL87Rkyg0lcg2BSPuIY2LAABcJCueSFTp 6D622mEwTUYogC6Td+SdMAH6WiwagJGpy/NBGhOkr18t0gjWT7P6iyRSJMkFto8r+V1U/T0PgEswo mNjKZD9GU1vtTWZbWeuYr1Vp7BRlFzgHNJ0WaZFSwzgReMex2Bro0lLuISbfc2PCSq4jmZIr+qouo Hnbn9mMfT3GqCjqbj4JQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji6k-000x6n-JU; Thu, 20 May 2021 12:48:59 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2Y-000u2e-Oo for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:39 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=i0IqN6c6HJkzO/xcD6oBcDU6/Hh8841smXSe77DfbNk=; b=abyN169iTDe/cgQdr9U4vOVy4q LMyDKgD/uqcrwHY0aX2mfMpPkP1RNlPnpbdTzvejJjcyrRazCdt37EHhCvAlx6nd6jtna6q0bRe5c r5NEmjE/FToz5tIlXYwd9Eml9jaffTdvPxIuhiheJyqBJtQbhMo9NkG7uZ/9CqlzKgpcDQJvap88s 9AIb2clzVM+ITgSu9wcXuolAeRWh2h5jBzZ1T4hVGRuwxA48gVWXU5QO9OQ7IFEKZf+KM0bboszQ1 ISmbo1piGH++DNcoO9Zvw5Rc5ISHGBoJDXZlUKaSNUMgbpqpg36zUgExZ+XWQ3WHGwemu+W2YxQ2O JwPunTsA==; Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2V-00GKfy-BP for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:37 +0000 Received: by mail-qv1-xf49.google.com with SMTP id l19-20020a0ce0930000b02901efdf1c21ecso10255572qvk.10 for ; Thu, 20 May 2021 05:44:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=i0IqN6c6HJkzO/xcD6oBcDU6/Hh8841smXSe77DfbNk=; b=IpKe4hAwROt0v4/uWmwLp7Nr1PPR0mclulTbDwsoL/M7AR/kFc47Ve/eRe2SXkRL7B znVhY2tUI8mbyKkUDo8t+z1hXvFEsMha1V7e6qLyd2kuQyhHDM3EoxL8tqLCORQNEiRq 4k0T7+iq0ozZR5z+TvwvzOQCKrA17gZfLNSWgwa0m8ho0FDT8U2fpU7T3K8+yhAMe++k Zj/cKIIZ+kGRMSLRlQEo3nKGz4qW6TotCzZZZsVRX0d46n+CMJhoi9I8eTzz+yx6Tc29 20RiRITbBbZfPZL8GpiuUin/NQLImYyx8JCwcuiQCPC+a7jxgP77hPTxVufL+MiIpjft lVxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=i0IqN6c6HJkzO/xcD6oBcDU6/Hh8841smXSe77DfbNk=; b=AV/4smQywhWeRH2g1+ZjGWDGQeViY6NT0uTXOi0uCj1t/Vpj/RTGcxWC8iu/KztyD4 AIKobYzUOEmz+7WyornPraRGK9ZVLAK2empOmTx/ExorO+f+r3wlLtntpZ2sK8CPctP8 abHAz0uA5OPvEiS79hK3+XSTVvz0udr3RON0Da5GmUW9CFKMMDvn3wU3SKYwbOoRaxDF 5JULmelpnH7ut9LhA6E0YF5m9X2iGn8v12REEAmIPYEaLETGyZ6vhRFeFmlswN3dmbtu u3VTqFn6AmRs829x5ejZCMj8X9tXp16y2x9w4x0HvKfiemjcGbtU+f+wqc1KveQhcuqU WpNQ== X-Gm-Message-State: AOAM531gitVbc6KpRqPgpQDrwZEVyXwawfjHVoV+6DwFMbgYtXg8Z4Mx QCbLU90hV8UAzdfxC5vHKgIo70ixmZhSjZFu9K+PC1MHTWryyAHt7yLeJEmpWgwU1KjlZLKIi+1 h35Dd2s3zpap0vLLzdImNonxoemppvYaDoX4FRBCXmy2PP8C+aDzQPy3YdJ+VeY0zRUtlUV8j+y ygZw== X-Google-Smtp-Source: ABdhPJwAU72+X/bQB3oKmbwrF45ZRmHXhJy++64QNb2sdG4e2GTNaevLW86q9Ly0QFgL+Zqi6nsdQwAKOw== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:ef81:: with SMTP id w1mr5376770qvr.32.1621514673729; Thu, 20 May 2021 05:44:33 -0700 (PDT) Date: Thu, 20 May 2021 13:44:00 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-13-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 12/18] arm64: __flush_dcache_area to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054435_423265_001FD1A9 X-CRM114-Status: GOOD ( 24.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba --- arch/arm64/include/asm/arch_gicv3.h | 3 ++- arch/arm64/include/asm/cacheflush.h | 8 ++++---- arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 3 ++- arch/arm64/kernel/hibernate.c | 18 +++++++++++------- arch/arm64/kernel/idreg-override.c | 3 ++- arch/arm64/kernel/kaslr.c | 12 +++++++++--- arch/arm64/kernel/machine_kexec.c | 20 +++++++++++++------- arch/arm64/kernel/smp.c | 8 ++++++-- arch/arm64/kernel/smp_spin_table.c | 7 ++++--- arch/arm64/kvm/hyp/nvhe/cache.S | 1 - arch/arm64/kvm/hyp/nvhe/setup.c | 3 ++- arch/arm64/kvm/hyp/pgtable.c | 13 ++++++++++--- arch/arm64/mm/cache.S | 9 ++++----- 14 files changed, 70 insertions(+), 40 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index 934b9be582d2..ed1cc9d8e6df 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -124,7 +124,8 @@ static inline u32 gic_read_rpr(void) #define gic_read_lpir(c) readq_relaxed(c) #define gic_write_lpir(v, c) writeq_relaxed(v, c) -#define gic_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l)) +#define gic_flush_dcache_to_poc(a,l) \ + __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) #define gits_read_baser(c) readq_relaxed(c) #define gits_write_baser(v, c) writeq_relaxed(v, c) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 157234706817..695f88864784 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -50,15 +50,15 @@ * - start - virtual start address * - end - virtual end address * - * __flush_dcache_area(kaddr, size) + * __flush_dcache_area(start, end) * * Ensure that the data held in page is written back. - * - kaddr - page address - * - size - region size + * - start - virtual start address + * - end - virtual end address */ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); -extern void __flush_dcache_area(void *addr, size_t len); +extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pop(void *addr, size_t len); diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 3578aba9c608..0ae2397076fd 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -137,7 +137,7 @@ void efi_virtmap_unload(void); static inline void efi_capsule_flush_cache_range(void *addr, int size) { - __flush_dcache_area(addr, size); + __flush_dcache_area((unsigned long)addr, (unsigned long)addr + size); } #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 25ed956f9af1..33293d5855af 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -180,7 +180,8 @@ static inline void *__kvm_vector_slot2addr(void *base, struct kvm; -#define kvm_flush_dcache_to_poc(a,l) __flush_dcache_area((a), (l)) +#define kvm_flush_dcache_to_poc(a,l) \ + __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b1cef371df2b..b40ddce71507 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -240,8 +240,6 @@ static int create_safe_exec_page(void *src_start, size_t length, return 0; } -#define dcache_clean_range(start, end) __flush_dcache_area(start, (end - start)) - #ifdef CONFIG_ARM64_MTE static DEFINE_XARRAY(mte_pages); @@ -383,13 +381,18 @@ int swsusp_arch_suspend(void) ret = swsusp_save(); } else { /* Clean kernel core startup/idle code to PoC*/ - dcache_clean_range(__mmuoff_data_start, __mmuoff_data_end); - dcache_clean_range(__idmap_text_start, __idmap_text_end); + __flush_dcache_area((unsigned long)__mmuoff_data_start, + (unsigned long)__mmuoff_data_end); + __flush_dcache_area((unsigned long)__idmap_text_start, + (unsigned long)__idmap_text_end); /* Clean kvm setup code to PoC? */ if (el2_reset_needed()) { - dcache_clean_range(__hyp_idmap_text_start, __hyp_idmap_text_end); - dcache_clean_range(__hyp_text_start, __hyp_text_end); + __flush_dcache_area( + (unsigned long)__hyp_idmap_text_start, + (unsigned long)__hyp_idmap_text_end); + __flush_dcache_area((unsigned long)__hyp_text_start, + (unsigned long)__hyp_text_end); } swsusp_mte_restore_tags(); @@ -474,7 +477,8 @@ int swsusp_arch_resume(void) * The hibernate exit text contains a set of el2 vectors, that will * be executed at el2 with the mmu off in order to reload hyp-stub. */ - __flush_dcache_area(hibernate_exit, exit_size); + __flush_dcache_area((unsigned long)hibernate_exit, + (unsigned long)hibernate_exit + exit_size); /* * KASLR will cause the el2 vectors to be in a different location in diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index e628c8ce1ffe..3dd515baf526 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -237,7 +237,8 @@ asmlinkage void __init init_feature_override(void) for (i = 0; i < ARRAY_SIZE(regs); i++) { if (regs[i]->override) - __flush_dcache_area(regs[i]->override, + __flush_dcache_area((unsigned long)regs[i]->override, + (unsigned long)regs[i]->override + sizeof(*regs[i]->override)); } } diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 341342b207f6..49cccd03cb37 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -72,7 +72,9 @@ u64 __init kaslr_early_init(void) * we end up running with module randomization disabled. */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - __flush_dcache_area(&module_alloc_base, sizeof(module_alloc_base)); + __flush_dcache_area((unsigned long)&module_alloc_base, + (unsigned long)&module_alloc_base + + sizeof(module_alloc_base)); /* * Try to map the FDT early. If this fails, we simply bail, @@ -170,8 +172,12 @@ u64 __init kaslr_early_init(void) module_alloc_base += (module_range * (seed & ((1 << 21) - 1))) >> 21; module_alloc_base &= PAGE_MASK; - __flush_dcache_area(&module_alloc_base, sizeof(module_alloc_base)); - __flush_dcache_area(&memstart_offset_seed, sizeof(memstart_offset_seed)); + __flush_dcache_area((unsigned long)&module_alloc_base, + (unsigned long)&module_alloc_base + + sizeof(module_alloc_base)); + __flush_dcache_area((unsigned long)&memstart_offset_seed, + (unsigned long)&memstart_offset_seed + + sizeof(memstart_offset_seed)); return offset; } diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index a03944fd0cd4..3e79110c8f3a 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -72,7 +72,9 @@ int machine_kexec_post_load(struct kimage *kimage) * For execution with the MMU off, reloc_code needs to be cleaned to the * PoC and invalidated from the I-cache. */ - __flush_dcache_area(reloc_code, arm64_relocate_new_kernel_size); + __flush_dcache_area((unsigned long)reloc_code, + (unsigned long)reloc_code + + arm64_relocate_new_kernel_size); invalidate_icache_range((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); @@ -106,16 +108,18 @@ static void kexec_list_flush(struct kimage *kimage) for (entry = &kimage->head; ; entry++) { unsigned int flag; - void *addr; + unsigned long addr; /* flush the list entries. */ - __flush_dcache_area(entry, sizeof(kimage_entry_t)); + __flush_dcache_area((unsigned long)entry, + (unsigned long)entry + + sizeof(kimage_entry_t)); flag = *entry & IND_FLAGS; if (flag == IND_DONE) break; - addr = phys_to_virt(*entry & PAGE_MASK); + addr = (unsigned long)phys_to_virt(*entry & PAGE_MASK); switch (flag) { case IND_INDIRECTION: @@ -124,7 +128,7 @@ static void kexec_list_flush(struct kimage *kimage) break; case IND_SOURCE: /* flush the source pages. */ - __flush_dcache_area(addr, PAGE_SIZE); + __flush_dcache_area(addr, addr + PAGE_SIZE); break; case IND_DESTINATION: break; @@ -151,8 +155,10 @@ static void kexec_segment_flush(const struct kimage *kimage) kimage->segment[i].memsz, kimage->segment[i].memsz / PAGE_SIZE); - __flush_dcache_area(phys_to_virt(kimage->segment[i].mem), - kimage->segment[i].memsz); + __flush_dcache_area( + (unsigned long)phys_to_virt(kimage->segment[i].mem), + (unsigned long)phys_to_virt(kimage->segment[i].mem) + + kimage->segment[i].memsz); } } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index dcd7041b2b07..5fcdee331087 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -122,7 +122,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) secondary_data.task = idle; secondary_data.stack = task_stack_page(idle) + THREAD_SIZE; update_cpu_boot_status(CPU_MMU_OFF); - __flush_dcache_area(&secondary_data, sizeof(secondary_data)); + __flush_dcache_area((unsigned long)&secondary_data, + (unsigned long)&secondary_data + + sizeof(secondary_data)); /* Now bring the CPU into our world */ ret = boot_secondary(cpu, idle); @@ -143,7 +145,9 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) pr_crit("CPU%u: failed to come online\n", cpu); secondary_data.task = NULL; secondary_data.stack = NULL; - __flush_dcache_area(&secondary_data, sizeof(secondary_data)); + __flush_dcache_area((unsigned long)&secondary_data, + (unsigned long)&secondary_data + + sizeof(secondary_data)); status = READ_ONCE(secondary_data.status); if (status == CPU_MMU_OFF) status = READ_ONCE(__early_cpu_boot_status); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index c45a83512805..58d804582a35 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -36,7 +36,7 @@ static void write_pen_release(u64 val) unsigned long size = sizeof(secondary_holding_pen_release); secondary_holding_pen_release = val; - __flush_dcache_area(start, size); + __flush_dcache_area((unsigned long)start, (unsigned long)start + size); } @@ -90,8 +90,9 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(pa_holding_pen, release_addr); - __flush_dcache_area((__force void *)release_addr, - sizeof(*release_addr)); + __flush_dcache_area((__force unsigned long)release_addr, + (__force unsigned long)release_addr + + sizeof(*release_addr)); /* * Send an event to wake up the secondary CPU. diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 3bcfa3cac46f..36cef6915428 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -8,7 +8,6 @@ #include SYM_FUNC_START_PI(__flush_dcache_area) - add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 7488f53b0aa2..5dffe928f256 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -134,7 +134,8 @@ static void update_nvhe_init_params(void) for (i = 0; i < hyp_nr_cpus; i++) { params = per_cpu_ptr(&kvm_init_params, i); params->pgd_pa = __hyp_pa(pkvm_pgtable.pgd); - __flush_dcache_area(params, sizeof(*params)); + __flush_dcache_area((unsigned long)params, + (unsigned long)params + sizeof(*params)); } } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index c37c1dc4feaf..10d2f04013d4 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -839,8 +839,11 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, stage2_put_pte(ptep, mmu, addr, level, mm_ops); if (need_flush) { - __flush_dcache_area(kvm_pte_follow(pte, mm_ops), - kvm_granule_size(level)); + kvm_pte_t *pte_follow = kvm_pte_follow(pte, mm_ops); + + __flush_dcache_area((unsigned long)pte_follow, + (unsigned long)pte_follow + + kvm_granule_size(level)); } if (childp) @@ -988,11 +991,15 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct kvm_pgtable *pgt = arg; struct kvm_pgtable_mm_ops *mm_ops = pgt->mm_ops; kvm_pte_t pte = *ptep; + kvm_pte_t *pte_follow; if (!kvm_pte_valid(pte) || !stage2_pte_cacheable(pgt, pte)) return 0; - __flush_dcache_area(kvm_pte_follow(pte, mm_ops), kvm_granule_size(level)); + pte_follow = kvm_pte_follow(pte, mm_ops); + __flush_dcache_area((unsigned long)pte_follow, + (unsigned long)pte_follow + + kvm_granule_size(level)); return 0; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 3b5461a32b85..35abc8d77c4e 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -106,16 +106,15 @@ alternative_else_nop_endif SYM_FUNC_END(invalidate_icache_range) /* - * __flush_dcache_area(kaddr, size) + * __flush_dcache_area(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned and invalidated to the PoC. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_PI(__flush_dcache_area) - add x1, x0, x1 dcache_by_line_op civac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__flush_dcache_area) From patchwork Thu May 20 12:44:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ECBCC433ED for ; Thu, 20 May 2021 12:51:03 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F87F61001 for ; Thu, 20 May 2021 12:51:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F87F61001 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=t7mEzqEwl3W3FNeIpYhJdLtmZ0YU18zcXXuk68+0OCM=; b=PXF1FG1v4rjOos AWSwuZb0ybKc1/9ioRj+eK9V2rXbGHKuDTLvrSx2GRkke42UA26T6surIHMOfpxuomCWsm2M3aECs kTza/gEBfszSKGFqWoMJANwoYto2D0uzmE6bundeS4UD4kZplxLAdSwV/Etqsm3BqMPtZPxHKerzz fEZijqU5uPk9o6BdYsXkYENoH/02ztNlQ7vKke2Iblnu4gBIvxT2LGdcAsFkMMz37KTPat5wnXDx2 g7WCr6i+zTHrf3Ju5nlYqWqfy9FAKzQmpcMIzFuZLUtDREnlHFoQ53PlOGCc2LpynVUzop4GYi/gF D3YJjNXdjRp2vz4svRHQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji7F-000xYh-LD; Thu, 20 May 2021 12:49:29 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2a-000u2t-PM for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:40 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=OZNPucg3lMOHZSQE7fP/86DhLlvq3G94MWPFTy7jSkM=; b=U6jLIlJb466FDxYzmP0kYAiVqP wZ6SX67m7WcaIJ/iLJTkoZlE0VBSWDT/CxcsNhEktafSuxyje1XupCGdblw09Ggeqvj+0uurwsE+7 FidowoReeuXTLxuBcUxj0zZOlC5+UNoAiPq3WzR+vax56wtqEy4Hd/JaeUOWYhCS6Lw6dnPFx+e10 JkJvHj2F/UwMUxdzgIZ/JcY8yzAbpbGM1mawHTdElC4Kn0EBExrNTrXWPvh7ZfzFNz/hJ5X4VN1sw lH/1KyKfWj1C78lLS8AZzizqfvPqCPb2Zfz7p3caLrhhK5vmQe/85194cUqoKUZR3SKMs/N2uoqHs 9CZBtdGA==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2Y-00GKgL-6D for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:39 +0000 Received: by mail-wm1-x34a.google.com with SMTP id z1-20020a1c7e010000b0290179806d11d4so1185098wmc.3 for ; Thu, 20 May 2021 05:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=OZNPucg3lMOHZSQE7fP/86DhLlvq3G94MWPFTy7jSkM=; b=IJNbKM2LNEHFehc2kcQ89mDybUlCpQe3ZWMaP6ZSA23tvZ3Dmk3NkvLwgmcJmIe4lj tKlk4Rkw5m1RyUw1DxgZS/11C8/EYqHAmBYakzoQhWA9aegnbah7yVLRaFW+uaDJWVR7 kEuIGy5ZEyHrgrcn4lROm0VlmIJVrT3EJ3qEQKRRNbepqIx/fJ7Dgm3FuKHFTL0Ah6KZ faQc90rxwyFFpTnut1ZZRoRPKIJf1DwMnI/Sp8bM/NiPKcfZjyVjl+Hdn11xJmqPsMbo XjPlloNx5awzH+e5OzJGNi6V4MyNoZ+EW58J96Rbhkh/nkc41zKJd8Fvko+ZQBNo5gKw ZoCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=OZNPucg3lMOHZSQE7fP/86DhLlvq3G94MWPFTy7jSkM=; b=aiVNuUoMPmC8UkMK02l9k6n40gtpfbpUVWbaLDFKQo3mQ46XhDDjKrRvS/fDsQUsrO F/LV4jpM0MJDEknHBZVJB6CtRrviFs6aGj093ZrvI024nWjwL29QK72SofwwFTPxTfZ/ fnbn7Nk/3oau/TnP1oR+mivfBr75wF9T3aUI3Imng/x2uYAhE7qrWbFoNoxHdz6WHjVv 8bclXXue5rhkeoJtJd0leNKGucDBmtVuSB+sKkC164UBSeFa7B52DMp2wdcT7b7yHJBx Dwsm7W63MDQ5asR1Q0ycW82gCjHS+UXJhLjhiJAUGsOyXXu03jZyzdaMHHW3Z5iAdt57 BpqQ== X-Gm-Message-State: AOAM5334ViZyzn/Jjr4H0mmpK8o5VTb4VfNzpXOti6hm/WscJI6wt+oY fPSNiUOjo3osWWjy8F7ai9T/wIXDd2te+aJUo9xe8SIq0gxEo7WZ1k2+LRGEVjduY+Kke8V0xiG gzjBQ3oIbsLArd0jApJH1OP+8o4tdHafUp6VjeJSoGgkgzX3C4E65sm98koZNnlVWG4rUQi3+sw dk1g== X-Google-Smtp-Source: ABdhPJyuRtRXr/bofhUW1lbOmxF7n0dCyxD+BilItdApL89YW0OFrZ0wLEnhTHneUTpOgfWCwkdXk+nXXA== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c090:: with SMTP id r16mr4015969wmh.112.1621514675927; Thu, 20 May 2021 05:44:35 -0700 (PDT) Date: Thu, 20 May 2021 13:44:01 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-14-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 13/18] arm64: __clean_dcache_area_poc to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054438_250874_20C03E6D X-CRM114-Status: GOOD ( 15.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. Because the code is shared with __dma_clean_area, it changes the parameters for that as well. However, __dma_clean_area is local to cache.S, so no other users are affected. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/efi-entry.S | 5 +++-- arch/arm64/mm/cache.S | 16 +++++++--------- 3 files changed, 11 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 695f88864784..3255878d6f30 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -60,7 +60,7 @@ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); -extern void __clean_dcache_area_poc(void *addr, size_t len); +extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 0073b24b5d25..72e6a580290a 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -28,6 +28,7 @@ SYM_CODE_START(efi_enter_kernel) * stale icache entries from before relocation. */ ldr w1, =kernel_size + add x1, x0, x1 bl __clean_dcache_area_poc ic ialluis @@ -36,7 +37,7 @@ SYM_CODE_START(efi_enter_kernel) * so that we can safely disable the MMU and caches. */ adr x0, 0f - ldr w1, 3f + adr x1, 3f bl __clean_dcache_area_poc 0: /* Turn off Dcache and MMU */ @@ -65,4 +66,4 @@ SYM_CODE_START(efi_enter_kernel) mov x3, xzr br x19 SYM_CODE_END(efi_enter_kernel) -3: .long . - 0b +3: diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 35abc8d77c4e..9a9c44bb26d2 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -178,24 +178,23 @@ SYM_FUNC_END_PI(__inval_dcache_area) SYM_FUNC_END(__dma_inv_area) /* - * __clean_dcache_area_poc(kaddr, size) + * __clean_dcache_area_poc(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoC. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_LOCAL(__dma_clean_area) SYM_FUNC_START_PI(__clean_dcache_area_poc) /* FALLTHROUGH */ /* - * __dma_clean_area(start, size) + * __dma_clean_area(start, end) * - start - virtual start address of region - * - size - size in question + * - end - virtual end address of region */ - add x1, x0, x1 dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_poc) @@ -211,10 +210,10 @@ SYM_FUNC_END(__dma_clean_area) * - size - size in question */ SYM_FUNC_START_PI(__clean_dcache_area_pop) + add x1, x0, x1 alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif - add x1, x0, x1 dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret SYM_FUNC_END_PI(__clean_dcache_area_pop) @@ -243,7 +242,6 @@ SYM_FUNC_START_PI(__dma_map_area) add x1, x0, x1 cmp w2, #DMA_FROM_DEVICE b.eq __dma_inv_area - sub x1, x1, x0 b __dma_clean_area SYM_FUNC_END_PI(__dma_map_area) From patchwork Thu May 20 12:44:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ADBFC433B4 for ; Thu, 20 May 2021 12:51:40 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F692600D1 for ; Thu, 20 May 2021 12:51:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F692600D1 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=I69JHA8CIy+9d0ecA+FiJiyh6Ag7q8UNQsUm/FQVSfY=; b=A4wno0rjYsRCUs 4k+Bh+jafWLIqgEl9rTcz2sKTpY3ZPX8JaaxMTvrPOyYeOXzE2eRaG+7/NDwz5CI1z6/sero1X5ps hYQT10YthNjTr8Oj0PYFIpGcsgDHK5q3iOQzKW9Dttl0ZZu5U97Gg5MbXyackF8U5QqxB3PB7mi1B cbzAjVFsnyBHMiY5SRUAkxBkj51CoIv0GynUFTJnptoccJkk2rC4be5Xvfqrrg4Qh4jL2632MhWAM cqSmnGnVUSzdwZpuxmDcDRYKLdsmq8TrZ+0b5miUaJphT0UNyGkwsZDo+hxXt7bcC5y9ov8YnNlxz 4+TBW2q6+h3yvg+d5G/Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji7h-000xqV-IJ; Thu, 20 May 2021 12:49:58 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2c-000u38-H3 for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:42 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=kTsEENXDmPZEUuN+noYdKA7b/Q6OpBcy7Qni/3TfSaE=; b=xL7w9o5PjMl8jZZACOr6IRSpx3 f7NSivruvL88TNgYTJQ7q0nHRG6avt/OsTQiYv14Kg/RqP1PF5DsEHfNkkIGHpVccil+ImmAEaMab h2blUZN4PtuqElfbD6hCCFflSRtPx3GEnN0FwW8o9Cnpf5RdqVKHi5d+sjgK/tQ5JbShtuxkHTUyr l0KXo/y9YUp3/SMCC02Najrgaofrxunm3ujEEwMG+/E5oA0pPQehz2SsZ1muSkKSfzg52AktDH71d IkxHMrfPKmna6NglSRqKDrnCe3VdYOi1QPFhBxF7n9OxblKRvzJRX8E8D0ILK5usSpXLGSRQN7xxh pP+pO/MA==; Received: from mail-wr1-x449.google.com ([2a00:1450:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2Z-00GKgi-V9 for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:41 +0000 Received: by mail-wr1-x449.google.com with SMTP id j33-20020adf91240000b029010e4009d2ffso8613495wrj.0 for ; Thu, 20 May 2021 05:44:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=kTsEENXDmPZEUuN+noYdKA7b/Q6OpBcy7Qni/3TfSaE=; b=f/8uTZIVRRkyOOBfvNRHVuffPY+EeZ/ZibLU3yJHqqxUj5GX3oVt2661UM3IzvPw24 h/Gk2Vk2Ny7XEX7UQ0n84xnt6mJ7E1Woy9zWjNrxkb0U/1V6IGlW28GCLjSNJSd5prfo dB+pURw8v9mM0DZjmW3bbSk/JPlhZkbZTKpZWRuwOl9aEvTy1b+2xvAzdcCbmwGSI/8i uu/83hFbMlvcGC+74MCpnrEgIHuYzPh8YkeNYBj8Z8jvcPJxOcsDGBUP59YgBq8voBwK /YMmormWZPgkdWIcwLIu4PdPcd/qEf6jy5ypRTe4etcunPeHDdQNN2CKFUTp2bcdaO9n 0mKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=kTsEENXDmPZEUuN+noYdKA7b/Q6OpBcy7Qni/3TfSaE=; b=t69IIaw328lPNb+yqaGRkKZnp5OjM4LwvL/0ElTnfAkTulmB6C12gd1HgUP7TH6hxE KJeDg3Tv7COJ/A3s8mTrMvDouQgaVb7zF6ggTieZmUS2L/u6dyq+YOWXcqbsdOfojsiE i5ldqY6zvr/qyUN92jmVkEyWFYXgm0JYQlcTMfYoYvod39oynBjo/UN1oU95E4VPfQXD u9MsBZ4LFN7B1ehhe2BGzkgPw/COvhm2ULykyoj3RcnTW8cwz0rhhpxhLU4cwtlqnypQ VwD96pQcPGTvHeQ35O5yzaXEPMO3aDb4yBcAlBJhy4WX64lcREC5k8VjKl9rpVqdALkx dJpw== X-Gm-Message-State: AOAM5309OPVsVSOcjms3vQ5LBYuyuXFERNWA/EU/ZGqQvisFoNGuxBXr DmnMfHQy6un36Z3ni0OcpIW1taxP7TU+ICMhA6K3b+7CfMosyMa1eKR8INuHvQGwuHNPg4dKkRu hR37tAn32GQkXjaIPQnPXlY2/La4Ma+U7jQ1HapCLCaAHMlc7ZD1OLkpcQvUIShxZLXuK51AHrk UE9A== X-Google-Smtp-Source: ABdhPJyuB5dIpvfZABUyHh4xUXycaJwowaYU+7tPLrX23BRooeuvKYYmldI7zCR4/UcMsYjU6OIWhd7MLQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a5d:4b50:: with SMTP id w16mr3988948wrs.276.1621514678023; Thu, 20 May 2021 05:44:38 -0700 (PDT) Date: Thu, 20 May 2021 13:44:02 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-15-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 14/18] arm64: __clean_dcache_area_pop to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054440_022458_AF292BDC X-CRM114-Status: GOOD ( 14.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/lib/uaccess_flushcache.c | 4 ++-- arch/arm64/mm/cache.S | 9 ++++----- arch/arm64/mm/flush.c | 2 +- 4 files changed, 8 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 3255878d6f30..fa5641868d65 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -61,7 +61,7 @@ extern void invalidate_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pop(void *addr, size_t len); +extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); extern void sync_icache_aliases(void *kaddr, unsigned long len); diff --git a/arch/arm64/lib/uaccess_flushcache.c b/arch/arm64/lib/uaccess_flushcache.c index c83bb5a4aad2..62ea989effe8 100644 --- a/arch/arm64/lib/uaccess_flushcache.c +++ b/arch/arm64/lib/uaccess_flushcache.c @@ -15,7 +15,7 @@ void memcpy_flushcache(void *dst, const void *src, size_t cnt) * barrier to order the cache maintenance against the memcpy. */ memcpy(dst, src, cnt); - __clean_dcache_area_pop(dst, cnt); + __clean_dcache_area_pop((unsigned long)dst, (unsigned long)dst + cnt); } EXPORT_SYMBOL_GPL(memcpy_flushcache); @@ -33,6 +33,6 @@ unsigned long __copy_user_flushcache(void *to, const void __user *from, rc = raw_copy_from_user(to, from, n); /* See above */ - __clean_dcache_area_pop(to, n - rc); + __clean_dcache_area_pop((unsigned long)to, (unsigned long)to + n - rc); return rc; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 9a9c44bb26d2..b72fbae4b8e9 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -201,16 +201,15 @@ SYM_FUNC_END_PI(__clean_dcache_area_poc) SYM_FUNC_END(__dma_clean_area) /* - * __clean_dcache_area_pop(kaddr, size) + * __clean_dcache_area_pop(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoP. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START_PI(__clean_dcache_area_pop) - add x1, x0, x1 alternative_if_not ARM64_HAS_DCPOP b __clean_dcache_area_poc alternative_else_nop_endif diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 4e3505c2bea6..5aba7fe42d4b 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -82,7 +82,7 @@ void arch_wb_cache_pmem(void *addr, size_t size) { /* Ensure order against any prior non-cacheable writes */ dmb(osh); - __clean_dcache_area_pop(addr, size); + __clean_dcache_area_pop((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); From patchwork Thu May 20 12:44:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2CACC433ED for ; Thu, 20 May 2021 12:52:06 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 55E12611AD for ; Thu, 20 May 2021 12:52:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 55E12611AD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=hQ1RKaZajx1gxs/ArStsfRP/haOPd5WBn6gAR8HCmmg=; b=OXrale+w1osJu8 AFtBSNHoZGLDef3AMKXxt9GfG2XP9LcTPX1Db8K5bpZNU5BVlPuNHjR2K25DmVNPh5+uCDMPGS2Or PtvtsHQ4ejTIMEEVHiloIVl4zcM4Xeg/jiH6gKsZAqywWWPjZJJjqkeYhJw+DWUOXQt4XRuiLAAZD mvE50HNIHVNL371sHJPsGeiND0NEt12enrSgm4xN4xm3oyazYo0lmFAH6IKe343rdhM7qAkN+/NKr EGgu5+c1tcHcqBIpb98GMi+lN/ApJ47kGyN4cC1RbUxkbaJx81PRSZcXEx0em0Bmx2AnUEQrauvRZ x4t6xPh5rMWTVr+kuQNg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji85-000yDS-Rc; Thu, 20 May 2021 12:50:22 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2e-000u3L-O0 for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=YVujAnizNIUy1/K24Db/sNFlmhnU5DJYyr7mqgK3N8A=; b=VqNBbfEDMN6xcTpgey4Bk+yimm cNHQtKsHA/v2Y6S9oL3QKWp5cM7Fb8E8HT3l9AKEHyN7C9fhO9Fq9peISJ9qAlyySmxzvo1YIoQN/ BMWubJUeJQoJYN0bMkO96xVsxnx7w8CkNd3XpudtxAIT9twJPj1ZqPk4wYwJ1cd4ylhmMDU29Po7m j8XoLSW59qfwaL3AYtvZ6ks8CkxrWbPUDWr7eN/J6rKQLUFUvdVhEPtkiPbeKiLEONGK7R8fqpIO4 ySkxU1+FGrvt/7h9NNK9wdqHlBQV3sumfxxIzKSMmw3mxquz+MvNvhKa5CRP+8oUCNJbH72w9Cf+w T1Pg9QqA==; Received: from mail-wm1-x34a.google.com ([2a00:1450:4864:20::34a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2c-00GKgz-8F for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:43 +0000 Received: by mail-wm1-x34a.google.com with SMTP id k124-20020a1ca1820000b0290175dd5429afso2289634wme.7 for ; Thu, 20 May 2021 05:44:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=YVujAnizNIUy1/K24Db/sNFlmhnU5DJYyr7mqgK3N8A=; b=hb1sJKeatAES56YCQyFjs7vKcr5uYAOPVrlA2O/7CwDuABYwnJJLdMWugaJ9Kg3Rs/ 6yPI3CyvuC3uLKig4dSep/ek/CqxJ5d7JCncfV/EgWSFQB4HiQ38S1N6ihaPCKMP5nhE vPLwotyguZ/qB2E9BC+7VkRppN0LORemNUhvpCUML8bfMSM1dbseXzb/0RCJBiU+ckK4 O8eBS6YgNHabqk/1BbrBIUqsGym1M6ydK4O/HeSWKev+VuB+BEAqMznEAoAa6CX1VZLp 1oIeQ0PlhIsdD9C/oWLJl1HXMXVfn15yh2BHVY/l674cSqdMbASnez/zp076csIO87Oe QOBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=YVujAnizNIUy1/K24Db/sNFlmhnU5DJYyr7mqgK3N8A=; b=nj7LtjK9TGpCgOUA+FfNVA955oVO3QHHoxp7TSF735YdbCjQsl9bInQLpN9WFPedeY 8wTnCm7LdrhwAPPZezmmFiMI+PkuiO0RfFSQajq18FeOQksKwmCzhhx+KROsemwqyQLN gzhNOk+vqmIVw9iwaCsN+yLeY0j0TUfHxoDYf+Kp1RmUhqu6KEqKUYp2rYnh1SnS92oB veXEep+SlLt/so3sK4cBAA9bKtewLLMbRhKPraZbAigdjVACIMrwZRihe9XO+eVlYhyD RracBMS9tb4Qd/ucGgAG8DJFkJrB39YLBp85X8hWaQDnrLkriU+ml5MJbMXipLVUA1kZ Qo7w== X-Gm-Message-State: AOAM532sb8fAGglqAg+D4KdxdnHlXGWH0BFtd89wa+irU33fBDBIitfW Nc9FFVpzzPGTL0oRX8qL+FWxv2eAEKEugBJtGKe0JrMkKjvIuMriB7YXrnTnayrC6vd/PcXdmke Sq5NPLL5oggAbrGDGFm0b78KL19AUcdgxVl0KTyRBQfmO3u+UT6ijMpGUw5XpAtjXm+QGSMJUmc fm6A== X-Google-Smtp-Source: ABdhPJwciwPAJlGwaR/vECPIy33Adr4pEEVk23JDh8x3urVE8IYYyjQ1aFQmeJFqAjyHSkUuyAg1xcV76A== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a1c:7d17:: with SMTP id y23mr3837475wmc.167.1621514679981; Thu, 20 May 2021 05:44:39 -0700 (PDT) Date: Thu, 20 May 2021 13:44:03 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-16-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 15/18] arm64: __clean_dcache_area_pou to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054442_312948_7067568C X-CRM114-Status: GOOD ( 13.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/mm/cache.S | 9 ++++----- arch/arm64/mm/flush.c | 2 +- 3 files changed, 6 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index fa5641868d65..f86723047315 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -62,7 +62,7 @@ extern void __flush_dcache_area(unsigned long start, unsigned long end); extern void __inval_dcache_area(unsigned long start, unsigned long end); extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pou(void *addr, size_t len); +extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); extern long __flush_cache_user_range(unsigned long start, unsigned long end); extern void sync_icache_aliases(void *kaddr, unsigned long len); diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index b72fbae4b8e9..b70a6699c02b 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -120,20 +120,19 @@ SYM_FUNC_START_PI(__flush_dcache_area) SYM_FUNC_END_PI(__flush_dcache_area) /* - * __clean_dcache_area_pou(kaddr, size) + * __clean_dcache_area_pou(start, end) * - * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoU. * - * - kaddr - kernel address - * - size - size in question + * - start - virtual start address of region + * - end - virtual end address of region */ SYM_FUNC_START(__clean_dcache_area_pou) alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif - add x1, x0, x1 dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret SYM_FUNC_END(__clean_dcache_area_pou) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 5aba7fe42d4b..a69d745fb1dc 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -19,7 +19,7 @@ void sync_icache_aliases(void *kaddr, unsigned long len) unsigned long addr = (unsigned long)kaddr; if (icache_is_aliasing()) { - __clean_dcache_area_pou(kaddr, len); + __clean_dcache_area_pou(kaddr, kaddr + len); __flush_icache_all(); } else { /* From patchwork Thu May 20 12:44:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A832EC43461 for ; Thu, 20 May 2021 12:52:44 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 347E361184 for ; Thu, 20 May 2021 12:52:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 347E361184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2ZUVNRwgLappwXp0OLjrofB8nVRqvj0wEXIJBE86/O4=; b=optxXEeThdS77/ fenaKrmaTpcYOB0LI/uC/C9nN1UCYMf2IiRLIJw3eIkXRk5Q9NbHmQM6cKJCLYPv8/swkCHvXgtAn BUY5M6R2sureGumq035x1akRLGHYZUc/r05Kw1NMcPL90a7IkqbRnH+IlgKOlvadmX777Man+kmxU DURzCWS3F+G1Uv1IrnKO0c/YnuL6zR/amm1mFguYLhCLKYMVq+pZ1dtvTErP44Aa9bjUG1Ft6G7Uj Y7vwDzrOloeMeO0HTsAZ4oPzvhHRQDZA6u1qRSzZ9pBIiW3M8ionjNpljjY6nvcwL3q2nH192MDuM AiWTFfy5aP7cxc1gIgOQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji8X-000yYY-C0; Thu, 20 May 2021 12:50:49 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2h-000u4M-4T for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=tEummeh6I6frwEp3TLdr5cPM5KsDvZ8aCtlKbBiCaX4=; b=026LF48c1MT9GJ3UJ1ysxwBdeP knswkcdlY8IT4ay8aj/BmlXFVbqMgcchkEd4pyV3Vud6wqkRvXqBaNfbvuZXx+7pbe23Pk47sJIf0 9RwxQ0K9orfLmfBMVrHWUraomMi5u8lcAslV6su0bn8iw+iT2WnGSkuN5VL9VCufyEJtxCXyPw8BM R/G3r2IVwW/xflyaCZT/oDkziXslS2FozX5hPX+mEy5umFavvKX91Kzx6lq1avXtru3oD0N46eySO LRvMTrHi6PaqXyfL+Yr1xJEM/C3Nco2fKoa5RXWlQEIQh/CJlSMO6yPDvmob5CO61MvXhnkTGpYq+ jUuzf8GA==; Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2e-00GKhS-Gy for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:45 +0000 Received: by mail-qk1-x74a.google.com with SMTP id l6-20020a3770060000b02902fa5329f2b4so12352889qkc.18 for ; Thu, 20 May 2021 05:44:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tEummeh6I6frwEp3TLdr5cPM5KsDvZ8aCtlKbBiCaX4=; b=JSxiTOhm+FGVW59PyZzNTNHuOmNmO6TfsDJRdhMP+pXJ/fwaRpVGPwvsszFaDooLar ca8UPnkaWr3wUljEFoxzrUTK4nLGhHRBnoqEsN05Mk0aS+7EJNzpXTLlS5T9hxQxdkR2 b2qQPKxciRctd2MagBYqsdHIit2jYS2xKnrUlK6fDdr8VJDUqwgGu4eEmb4n+Jl7Nmjh l6IlTnAleStnUPuMZkl+fxswyk9hysUPOXbTmX2CoCGYm2ywy30xL7CHuNcyw96CDE4y OMJJ6EE6hnNBWv6M01MB2H0i+v2IjhNxapOnKgLt3+WEJ0YsFYX+kj0533sDaF68njsl ZOig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tEummeh6I6frwEp3TLdr5cPM5KsDvZ8aCtlKbBiCaX4=; b=YM0BKvYUqAWjbhyLn6ejcUZ4mRrG//DiITE/18GFlz4zISGOwD41nTT8TM1alm4Sy2 IoiXkvSCCW2mPArvyU3/LIPRxcz71XGFP4C2j9Yh9P0EHk+TvplMXutvfbXEZmqLYy2V 1hewFKVLcMYmrebA1FG0dlxKPwkHTC+Wz4ZX9pPcBiE8DCd0GLCmUn5VNWS2Q0iy1Ou4 ub3fULlH58tXEkQGQrj0zoqwxtE/xQk15nOQEuG743y2L1ja/ZCeJX7CR+sDS5YvjG9W CaroaZWl8DvSzZOiyVVjgjgrfuX7yHTCGfm0DghPytguu1O7XmpglZgZWIfcVzubyRXk KGvA== X-Gm-Message-State: AOAM531y6kMpyKW4nt9LkSPjD/xzFK2L8v/1EYYHig5nPzcUKyv2jY5s PpRdDOF05JSxY2x04Q5CaMh6C6Tajd1YHMO55KX3DjYOqynqF67xpVFpwU+pZz2lt5m1KBQUGYm UHFOtZEfNwxtmdvGya/PPJH8JM3zO6AwYJtXsJJLtHFdNwo5zxhLyGohPnqmc4I3ol//NnY3BuC KoOw== X-Google-Smtp-Source: ABdhPJxN4Yvg2o2Hd92+w9PZzanYY95ScHSk3i0BtPwbWVen/MDmPf4vN5nqQTxn0CCXcdHJOlgN85UE5g== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a0c:edcf:: with SMTP id i15mr5455879qvr.10.1621514682038; Thu, 20 May 2021 05:44:42 -0700 (PDT) Date: Thu, 20 May 2021 13:44:04 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-17-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 16/18] arm64: sync_icache_aliases to take end parameter instead of size From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054444_584572_ECBC7455 X-CRM114-Status: GOOD ( 14.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To be consistent with other functions with similar names and functionality in cacheflush.h, cache.S, and cachetlb.rst, change to specify the range in terms of start and end, as opposed to start and size. No functional change intended. Reported-by: Will Deacon Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 2 +- arch/arm64/kernel/probes/uprobes.c | 2 +- arch/arm64/mm/flush.c | 21 +++++++++++---------- 3 files changed, 13 insertions(+), 12 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index f86723047315..70b389a8dea5 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -64,7 +64,7 @@ extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); extern long __flush_cache_user_range(unsigned long start, unsigned long end); -extern void sync_icache_aliases(void *kaddr, unsigned long len); +extern void sync_icache_aliases(unsigned long start, unsigned long end); static inline void flush_icache_range(unsigned long start, unsigned long end) { diff --git a/arch/arm64/kernel/probes/uprobes.c b/arch/arm64/kernel/probes/uprobes.c index 2c247634552b..9be668f3f034 100644 --- a/arch/arm64/kernel/probes/uprobes.c +++ b/arch/arm64/kernel/probes/uprobes.c @@ -21,7 +21,7 @@ void arch_uprobe_copy_ixol(struct page *page, unsigned long vaddr, memcpy(dst, src, len); /* flush caches (dcache/icache) */ - sync_icache_aliases(dst, len); + sync_icache_aliases((unsigned long)dst, (unsigned long)dst + len); kunmap_atomic(xol_page_kaddr); } diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index a69d745fb1dc..143f625e7727 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -14,28 +14,26 @@ #include #include -void sync_icache_aliases(void *kaddr, unsigned long len) +void sync_icache_aliases(unsigned long start, unsigned long end) { - unsigned long addr = (unsigned long)kaddr; - if (icache_is_aliasing()) { - __clean_dcache_area_pou(kaddr, kaddr + len); + __clean_dcache_area_pou(start, end); __flush_icache_all(); } else { /* * Don't issue kick_all_cpus_sync() after I-cache invalidation * for user mappings. */ - __flush_icache_range(addr, addr + len); + __flush_icache_range(start, end); } } static void flush_ptrace_access(struct vm_area_struct *vma, struct page *page, - unsigned long uaddr, void *kaddr, - unsigned long len) + unsigned long uaddr, unsigned long start, + unsigned long end) { if (vma->vm_flags & VM_EXEC) - sync_icache_aliases(kaddr, len); + sync_icache_aliases(start, end); } /* @@ -48,7 +46,8 @@ void copy_to_user_page(struct vm_area_struct *vma, struct page *page, unsigned long len) { memcpy(dst, src, len); - flush_ptrace_access(vma, page, uaddr, dst, len); + flush_ptrace_access(vma, page, uaddr, (unsigned long)dst, + (unsigned long)dst + len); } void __sync_icache_dcache(pte_t pte) @@ -56,7 +55,9 @@ void __sync_icache_dcache(pte_t pte) struct page *page = pte_page(pte); if (!test_and_set_bit(PG_dcache_clean, &page->flags)) - sync_icache_aliases(page_address(page), page_size(page)); + sync_icache_aliases((unsigned long)page_address(page), + (unsigned long)page_address(page) + + page_size(page)); } EXPORT_SYMBOL_GPL(__sync_icache_dcache); From patchwork Thu May 20 12:44:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F289C433B4 for ; Thu, 20 May 2021 12:52:58 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EBA476101D for ; Thu, 20 May 2021 12:52:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EBA476101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eCo1vQBlHR7xC19kdlgSCxfKhoTKIz/uL7rwwxUGl/8=; b=jOkmRb/aYCbdSp Y3OqAWSU3s4ypBqeSSSr4ZjjkJxzKXAb1EWuhEldXmLeaSguKF7A4y5IkJWL5BBfWx6ENInbVPkAo l51Zzpv60jXZ7+9ioVAkBeFv1rnIiohcopk/JdSD5l5w3VF+Jr8TFjxK4CWwCxzk6CZQxGqzQZZRQ WD44Yb+6tNOdwVQZVqfOq67e/YC/u2r+FNNc2p6JB3WU9qKwsFtJ4D3pDR5dcKmzTgOEKOTxBiPha 5/685NapWaXCsxJoO6vzmYxyMWRrk5Bc9OrWz1WL3mJs8SIHR26Awje2n3dm9NhTEILopTd7x6coD OSAWG+CbRWHBboslQtwQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji8y-000yqi-RO; Thu, 20 May 2021 12:51:17 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2i-000u4W-EV for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=JUJRBpAmSgW3iUmlRzcNOvrsxGJGypryDJgOqUy8erw=; b=XxHuthUHnb8xDEdRjUxth0Y46F vZjceUMxHHCB2ande67u64O6z0rrCh3vk1KGgcIsW42/L76K3uYeZ92hiAi4XqD9iROxAvPOxynpH k/i+4BfrHPzvGpPQj+JeD0uDy8Doz3NQzBi/SayYF2RhBV1GFix99tjuBkmpSqUFg2umMJ83hDYUJ IHbvo5IyocEPQx67at5pqCrgq/oJ7Mf/6ylLsw23smDqoBmwAFdWQ49P8rexnIyRpoKClrBRMjiX+ Pc95HR3Fv9oi7lsSwLma/wMOGD/IMxkX3A5fhSRQ2Ueux3hGdwoiRbA9/+iaCAsxkYFoNSAnUUARe lNUh3I/w==; Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2f-00GKhi-TV for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:47 +0000 Received: by mail-qt1-x849.google.com with SMTP id b17-20020ac854110000b02901f279c73d75so8301520qtq.2 for ; Thu, 20 May 2021 05:44:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=JUJRBpAmSgW3iUmlRzcNOvrsxGJGypryDJgOqUy8erw=; b=SwFc475/zUvR5CwVxipNoCMsL/mzIR2O0wmA85yOmH4sKMrdwJYI18PEV8phpTGWwN 85VAj5bYIlAwoFUJWfiJr1MDCV6kcXy3KywRF2JTJh1JaEzXJV5B8UODxuVJR9Tku8vx k+xkwijhyRal/ePXmWpAzeU2FbnL4l5OE80l4YT3XhPZK5VvDyfrnsmyc32ZADZAv0QB pbo4v3xTh8yfU9R5yTe4G2vKuSOJVVclNYOS5kRsCMJpQOufRQ4EJSk3pWwbYJ7OqKwv x5UA0xwxv2XjENKVgXgAH0W8gZ73Aw2WPBraGVlD6CdHkVO2J3i3rIbGRCGMpmNlh3M0 R2UA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=JUJRBpAmSgW3iUmlRzcNOvrsxGJGypryDJgOqUy8erw=; b=cICnVvY+ciSHAhtJCTahyXFwzOGUZIgGXIXC359EPKAu18jU5fLq3IN7U4BO+ugcQa MVNXlfsqxI8xQeCHauVraUHjJ34frBAG37sZPQCgSWycWzNd+fctpeYag0oyjcDkaVTr F9hPOXJNtPVLd1VBxhZ+1g1Cu1wWk2t4OAd+p+81ggxowjZUIZvpCGekYSSGtm6coxYn iU/Ia1OjPmCaWlcuQ784iFoo26mQfDUJtcPHUPoKwLw6zjLjk3C3rsj0uPHs7Shvok6j D8LtMJQzgM1rhR4Fmixbx0c609AvaVOtEf9K2EDqQDuedZcMEeFKUMc8tK78H42t5hJv wQCQ== X-Gm-Message-State: AOAM531H1QX6DEM/PrEko2DrNZ8IsdL39CsZLXW4qLFQHaMjxSS48vlk vjodSAMMR5C8JOdHmOgaLT64UIs+AvmwjLAZ9H5IMxI70FOEG1lSvbD30lwD3OaX93DYemTRBy3 gdgVP5l2zozgdK02xA5/wWHhzqkagvq9jIZUmen5pluzrp7OuPkSIMVbH5qKUpoBwWTkCXM3MSZ jWNA== X-Google-Smtp-Source: ABdhPJwljOoYVtFR+khH0MWktMIki/YtdBwegkDxjm60r/0nZAGgvkCZFWg1Ap4Re070w/9p44wlkqQphg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:ad4:5112:: with SMTP id g18mr5399708qvp.26.1621514684354; Thu, 20 May 2021 05:44:44 -0700 (PDT) Date: Thu, 20 May 2021 13:44:05 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-18-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 17/18] arm64: Fix cache maintenance function comments From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054445_977912_3945158F X-CRM114-Status: GOOD ( 11.73 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Fix and expand comments for the cache maintenance functions in cacheflush.h. Adds comments to functions that weren't described before. Explains what the functions do using Arm Architecture Reference Manual terminology. No functional change intended. Signed-off-by: Fuad Tabba Acked-by: Mark Rutland --- arch/arm64/include/asm/cacheflush.h | 43 +++++++++++++++++++---------- 1 file changed, 28 insertions(+), 15 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 70b389a8dea5..4b91d3530013 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -30,31 +30,44 @@ * the implementation assumes non-aliasing VIPT D-cache and (aliasing) * VIPT I-cache. * - * flush_icache_range(start, end) - * - * Ensure coherency between the I-cache and the D-cache in the - * region described by start, end. + * All functions below apply to the region described by [start, end) * - start - virtual start address * - end - virtual end address * - * invalidate_icache_range(start, end) + * __flush_icache_range(start, end) * - * Invalidate the I-cache in the region described by start, end. - * - start - virtual start address - * - end - virtual end address + * Ensure coherency between the I-cache and the D-cache region to + * the Point of Unification. * * __flush_cache_user_range(start, end) * - * Ensure coherency between the I-cache and the D-cache in the - * region described by start, end. - * - start - virtual start address - * - end - virtual end address + * Ensure coherency between the I-cache and the D-cache region to + * the Point of Unification. + * Use only if the region might access user memory. + * + * invalidate_icache_range(start, end) + * + * Invalidate I-cache region to the Point of Unification. * * __flush_dcache_area(start, end) * - * Ensure that the data held in page is written back. - * - start - virtual start address - * - end - virtual end address + * Clean and invalidate D-cache region to the Point of Coherence. + * + * __inval_dcache_area(start, end) + * + * Invalidate D-cache region to the Point of Coherence. + * + * __clean_dcache_area_poc(start, end) + * + * Clean D-cache region to the Point of Coherence. + * + * __clean_dcache_area_pop(start, end) + * + * Clean D-cache region to the Point of Persistence. + * + * __clean_dcache_area_pou(start, end) + * + * Clean D-cache region to the Point of Unification. */ extern void __flush_icache_range(unsigned long start, unsigned long end); extern void invalidate_icache_range(unsigned long start, unsigned long end); From patchwork Thu May 20 12:44:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E475C433B4 for ; Thu, 20 May 2021 12:54:41 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C07FA6101D for ; Thu, 20 May 2021 12:54:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C07FA6101D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eWqnU3CMhIpvP6QTFY+AfWO//BcrHczV2HoxD0rE1Vg=; b=jbvO5nD4BawtOw S15u93QgGkWUb/tNYSdZdz6m5MIvu9cp3rTUCHsm2FxcIxCFidb06QxrhkPX9LNtRz3k2Bf9yM4Wz P+84T8e2IePgFSiTA2Ljr5xTKlv6iAsgwZ8bxx2+/YRust4OFaCeTug6rXjmvhJibSlEce5YS9jxH cxGnb+6b7g6za5Ppq6LcMxmV2U0GNXGV/TtO4XzYsB4v39v3DMj8W+Ygq+xGhRoD73MMVsEfvGfYl DLM0lNe1ahR3xR4+NNerfdz+kNSAvzOX//FrQcWcZE/mbs2Gq9+iPrFO4cAKP5CbOb9FqvqPrdCta JDgglXxu9ODw59464fVw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ljiAQ-000zjn-Lm; Thu, 20 May 2021 12:52:47 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2m-000u6C-Rw for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Wzx3cv7OIkpN5sWucp+PFlfLE2hk86a+/dyYzLf9Frw=; b=u/8nZohjYj9XIf9H18DJv9tiCG hHsIuhF17vUoLpP333BfOskKom4UnPbFN59LaOlFBAsLcvk15TcMjJp8UIugiOAeQQ05ksjPXhzdM BuCsY3390M3r3K8SNPNGHvle1NmbfaowsKHq71xD/mHBqHuj7099DBO5w4ADimbpWPhNNi5M1cU3r wVnb6xLGra/Kk5mS6cx48I/28HkrsmxVZtGMvbbOLqHmf+aZDs5hOBDEK2YGLi0w9C1KPukx0O0I+ SPykrSzYgMb/HF7E/69vW/qyDz1OF60ztr3SFAZbXcfhOE8zvESuzoAWi7UT++VstU/XnFqv0EqdN WFoFWkuQ==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2i-00GKhy-Ag for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:51 +0000 Received: by mail-wr1-x44a.google.com with SMTP id f19-20020adfb6130000b02901121afc9a31so1899367wre.10 for ; Thu, 20 May 2021 05:44:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Wzx3cv7OIkpN5sWucp+PFlfLE2hk86a+/dyYzLf9Frw=; b=rItdQVuUoVUuaQm+cqWLO+YArGsRBswpXw31n3RJNJ/5MLzzlAvc9iBVAF7qHeRQpb LuNy/y6ltzIgrNJ2bQ7E6X7VmmjpWWJZSZvJOqRAi5xawh1m7eDiffQCOlkYhozmBKk/ x6loqOySVGkRNmjmOC3cdXLhsh6l4I/hkX4GGY5aV0ifNrpIPPZS+nimYsLhWsqgd3YO HBrPb/yaeCiYSOP73ZvJHS+4qbQVTnHLj2wXDwEQpVpPHkQrpLYwPOM6tLLWPPmpJBI3 N2EAYJE/PyRXHGlMr6z3i8XPiPpmbmhtD5DDspQauuK3qc2qYOwvDUeS6dsOKucoeQYp XiCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Wzx3cv7OIkpN5sWucp+PFlfLE2hk86a+/dyYzLf9Frw=; b=sSyILynKlwvbRwLnYBFPfXXeMDg5neu05T2cdaUy/9hy/HPaUsg+2HWdC/vRuyg3bl yho176rozI7EAtGCfvcNwmbPQZR2KNTaf6Q8Wvk1moqE/peuKezbAU0UiaLp7gWNYj0h hf+AXYKpNEuM13bZtSArOCB8Iee3QdvA2y+LeV2Q2D4P+xsf6NslHNf99lrIzTOs7mq3 XRHK8nKilZU/2V+G2tJaPeoY8O8VQkwYWeiqi15h72NUgM7OHriwSnwMRXtC6VrS05PX 7IKkvOz3PJju0UEIepH939Z25WtSJux6RRXB8eutrh60SkzoQDFuyhgb9ts0LETPdeUz zLMg== X-Gm-Message-State: AOAM533IXbZVF/8zt6irM1BGKrNc8zBGkMIpzdPqgzquyA7jh1G9lvIG xZ7/wcmSIyIqyCH6locBPW1IwX2caSGooolKbxNZaBecIld5BWgMYVaf1P19yK9PZgRqHnuQ9ev gMXND+mbTObbv+lAT58fmK/wtSr3HHE89NzN8Dlo4GNUQhT9rwgWIbSZLDBhoH88OgB/OEM9bE+ LIAw== X-Google-Smtp-Source: ABdhPJyIqSw8ieKoBn4iXAMDFg6mOcfVTWb+qa3otd9IE7MvwBF+Z3ELygikYC8OTbJHpSXvHCsOQMryAg== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:a7b:c346:: with SMTP id l6mr3422302wmj.109.1621514686440; Thu, 20 May 2021 05:44:46 -0700 (PDT) Date: Thu, 20 May 2021 13:44:06 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-19-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 18/18] arm64: Rename arm64-internal cache maintenance functions From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054448_417097_B09B1AAC X-CRM114-Status: GOOD ( 26.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Although naming across the codebase isn't that consistent, it tends to follow certain patterns. Moreover, the term "flush" isn't defined in the Arm Architecture reference manual, and might be interpreted to mean clean, invalidate, or both for a cache. Rename arm64-internal functions to make the naming internally consistent, as well as making it consistent with the Arm ARM, by specifying whether it applies to the instruction, data, or both caches, whether the operation is a clean, invalidate, or both. Also specify which point the operation applies to, i.e., to the point of unification (PoU), coherence (PoC), or persistence (PoP). This commit applies the following sed transformation to all files under arch/arm64: "s/\b__flush_cache_range\b/caches_clean_inval_pou_macro/g;"\ "s/\b__flush_icache_range\b/caches_clean_inval_pou/g;"\ "s/\binvalidate_icache_range\b/icache_inval_pou/g;"\ "s/\b__flush_dcache_area\b/dcache_clean_inval_poc/g;"\ "s/\b__inval_dcache_area\b/dcache_inval_poc/g;"\ "s/__clean_dcache_area_poc\b/dcache_clean_poc/g;"\ "s/\b__clean_dcache_area_pop\b/dcache_clean_pop/g;"\ "s/\b__clean_dcache_area_pou\b/dcache_clean_pou/g;"\ "s/\b__flush_cache_user_range\b/caches_clean_inval_user_pou/g;"\ "s/\b__flush_icache_all\b/icache_inval_all_pou/g;" Note that __clean_dcache_area_poc is deliberately missing a word boundary check at the beginning in order to match the efistub symbols in image-vars.h. Also note that, despite its name, __flush_icache_range operates on both instruction and data caches. The name change here reflects that. No functional change intended. Signed-off-by: Fuad Tabba Acked-by: Mark Rutland Tested-by: Mark Rutland --- arch/arm64/include/asm/arch_gicv3.h | 2 +- arch/arm64/include/asm/cacheflush.h | 36 +++++++++--------- arch/arm64/include/asm/efi.h | 2 +- arch/arm64/include/asm/kvm_mmu.h | 6 +-- arch/arm64/kernel/alternative.c | 2 +- arch/arm64/kernel/efi-entry.S | 4 +- arch/arm64/kernel/head.S | 8 ++-- arch/arm64/kernel/hibernate-asm.S | 4 +- arch/arm64/kernel/hibernate.c | 12 +++--- arch/arm64/kernel/idreg-override.c | 2 +- arch/arm64/kernel/image-vars.h | 2 +- arch/arm64/kernel/insn.c | 2 +- arch/arm64/kernel/kaslr.c | 6 +-- arch/arm64/kernel/machine_kexec.c | 10 ++--- arch/arm64/kernel/smp.c | 4 +- arch/arm64/kernel/smp_spin_table.c | 4 +- arch/arm64/kernel/sys_compat.c | 2 +- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/cache.S | 4 +- arch/arm64/kvm/hyp/nvhe/setup.c | 2 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 2 +- arch/arm64/kvm/hyp/pgtable.c | 4 +- arch/arm64/lib/uaccess_flushcache.c | 4 +- arch/arm64/mm/cache.S | 58 ++++++++++++++--------------- arch/arm64/mm/flush.c | 12 +++--- 25 files changed, 98 insertions(+), 98 deletions(-) diff --git a/arch/arm64/include/asm/arch_gicv3.h b/arch/arm64/include/asm/arch_gicv3.h index ed1cc9d8e6df..4ad22c3135db 100644 --- a/arch/arm64/include/asm/arch_gicv3.h +++ b/arch/arm64/include/asm/arch_gicv3.h @@ -125,7 +125,7 @@ static inline u32 gic_read_rpr(void) #define gic_write_lpir(v, c) writeq_relaxed(v, c) #define gic_flush_dcache_to_poc(a,l) \ - __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) + dcache_clean_inval_poc((unsigned long)(a), (unsigned long)(a)+(l)) #define gits_read_baser(c) readq_relaxed(c) #define gits_write_baser(v, c) writeq_relaxed(v, c) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 4b91d3530013..885bda37b805 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -34,54 +34,54 @@ * - start - virtual start address * - end - virtual end address * - * __flush_icache_range(start, end) + * caches_clean_inval_pou(start, end) * * Ensure coherency between the I-cache and the D-cache region to * the Point of Unification. * - * __flush_cache_user_range(start, end) + * caches_clean_inval_user_pou(start, end) * * Ensure coherency between the I-cache and the D-cache region to * the Point of Unification. * Use only if the region might access user memory. * - * invalidate_icache_range(start, end) + * icache_inval_pou(start, end) * * Invalidate I-cache region to the Point of Unification. * - * __flush_dcache_area(start, end) + * dcache_clean_inval_poc(start, end) * * Clean and invalidate D-cache region to the Point of Coherence. * - * __inval_dcache_area(start, end) + * dcache_inval_poc(start, end) * * Invalidate D-cache region to the Point of Coherence. * - * __clean_dcache_area_poc(start, end) + * dcache_clean_poc(start, end) * * Clean D-cache region to the Point of Coherence. * - * __clean_dcache_area_pop(start, end) + * dcache_clean_pop(start, end) * * Clean D-cache region to the Point of Persistence. * - * __clean_dcache_area_pou(start, end) + * dcache_clean_pou(start, end) * * Clean D-cache region to the Point of Unification. */ -extern void __flush_icache_range(unsigned long start, unsigned long end); -extern void invalidate_icache_range(unsigned long start, unsigned long end); -extern void __flush_dcache_area(unsigned long start, unsigned long end); -extern void __inval_dcache_area(unsigned long start, unsigned long end); -extern void __clean_dcache_area_poc(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pop(unsigned long start, unsigned long end); -extern void __clean_dcache_area_pou(unsigned long start, unsigned long end); -extern long __flush_cache_user_range(unsigned long start, unsigned long end); +extern void caches_clean_inval_pou(unsigned long start, unsigned long end); +extern void icache_inval_pou(unsigned long start, unsigned long end); +extern void dcache_clean_inval_poc(unsigned long start, unsigned long end); +extern void dcache_inval_poc(unsigned long start, unsigned long end); +extern void dcache_clean_poc(unsigned long start, unsigned long end); +extern void dcache_clean_pop(unsigned long start, unsigned long end); +extern void dcache_clean_pou(unsigned long start, unsigned long end); +extern long caches_clean_inval_user_pou(unsigned long start, unsigned long end); extern void sync_icache_aliases(unsigned long start, unsigned long end); static inline void flush_icache_range(unsigned long start, unsigned long end) { - __flush_icache_range(start, end); + caches_clean_inval_pou(start, end); /* * IPI all online CPUs so that they undergo a context synchronization @@ -135,7 +135,7 @@ extern void copy_to_user_page(struct vm_area_struct *, struct page *, #define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1 extern void flush_dcache_page(struct page *); -static __always_inline void __flush_icache_all(void) +static __always_inline void icache_inval_all_pou(void) { if (cpus_have_const_cap(ARM64_HAS_CACHE_DIC)) return; diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 0ae2397076fd..1bed37eb013a 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -137,7 +137,7 @@ void efi_virtmap_unload(void); static inline void efi_capsule_flush_cache_range(void *addr, int size) { - __flush_dcache_area((unsigned long)addr, (unsigned long)addr + size); + dcache_clean_inval_poc((unsigned long)addr, (unsigned long)addr + size); } #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 33293d5855af..f4cbfa9025a8 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -181,7 +181,7 @@ static inline void *__kvm_vector_slot2addr(void *base, struct kvm; #define kvm_flush_dcache_to_poc(a,l) \ - __flush_dcache_area((unsigned long)(a), (unsigned long)(a)+(l)) + dcache_clean_inval_poc((unsigned long)(a), (unsigned long)(a)+(l)) static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) { @@ -209,12 +209,12 @@ static inline void __invalidate_icache_guest_page(kvm_pfn_t pfn, { if (icache_is_aliasing()) { /* any kind of VIPT cache */ - __flush_icache_all(); + icache_inval_all_pou(); } else if (is_kernel_in_hyp_mode() || !icache_is_vpipt()) { /* PIPT or VPIPT at EL2 (see comment in __kvm_tlb_flush_vmid_ipa) */ void *va = page_address(pfn_to_page(pfn)); - invalidate_icache_range((unsigned long)va, + icache_inval_pou((unsigned long)va, (unsigned long)va + size); } } diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index c906d20c7b52..3fb79b76e9d9 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -181,7 +181,7 @@ static void __nocfi __apply_alternatives(struct alt_region *region, bool is_modu */ if (!is_module) { dsb(ish); - __flush_icache_all(); + icache_inval_all_pou(); isb(); /* Ignore ARM64_CB bit from feature mask */ diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 72e6a580290a..6668bad21f86 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -29,7 +29,7 @@ SYM_CODE_START(efi_enter_kernel) */ ldr w1, =kernel_size add x1, x0, x1 - bl __clean_dcache_area_poc + bl dcache_clean_poc ic ialluis /* @@ -38,7 +38,7 @@ SYM_CODE_START(efi_enter_kernel) */ adr x0, 0f adr x1, 3f - bl __clean_dcache_area_poc + bl dcache_clean_poc 0: /* Turn off Dcache and MMU */ mrs x0, CurrentEL diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 8df0ac8d9123..6928cb67d3a0 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -118,7 +118,7 @@ SYM_CODE_START_LOCAL(preserve_boot_args) // MMU off add x1, x0, #0x20 // 4 x 8 bytes - b __inval_dcache_area // tail call + b dcache_inval_poc // tail call SYM_CODE_END(preserve_boot_args) /* @@ -268,7 +268,7 @@ SYM_FUNC_START_LOCAL(__create_page_tables) */ adrp x0, init_pg_dir adrp x1, init_pg_end - bl __inval_dcache_area + bl dcache_inval_poc /* * Clear the init page tables. @@ -381,11 +381,11 @@ SYM_FUNC_START_LOCAL(__create_page_tables) adrp x0, idmap_pg_dir adrp x1, idmap_pg_end - bl __inval_dcache_area + bl dcache_inval_poc adrp x0, init_pg_dir adrp x1, init_pg_end - bl __inval_dcache_area + bl dcache_inval_poc ret x28 SYM_FUNC_END(__create_page_tables) diff --git a/arch/arm64/kernel/hibernate-asm.S b/arch/arm64/kernel/hibernate-asm.S index ef2ab7caf815..81c0186a5e32 100644 --- a/arch/arm64/kernel/hibernate-asm.S +++ b/arch/arm64/kernel/hibernate-asm.S @@ -45,7 +45,7 @@ * Because this code has to be copied to a 'safe' page, it can't call out to * other functions by PC-relative address. Also remember that it may be * mid-way through over-writing other functions. For this reason it contains - * code from __flush_icache_range() and uses the copy_page() macro. + * code from caches_clean_inval_pou() and uses the copy_page() macro. * * This 'safe' page is mapped via ttbr0, and executed from there. This function * switches to a copy of the linear map in ttbr1, performs the restore, then @@ -87,7 +87,7 @@ SYM_CODE_START(swsusp_arch_suspend_exit) copy_page x0, x1, x2, x3, x4, x5, x6, x7, x8, x9 add x1, x10, #PAGE_SIZE - /* Clean the copied page to PoU - based on __flush_icache_range() */ + /* Clean the copied page to PoU - based on caches_clean_inval_pou() */ raw_dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x10, x3 diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c index b40ddce71507..46a0b4d6e251 100644 --- a/arch/arm64/kernel/hibernate.c +++ b/arch/arm64/kernel/hibernate.c @@ -210,7 +210,7 @@ static int create_safe_exec_page(void *src_start, size_t length, return -ENOMEM; memcpy(page, src_start, length); - __flush_icache_range((unsigned long)page, (unsigned long)page + length); + caches_clean_inval_pou((unsigned long)page, (unsigned long)page + length); rc = trans_pgd_idmap_page(&trans_info, &trans_ttbr0, &t0sz, page); if (rc) return rc; @@ -381,17 +381,17 @@ int swsusp_arch_suspend(void) ret = swsusp_save(); } else { /* Clean kernel core startup/idle code to PoC*/ - __flush_dcache_area((unsigned long)__mmuoff_data_start, + dcache_clean_inval_poc((unsigned long)__mmuoff_data_start, (unsigned long)__mmuoff_data_end); - __flush_dcache_area((unsigned long)__idmap_text_start, + dcache_clean_inval_poc((unsigned long)__idmap_text_start, (unsigned long)__idmap_text_end); /* Clean kvm setup code to PoC? */ if (el2_reset_needed()) { - __flush_dcache_area( + dcache_clean_inval_poc( (unsigned long)__hyp_idmap_text_start, (unsigned long)__hyp_idmap_text_end); - __flush_dcache_area((unsigned long)__hyp_text_start, + dcache_clean_inval_poc((unsigned long)__hyp_text_start, (unsigned long)__hyp_text_end); } @@ -477,7 +477,7 @@ int swsusp_arch_resume(void) * The hibernate exit text contains a set of el2 vectors, that will * be executed at el2 with the mmu off in order to reload hyp-stub. */ - __flush_dcache_area((unsigned long)hibernate_exit, + dcache_clean_inval_poc((unsigned long)hibernate_exit, (unsigned long)hibernate_exit + exit_size); /* diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c index 3dd515baf526..53a381a7f65d 100644 --- a/arch/arm64/kernel/idreg-override.c +++ b/arch/arm64/kernel/idreg-override.c @@ -237,7 +237,7 @@ asmlinkage void __init init_feature_override(void) for (i = 0; i < ARRAY_SIZE(regs); i++) { if (regs[i]->override) - __flush_dcache_area((unsigned long)regs[i]->override, + dcache_clean_inval_poc((unsigned long)regs[i]->override, (unsigned long)regs[i]->override + sizeof(*regs[i]->override)); } diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h index bcf3c2755370..c96a9a0043bf 100644 --- a/arch/arm64/kernel/image-vars.h +++ b/arch/arm64/kernel/image-vars.h @@ -35,7 +35,7 @@ __efistub_strnlen = __pi_strnlen; __efistub_strcmp = __pi_strcmp; __efistub_strncmp = __pi_strncmp; __efistub_strrchr = __pi_strrchr; -__efistub___clean_dcache_area_poc = __pi___clean_dcache_area_poc; +__efistub_dcache_clean_poc = __pi_dcache_clean_poc; #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) __efistub___memcpy = __pi_memcpy; diff --git a/arch/arm64/kernel/insn.c b/arch/arm64/kernel/insn.c index 6c0de2f60ea9..51cb8dc98d00 100644 --- a/arch/arm64/kernel/insn.c +++ b/arch/arm64/kernel/insn.c @@ -198,7 +198,7 @@ int __kprobes aarch64_insn_patch_text_nosync(void *addr, u32 insn) ret = aarch64_insn_write(tp, insn); if (ret == 0) - __flush_icache_range((uintptr_t)tp, + caches_clean_inval_pou((uintptr_t)tp, (uintptr_t)tp + AARCH64_INSN_SIZE); return ret; diff --git a/arch/arm64/kernel/kaslr.c b/arch/arm64/kernel/kaslr.c index 49cccd03cb37..cfa2cfde3019 100644 --- a/arch/arm64/kernel/kaslr.c +++ b/arch/arm64/kernel/kaslr.c @@ -72,7 +72,7 @@ u64 __init kaslr_early_init(void) * we end up running with module randomization disabled. */ module_alloc_base = (u64)_etext - MODULES_VSIZE; - __flush_dcache_area((unsigned long)&module_alloc_base, + dcache_clean_inval_poc((unsigned long)&module_alloc_base, (unsigned long)&module_alloc_base + sizeof(module_alloc_base)); @@ -172,10 +172,10 @@ u64 __init kaslr_early_init(void) module_alloc_base += (module_range * (seed & ((1 << 21) - 1))) >> 21; module_alloc_base &= PAGE_MASK; - __flush_dcache_area((unsigned long)&module_alloc_base, + dcache_clean_inval_poc((unsigned long)&module_alloc_base, (unsigned long)&module_alloc_base + sizeof(module_alloc_base)); - __flush_dcache_area((unsigned long)&memstart_offset_seed, + dcache_clean_inval_poc((unsigned long)&memstart_offset_seed, (unsigned long)&memstart_offset_seed + sizeof(memstart_offset_seed)); diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index 3e79110c8f3a..03ceabe4d912 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -72,10 +72,10 @@ int machine_kexec_post_load(struct kimage *kimage) * For execution with the MMU off, reloc_code needs to be cleaned to the * PoC and invalidated from the I-cache. */ - __flush_dcache_area((unsigned long)reloc_code, + dcache_clean_inval_poc((unsigned long)reloc_code, (unsigned long)reloc_code + arm64_relocate_new_kernel_size); - invalidate_icache_range((uintptr_t)reloc_code, + icache_inval_pou((uintptr_t)reloc_code, (uintptr_t)reloc_code + arm64_relocate_new_kernel_size); @@ -111,7 +111,7 @@ static void kexec_list_flush(struct kimage *kimage) unsigned long addr; /* flush the list entries. */ - __flush_dcache_area((unsigned long)entry, + dcache_clean_inval_poc((unsigned long)entry, (unsigned long)entry + sizeof(kimage_entry_t)); @@ -128,7 +128,7 @@ static void kexec_list_flush(struct kimage *kimage) break; case IND_SOURCE: /* flush the source pages. */ - __flush_dcache_area(addr, addr + PAGE_SIZE); + dcache_clean_inval_poc(addr, addr + PAGE_SIZE); break; case IND_DESTINATION: break; @@ -155,7 +155,7 @@ static void kexec_segment_flush(const struct kimage *kimage) kimage->segment[i].memsz, kimage->segment[i].memsz / PAGE_SIZE); - __flush_dcache_area( + dcache_clean_inval_poc( (unsigned long)phys_to_virt(kimage->segment[i].mem), (unsigned long)phys_to_virt(kimage->segment[i].mem) + kimage->segment[i].memsz); diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index 5fcdee331087..9b4c1118194d 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -122,7 +122,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) secondary_data.task = idle; secondary_data.stack = task_stack_page(idle) + THREAD_SIZE; update_cpu_boot_status(CPU_MMU_OFF); - __flush_dcache_area((unsigned long)&secondary_data, + dcache_clean_inval_poc((unsigned long)&secondary_data, (unsigned long)&secondary_data + sizeof(secondary_data)); @@ -145,7 +145,7 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) pr_crit("CPU%u: failed to come online\n", cpu); secondary_data.task = NULL; secondary_data.stack = NULL; - __flush_dcache_area((unsigned long)&secondary_data, + dcache_clean_inval_poc((unsigned long)&secondary_data, (unsigned long)&secondary_data + sizeof(secondary_data)); status = READ_ONCE(secondary_data.status); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index 58d804582a35..7e1624ecab3c 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -36,7 +36,7 @@ static void write_pen_release(u64 val) unsigned long size = sizeof(secondary_holding_pen_release); secondary_holding_pen_release = val; - __flush_dcache_area((unsigned long)start, (unsigned long)start + size); + dcache_clean_inval_poc((unsigned long)start, (unsigned long)start + size); } @@ -90,7 +90,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * the boot protocol. */ writeq_relaxed(pa_holding_pen, release_addr); - __flush_dcache_area((__force unsigned long)release_addr, + dcache_clean_inval_poc((__force unsigned long)release_addr, (__force unsigned long)release_addr + sizeof(*release_addr)); diff --git a/arch/arm64/kernel/sys_compat.c b/arch/arm64/kernel/sys_compat.c index 265fe3eb1069..db5159a3055f 100644 --- a/arch/arm64/kernel/sys_compat.c +++ b/arch/arm64/kernel/sys_compat.c @@ -41,7 +41,7 @@ __do_compat_cache_op(unsigned long start, unsigned long end) dsb(ish); } - ret = __flush_cache_user_range(start, start + chunk); + ret = caches_clean_inval_user_pou(start, start + chunk); if (ret) return ret; diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 1cb39c0803a4..c1953f65ca0e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1064,7 +1064,7 @@ static int kvm_arch_vcpu_ioctl_vcpu_init(struct kvm_vcpu *vcpu, if (!cpus_have_final_cap(ARM64_HAS_STAGE2_FWB)) stage2_unmap_vm(vcpu->kvm); else - __flush_icache_all(); + icache_inval_all_pou(); } vcpu_reset_hcr(vcpu); diff --git a/arch/arm64/kvm/hyp/nvhe/cache.S b/arch/arm64/kvm/hyp/nvhe/cache.S index 36cef6915428..958734f4d6b0 100644 --- a/arch/arm64/kvm/hyp/nvhe/cache.S +++ b/arch/arm64/kvm/hyp/nvhe/cache.S @@ -7,7 +7,7 @@ #include #include -SYM_FUNC_START_PI(__flush_dcache_area) +SYM_FUNC_START_PI(dcache_clean_inval_poc) dcache_by_line_op civac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__flush_dcache_area) +SYM_FUNC_END_PI(dcache_clean_inval_poc) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setup.c index 5dffe928f256..8143ebd4fb72 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -134,7 +134,7 @@ static void update_nvhe_init_params(void) for (i = 0; i < hyp_nr_cpus; i++) { params = per_cpu_ptr(&kvm_init_params, i); params->pgd_pa = __hyp_pa(pkvm_pgtable.pgd); - __flush_dcache_area((unsigned long)params, + dcache_clean_inval_poc((unsigned long)params, (unsigned long)params + sizeof(*params)); } } diff --git a/arch/arm64/kvm/hyp/nvhe/tlb.c b/arch/arm64/kvm/hyp/nvhe/tlb.c index 83dc3b271bc5..38ed0f6f2703 100644 --- a/arch/arm64/kvm/hyp/nvhe/tlb.c +++ b/arch/arm64/kvm/hyp/nvhe/tlb.c @@ -104,7 +104,7 @@ void __kvm_tlb_flush_vmid_ipa(struct kvm_s2_mmu *mmu, * you should be running with VHE enabled. */ if (icache_is_vpipt()) - __flush_icache_all(); + icache_inval_all_pou(); __tlb_switch_to_host(&cxt); } diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 10d2f04013d4..e9ad7fb28ee3 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -841,7 +841,7 @@ static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, if (need_flush) { kvm_pte_t *pte_follow = kvm_pte_follow(pte, mm_ops); - __flush_dcache_area((unsigned long)pte_follow, + dcache_clean_inval_poc((unsigned long)pte_follow, (unsigned long)pte_follow + kvm_granule_size(level)); } @@ -997,7 +997,7 @@ static int stage2_flush_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, return 0; pte_follow = kvm_pte_follow(pte, mm_ops); - __flush_dcache_area((unsigned long)pte_follow, + dcache_clean_inval_poc((unsigned long)pte_follow, (unsigned long)pte_follow + kvm_granule_size(level)); return 0; diff --git a/arch/arm64/lib/uaccess_flushcache.c b/arch/arm64/lib/uaccess_flushcache.c index 62ea989effe8..baee22961bdb 100644 --- a/arch/arm64/lib/uaccess_flushcache.c +++ b/arch/arm64/lib/uaccess_flushcache.c @@ -15,7 +15,7 @@ void memcpy_flushcache(void *dst, const void *src, size_t cnt) * barrier to order the cache maintenance against the memcpy. */ memcpy(dst, src, cnt); - __clean_dcache_area_pop((unsigned long)dst, (unsigned long)dst + cnt); + dcache_clean_pop((unsigned long)dst, (unsigned long)dst + cnt); } EXPORT_SYMBOL_GPL(memcpy_flushcache); @@ -33,6 +33,6 @@ unsigned long __copy_user_flushcache(void *to, const void __user *from, rc = raw_copy_from_user(to, from, n); /* See above */ - __clean_dcache_area_pop((unsigned long)to, (unsigned long)to + n - rc); + dcache_clean_pop((unsigned long)to, (unsigned long)to + n - rc); return rc; } diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index b70a6699c02b..e799a4999299 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -15,7 +15,7 @@ #include /* - * __flush_cache_range(start,end) [fixup] + * caches_clean_inval_pou_macro(start,end) [fixup] * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -25,7 +25,7 @@ * - end - virtual end address of region * - fixup - optional label to branch to on user fault */ -.macro __flush_cache_range, fixup +.macro caches_clean_inval_pou_macro, fixup alternative_if ARM64_HAS_CACHE_IDC dsb ishst b .Ldc_skip_\@ @@ -50,7 +50,7 @@ alternative_else_nop_endif .endm /* - * __flush_icache_range(start,end) + * caches_clean_inval_pou(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -59,13 +59,13 @@ alternative_else_nop_endif * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__flush_icache_range) - __flush_cache_range +SYM_FUNC_START(caches_clean_inval_pou) + caches_clean_inval_pou_macro ret -SYM_FUNC_END(__flush_icache_range) +SYM_FUNC_END(caches_clean_inval_pou) /* - * __flush_cache_user_range(start,end) + * caches_clean_inval_user_pou(start,end) * * Ensure that the I and D caches are coherent within specified region. * This is typically used when code has been written to a memory region, @@ -74,10 +74,10 @@ SYM_FUNC_END(__flush_icache_range) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__flush_cache_user_range) +SYM_FUNC_START(caches_clean_inval_user_pou) uaccess_ttbr0_enable x2, x3, x4 - __flush_cache_range 2f + caches_clean_inval_pou_macro 2f mov x0, xzr 1: uaccess_ttbr0_disable x1, x2 @@ -85,17 +85,17 @@ SYM_FUNC_START(__flush_cache_user_range) 2: mov x0, #-EFAULT b 1b -SYM_FUNC_END(__flush_cache_user_range) +SYM_FUNC_END(caches_clean_inval_user_pou) /* - * invalidate_icache_range(start,end) + * icache_inval_pou(start,end) * * Ensure that the I cache is invalid within specified region. * * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(invalidate_icache_range) +SYM_FUNC_START(icache_inval_pou) alternative_if ARM64_HAS_CACHE_DIC isb ret @@ -103,10 +103,10 @@ alternative_else_nop_endif invalidate_icache_by_line x0, x1, x2, x3 ret -SYM_FUNC_END(invalidate_icache_range) +SYM_FUNC_END(icache_inval_pou) /* - * __flush_dcache_area(start, end) + * dcache_clean_inval_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned and invalidated to the PoC. @@ -114,13 +114,13 @@ SYM_FUNC_END(invalidate_icache_range) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START_PI(__flush_dcache_area) +SYM_FUNC_START_PI(dcache_clean_inval_poc) dcache_by_line_op civac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__flush_dcache_area) +SYM_FUNC_END_PI(dcache_clean_inval_poc) /* - * __clean_dcache_area_pou(start, end) + * dcache_clean_pou(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoU. @@ -128,17 +128,17 @@ SYM_FUNC_END_PI(__flush_dcache_area) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START(__clean_dcache_area_pou) +SYM_FUNC_START(dcache_clean_pou) alternative_if ARM64_HAS_CACHE_IDC dsb ishst ret alternative_else_nop_endif dcache_by_line_op cvau, ish, x0, x1, x2, x3 ret -SYM_FUNC_END(__clean_dcache_area_pou) +SYM_FUNC_END(dcache_clean_pou) /* - * __inval_dcache_area(start, end) + * dcache_inval_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are invalidated. Any partial lines at the ends of the interval are @@ -148,7 +148,7 @@ SYM_FUNC_END(__clean_dcache_area_pou) * - end - kernel end address of region */ SYM_FUNC_START_LOCAL(__dma_inv_area) -SYM_FUNC_START_PI(__inval_dcache_area) +SYM_FUNC_START_PI(dcache_inval_poc) /* FALLTHROUGH */ /* @@ -173,11 +173,11 @@ SYM_FUNC_START_PI(__inval_dcache_area) b.lo 2b dsb sy ret -SYM_FUNC_END_PI(__inval_dcache_area) +SYM_FUNC_END_PI(dcache_inval_poc) SYM_FUNC_END(__dma_inv_area) /* - * __clean_dcache_area_poc(start, end) + * dcache_clean_poc(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoC. @@ -186,7 +186,7 @@ SYM_FUNC_END(__dma_inv_area) * - end - virtual end address of region */ SYM_FUNC_START_LOCAL(__dma_clean_area) -SYM_FUNC_START_PI(__clean_dcache_area_poc) +SYM_FUNC_START_PI(dcache_clean_poc) /* FALLTHROUGH */ /* @@ -196,11 +196,11 @@ SYM_FUNC_START_PI(__clean_dcache_area_poc) */ dcache_by_line_op cvac, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__clean_dcache_area_poc) +SYM_FUNC_END_PI(dcache_clean_poc) SYM_FUNC_END(__dma_clean_area) /* - * __clean_dcache_area_pop(start, end) + * dcache_clean_pop(start, end) * * Ensure that any D-cache lines for the interval [start, end) * are cleaned to the PoP. @@ -208,13 +208,13 @@ SYM_FUNC_END(__dma_clean_area) * - start - virtual start address of region * - end - virtual end address of region */ -SYM_FUNC_START_PI(__clean_dcache_area_pop) +SYM_FUNC_START_PI(dcache_clean_pop) alternative_if_not ARM64_HAS_DCPOP - b __clean_dcache_area_poc + b dcache_clean_poc alternative_else_nop_endif dcache_by_line_op cvap, sy, x0, x1, x2, x3 ret -SYM_FUNC_END_PI(__clean_dcache_area_pop) +SYM_FUNC_END_PI(dcache_clean_pop) /* * __dma_flush_area(start, size) diff --git a/arch/arm64/mm/flush.c b/arch/arm64/mm/flush.c index 143f625e7727..5fea9a3f6663 100644 --- a/arch/arm64/mm/flush.c +++ b/arch/arm64/mm/flush.c @@ -17,14 +17,14 @@ void sync_icache_aliases(unsigned long start, unsigned long end) { if (icache_is_aliasing()) { - __clean_dcache_area_pou(start, end); - __flush_icache_all(); + dcache_clean_pou(start, end); + icache_inval_all_pou(); } else { /* * Don't issue kick_all_cpus_sync() after I-cache invalidation * for user mappings. */ - __flush_icache_range(start, end); + caches_clean_inval_pou(start, end); } } @@ -76,20 +76,20 @@ EXPORT_SYMBOL(flush_dcache_page); /* * Additional functions defined in assembly. */ -EXPORT_SYMBOL(__flush_icache_range); +EXPORT_SYMBOL(caches_clean_inval_pou); #ifdef CONFIG_ARCH_HAS_PMEM_API void arch_wb_cache_pmem(void *addr, size_t size) { /* Ensure order against any prior non-cacheable writes */ dmb(osh); - __clean_dcache_area_pop((unsigned long)addr, (unsigned long)addr + size); + dcache_clean_pop((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_wb_cache_pmem); void arch_invalidate_pmem(void *addr, size_t size) { - __inval_dcache_area((unsigned long)addr, (unsigned long)addr + size); + dcache_inval_poc((unsigned long)addr, (unsigned long)addr + size); } EXPORT_SYMBOL_GPL(arch_invalidate_pmem); #endif