From patchwork Thu May 20 12:43:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fuad Tabba X-Patchwork-Id: 12270135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04CB4C433ED for ; Thu, 20 May 2021 12:46:37 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8809261184 for ; Thu, 20 May 2021 12:46:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8809261184 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=nnM6OPrjc++p9jTFIaoEx0we77dJM0ucyFc9lYdrAXc=; b=pmDOQ907BzuUPp ifpJCuACi93XQV0Jpv9HAEgjPFwa9zYgqyN051ad44f4fWljXsE8YIcRFXrAWxOT52s4Ne+CdVzkx 4IyRj8XIYn6q1gVW17SkP12wzYu//bqP5mQME1XYy3ErVa42fgHmamTrOeFOApuT5KFlrLTQs8K4p VtuhXb2ajRF06qTsgmSXUPlegi+omfF9da9RoOuOrKWzxLIcHhBZvXwulu3oTeGSZp4syPv3myPQ0 uRTe8/9o0mVf33MzkyOTLp4Fb9K//I0AhQrY9en/q+ZoGeDY/LEVzkePU3o0QqesTT1ajfN9V3Nt9 +FWOl7mUhQ9XAnSquZQw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji37-000uKE-Ht; Thu, 20 May 2021 12:45:13 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lji2D-000tyK-Nt for linux-arm-kernel@desiato.infradead.org; Thu, 20 May 2021 12:44:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=KDPGYyPzMnkWgOQgmBvxX+NxkJicVg2L+NLCrWpY0nE=; b=gMO8AK1AWoiucJaJpVETY7ckHU Hopymvay5rknzvgqhuMCcJtebdhFj5U/LBkt9yv9xW0aMpWFArsZwAs70PvOkLpi/gQQZd81rQJPN gC9AZHX19LT35FVw/NrIROykVcHdp11fS//HuhyeBZMrVcELpla1EktBrj9wRYRwXyDW6o80XeEN2 3+dV/EdlW7MYPXJj8n7CBu9orugEGJJOo8GfXN4F7gkOxh/2ZL39H61YIIcwABienZgy/z6T61//W lydAlR/NF7VueUgb67bRqeZRbi+kzTnb+pYBaMJUZ27F9KqL4Z+pixmP2ZLFzsjCeEeKJb+UFAaYr DU5Y4R1A==; Received: from mail-wr1-x44a.google.com ([2a00:1450:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lji2B-00GKc9-4c for linux-arm-kernel@lists.infradead.org; Thu, 20 May 2021 12:44:16 +0000 Received: by mail-wr1-x44a.google.com with SMTP id u20-20020a0560001614b02901115c8f2d89so8501160wrb.3 for ; Thu, 20 May 2021 05:44:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KDPGYyPzMnkWgOQgmBvxX+NxkJicVg2L+NLCrWpY0nE=; b=mkEtYX5k5nSIEoECJzSwKj0OD0FbogkJ8FQPnY/DUeNP6hiO620sfX74a+mOZDUhRu T6E67WjKhdyHZBxcYEa2iznAfgc+XKWUSs+behUn3G85NUdzwqJEs8yN7MzpHHq3evtc 86vjfL2Rxt+njmoCfiOmg1faSP8Xp1MsAFzsnTl+zIPgJ8QUsXjXvNFpC1ppOST+Q0+L tSEUxJ8WjfPhcia3WEqJN7TwlBkVbHZ1Xzct3LxOcx009jHSGgOrDe/ikMpuo8kq4Gp8 Fvb3o51h0EvpnKsLeDq3tapFiY0fOwwn1GzoCzHPCj8b0z7aVU+v8oFAPo30S5HwlMu7 vWqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KDPGYyPzMnkWgOQgmBvxX+NxkJicVg2L+NLCrWpY0nE=; b=HD7+CRTakjFZYYGav3m224ZtvdFAjLubtnpCtZXYy+WCH+TMBEZaDNtFE0Cr8JuZeb qYqTnFVnoZR87/dBwdku4DZh27PlOei5wOVKSKxGNqtJjd8O+IAVMFiqi1GX14rhgVI/ j3a5ynj+BrcLTwqcZsO3FHDbzWTn8AoIZPT/S4W1E1fPqGyXc3+6t6h964n8Sx4Nwmiy DJyEuqoVQbrjdy6cdKWTYv5U8f7wbTHsU52TlyaQo09GzkIAxaJ2d1CdOvTvn8YDSPLj jhi0uUH7StAwwEpsz6Qv6UvY1oezgQbdl5fsNiGkVfB31mHECok67bO2nZmQ1b/O/EFc zEmA== X-Gm-Message-State: AOAM531EKv7Cgb86Ehhmjm57rCh0z7WcxtYsmy0dSksF7ehy8IQfLCs0 qv2J6rxBXwzD94+KTHLpIx5oBlM43OD3vVcGy21NIdxARvj/9UXzyC0e+cBjs7X5MAhRzglRn9e vD8rD2fZBJ5fg9y1g/QOIEmLFY7/9KWfpEsW0NTVLreDIaULuCQ9zDZiSMxagUt0W7BKcqLr9ms jRpw== X-Google-Smtp-Source: ABdhPJz4PdNI9eZ8Mcbd15lAKmd0F27Unm31NjTDRF0bj180zMMuZ/D9TJzzeC0azjfnSA5tT+lI5jGszQ== X-Received: from tabba.c.googlers.com ([fda3:e722:ac3:10:28:9cb1:c0a8:482]) (user=tabba job=sendgmr) by 2002:adf:f212:: with SMTP id p18mr4026328wro.318.1621514652435; Thu, 20 May 2021 05:44:12 -0700 (PDT) Date: Thu, 20 May 2021 13:43:50 +0100 In-Reply-To: <20210520124406.2731873-1-tabba@google.com> Message-Id: <20210520124406.2731873-3-tabba@google.com> Mime-Version: 1.0 References: <20210520124406.2731873-1-tabba@google.com> X-Mailer: git-send-email 2.31.1.751.gd2f1c929bd-goog Subject: [PATCH v3 02/18] arm64: assembler: add conditional cache fixups From: Fuad Tabba To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, maz@kernel.org, ardb@kernel.org, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, robin.murphy@arm.com, tabba@google.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210520_054415_204386_4EE7DC60 X-CRM114-Status: GOOD ( 15.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Mark Rutland It would be helpful if we could use both `dcache_by_line_op` and `invalidate_icache_by_line` for user memory without accidentally fixing up unexpected faults when performing maintenance on kernel addresses. Let's make this possible by having both macros take an optional fixup label, and only generating an extable entry if a label is provided. At the same time, let's clean up the labels used to be globally unique using \@ as we do for other macros. There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Signed-off-by: Fuad Tabba Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: Fuad Tabba Cc: Will Deacon --- arch/arm64/include/asm/assembler.h | 39 +++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 6a0fbc599196..0a276b46ef50 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -130,15 +130,27 @@ alternative_endif .endm /* - * Emit an entry into the exception table + * Create an exception table entry for `insn`, which will branch to `fixup` + * when an unhandled fault is taken. */ - .macro _asm_extable, from, to + .macro _asm_extable, insn, fixup .pushsection __ex_table, "a" .align 3 - .long (\from - .), (\to - .) + .long (\insn - .), (\fixup - .) .popsection .endm +/* + * Create an exception table entry for `insn` if `fixup` is provided. Otherwise + * do nothing. + */ + .macro _cond_extable, insn, fixup + .ifnc \fixup, + _asm_extable \insn, \fixup + .endif + .endm + + #define USER(l, x...) \ 9999: x; \ _asm_extable 9999b, l @@ -383,6 +395,7 @@ alternative_cb_end * domain: domain used in dsb instruciton * addr: starting virtual address of the region * size: size of the region + * fixup: optional label to branch to on user fault * Corrupts: addr, size, tmp1, tmp2 */ .macro __dcache_op_workaround_clean_cache, op, addr @@ -393,12 +406,12 @@ alternative_else alternative_endif .endm - .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2 + .macro dcache_by_line_op op, domain, addr, size, tmp1, tmp2, fixup dcache_line_size \tmp1, \tmp2 add \size, \addr, \size sub \tmp2, \tmp1, #1 bic \addr, \addr, \tmp2 -9998: +.Ldcache_op\@: .ifc \op, cvau __dcache_op_workaround_clean_cache \op, \addr .else @@ -418,8 +431,10 @@ alternative_endif .endif add \addr, \addr, \tmp1 cmp \addr, \size - b.lo 9998b + b.lo .Ldcache_op\@ dsb \domain + + _cond_extable .Ldcache_op\@, \fixup .endm /* @@ -427,20 +442,22 @@ alternative_endif * [start, end) * * start, end: virtual addresses describing the region - * label: A label to branch to on user fault. + * fixup: optional label to branch to on user fault * Corrupts: tmp1, tmp2 */ - .macro invalidate_icache_by_line start, end, tmp1, tmp2, label + .macro invalidate_icache_by_line start, end, tmp1, tmp2, fixup icache_line_size \tmp1, \tmp2 sub \tmp2, \tmp1, #1 bic \tmp2, \start, \tmp2 -9997: -USER(\label, ic ivau, \tmp2) // invalidate I line PoU +.Licache_op\@: + ic ivau, \tmp2 // invalidate I line PoU add \tmp2, \tmp2, \tmp1 cmp \tmp2, \end - b.lo 9997b + b.lo .Licache_op\@ dsb ish isb + + _cond_extable .Licache_op\@, \fixup .endm /*