From patchwork Mon Dec 6 00:47:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 12694856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 13A03C433F5 for ; Mon, 6 Dec 2021 00:49:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=A+Uc4g3GC5kzHOZTE3xIxo412qPWALpMWg7q6rE5IEQ=; b=DrP9zRda8qrNJspB5vpWwYL6J7 efJgkkW6+xVnuC/xdfWgYcrd5fzjGfoXyij29OVBdh6GQ0qDaR9DSv7am5DkwUInfoYHRyOT771so 9sxSWzYLapmhqVCRmxWcI0GOg5oFbBBFaDwoZ+Nis9uYfWLzT/J26Nc0GQYMn5e7NSVag0f7jIefe A57lMu4yH+5bHoimr8icR1aGITXHulkQ+fy/2SGBkZ/9sFnwu5LZkQgm5M3wt89d85cJfLpCPcEu/ IHjPFRt+dcWsq4Nm/mxe0HjJKtMdD2lFLWIs7rkaU19+6st93153sMxyTJlcgKqLuy46lUkrGS7y/ uddTEPOQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mu2Ay-002GSU-VR; Mon, 06 Dec 2021 00:48:17 +0000 Received: from mail-pj1-x1049.google.com ([2607:f8b0:4864:20::1049]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mu2Ag-002GL9-O0 for linux-arm-kernel@lists.infradead.org; Mon, 06 Dec 2021 00:48:00 +0000 Received: by mail-pj1-x1049.google.com with SMTP id ce19-20020a17090aff1300b001a6f72e2dbdso3985231pjb.7 for ; Sun, 05 Dec 2021 16:47:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Juc0qnZj8TlomGHzjSRBMmD/3w+jNx2TwdjwTE6BS98=; b=hWASvvEZtys46W/X8EPsrSiN97sWwQWsAIj49tLDVsttqi5umZE0LsPuMSopJRUewn xOokNd7G4qpHNxLJaV/QOOR7XWu4l+XVW/9L1jfD/5otmIpvXUUq0aBv/fgf7hQDZ7gQ 9ttMrbu+Rg0RJZjNTv34F2MUu6Uo5F9ORaqpRzEdIfLp40QqznuSjpmA1O/Y9xtpd/g9 Yp0nHPsueGZUhgnNkzRALzezkN98hJq3HpF2yKZo4lz8ikZqxWPwq5npVe3nw6i1QmUX JZAH2PZs9Txrl+pV4MfbPL+TT6a6udy+W8NXKJR5F7cB428JUGD+PhRAYhSlG3qIkg3/ rlUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Juc0qnZj8TlomGHzjSRBMmD/3w+jNx2TwdjwTE6BS98=; b=Btu/jNteNmDqtAcO0MABt2Je0ax8W5QqP9O7O7UezIcnIoMVJLD6UOEscEgIPaECN2 zpHBafjsHDFPAdQ+WhD63QhYhzgTtj/Zw6zE7C7kDZ8fL11ZuFu4OrKwCoLe2cmryGml Jg7SK/oC03gxwDEs/ePPb10qG5O2Y4n60Q3MJGqTr9R3szWSbJ8OlQcXj2hNRIwcrq8O WUKI1HWNzSs37c4qEZ/JAJDlb5f5MTubSap69ZjThLhi7zaomSSLiuCwtUndaQK5UnHq YoTBnu7OM8Phka58bQBWFMZ58vnlcJEh2I5uJr3jPdsMVorc31hgC1WYA/UUfzM6wi2T x4bQ== X-Gm-Message-State: AOAM532Rijh4jZtmBqjnERzCpP6J9QhQvMLGFQzT7feJWM+cEMp0hAZW z5vpFf772Zniu+i80loLNjz/9WvTk8I= X-Google-Smtp-Source: ABdhPJytq7YmOnRIwb0exEbD+4p7g7hb2gfAKiUucUCocWiTk/gwO1Fi9Jj8cfb+NQuimHJC4pbdzVvtfMk= X-Received: from reiji-vws-sp.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3d59]) (user=reijiw job=sendgmr) by 2002:a05:6a00:b83:b0:49f:b555:1183 with SMTP id g3-20020a056a000b8300b0049fb5551183mr33536263pfj.32.1638751677472; Sun, 05 Dec 2021 16:47:57 -0800 (PST) Date: Sun, 5 Dec 2021 16:47:36 -0800 In-Reply-To: <20211206004736.1520989-1-reijiw@google.com> Message-Id: <20211206004736.1520989-3-reijiw@google.com> Mime-Version: 1.0 References: <20211206004736.1520989-1-reijiw@google.com> X-Mailer: git-send-email 2.34.1.400.ga245620fadb-goog Subject: [PATCH v4 2/2] arm64: mte: DC {GVA,GZVA} shouldn't be used when DCZID_EL0.DZP == 1 From: Reiji Watanabe To: Catalin Marinas , Will Deacon Cc: Marc Zyngier , Robin Murphy , Mark Rutland , Peter Collingbourne , Evgenii Stepanov , guanghuifeng@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, Peter Shier , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211205_164758_812412_14C462A6 X-CRM114-Status: GOOD ( 12.13 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, mte_set_mem_tag_range() and mte_zero_clear_page_tags() use DC {GVA,GZVA} unconditionally. But, they should make sure that DCZID_EL0.DZP, which indicates whether or not use of those instructions is prohibited, is zero when using those instructions. Use ST{G,ZG,Z2G} instead when DCZID_EL0.DZP == 1. Fixes: 013bb59dbb7c ("arm64: mte: handle tags zeroing at page allocation time") Fixes: 3d0cca0b02ac ("kasan: speed up mte_set_mem_tag_range") Signed-off-by: Reiji Watanabe --- arch/arm64/include/asm/mte-kasan.h | 8 +++++--- arch/arm64/lib/mte.S | 8 +++++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 478b9bcf69ad..e4704a403237 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -84,10 +84,12 @@ static inline void __dc_gzva(u64 p) static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, bool init) { - u64 curr, mask, dczid_bs, end1, end2, end3; + u64 curr, mask, dczid, dczid_bs, dczid_dzp, end1, end2, end3; /* Read DC G(Z)VA block size from the system register. */ - dczid_bs = 4ul << (read_cpuid(DCZID_EL0) & 0xf); + dczid = read_cpuid(DCZID_EL0); + dczid_bs = 4ul << (dczid & 0xf); + dczid_dzp = (dczid >> 4) & 1; curr = (u64)__tag_set(addr, tag); mask = dczid_bs - 1; @@ -106,7 +108,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, */ #define SET_MEMTAG_RANGE(stg_post, dc_gva) \ do { \ - if (size >= 2 * dczid_bs) { \ + if (!dczid_dzp && size >= 2 * dczid_bs) {\ do { \ curr = stg_post(curr); \ } while (curr < end1); \ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index e83643b3995f..f531dcb95174 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) * x0 - address to the beginning of the page */ SYM_FUNC_START(mte_zero_clear_page_tags) + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag mrs x1, dczid_el0 + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag 1: dc gzva, x0 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stz2g x0, [x0], #(MTE_GRANULE_SIZE * 2) + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END(mte_zero_clear_page_tags) /*