From patchwork Fri Nov 19 06:09:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Reiji Watanabe X-Patchwork-Id: 12693104 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 715D0C433F5 for ; Fri, 19 Nov 2021 06:11:25 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3D0BE61A6C for ; Fri, 19 Nov 2021 06:11:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3D0BE61A6C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=PPZL9l+XvuYCquGrWUsRvStj1FXYHZCkA7qj81HPehc=; b=KdHblHgT2WGgGmH29DDyBTSijZ b+SQkg/qiIsNExH98J0k3TLAYIapJyHQrCp+NcKWjSgpiGZg1SXUs4UG9eJP4/CR0ZoaPETvnlbVW E7wcgO75FB3Kq7Bho+q0KoqcuxZY4OveCWpyA891qmJd4vJs6SDID5Il6OdglGaawLY9MJ5GM7xZ8 WEc7rpQKmLKyD/6w3tzQ/cPKyc1dRMg2Vodq3litIV3JXc3nwSM2XwLvWdyXyskj2MJNbR2zDe0+V y3P+Lp1I/bj67drGRFdGFb8RdARrYeFuhlVrvuVeAE6r1T2Wfp9EBVYfKsEhC0enRC2s87/apQD3G ju/7uSvg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mnx61-009W3F-0b; Fri, 19 Nov 2021 06:10:01 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mnx5x-009W2Z-JW for linux-arm-kernel@lists.infradead.org; Fri, 19 Nov 2021 06:09:58 +0000 Received: by mail-pf1-x44a.google.com with SMTP id 184-20020a6217c1000000b0049f9aad0040so5056347pfx.21 for ; Thu, 18 Nov 2021 22:09:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fcckAcmgEQbovsGgGLuMRFH/grtBIDbLvh2T+PSrY5U=; b=gcHbxi7tqCg0M8sof+1DwqiDeWRQOxX7DjUts1IvXWBONHNvCB4WsZ9nWA3ZbCO4Gj QxEYmrntmKicPAE3daqJO9eamcQYYyr4hLZAeUZRgjAp+q3GTuakL+ULzbn/HwY2adZe AxrCvdlI3wFTEmnWuUu1gjcOkpRu3TZHuW0EbCm1FiI+VqYfCnpYUx+CX3sE4PeXN9Sw KAUmE5sFVIEbm1ALgl4ZCLCV9D/c3bAyuff8mjQZ9IiEqiAnnTvRMZNXm3y6wBBrwS5t Is1Kztak3YI2uBXxPDMBQnPbLZY5J3TC4GX2N4xnsSPPd+vR/GUgoFi3Qp9PVWru9XMK UdJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fcckAcmgEQbovsGgGLuMRFH/grtBIDbLvh2T+PSrY5U=; b=Y39W24chUs5qSv1sekMt3xJ4i5w+Amn/LUse+t+xejC5CLi0bdczNv/McRGMFQ7w2b cBtIz7QF3fixCQD7r7rgQ88CHpUDNOclRZLY0DQOU+lBlIbe915ySDuZvN0cUIYs39IG 1/44cNqaU+6qnnNf5kf5lqUttXkbofm9JuaYXskC7TrGctFBtVmbYhF1PrkQZ3zieG2H PCRi4GgCvGShENiNHPiQepWvV2Uls03L7PfLxclgV6k/ATZj2hn3lMiAvGoUEkZamMmP /EdjSiMhPCZI0BicIdruj90PHp0/0do14ZJeZRxPAJRMKoeEe57Ow/N5uYn1rtyb+xzc ghrA== X-Gm-Message-State: AOAM532eKetCqnTbhCEd3fyTAWoHAJl/MOylXV6ikb2UcqscaFarPsIb PSnDIVyoAqmfvSxMS+YLHpMEhY0owdQ= X-Google-Smtp-Source: ABdhPJzFXX4Xk5uYFpfMpyJJsoCcHY+O6x2k6zcowYF3iwK+5iR2utuIb2fGErLrP+349WkBpKhJzKfDyQc= X-Received: from reiji-vws-sp.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3d59]) (user=reijiw job=sendgmr) by 2002:a05:6a00:10d2:b0:44d:f03e:46c7 with SMTP id d18-20020a056a0010d200b0044df03e46c7mr61693766pfu.0.1637302196285; Thu, 18 Nov 2021 22:09:56 -0800 (PST) Date: Thu, 18 Nov 2021 22:09:27 -0800 In-Reply-To: <20211119060927.3185381-1-reijiw@google.com> Message-Id: <20211119060927.3185381-3-reijiw@google.com> Mime-Version: 1.0 References: <20211119060927.3185381-1-reijiw@google.com> X-Mailer: git-send-email 2.34.0.rc2.393.gf8c9666880-goog Subject: [PATCH v3 2/2] arm64: mte: DC {GVA,GZVA} shouldn't be used when DCZID_EL0.DZP == 1 From: Reiji Watanabe To: Catalin Marinas , Will Deacon Cc: Marc Zyngier , Robin Murphy , Mark Rutland , Peter Collingbourne , Evgenii Stepanov , linux-arm-kernel@lists.infradead.org, guanghuifeng@linux.alibaba.com, Peter Shier , Ricardo Koller , Oliver Upton , Jing Zhang , Raghavendra Rao Anata , Reiji Watanabe X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211118_220957_672028_682FCAAC X-CRM114-Status: GOOD ( 12.95 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently, mte_set_mem_tag_range() and mte_zero_clear_page_tags() use DC {GVA,GZVA} unconditionally. But, they should make sure that DCZID_EL0.DZP, which indicates whether or not use of those instructions is prohibited, is zero when using those instructions. Use ST{G,ZG} instead when DCZID_EL0.DZP == 1. Fixes: 013bb59dbb7c ("arm64: mte: handle tags zeroing at page allocation time") Fixes: 3d0cca0b02ac ("kasan: speed up mte_set_mem_tag_range") Signed-off-by: Reiji Watanabe --- arch/arm64/include/asm/mte-kasan.h | 8 +++++--- arch/arm64/lib/mte.S | 8 +++++++- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 478b9bcf69ad..e4704a403237 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -84,10 +84,12 @@ static inline void __dc_gzva(u64 p) static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, bool init) { - u64 curr, mask, dczid_bs, end1, end2, end3; + u64 curr, mask, dczid, dczid_bs, dczid_dzp, end1, end2, end3; /* Read DC G(Z)VA block size from the system register. */ - dczid_bs = 4ul << (read_cpuid(DCZID_EL0) & 0xf); + dczid = read_cpuid(DCZID_EL0); + dczid_bs = 4ul << (dczid & 0xf); + dczid_dzp = (dczid >> 4) & 1; curr = (u64)__tag_set(addr, tag); mask = dczid_bs - 1; @@ -106,7 +108,7 @@ static inline void mte_set_mem_tag_range(void *addr, size_t size, u8 tag, */ #define SET_MEMTAG_RANGE(stg_post, dc_gva) \ do { \ - if (size >= 2 * dczid_bs) { \ + if (!dczid_dzp && size >= 2 * dczid_bs) {\ do { \ curr = stg_post(curr); \ } while (curr < end1); \ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index e83643b3995f..e62c048af337 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -43,17 +43,23 @@ SYM_FUNC_END(mte_clear_page_tags) * x0 - address to the beginning of the page */ SYM_FUNC_START(mte_zero_clear_page_tags) + and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag mrs x1, dczid_el0 + tbnz x1, #4, 2f // Branch if DC GZVA is prohibited and w1, w1, #0xf mov x2, #4 lsl x1, x2, x1 - and x0, x0, #(1 << MTE_TAG_SHIFT) - 1 // clear the tag 1: dc gzva, x0 add x0, x0, x1 tst x0, #(PAGE_SIZE - 1) b.ne 1b ret + +2: stzg x0, [x0], #16 + tst x0, #(PAGE_SIZE - 1) + b.ne 2b + ret SYM_FUNC_END(mte_zero_clear_page_tags) /*