From patchwork Thu Jan 7 17:29:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12004563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81106C433DB for ; Thu, 7 Jan 2021 17:31:52 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4029022C9F for ; Thu, 7 Jan 2021 17:31:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4029022C9F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=l1i1ZeLEw29jcMUbndzvkv7au1g9X3yTBOMKT0pBENg=; b=C1ZGjYuqJoIduNUFjh9XkYYPV FN9tTSxxNEA0DUBdoUp07KJJydBcKHVFOdJNtIT5Y/PfSvf5hUhNuTb25EiA4idy/cswRTU1lAxu6 iMYbXFd9RTLTbFKZOYH5ZZqdia82wvoiH4gjoxA8floWClcMJEshvFTIoLMCnrMpWJ34lawprTusc v7Ed7K1jg+vFNmdvMUgTn2/+QCBVrm5XaHPu7CbGBB+WutzPTVXteKrQjeVzRyky8187hljBKModE no7HAymmFqiNj/KLma83LM5I+5R4IPs+HByFBxLNDXvN7g2DKZFjzHWI/Us7vmUjvaKYaJ7I5Am9y EKl92Dc+w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7L-0005KX-E6; Thu, 07 Jan 2021 17:30:35 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7D-0005Iy-Iw for linux-arm-kernel@lists.infradead.org; Thu, 07 Jan 2021 17:30:28 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A4EE0ED1; Thu, 7 Jan 2021 09:30:22 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id EAA443F719; Thu, 7 Jan 2021 09:30:20 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 1/4] kasan, arm64: Add KASAN light mode Date: Thu, 7 Jan 2021 17:29:05 +0000 Message-Id: <20210107172908.42686-2-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210107172908.42686-1-vincenzo.frascino@arm.com> References: <20210107172908.42686-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210107_123027_785990_BBA106D2 X-CRM114-Status: GOOD ( 22.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marco Elver , Catalin Marinas , Branislav Rankov , Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Will Deacon , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Architectures supported by KASAN HW can provide a light mode of execution. On an MTE enabled arm64 hw for example this can be identified with the asynch mode of execution. If an async exception occurs, the arm64 core updates a register which is asynchronously detected the next time in which the kernel is accessed. KASAN requires a specific mode of execution to make use of this hw feature. Add KASAN HW light execution mode. Note: This patch adds the KASAN_ARG_MODE_LIGHT config option and the "light" kernel command line option to enable the described feature. This patch introduces the kasan_def.h header to make easier to propagate the relevant enumerations to the architectural code. Cc: Dmitry Vyukov Cc: Andrey Ryabinin Cc: Alexander Potapenko Cc: Andrey Konovalov Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/memory.h | 2 +- arch/arm64/include/asm/mte-kasan.h | 5 +++-- arch/arm64/kernel/mte.c | 2 +- include/linux/kasan.h | 1 + include/linux/kasan_def.h | 25 +++++++++++++++++++++++++ mm/kasan/hw_tags.c | 24 ++++-------------------- mm/kasan/kasan.h | 2 +- 7 files changed, 36 insertions(+), 25 deletions(-) create mode 100644 include/linux/kasan_def.h diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 18fce223b67b..3a7c5beb7096 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -231,7 +231,7 @@ static inline const void *__tag_set(const void *addr, u8 tag) } #ifdef CONFIG_KASAN_HW_TAGS -#define arch_enable_tagging() mte_enable_kernel() +#define arch_enable_tagging(mode) mte_enable_kernel(mode) #define arch_init_tags(max_tag) mte_init_tags(max_tag) #define arch_get_random_tag() mte_get_random_tag() #define arch_get_mem_tag(addr) mte_get_mem_tag(addr) diff --git a/arch/arm64/include/asm/mte-kasan.h b/arch/arm64/include/asm/mte-kasan.h index 26349a4b5e2e..79e612c31186 100644 --- a/arch/arm64/include/asm/mte-kasan.h +++ b/arch/arm64/include/asm/mte-kasan.h @@ -9,6 +9,7 @@ #ifndef __ASSEMBLY__ +#include #include /* @@ -29,7 +30,7 @@ u8 mte_get_mem_tag(void *addr); u8 mte_get_random_tag(void); void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag); -void mte_enable_kernel(void); +void mte_enable_kernel(enum kasan_arg_mode mode); void mte_init_tags(u64 max_tag); #else /* CONFIG_ARM64_MTE */ @@ -52,7 +53,7 @@ static inline void *mte_set_mem_tag_range(void *addr, size_t size, u8 tag) return addr; } -static inline void mte_enable_kernel(void) +static inline void mte_enable_kernel(enum kasan_arg_mode mode) { } diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index dc9ada64feed..24a273d47df1 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -151,7 +151,7 @@ void mte_init_tags(u64 max_tag) write_sysreg_s(SYS_GCR_EL1_RRND | gcr_kernel_excl, SYS_GCR_EL1); } -void mte_enable_kernel(void) +void mte_enable_kernel(enum kasan_arg_mode mode) { /* Enable MTE Sync Mode for EL1. */ sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 5e0655fb2a6f..026031444217 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -2,6 +2,7 @@ #ifndef _LINUX_KASAN_H #define _LINUX_KASAN_H +#include #include #include diff --git a/include/linux/kasan_def.h b/include/linux/kasan_def.h new file mode 100644 index 000000000000..e774b0122980 --- /dev/null +++ b/include/linux/kasan_def.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_KASAN_DEF_H +#define _LINUX_KASAN_DEF_H + +enum kasan_arg_mode { + KASAN_ARG_MODE_DEFAULT, + KASAN_ARG_MODE_OFF, + KASAN_ARG_MODE_LIGHT, + KASAN_ARG_MODE_PROD, + KASAN_ARG_MODE_FULL, +}; + +enum kasan_arg_stacktrace { + KASAN_ARG_STACKTRACE_DEFAULT, + KASAN_ARG_STACKTRACE_OFF, + KASAN_ARG_STACKTRACE_ON, +}; + +enum kasan_arg_fault { + KASAN_ARG_FAULT_DEFAULT, + KASAN_ARG_FAULT_REPORT, + KASAN_ARG_FAULT_PANIC, +}; + +#endif /* _LINUX_KASAN_DEF_H */ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 55bd6f09c70f..d5c6ad8b2c44 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -19,25 +19,6 @@ #include "kasan.h" -enum kasan_arg_mode { - KASAN_ARG_MODE_DEFAULT, - KASAN_ARG_MODE_OFF, - KASAN_ARG_MODE_PROD, - KASAN_ARG_MODE_FULL, -}; - -enum kasan_arg_stacktrace { - KASAN_ARG_STACKTRACE_DEFAULT, - KASAN_ARG_STACKTRACE_OFF, - KASAN_ARG_STACKTRACE_ON, -}; - -enum kasan_arg_fault { - KASAN_ARG_FAULT_DEFAULT, - KASAN_ARG_FAULT_REPORT, - KASAN_ARG_FAULT_PANIC, -}; - static enum kasan_arg_mode kasan_arg_mode __ro_after_init; static enum kasan_arg_stacktrace kasan_arg_stacktrace __ro_after_init; static enum kasan_arg_fault kasan_arg_fault __ro_after_init; @@ -60,6 +41,8 @@ static int __init early_kasan_mode(char *arg) if (!strcmp(arg, "off")) kasan_arg_mode = KASAN_ARG_MODE_OFF; + else if (!strcmp(arg, "light")) + kasan_arg_mode = KASAN_ARG_MODE_LIGHT; else if (!strcmp(arg, "prod")) kasan_arg_mode = KASAN_ARG_MODE_PROD; else if (!strcmp(arg, "full")) @@ -118,7 +101,7 @@ void kasan_init_hw_tags_cpu(void) return; hw_init_tags(KASAN_TAG_MAX); - hw_enable_tagging(); + hw_enable_tagging(kasan_arg_mode); } /* kasan_init_hw_tags() is called once on boot CPU. */ @@ -145,6 +128,7 @@ void __init kasan_init_hw_tags(void) case KASAN_ARG_MODE_OFF: /* If KASAN is disabled, do nothing. */ return; + case KASAN_ARG_MODE_LIGHT: case KASAN_ARG_MODE_PROD: static_branch_enable(&kasan_flag_enabled); break; diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index cc4d9e1d49b1..78c09279327e 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -284,7 +284,7 @@ static inline const void *arch_kasan_set_tag(const void *addr, u8 tag) #define arch_set_mem_tag_range(addr, size, tag) ((void *)(addr)) #endif -#define hw_enable_tagging() arch_enable_tagging() +#define hw_enable_tagging(mode) arch_enable_tagging(mode) #define hw_init_tags(max_tag) arch_init_tags(max_tag) #define hw_get_random_tag() arch_get_random_tag() #define hw_get_mem_tag(addr) arch_get_mem_tag(addr) From patchwork Thu Jan 7 17:29:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12004561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE08CC43381 for ; Thu, 7 Jan 2021 17:31:48 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8EC65233CE for ; Thu, 7 Jan 2021 17:31:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8EC65233CE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Si+WzZ9Mtk9L+6cHueW2pbKI1eA943XY2jvh/xVfgwc=; b=QQxRpeFlHfosIQhlTmkUPpI2p DkE+CZ77zg+PydFd1tgLE4CIFZgEsFLaMW9iBXEIyT7GCN27qAm2Qbb/LroDbkANQWU9JS72nFYfh JLw0aAJcWeUGiwlPes09kXYbm4kUq7HlES03UjMcGDVV69BofY95SaxDviZHXpKufjVoX9yk31Xsz QMw61mjvap4jFxC2gI26a00U3CpNk+RVeYFUpgWtDWhFvDBzQ7tPFb85kc5td7tNiwMfozfM83CEF goRmDt14pt5gb9JJf/rS423lt+bYHRFUM9Swt0x0wyzzYiX+QX6Dq6lMxEWRxfJCthydEilh6XgeZ HrFdAPuiQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7N-0005Kr-0c; Thu, 07 Jan 2021 17:30:37 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7F-0005Jb-7h for linux-arm-kernel@lists.infradead.org; Thu, 07 Jan 2021 17:30:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7B05F106F; Thu, 7 Jan 2021 09:30:24 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CA71E3F719; Thu, 7 Jan 2021 09:30:22 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 2/4] arm64: mte: Add asynchronous mode support Date: Thu, 7 Jan 2021 17:29:06 +0000 Message-Id: <20210107172908.42686-3-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210107172908.42686-1-vincenzo.frascino@arm.com> References: <20210107172908.42686-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210107_123029_409872_8EF64F55 X-CRM114-Status: GOOD ( 13.89 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marco Elver , Catalin Marinas , Branislav Rankov , Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Will Deacon , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MTE provides an asynchronous mode for detecting tag exceptions. In particular instead of triggering a fault the arm64 core updates a register which is checked by the kernel at the first entry after the tag exception has occurred. Add support for MTE asynchronous mode. The exception handling mechanism will be added with a future patch. Note: KASAN HW activates async mode via kasan.mode kernel parameter. The default mode is set to synchronous. Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/kernel/mte.c | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 24a273d47df1..5d992e16b420 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -153,8 +153,35 @@ void mte_init_tags(u64 max_tag) void mte_enable_kernel(enum kasan_arg_mode mode) { - /* Enable MTE Sync Mode for EL1. */ - sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + const char *m; + + /* Preset parameter values based on the mode. */ + switch (mode) { + case KASAN_ARG_MODE_OFF: + return; + case KASAN_ARG_MODE_LIGHT: + /* Enable MTE Async Mode for EL1. */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_ASYNC); + m = "asynchronous"; + break; + case KASAN_ARG_MODE_DEFAULT: + case KASAN_ARG_MODE_PROD: + case KASAN_ARG_MODE_FULL: + /* Enable MTE Sync Mode for EL1. */ + sysreg_clear_set(sctlr_el1, SCTLR_ELx_TCF_MASK, SCTLR_ELx_TCF_SYNC); + m = "synchronous"; + break; + default: + /* + * kasan mode should be always set hence we should + * not reach this condition. + */ + WARN_ON_ONCE(1); + return; + } + + pr_info_once("MTE: enabled in %s mode at EL1\n", m); + isb(); } From patchwork Thu Jan 7 17:29:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12004557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CC6DFC433DB for ; Thu, 7 Jan 2021 17:31:48 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8B39722C9F for ; Thu, 7 Jan 2021 17:31:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8B39722C9F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=W9RyUKn+RNO3FZpzDbIwUCTPHjecnzfV0eclZJQCqDk=; b=hbj09GONVROEy20bH52TUA4yo i/pdGmrJvS9HpfzR6fhQ7Bc1J2BP8cZCgoGrmVQ3IUBk+3Aa7f0Qe3s8muktzwOG06++WlS+gP3zq J8wyqzwko4mQC2VFsEkNEKv51ge2obnslN7HYQoKQngoEJWJv3TqfjRLJ0tVvAHjVnvD7+VPJKmug 32BfrYecpj3GMc+tYyjTuPK7uDM8Q4Y1z8+eonW/wnGPfuar8pND83tQQ3pmb2uF7v5eYHZYtFxpx Fa3C00figImYDbaHQJahc7zCn1cpRnGRnPBX/nlzmH6xhtRB2kbdlOjFeo0RfpBq481lqhduAdxMw IdieBxmnA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7I-0005KB-Ep; Thu, 07 Jan 2021 17:30:32 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7D-0005Ix-In for linux-arm-kernel@lists.infradead.org; Thu, 07 Jan 2021 17:30:28 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 59D58113E; Thu, 7 Jan 2021 09:30:26 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AA6C83F719; Thu, 7 Jan 2021 09:30:24 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 3/4] arm64: mte: Enable async tag check fault Date: Thu, 7 Jan 2021 17:29:07 +0000 Message-Id: <20210107172908.42686-4-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210107172908.42686-1-vincenzo.frascino@arm.com> References: <20210107172908.42686-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210107_123027_766426_E5E7F166 X-CRM114-Status: GOOD ( 19.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marco Elver , Catalin Marinas , Branislav Rankov , Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Will Deacon , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org MTE provides a mode that asynchronously updates the TFSR_EL1 register when a tag check exception is detected. To take advantage of this mode the kernel has to verify the status of the register at: 1. Context switching 2. Return to user/EL0 (Not required in entry from EL0 since the kernel did not run) 3. Kernel entry from EL1 4. Kernel exit to EL1 If the register is non-zero a trace is reported. Add the required features for EL1 detection and reporting. Note: ITFSB bit is set in the SCTLR_EL1 register hence it guaranties that the indirect writes to TFSR_EL1 are synchronized at exception entry to EL1. On the context switch path the synchronization is guarantied by the dsb() in __switch_to(). Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/mte.h | 4 ++++ arch/arm64/kernel/entry-common.c | 6 ++++++ arch/arm64/kernel/mte.c | 37 ++++++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index d02aff9f493d..a60d3718baae 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -39,6 +39,7 @@ void mte_free_tag_storage(char *storage); /* track which pages have valid allocation tags */ #define PG_mte_tagged PG_arch_2 +void mte_check_tfsr_el1(void); void mte_sync_tags(pte_t *ptep, pte_t pte); void mte_copy_page_tags(void *kto, const void *kfrom); void flush_mte_state(void); @@ -56,6 +57,9 @@ void mte_assign_mem_tag_range(void *addr, size_t size); /* unused if !CONFIG_ARM64_MTE, silence the compiler */ #define PG_mte_tagged 0 +static inline void mte_check_tfsr_el1(void) +{ +} static inline void mte_sync_tags(pte_t *ptep, pte_t pte) { } diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 5346953e4382..74b020ce72d7 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -37,6 +37,8 @@ static void noinstr enter_from_kernel_mode(struct pt_regs *regs) lockdep_hardirqs_off(CALLER_ADDR0); rcu_irq_enter_check_tick(); trace_hardirqs_off_finish(); + + mte_check_tfsr_el1(); } /* @@ -47,6 +49,8 @@ static void noinstr exit_to_kernel_mode(struct pt_regs *regs) { lockdep_assert_irqs_disabled(); + mte_check_tfsr_el1(); + if (interrupts_enabled(regs)) { if (regs->exit_rcu) { trace_hardirqs_on_prepare(); @@ -243,6 +247,8 @@ asmlinkage void noinstr enter_from_user_mode(void) asmlinkage void noinstr exit_to_user_mode(void) { + mte_check_tfsr_el1(); + trace_hardirqs_on_prepare(); lockdep_hardirqs_on_prepare(CALLER_ADDR0); user_enter_irqoff(); diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 5d992e16b420..26030f0b79fe 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -185,6 +185,34 @@ void mte_enable_kernel(enum kasan_arg_mode mode) isb(); } +void mte_check_tfsr_el1(void) +{ + u64 tfsr_el1; + + if (!IS_ENABLED(CONFIG_KASAN_HW_TAGS)) + return; + + if (!system_supports_mte()) + return; + + tfsr_el1 = read_sysreg_s(SYS_TFSR_EL1); + + /* + * The kernel should never hit the condition TF0 == 1 + * at this point because for the futex code we set + * PSTATE.TCO. + */ + WARN_ON(tfsr_el1 & SYS_TFSR_EL1_TF0); + + if (tfsr_el1 & SYS_TFSR_EL1_TF1) { + write_sysreg_s(0, SYS_TFSR_EL1); + isb(); + + pr_err("MTE: Asynchronous tag exception detected!"); + } +} +NOKPROBE_SYMBOL(mte_check_tfsr_el1); + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ @@ -250,6 +278,15 @@ void mte_thread_switch(struct task_struct *next) /* avoid expensive SCTLR_EL1 accesses if no change */ if (current->thread.sctlr_tcf0 != next->thread.sctlr_tcf0) update_sctlr_el1_tcf0(next->thread.sctlr_tcf0); + + /* + * Check if an async tag exception occurred at EL1. + * + * Note: On the context switch patch we rely on the dsb() present + * in __switch_to() to guaranty that the indirect writes to TFSR_EL1 + * are synchronized before this point. + */ + mte_check_tfsr_el1(); } void mte_suspend_exit(void) From patchwork Thu Jan 7 17:29:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincenzo Frascino X-Patchwork-Id: 12004565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C920C433E0 for ; Thu, 7 Jan 2021 17:31:57 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C47CB22C9F for ; Thu, 7 Jan 2021 17:31:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C47CB22C9F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mO4W0ikFeZyMJ7OCRCDwTX9hFVsmyvwL/ZfL6kAoLII=; b=Ty7Te+0hCQJOR2HxlFXyeNgJm 5bxxdRY0RPlbnmQTFS20DvysTJDTnV3ldiafbD5XD/1vCIrGAytP45YnBQseJjKdetKEZuwKF+n07 gAw9q0gm6yDfyWChhmkr+ZAAKIRsvL8gCfCc3iomtTId8+dSWpmqKXKBhDQuj+JjOjFpUyUkDJ4Ua K1h3tk3pKssHE1kG/ukFDa9ySPQuVKR9gWOnRvCQc9TpJQvw6EzvSLxA6ykTkbYL+WOahKjYn5Oxx eX2MwS3Sbub7QxNoGKEjVTih/Mmou0x/pQ3R8kfzRzTaqFHdCWdqlfEA2oWyFTV1uju/bee4brFpr UyTyC6IBw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7U-0005MV-0L; Thu, 07 Jan 2021 17:30:44 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kxZ7R-0005Lw-S4 for linux-arm-kernel@lists.infradead.org; Thu, 07 Jan 2021 17:30:43 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3544911FB; Thu, 7 Jan 2021 09:30:28 -0800 (PST) Received: from e119884-lin.cambridge.arm.com (e119884-lin.cambridge.arm.com [10.1.196.72]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8A4E63F719; Thu, 7 Jan 2021 09:30:26 -0800 (PST) From: Vincenzo Frascino To: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com Subject: [PATCH v2 4/4] arm64: mte: Optimize mte_assign_mem_tag_range() Date: Thu, 7 Jan 2021 17:29:08 +0000 Message-Id: <20210107172908.42686-5-vincenzo.frascino@arm.com> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210107172908.42686-1-vincenzo.frascino@arm.com> References: <20210107172908.42686-1-vincenzo.frascino@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210107_123042_012451_C132A0E3 X-CRM114-Status: GOOD ( 12.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Marco Elver , Catalin Marinas , Branislav Rankov , Alexander Potapenko , Evgenii Stepanov , Andrey Konovalov , Andrey Ryabinin , Vincenzo Frascino , Will Deacon , Dmitry Vyukov Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org mte_assign_mem_tag_range() is called on production KASAN HW hot paths. It makes sense to optimize it in an attempt to reduce the overhead. Optimize mte_assign_mem_tag_range() based on the indications provided at [1]. [1] https://lore.kernel.org/r/CAAeHK+wCO+J7D1_T89DG+jJrPLk3X9RsGFKxJGd0ZcUFjQT-9Q@mail.gmail.com/ Cc: Catalin Marinas Cc: Will Deacon Signed-off-by: Vincenzo Frascino --- arch/arm64/include/asm/mte.h | 26 +++++++++++++++++++++++++- arch/arm64/lib/mte.S | 15 --------------- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/arch/arm64/include/asm/mte.h b/arch/arm64/include/asm/mte.h index a60d3718baae..c341ce6b9779 100644 --- a/arch/arm64/include/asm/mte.h +++ b/arch/arm64/include/asm/mte.h @@ -50,7 +50,31 @@ long get_mte_ctrl(struct task_struct *task); int mte_ptrace_copy_tags(struct task_struct *child, long request, unsigned long addr, unsigned long data); -void mte_assign_mem_tag_range(void *addr, size_t size); +static inline void mte_assign_mem_tag_range(void *addr, size_t size) +{ + u64 _addr = (u64)addr; + u64 _end = _addr + size; + + /* + * This function must be invoked from an MTE enabled context. + * + * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and + * size must be non-zero and MTE_GRANULE_SIZE aligned. + */ + do { + /* + * 'asm volatile' is required to prevent the compiler to move + * the statement outside of the loop. + */ + asm volatile(__MTE_PREAMBLE "stg %0, [%0]" + : + : "r" (_addr) + : "memory"); + + _addr += MTE_GRANULE_SIZE; + } while (_addr < _end); +} + #else /* CONFIG_ARM64_MTE */ diff --git a/arch/arm64/lib/mte.S b/arch/arm64/lib/mte.S index 9e1a12e10053..a0a650451510 100644 --- a/arch/arm64/lib/mte.S +++ b/arch/arm64/lib/mte.S @@ -150,18 +150,3 @@ SYM_FUNC_START(mte_restore_page_tags) ret SYM_FUNC_END(mte_restore_page_tags) -/* - * Assign allocation tags for a region of memory based on the pointer tag - * x0 - source pointer - * x1 - size - * - * Note: The address must be non-NULL and MTE_GRANULE_SIZE aligned and - * size must be non-zero and MTE_GRANULE_SIZE aligned. - */ -SYM_FUNC_START(mte_assign_mem_tag_range) -1: stg x0, [x0] - add x0, x0, #MTE_GRANULE_SIZE - subs x1, x1, #MTE_GRANULE_SIZE - b.gt 1b - ret -SYM_FUNC_END(mte_assign_mem_tag_range)