From patchwork Tue Jul 13 23:47:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12375403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D61B5C07E96 for ; Tue, 13 Jul 2021 23:50:13 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A48F06128E for ; Tue, 13 Jul 2021 23:50:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A48F06128E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=sc8XHNoSIiFtmeVfFDhAzUUb43Gnn2EbDS+wbzS7pDA=; b=PjRe1sRjwlV3nQaL9LDpZJcOUg GZCVdQpqMZ8J5qKaRNPdERyjui9OdR9f0Gt/BG4CUUXnr5zOx9KvwzMMSVmT6E+OGBf6rKBewLmVA dcgoNU5L4KkhXSoumr+ReIi7hXEDwsvCJr3aC0psr4ApPvga2xeqaw41EkZEHSD2qk7/O2bmjC0oV AMHNZdtbUFt4HF7LhnlolqPx25zbSTmEK1t848V01wviQ+CKKJly7uycfIFvkZn30ExP5K5/HWMgl 1Wtx2Y+a1EJkXGbucZFOK77ScqQ/cm8q++Mo4kYIvwVF9rRIXvU/L8CnrMEiiRuDOOc4qFiIvAb2B qg+EojmQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8V-00BggY-76; Tue, 13 Jul 2021 23:48:23 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8H-00BgeX-4v for linux-arm-kernel@lists.infradead.org; Tue, 13 Jul 2021 23:48:10 +0000 Received: by mail-qt1-x84a.google.com with SMTP id c18-20020a05622a0592b0290251fb198592so688698qtb.1 for ; Tue, 13 Jul 2021 16:48:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fOeV+y0DQThZkLyceAvM2kgMJy8cRVtWY5oYUssPk3s=; b=YrUZxeKbaX2gJWqTsoj5jJAysCmvHrqzKr4D5lAuzrZS05fsiBveLaVKUWeHnPpeS3 4K3JWE175/osUlYR/Sd4olcjvdTv6kv9OfvXaRFw729RP2PuODndMN0+qTidlUhMI1ZM gO/i2FDjWArKHwy0fEfDu5r6FeNrPjJdMqkPc+TgOYUtWb86LgUGfwv4Lq9QSc5r8JJQ veualAQm2lmHxMVwop5qQw/n6SAUocpleKkEEaZE73AjErq+FlzKItVLaF6BPigUcpv1 DTYgH5CZS7RyuhJPqNUq6rm/x8FAW8lv/zSwS+n243UddPldI6mSaQ8a60EVqt9Wihh8 SwSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fOeV+y0DQThZkLyceAvM2kgMJy8cRVtWY5oYUssPk3s=; b=nKYAQ3B/bKh21tqb54Me6wlzljBtrJojRaZo0VBnjkpiUS8hhV0U152HGLlvByu6B+ orDFk1loDGq5sNIG0oRMomAKSViAP+IdAQHFkY5m604JyR8YwJ//hacf72B6A2SwPrX4 4zgJKrH02kbKxEcBSWfZE0lbb/UtkfCgHQeDzaHMWenQ4oL+5NDGA/xFU7GS/EfeT0IJ e8HC9ZDPqgZYZubUMLlaJLFKlJm9s0t+GR+t8JTXeFr+KC3fI9fEgRCSkJ5+eEYP6BgA +DUupb+mI9hM+vBqtaLP/Qtt6ZEQapVr6bES2mXawTtZ8owajUYqJ4vNj1J6UhO2EBDL GzHw== X-Gm-Message-State: AOAM531bB8y6oNqCLU0MFyEh0Hfd/PLcypNF/3p8uUUBzogk4wPARMs/ inCzARjtIgiFYWgIQW+t4jua0ns= X-Google-Smtp-Source: ABdhPJzz2Ka3IDZG3ejflYJYEl792VbOvEA8STITaSyie2Ttgd3zmBQsNG658i1+UB5E4gGZ9D4UHIw= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:99b9:48f7:dbe6:8670]) (user=pcc job=sendgmr) by 2002:a0c:d605:: with SMTP id c5mr7885036qvj.7.1626220087431; Tue, 13 Jul 2021 16:48:07 -0700 (PDT) Date: Tue, 13 Jul 2021 16:47:57 -0700 In-Reply-To: <20210713234801.3858018-1-pcc@google.com> Message-Id: <20210713234801.3858018-2-pcc@google.com> Mime-Version: 1.0 References: <20210713234801.3858018-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog Subject: [PATCH v10 1/5] arm64: mte: rename gcr_user_excl to mte_ctrl From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210713_164809_236318_0948DE3F X-CRM114-Status: GOOD ( 15.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We are going to use this field to store more data. To prepare for that, rename it and change the users to rely on the bit position of gcr_user_excl in mte_ctrl. Link: https://linux-review.googlesource.com/id/Ie1fd18e480100655f5d22137f5b22f4f3a9f9e2e Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas Acked-by: Will Deacon --- arch/arm64/include/asm/processor.h | 5 ++++- arch/arm64/kernel/asm-offsets.c | 2 +- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/kernel/mte.c | 14 ++++++++------ 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 9df3feeee890..6322fb1714d5 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -16,6 +16,9 @@ */ #define NET_IP_ALIGN 0 +#define MTE_CTRL_GCR_USER_EXCL_SHIFT 0 +#define MTE_CTRL_GCR_USER_EXCL_MASK 0xffff + #ifndef __ASSEMBLY__ #include @@ -151,7 +154,7 @@ struct thread_struct { struct ptrauth_keys_kernel keys_kernel; #endif #ifdef CONFIG_ARM64_MTE - u64 gcr_user_excl; + u64 mte_ctrl; #endif u64 sctlr_user; }; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 0cb34ccb6e73..63d02cd67b44 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -49,7 +49,7 @@ int main(void) DEFINE(THREAD_KEYS_KERNEL, offsetof(struct task_struct, thread.keys_kernel)); #endif #ifdef CONFIG_ARM64_MTE - DEFINE(THREAD_GCR_EL1_USER, offsetof(struct task_struct, thread.gcr_user_excl)); + DEFINE(THREAD_MTE_CTRL, offsetof(struct task_struct, thread.mte_ctrl)); #endif BLANK(); DEFINE(S_X0, offsetof(struct pt_regs, regs[0])); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 3513984a88bd..ce59280355c5 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -182,7 +182,7 @@ alternative_else_nop_endif * the RRND (bit[16]) setting. */ mrs_s \tmp2, SYS_GCR_EL1 - bfi \tmp2, \tmp, #0, #16 + bfxil \tmp2, \tmp, #MTE_CTRL_GCR_USER_EXCL_SHIFT, #16 msr_s SYS_GCR_EL1, \tmp2 #endif .endm @@ -205,7 +205,7 @@ alternative_else_nop_endif alternative_if_not ARM64_MTE b 1f alternative_else_nop_endif - ldr \tmp, [\tsk, #THREAD_GCR_EL1_USER] + ldr \tmp, [\tsk, #THREAD_MTE_CTRL] mte_set_gcr \tmp, \tmp2 1: diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 125a10e413e9..d3884d09513d 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -199,7 +199,7 @@ static void update_gcr_el1_excl(u64 excl) static void set_gcr_el1_excl(u64 excl) { - current->thread.gcr_user_excl = excl; + current->thread.mte_ctrl = excl; /* * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value @@ -263,8 +263,8 @@ void mte_suspend_exit(void) long set_mte_ctrl(struct task_struct *task, unsigned long arg) { u64 sctlr = task->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK; - u64 gcr_excl = ~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & - SYS_GCR_EL1_EXCL_MASK; + u64 mte_ctrl = (~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & + SYS_GCR_EL1_EXCL_MASK) << MTE_CTRL_GCR_USER_EXCL_SHIFT; if (!system_supports_mte()) return 0; @@ -285,10 +285,10 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg) if (task != current) { task->thread.sctlr_user = sctlr; - task->thread.gcr_user_excl = gcr_excl; + task->thread.mte_ctrl = mte_ctrl; } else { set_task_sctlr_el1(sctlr); - set_gcr_el1_excl(gcr_excl); + set_gcr_el1_excl(mte_ctrl); } return 0; @@ -297,7 +297,9 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg) long get_mte_ctrl(struct task_struct *task) { unsigned long ret; - u64 incl = ~task->thread.gcr_user_excl & SYS_GCR_EL1_EXCL_MASK; + u64 mte_ctrl = task->thread.mte_ctrl; + u64 incl = (~mte_ctrl >> MTE_CTRL_GCR_USER_EXCL_SHIFT) & + SYS_GCR_EL1_EXCL_MASK; if (!system_supports_mte()) return 0; From patchwork Tue Jul 13 23:47:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12375405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D7F2C07E95 for ; Tue, 13 Jul 2021 23:50:28 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1CE1C6128E for ; Tue, 13 Jul 2021 23:50:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1CE1C6128E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=VkpG4NuZZCIcYUsMwelkqMaOhhznX+rPJbtxObvKK9M=; b=FOq0/ptMWIBdZ8hY88FBso5Nxg qg0cVY7vTPeAO4Za8Hz0NlVcCbAZm8FAmV9DpsvW6WVaakrqejD99twYo+4ultwhpazuFLUG+CaN7 /4iaHe1Icyvnbd+iqyME83E+tdd4QxDIZfansTW3M2a7aFWFba8quVrq5yK4gGljtqTeX7UOopowt CsPjtV1rMDCRYPgBcvyH1USEa96yutJtmKBW5dTU8W0zjJVSlZ8pMqcJo19IUgZDq7fOHXvlr+46u rtTDqolBwJO1lPYC2Ma80u1oVs7Qe8yp6jjLcei1McrRQdZnUTxwlrA5LwFoYFXL/4TGBPCiEofub 7NxxuN4Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8g-00BgiH-4U; Tue, 13 Jul 2021 23:48:34 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8J-00Bget-Nw for linux-arm-kernel@lists.infradead.org; Tue, 13 Jul 2021 23:48:13 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id x15-20020a25ce0f0000b029055bb0981111so150793ybe.7 for ; Tue, 13 Jul 2021 16:48:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=V8AvbU0TQRUNOZ6bECCv/o7+4i37teHt5vLXN692cCw=; b=hA/3EhFMK0WvYGSSNFKGBYGclNQf/R84Mas+FDK1Vl5w9diDBE/xtnBEQ7YGET7tHd 9t0A8BfndVsoww7k94WCRk7uHuyezq2Z35SF5uemq+Qz6AD6JwEQ2A6vHQNjQ/ExZKX+ sEP1e1lesQ3L4B+Kc9YgmTD3WzGhCUTel3Tp/zuwRIYkQpy4Kp7qZUA2LjApGJKymegh UlxQ1WwRL6Sp5HarLyght8t7iqXXxELHPs4/63ZWjUbrOqP+WIkpORg6RIQYosDEBNSV /EuXhL3NvZBzxjGZhTAw9ugc5M2HOu0l3q+gxKOKdehKGGUjLrciF8QBNWQ0zXts8Aht CizA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=V8AvbU0TQRUNOZ6bECCv/o7+4i37teHt5vLXN692cCw=; b=gv/gc4TPN6xOyE59IVkexjV4+cuI62K2ohwwHUmXh/oEbTbv93Sz7FZkG3XSibF9wL LfivcUFq8doEmgpX39o4xeruuNnEXog2+cPlclJYaHV8L7VJeA1JFrs+1kETB0zNyz+n kZJ+f1GcYDKDgSOOKFo16KER5TUe3IwlwKrkby74l76LKbbz6Icbi9pvKBSsU+vjyO8i /vSvIVSVdUCtl9dTtrzm53R4SeUO+nyKEze5+Zbl7mA8pS6fQw5LgLvQKBW2oZUvhylB q9RwZyP+C+YitCOYOc2rzT29OLwUdS6VFVX1kj5auWeOHEnvpqKXdg84j3p5QHlL3HmF mY/Q== X-Gm-Message-State: AOAM531ScMHDhv7Dyr/M3p6xrQvX8+RJLOBcxJoZvP60ph9zZy2JK664 1ijFiT0eFaMgUkRtbgGWjblhYBQ= X-Google-Smtp-Source: ABdhPJyDg4i/DdDzBouoDrp7vPyLRcfgKlMsD30AVaqcUgPdMmKj8E6w2C1pv3LkzjrwGJv8Epp6BTM= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:99b9:48f7:dbe6:8670]) (user=pcc job=sendgmr) by 2002:a25:3d41:: with SMTP id k62mr8994775yba.479.1626220089890; Tue, 13 Jul 2021 16:48:09 -0700 (PDT) Date: Tue, 13 Jul 2021 16:47:58 -0700 In-Reply-To: <20210713234801.3858018-1-pcc@google.com> Message-Id: <20210713234801.3858018-3-pcc@google.com> Mime-Version: 1.0 References: <20210713234801.3858018-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog Subject: [PATCH v10 2/5] arm64: mte: change ASYNC and SYNC TCF settings into bitfields From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210713_164811_827634_36319679 X-CRM114-Status: GOOD ( 16.55 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow the user program to specify both ASYNC and SYNC TCF modes by repurposing the existing constants as bitfields. This will allow the kernel to select one of the modes on behalf of the user program. With this patch the kernel will always select async mode, but a subsequent patch will make this configurable. Link: https://linux-review.googlesource.com/id/Icc5923c85a8ea284588cc399ae74fd19ec291230 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas Acked-by: Will Deacon --- v9: - make mte_update_sctlr_user static arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/kernel/mte.c | 70 ++++++++++++------------------ include/uapi/linux/prctl.h | 11 ++--- 3 files changed, 37 insertions(+), 47 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 6322fb1714d5..80ceb9cbdd60 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -19,6 +19,9 @@ #define MTE_CTRL_GCR_USER_EXCL_SHIFT 0 #define MTE_CTRL_GCR_USER_EXCL_MASK 0xffff +#define MTE_CTRL_TCF_SYNC (1UL << 16) +#define MTE_CTRL_TCF_ASYNC (1UL << 17) + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index d3884d09513d..53d89915029d 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -197,14 +197,19 @@ static void update_gcr_el1_excl(u64 excl) sysreg_clear_set_s(SYS_GCR_EL1, SYS_GCR_EL1_EXCL_MASK, excl); } -static void set_gcr_el1_excl(u64 excl) +static void mte_update_sctlr_user(struct task_struct *task) { - current->thread.mte_ctrl = excl; + unsigned long sctlr = task->thread.sctlr_user; + unsigned long pref = MTE_CTRL_TCF_ASYNC; + unsigned long mte_ctrl = task->thread.mte_ctrl; + unsigned long resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; - /* - * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value - * by mte_set_user_gcr() in kernel_exit, - */ + sctlr &= ~SCTLR_EL1_TCF0_MASK; + if (resolved_mte_tcf & MTE_CTRL_TCF_ASYNC) + sctlr |= SCTLR_EL1_TCF0_ASYNC; + else if (resolved_mte_tcf & MTE_CTRL_TCF_SYNC) + sctlr |= SCTLR_EL1_TCF0_SYNC; + task->thread.sctlr_user = sctlr; } void mte_thread_init_user(void) @@ -216,15 +221,16 @@ void mte_thread_init_user(void) dsb(ish); write_sysreg_s(0, SYS_TFSRE0_EL1); clear_thread_flag(TIF_MTE_ASYNC_FAULT); - /* disable tag checking */ - set_task_sctlr_el1((current->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK) | - SCTLR_EL1_TCF0_NONE); - /* reset tag generation mask */ - set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK); + /* disable tag checking and reset tag generation mask */ + current->thread.mte_ctrl = MTE_CTRL_GCR_USER_EXCL_MASK; + mte_update_sctlr_user(current); + set_task_sctlr_el1(current->thread.sctlr_user); } void mte_thread_switch(struct task_struct *next) { + mte_update_sctlr_user(next); + /* * Check if an async tag exception occurred at EL1. * @@ -262,33 +268,21 @@ void mte_suspend_exit(void) long set_mte_ctrl(struct task_struct *task, unsigned long arg) { - u64 sctlr = task->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK; u64 mte_ctrl = (~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & SYS_GCR_EL1_EXCL_MASK) << MTE_CTRL_GCR_USER_EXCL_SHIFT; if (!system_supports_mte()) return 0; - switch (arg & PR_MTE_TCF_MASK) { - case PR_MTE_TCF_NONE: - sctlr |= SCTLR_EL1_TCF0_NONE; - break; - case PR_MTE_TCF_SYNC: - sctlr |= SCTLR_EL1_TCF0_SYNC; - break; - case PR_MTE_TCF_ASYNC: - sctlr |= SCTLR_EL1_TCF0_ASYNC; - break; - default: - return -EINVAL; - } + if (arg & PR_MTE_TCF_ASYNC) + mte_ctrl |= MTE_CTRL_TCF_ASYNC; + if (arg & PR_MTE_TCF_SYNC) + mte_ctrl |= MTE_CTRL_TCF_SYNC; - if (task != current) { - task->thread.sctlr_user = sctlr; - task->thread.mte_ctrl = mte_ctrl; - } else { - set_task_sctlr_el1(sctlr); - set_gcr_el1_excl(mte_ctrl); + task->thread.mte_ctrl = mte_ctrl; + if (task == current) { + mte_update_sctlr_user(task); + set_task_sctlr_el1(task->thread.sctlr_user); } return 0; @@ -305,18 +299,10 @@ long get_mte_ctrl(struct task_struct *task) return 0; ret = incl << PR_MTE_TAG_SHIFT; - - switch (task->thread.sctlr_user & SCTLR_EL1_TCF0_MASK) { - case SCTLR_EL1_TCF0_NONE: - ret |= PR_MTE_TCF_NONE; - break; - case SCTLR_EL1_TCF0_SYNC: - ret |= PR_MTE_TCF_SYNC; - break; - case SCTLR_EL1_TCF0_ASYNC: + if (mte_ctrl & MTE_CTRL_TCF_ASYNC) ret |= PR_MTE_TCF_ASYNC; - break; - } + if (mte_ctrl & MTE_CTRL_TCF_SYNC) + ret |= PR_MTE_TCF_SYNC; return ret; } diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 18a9f59dc067..d3a5afb4c1ae 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -234,14 +234,15 @@ struct prctl_mm_map { #define PR_GET_TAGGED_ADDR_CTRL 56 # define PR_TAGGED_ADDR_ENABLE (1UL << 0) /* MTE tag check fault modes */ -# define PR_MTE_TCF_SHIFT 1 -# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_NONE 0 +# define PR_MTE_TCF_SYNC (1UL << 1) +# define PR_MTE_TCF_ASYNC (1UL << 2) +# define PR_MTE_TCF_MASK (PR_MTE_TCF_SYNC | PR_MTE_TCF_ASYNC) /* MTE tag inclusion mask */ # define PR_MTE_TAG_SHIFT 3 # define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT) +/* Unused; kept only for source compatibility */ +# define PR_MTE_TCF_SHIFT 1 /* Control reclaim behavior when allocating memory */ #define PR_SET_IO_FLUSHER 57 From patchwork Tue Jul 13 23:47:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12375407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC34FC07E95 for ; Tue, 13 Jul 2021 23:50:30 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B2B7F6128E for ; Tue, 13 Jul 2021 23:50:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B2B7F6128E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=/+oZEmQ5m61W3tQ4YnzKLlPbi2A0Et07oFaExncL4Es=; b=paxsWm3TtJ2jZoNaUMVZsUoAnD oQyKV4hmS+1k660cIvygKoB05JEKW3gNxALy8agSCwBwkRHgvxway5tpj0ma82mUkc+uynbTIa2Du JUezo8YOx7u+fcmTklWjQZwh78wAyMpdFllCQ25YpXqlWB5FuUmSEdDCvZcwK+gMC2fDrS221PLyj WyS44GpmuWr4V34+TlFAa8aT7/pwk6aIr2rLCM349D1rm7thzOWVEHmPXjhkZvDkifNUQkuG/EK+v h+Ln5k6yMaURskfbeNK+Nn/8HzfIlTfPPNYXAA7E6CNZ7GqjnyH5gYvPVXuOg9qSgGINAK//WAo0H aeZfjBiQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8q-00BgkP-QA; Tue, 13 Jul 2021 23:48:45 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8L-00BgfH-MX for linux-arm-kernel@lists.infradead.org; Tue, 13 Jul 2021 23:48:15 +0000 Received: by mail-qk1-x74a.google.com with SMTP id c3-20020a37b3030000b02903ad0001a2e8so18705395qkf.3 for ; Tue, 13 Jul 2021 16:48:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=9ryHViKZp1y1hFQGGllX7M8/cefBkqgQ+MUoHbikRZo=; b=wOQKfld1rwxmQQmuOuGAvQTtFKN2js98cD4lfRkwZdthi126IS9Qe33b7mvQvQ5Glv fyDJDM02hZdBG8S6l6Fst9AarUMRidN/colnamJ567lNeKH2bsCSyYxM5+48olb5F3n4 rer3wcCwp7a1bkYyP7zg/3PpxhJH8oBLNFnG3il+PclL47Nmc+ZG1Tj987GGKYL9MQyF /x/YgUqCPfoxiASPijXd2kUrWM6SMxJQmuayvVty/Ed/e3JHWuYQ8CXzRb/muXeBtnav JBjDIWBdf40kPB336DuBOgCwMKu8pj7tnSyyTp+1PP1ne/OGI/4F6o8mDQkehIkNH4vH poQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=9ryHViKZp1y1hFQGGllX7M8/cefBkqgQ+MUoHbikRZo=; b=QTzLUSV8tV8ciGALsalSJffZStAHgryb8dK6I7aMOlvktbsy6zfvNnIIzYossoeufz RInWJNZyZ3zE8R4NNywcx0acPjHNxU+um97/tkFfWnxAhmdWYxDM8m8b8MC75BU7ZlPd /9NLUvOqt98dDsMMk6DKpFZjawBjkBbPvZq3ohbaS2tcG0poGJQQ9l7uEcwmzONrhgyB +UVOJZzFIIaVEM8QTpyIpS2q+VKMdzcYT2WZLry9Y9w/FP+cOCeGbrKU0mbG1cm6gkKy mQPQfZgmKFoOlY6KISZNv2M2q0ELLq7OrgDcQi8jEM6llHtww9phRJvqyHEf2IAAcfdm to9w== X-Gm-Message-State: AOAM532PJskGnrUqY+FCKTFfrWDpPoB2irT9FV1R/2uxMRIys1ioA8WE OTQ78DumTfkXl0jp2QIvFYfrENA= X-Google-Smtp-Source: ABdhPJzqWRThXjhQAYuvB5wfAi7hHNddNkwDavJ/DKuKZ3/nEAVQZePVfx+BorLZFdUAsHFaNIJontE= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:99b9:48f7:dbe6:8670]) (user=pcc job=sendgmr) by 2002:ad4:52ea:: with SMTP id p10mr7512546qvu.45.1626220092310; Tue, 13 Jul 2021 16:48:12 -0700 (PDT) Date: Tue, 13 Jul 2021 16:47:59 -0700 In-Reply-To: <20210713234801.3858018-1-pcc@google.com> Message-Id: <20210713234801.3858018-4-pcc@google.com> Mime-Version: 1.0 References: <20210713234801.3858018-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog Subject: [PATCH v10 3/5] arm64: move preemption disablement to prctl handlers From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210713_164813_788973_780C507C X-CRM114-Status: GOOD ( 22.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In the next patch, we will start reading sctlr_user from mte_update_sctlr_user and subsequently writing a new value based on the task's TCF setting and potentially the per-CPU TCF preference. This means that we need to be careful to disable preemption around any code sequences that read from sctlr_user and subsequently write to sctlr_user and/or SCTLR_EL1, so that we don't end up writing a stale value (based on the previous CPU's TCF preference) to either of them. We currently have four such sequences, in the prctl handlers for PR_SET_TAGGED_ADDR_CTRL and PR_PAC_SET_ENABLED_KEYS, as well as in the task initialization code that resets the prctl settings. Change the prctl handlers to disable preemption in the handlers themselves rather than the functions that they call, and change the task initialization code to call the respective prctl handlers instead of setting sctlr_user directly. As a result of this change, we no longer need the helper function set_task_sctlr_el1, nor does its behavior make sense any more, so remove it. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ic0e8a0c00bb47d786c1e8011df0b7fe99bee4bb5 Acked-by: Will Deacon --- arch/arm64/include/asm/pointer_auth.h | 12 ++++++------ arch/arm64/include/asm/processor.h | 2 +- arch/arm64/kernel/mte.c | 8 ++++---- arch/arm64/kernel/pointer_auth.c | 10 ++++++---- arch/arm64/kernel/process.c | 21 +++++++-------------- 5 files changed, 24 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/pointer_auth.h b/arch/arm64/include/asm/pointer_auth.h index d50416be99be..592968f0bc22 100644 --- a/arch/arm64/include/asm/pointer_auth.h +++ b/arch/arm64/include/asm/pointer_auth.h @@ -10,6 +10,9 @@ #include #include +#define PR_PAC_ENABLED_KEYS_MASK \ + (PR_PAC_APIAKEY | PR_PAC_APIBKEY | PR_PAC_APDAKEY | PR_PAC_APDBKEY) + #ifdef CONFIG_ARM64_PTR_AUTH /* * Each key is a 128-bit quantity which is split across a pair of 64-bit @@ -113,9 +116,9 @@ static __always_inline void ptrauth_enable(void) \ /* enable all keys */ \ if (system_supports_address_auth()) \ - set_task_sctlr_el1(current->thread.sctlr_user | \ - SCTLR_ELx_ENIA | SCTLR_ELx_ENIB | \ - SCTLR_ELx_ENDA | SCTLR_ELx_ENDB); \ + ptrauth_set_enabled_keys(current, \ + PR_PAC_ENABLED_KEYS_MASK, \ + PR_PAC_ENABLED_KEYS_MASK); \ } while (0) #define ptrauth_thread_switch_user(tsk) \ @@ -139,7 +142,4 @@ static __always_inline void ptrauth_enable(void) #define ptrauth_thread_switch_kernel(tsk) #endif /* CONFIG_ARM64_PTR_AUTH */ -#define PR_PAC_ENABLED_KEYS_MASK \ - (PR_PAC_APIAKEY | PR_PAC_APIBKEY | PR_PAC_APDAKEY | PR_PAC_APDBKEY) - #endif /* __ASM_POINTER_AUTH_H */ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 80ceb9cbdd60..ebb3b1aefed7 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -257,7 +257,7 @@ extern void release_thread(struct task_struct *); unsigned long get_wchan(struct task_struct *p); -void set_task_sctlr_el1(u64 sctlr); +void update_sctlr_el1(u64 sctlr); /* Thread switching */ extern struct task_struct *cpu_switch_to(struct task_struct *prev, diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 53d89915029d..432d9b641e9c 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -222,9 +222,7 @@ void mte_thread_init_user(void) write_sysreg_s(0, SYS_TFSRE0_EL1); clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking and reset tag generation mask */ - current->thread.mte_ctrl = MTE_CTRL_GCR_USER_EXCL_MASK; - mte_update_sctlr_user(current); - set_task_sctlr_el1(current->thread.sctlr_user); + set_mte_ctrl(current, 0); } void mte_thread_switch(struct task_struct *next) @@ -281,8 +279,10 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg) task->thread.mte_ctrl = mte_ctrl; if (task == current) { + preempt_disable(); mte_update_sctlr_user(task); - set_task_sctlr_el1(task->thread.sctlr_user); + update_sctlr_el1(task->thread.sctlr_user); + preempt_enable(); } return 0; diff --git a/arch/arm64/kernel/pointer_auth.c b/arch/arm64/kernel/pointer_auth.c index 60901ab0a7fe..2708b620b4ae 100644 --- a/arch/arm64/kernel/pointer_auth.c +++ b/arch/arm64/kernel/pointer_auth.c @@ -67,7 +67,7 @@ static u64 arg_to_enxx_mask(unsigned long arg) int ptrauth_set_enabled_keys(struct task_struct *tsk, unsigned long keys, unsigned long enabled) { - u64 sctlr = tsk->thread.sctlr_user; + u64 sctlr; if (!system_supports_address_auth()) return -EINVAL; @@ -78,12 +78,14 @@ int ptrauth_set_enabled_keys(struct task_struct *tsk, unsigned long keys, if ((keys & ~PR_PAC_ENABLED_KEYS_MASK) || (enabled & ~keys)) return -EINVAL; + preempt_disable(); + sctlr = tsk->thread.sctlr_user; sctlr &= ~arg_to_enxx_mask(keys); sctlr |= arg_to_enxx_mask(enabled); + tsk->thread.sctlr_user = sctlr; if (tsk == current) - set_task_sctlr_el1(sctlr); - else - tsk->thread.sctlr_user = sctlr; + update_sctlr_el1(sctlr); + preempt_enable(); return 0; } diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index b4bb67f17a2c..c548eec87810 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -527,7 +527,13 @@ static void erratum_1418040_thread_switch(struct task_struct *prev, write_sysreg(val, cntkctl_el1); } -static void update_sctlr_el1(u64 sctlr) +/* + * __switch_to() checks current->thread.sctlr_user as an optimisation. Therefore + * this function must be called with preemption disabled and the update to + * sctlr_user must be made in the same preemption disabled block so that + * __switch_to() does not see the variable update before the SCTLR_EL1 one. + */ +void update_sctlr_el1(u64 sctlr) { /* * EnIA must not be cleared while in the kernel as this is necessary for @@ -539,19 +545,6 @@ static void update_sctlr_el1(u64 sctlr) isb(); } -void set_task_sctlr_el1(u64 sctlr) -{ - /* - * __switch_to() checks current->thread.sctlr as an - * optimisation. Disable preemption so that it does not see - * the variable update before the SCTLR_EL1 one. - */ - preempt_disable(); - current->thread.sctlr_user = sctlr; - update_sctlr_el1(sctlr); - preempt_enable(); -} - /* * Thread switching. */ From patchwork Tue Jul 13 23:48:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12375409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 504AEC07E95 for ; Tue, 13 Jul 2021 23:50:43 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1D7AF6100C for ; Tue, 13 Jul 2021 23:50:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1D7AF6100C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=WkLbNVdXl+I9OF0a5OmGI98rHy4Kedxneq64UrJXAmo=; b=l97e1SXE8MIH8WHtUkqCxqCnDm vGnSTWb+PEC4gOQxL5QYcYHZVx9Vc9MyLBZ15p1Jhkk0kyge9lmy95q5aZeAto+pB1BbSIysVvGV/ 8a3m54F22tBNwNufmkirveuu8suOjmP3q0c76PpBqSmX/puinXfj7+Fs/WOTV10uPL6LqgnbWQoyy wB930bjlEOBqtpRxMeD+ojp9x++osDRolD9iDPOywtPjdqT1yn3Nwlso+bvYJ/ULnD74RVA1SK7cp tiqxoUA2tCBtEhVIIlDwBASkU7gELdqIShTEqNXtPN4CopQ4X60XEGOF6eOxr8i8bLbQSHobeHj4C vqIKcaxA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S96-00BgoD-O4; Tue, 13 Jul 2021 23:49:01 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8O-00Bgfb-Ht for linux-arm-kernel@lists.infradead.org; Tue, 13 Jul 2021 23:48:18 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id t15-20020a056214036fb02902dc72b5283dso221322qvu.23 for ; Tue, 13 Jul 2021 16:48:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=0I/fw/N5vkRou4jOquJnkoOvJZrGZlJhMkd9V4c9gW8=; b=U/YW1wOjj6cTKNDZQk4o0IMEjot2yut30LzM61JjqOuEVePBOnuVprKz88BO49YhuE LL2bmDNSp111qCvMsWiWjymjH7YzCyiWAsnYOqzvXXdDsgoNN3E6sCWK7ABkv0wX/mb9 2u982Hii0nYssLo3nwFrpw7kiCBeGUCoFZ4UhWbt8Bt5d8c8Q/MlYRKsoTqNluSSDeKl W2HQ8iB0z7LP+qHbQB6e1emOLmmJMidf2Uy+9oQGS7eUKmWbUzINX/VJyM71Mei9hkjo ao+YvHJImGWplCXNteWOn6t2ygoL1h+jwCGjw4kSiuBFQV/kunPnk5ENcu3CSKCyFlMj ldZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=0I/fw/N5vkRou4jOquJnkoOvJZrGZlJhMkd9V4c9gW8=; b=tKsQC6B9jlNYRTtLu3nfr+AORfMhsY5xk/2us1hLqKqHQGl9NJCBSI8d1T5v8i2IxI lpGBzpvoSc0kIFm77O6fjDTAhW/+D4+4RfV5d9XYMjCEVwLJXvELEoge2jpqYTFPYXQh ujTR7nMX4/3pzB+ZKVDxfh60wE2t8KNvWl68k9NunkAG5P8IHqjLW8sX6iescjV5RDbU dAeBH3Ow4ZqQdyoVvAlfXf0uDIEap2oFthU0dO609BO8yJ7ezSUvVVc02ThS3tF2n0dG MMxvEq3S5EolYMJgH1q0T/PC1F6T+begID7+9IzszOr9N8AeLjlT+YHIBsj8iefWflzi fgIw== X-Gm-Message-State: AOAM5310YbO2NNEN+8xEnLRdVeog6HFAqvHnwfmiQzb5mTqrqOiIgBGm oOcP1kZpLgqnmScDku7+Pb8ZoHQ= X-Google-Smtp-Source: ABdhPJyQo4s+Fanu8SP0gzCwsHt+ldWMPq7w08L+R2/xlGJnu2kviDTJl4R1eQn8CdrNd1Yr51BaG9k= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:99b9:48f7:dbe6:8670]) (user=pcc job=sendgmr) by 2002:a05:6214:e6e:: with SMTP id jz14mr7557531qvb.55.1626220094704; Tue, 13 Jul 2021 16:48:14 -0700 (PDT) Date: Tue, 13 Jul 2021 16:48:00 -0700 In-Reply-To: <20210713234801.3858018-1-pcc@google.com> Message-Id: <20210713234801.3858018-5-pcc@google.com> Mime-Version: 1.0 References: <20210713234801.3858018-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog Subject: [PATCH v10 4/5] arm64: mte: introduce a per-CPU tag checking mode preference From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210713_164816_634056_1867A045 X-CRM114-Status: GOOD ( 16.34 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a per-CPU sysfs node, mte_tcf_preferred, that allows the preferred tag checking mode to be configured. The current possible values are async and sync. Link: https://linux-review.googlesource.com/id/I7493dcd533a2785a1437b16c3f6b50919f840854 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas Acked-by: Will Deacon --- v10: - stop calling smp_call_function_single arch/arm64/kernel/mte.c | 65 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 432d9b641e9c..50ae1a7435ae 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -4,6 +4,7 @@ */ #include +#include #include #include #include @@ -26,6 +27,8 @@ u64 gcr_kernel_excl __ro_after_init; static bool report_fault_once = true; +static DEFINE_PER_CPU_READ_MOSTLY(u64, mte_tcf_preferred); + #ifdef CONFIG_KASAN_HW_TAGS /* Whether the MTE asynchronous mode is enabled. */ DEFINE_STATIC_KEY_FALSE(mte_async_mode); @@ -199,11 +202,18 @@ static void update_gcr_el1_excl(u64 excl) static void mte_update_sctlr_user(struct task_struct *task) { + /* + * This must be called with preemption disabled and can only be called + * on the current or next task since the CPU must match where the thread + * is going to run. The caller is responsible for calling + * update_sctlr_el1() later in the same preemption disabled block. + */ unsigned long sctlr = task->thread.sctlr_user; - unsigned long pref = MTE_CTRL_TCF_ASYNC; unsigned long mte_ctrl = task->thread.mte_ctrl; - unsigned long resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; + unsigned long pref, resolved_mte_tcf; + pref = __this_cpu_read(mte_tcf_preferred); + resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; sctlr &= ~SCTLR_EL1_TCF0_MASK; if (resolved_mte_tcf & MTE_CTRL_TCF_ASYNC) sctlr |= SCTLR_EL1_TCF0_ASYNC; @@ -441,3 +451,54 @@ int mte_ptrace_copy_tags(struct task_struct *child, long request, return ret; } + +static ssize_t mte_tcf_preferred_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + switch (per_cpu(mte_tcf_preferred, dev->id)) { + case MTE_CTRL_TCF_ASYNC: + return sysfs_emit(buf, "async\n"); + case MTE_CTRL_TCF_SYNC: + return sysfs_emit(buf, "sync\n"); + default: + return sysfs_emit(buf, "???\n"); + } +} + +static ssize_t mte_tcf_preferred_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + u64 tcf; + + if (sysfs_streq(buf, "async")) + tcf = MTE_CTRL_TCF_ASYNC; + else if (sysfs_streq(buf, "sync")) + tcf = MTE_CTRL_TCF_SYNC; + else + return -EINVAL; + + device_lock(dev); + per_cpu(mte_tcf_preferred, dev->id) = tcf; + device_unlock(dev); + + return count; +} +static DEVICE_ATTR_RW(mte_tcf_preferred); + +static int register_mte_tcf_preferred_sysctl(void) +{ + unsigned int cpu; + + if (!system_supports_mte()) + return 0; + + for_each_possible_cpu(cpu) { + per_cpu(mte_tcf_preferred, cpu) = MTE_CTRL_TCF_ASYNC; + device_create_file(get_cpu_device(cpu), + &dev_attr_mte_tcf_preferred); + } + + return 0; +} +subsys_initcall(register_mte_tcf_preferred_sysctl); From patchwork Tue Jul 13 23:48:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12375411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7787C07E95 for ; Tue, 13 Jul 2021 23:51:03 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B0D946128E for ; Tue, 13 Jul 2021 23:51:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0D946128E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=zQemMbpjTaKvWKizJBwOsxWvP6pyevbU8pAUsLuvrzI=; b=uga64sjcO12O6lhQSUtVU3bNEx ckW8ooOxwJARyi42ca1aELcTlIVFlZ25mE/pjVqzAs2JVm6mGt9pqXGbaQPBwHeQmWM3EFfx95Ujm ZgB/PbW+OHwQSHBe5Io+Aye4kk/8mt1RuhSbWFMDkm4rPZGN7vuEVKufXaxbZuNzCs/F20cjYcdiD FgNRGgldJum8ZBn/d6FXFGGk6wpJk0XASEeY2gixlIxBDoSj3dsniOI74GJ+Y3tjS9xiQs0HI3JlQ fpDcYzdwnnhywyJwfnV0XQrDsilRqajwGoRKlbc8O8wmpz0LVI9ul1miLh8v6J+FggpLiQABhHust C8M5lSxQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S9T-00Bgxj-8e; Tue, 13 Jul 2021 23:49:23 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m3S8R-00Bgfw-44 for linux-arm-kernel@lists.infradead.org; Tue, 13 Jul 2021 23:48:21 +0000 Received: by mail-yb1-xb49.google.com with SMTP id x5-20020a0569021025b029055b9b68cd3eso149755ybt.8 for ; Tue, 13 Jul 2021 16:48:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=N62IsDn/uNo2BIrbX/CKAnFDZcRj13ue8EMwn2XqY8U=; b=AF7sX5PAZEIfRdUWHDt1ZNbOFZMBatW0d86UZxJWeL7jnQ7Qvff/Zz1aKDqrZrfe4D W558dtcwmxYx8/ujW2lnYQOkVgN+1T3sbTV2mYMDngNgBXy2gpr+eE2wcsMntg1SHU2c f0VB4ov7HznJeSRsZG8VMu/sJDgEPipuQuiBA7DMqGsCoozOSBlrUI4fp05XLg1z992g 8eQZta/aGZy/jy30vTdtunjLsRMQMPDn8lpKaBxjwMR9rNyUr446S7y8LNDf2R444F1l GYLbe5ClpcE2WsfxQVAbH7NS3NLhejw/au0ZAFiMH4VE079aCli1/OpLWESDeSemuG4R S4dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=N62IsDn/uNo2BIrbX/CKAnFDZcRj13ue8EMwn2XqY8U=; b=sZRS4z33HNhRzRXOZv6fToqq1CbkOOUFATpyM7noSzrE32RX3Ev9UtyWJmqbWNsG74 /oUB3TEkLBOVwyWCaEq8NOn1RZopMaCeRvht3zvc6no0E+FtyI3Q1hncZoHyON5QslMS Yo/MPbSmPUf5Fkd7BH78A9V9MTcBQZ+iEFLm1QXEQvdRBFtYfuNyniJkK64sTJEgn/kU uwJsBCo2CRuYhs7fSHFNtBVYCLXAeCPlL7HQCIsuGNZ9REELoyd3ydjth8MHPXjuKKXf +lct+ck6egQaPUf4Ki9mC2mMOanEWB8uQ7tGYRlGc6zoFKK5qbaoRZ7ld1ewU2OyYLNa XxNA== X-Gm-Message-State: AOAM530OI0umWl8O+YUEr/7i2+fMdCqky6Pq7OBUliVAjEW5wB+zwy85 hryPIg6v7thrsLG74wsx2SHSbng= X-Google-Smtp-Source: ABdhPJyIN6eBCRP+grvB50q+dC1uzVREqq5fg4GuVOrnfQ8g04jKDJBVZsToX9F4fI17Ew9KK2IXgvE= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:99b9:48f7:dbe6:8670]) (user=pcc job=sendgmr) by 2002:a25:ada1:: with SMTP id z33mr9452281ybi.438.1626220097369; Tue, 13 Jul 2021 16:48:17 -0700 (PDT) Date: Tue, 13 Jul 2021 16:48:01 -0700 In-Reply-To: <20210713234801.3858018-1-pcc@google.com> Message-Id: <20210713234801.3858018-6-pcc@google.com> Mime-Version: 1.0 References: <20210713234801.3858018-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.93.g670b81a890-goog Subject: [PATCH v10 5/5] Documentation: document the preferred tag checking mode feature From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210713_164819_246759_83D836FA X-CRM114-Status: GOOD ( 20.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Document the functionality added in the previous patches. Link: https://linux-review.googlesource.com/id/I48217cc3e8b8da33abc08cbaddc11cf4360a1b86 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas Acked-by: Will Deacon Acked-by: Will Deacon --- v10: - document that setting the sysfs node may not take effect immediately - unabbreviate month name v9: - add documentation for sysfs node under Documentation/ABI .../ABI/testing/sysfs-devices-system-cpu | 18 +++++++ .../arm64/memory-tagging-extension.rst | 48 ++++++++++++++++--- 2 files changed, 59 insertions(+), 7 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu index fe13baa53c59..6e667338df9f 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -640,3 +640,21 @@ Description: SPURR ticks for cpuX when it was idle. This sysfs interface exposes the number of SPURR ticks for cpuX when it was idle. + +What: /sys/devices/system/cpu/cpuX/mte_tcf_preferred +Date: July 2021 +Contact: Linux ARM Kernel Mailing list +Description: Preferred MTE tag checking mode + + When a user program specifies more than one MTE tag checking + mode, this sysfs node is used to specify which mode should + be preferred when running on that CPU. Possible values: + + ================ ============================================== + "sync" Prefer synchronous mode + "async" Prefer asynchronous mode + ================ ============================================== + + Changes to this sysfs node may not take effect immediately. + + See also: Documentation/arm64/memory-tagging-extension.rst diff --git a/Documentation/arm64/memory-tagging-extension.rst b/Documentation/arm64/memory-tagging-extension.rst index b540178a93f8..7b99c8f428eb 100644 --- a/Documentation/arm64/memory-tagging-extension.rst +++ b/Documentation/arm64/memory-tagging-extension.rst @@ -77,14 +77,20 @@ configurable behaviours: address is unknown). The user can select the above modes, per thread, using the -``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where -``flags`` contain one of the following values in the ``PR_MTE_TCF_MASK`` +``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where ``flags`` +contains any number of the following values in the ``PR_MTE_TCF_MASK`` bit-field: -- ``PR_MTE_TCF_NONE`` - *Ignore* tag check faults +- ``PR_MTE_TCF_NONE``  - *Ignore* tag check faults + (ignored if combined with other options) - ``PR_MTE_TCF_SYNC`` - *Synchronous* tag check fault mode - ``PR_MTE_TCF_ASYNC`` - *Asynchronous* tag check fault mode +If no modes are specified, tag check faults are ignored. If a single +mode is specified, the program will run in that mode. If multiple +modes are specified, the mode is selected as described in the "Per-CPU +preferred tag checking modes" section below. + The current tag check fault mode can be read using the ``prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0)`` system call. @@ -120,13 +126,39 @@ in the ``PR_MTE_TAG_MASK`` bit-field. interface provides an include mask. An include mask of ``0`` (exclusion mask ``0xffff``) results in the CPU always generating tag ``0``. +Per-CPU preferred tag checking mode +----------------------------------- + +On some CPUs the performance of MTE in stricter tag checking modes +is similar to that of less strict tag checking modes. This makes it +worthwhile to enable stricter checks on those CPUs when a less strict +checking mode is requested, in order to gain the error detection +benefits of the stricter checks without the performance downsides. To +support this scenario, a privileged user may configure a stricter +tag checking mode as the CPU's preferred tag checking mode. + +The preferred tag checking mode for each CPU is controlled by +``/sys/devices/system/cpu/cpu/mte_tcf_preferred``, to which a +privileged user may write the value ``async`` or ``sync``. The default +preferred mode for each CPU is ``async``. + +To allow a program to potentially run in the CPU's preferred tag +checking mode, the user program may set multiple tag check fault mode +bits in the ``flags`` argument to the ``prctl(PR_SET_TAGGED_ADDR_CTRL, +flags, 0, 0, 0)`` system call. If the CPU's preferred tag checking +mode is in the task's set of provided tag checking modes (this will +always be the case at present because the kernel only supports two +tag checking modes, but future kernels may support more modes), that +mode will be selected. Otherwise, one of the modes in the task's mode +set will be selected in a currently unspecified manner. + Initial process state --------------------- On ``execve()``, the new process has the following configuration: - ``PR_TAGGED_ADDR_ENABLE`` set to 0 (disabled) -- Tag checking mode set to ``PR_MTE_TCF_NONE`` +- No tag checking modes are selected (tag check faults ignored) - ``PR_MTE_TAG_MASK`` set to 0 (all tags excluded) - ``PSTATE.TCO`` set to 0 - ``PROT_MTE`` not set on any of the initial memory maps @@ -251,11 +283,13 @@ Example of correct usage return EXIT_FAILURE; /* - * Enable the tagged address ABI, synchronous MTE tag check faults and - * allow all non-zero tags in the randomly generated set. + * Enable the tagged address ABI, synchronous or asynchronous MTE + * tag check faults (based on per-CPU preference) and allow all + * non-zero tags in the randomly generated set. */ if (prctl(PR_SET_TAGGED_ADDR_CTRL, - PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | (0xfffe << PR_MTE_TAG_SHIFT), + PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | PR_MTE_TCF_ASYNC | + (0xfffe << PR_MTE_TAG_SHIFT), 0, 0, 0)) { perror("prctl() failed"); return EXIT_FAILURE;