From patchwork Tue Jul 27 20:52:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12403601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A98E1C432BE for ; Tue, 27 Jul 2021 20:55:26 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 761F06056B for ; Tue, 27 Jul 2021 20:55:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 761F06056B Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=EwjreblxJ6HDwZ10Klc7RmMjXhoabtoxhL+EMXMghj0=; b=bpQIEAnSMASEWWm40QSaAEBUMx 0zJPzzhT3hSX8ck+4uDIb6J0e2QlnYBbsXOtuIJWxYb1r53lX5fiSoi71LLAWydG4rRuPPiTEupPS 63MFNitkxb7fsMfBQvnKu7BoveLWUZnV83eQ8PKt4/t/e3yYDJb2AilOgp+aY11mYZ+JUFn5gODjo k3i9O8ZgIQrJ33A6kbPZgWv9AziNqQ9tgDUeOMJMAabwv73zbylIH9E4m+o3hlcstiJVzEADs4R+M DlNQlH7Xi+NFzgtfCq9pU7C80Iff77Ve64tEwUA76zExEhAJhLvj4emaPSmDzvvNU8piTzlhhExVm Q0ru7Tew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U4p-00GHQh-TX; Tue, 27 Jul 2021 20:53:24 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U4a-00GHNC-O2 for linux-arm-kernel@lists.infradead.org; Tue, 27 Jul 2021 20:53:10 +0000 Received: by mail-qt1-x849.google.com with SMTP id o7-20020ac87c470000b029025f8084df09so7095141qtv.6 for ; Tue, 27 Jul 2021 13:53:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=yXg/FAbxbVRci52SwcGooeYpqQ7qDxcgWG1wcwzKePw=; b=mMuco8UBnG0USvWxtfdfqyt/DdcogL3yMfnSrNJBwIluIYtghAGSCctdQmTm40gsZ8 AJjkRciTaYH0ENXwgSH3eBw9xhwZxJMetoFLNJM94zh9Gka4tfdwcvuGXoGEhk8L/jbn jALmdCVdhGqDIOQJ3Ezyjoos5SuugxmaRTdwaBnqTZyHPekgTorSlHh/FL32D6ocpngd XwtBhfxGa9bdQ/V6n7i/Bts+OYny9wLe7KybxaEX7SN1iyx18pK2zSX3Fyr8+OuTSZ8q spDkpEnpFW+18X9fctrfOED9EsHzeICyayS0NDarCpTRZSVpFPgkXF4ZRuKOI6UjlkBk Qc4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=yXg/FAbxbVRci52SwcGooeYpqQ7qDxcgWG1wcwzKePw=; b=H+L6XNwCjsqyiA2mbzL1uzeUBb3/iGHRjKUkdZ9rQ5+aqa9cKm3aW0Mlkpx+Xbh204 33dN+uRBs17hoTxgc/KlusZfw1WUW30Co1bmFGqZb4cHFkuxnBftVEIyi02DyhKw6RLU Zb1bG5DacNv3fhxZWOyY9Ff9ENTjtPqk6KtiooF4eblqTaSUpbaLj8z6i4DfLGEnTwi9 qJBJpmZzBWtBHSV5+ojoiGbHLHRLc0lkVkoh8HHEsVh+qq0pkL3UUyVjdI1CItiMQv6D 1eu1etM8HH0mf4La+GB5AjPRxqZLzd7UErPj9OOMfTU/9xA98/XXkNHg1pGRLzz2bNUO oraQ== X-Gm-Message-State: AOAM5308rz+bq1ah6qlFumZKxA2To6aCP7RyfoVvwRVkqpR2/plBk44Q /H9NHyVNU5uuotLmTwc/mITCNzw= X-Google-Smtp-Source: ABdhPJwQcRaQSAff5ADA+rH+qOc00nGO+JDMZ3rVQZcP4FLcNYlGcMll1AHgLPBty2mJeZzcnz0c828= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:538:aa52:9b59:413b]) (user=pcc job=sendgmr) by 2002:a0c:eece:: with SMTP id h14mr24780719qvs.12.1627419186744; Tue, 27 Jul 2021 13:53:06 -0700 (PDT) Date: Tue, 27 Jul 2021 13:52:55 -0700 In-Reply-To: <20210727205300.2554659-1-pcc@google.com> Message-Id: <20210727205300.2554659-2-pcc@google.com> Mime-Version: 1.0 References: <20210727205300.2554659-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v11 1/5] arm64: mte: rename gcr_user_excl to mte_ctrl From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210727_135308_828482_E6DCAC94 X-CRM114-Status: GOOD ( 14.49 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We are going to use this field to store more data. To prepare for that, rename it and change the users to rely on the bit position of gcr_user_excl in mte_ctrl. Link: https://linux-review.googlesource.com/id/Ie1fd18e480100655f5d22137f5b22f4f3a9f9e2e Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas --- arch/arm64/include/asm/processor.h | 5 ++++- arch/arm64/kernel/asm-offsets.c | 2 +- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/kernel/mte.c | 14 ++++++++------ 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index b6517fd03d7b..54d34276fa91 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -16,6 +16,9 @@ */ #define NET_IP_ALIGN 0 +#define MTE_CTRL_GCR_USER_EXCL_SHIFT 0 +#define MTE_CTRL_GCR_USER_EXCL_MASK 0xffff + #ifndef __ASSEMBLY__ #include @@ -153,7 +156,7 @@ struct thread_struct { #endif #endif #ifdef CONFIG_ARM64_MTE - u64 gcr_user_excl; + u64 mte_ctrl; #endif u64 sctlr_user; }; diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index c85670692afa..551427ae8cc5 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -52,7 +52,7 @@ int main(void) DEFINE(THREAD_KEYS_KERNEL, offsetof(struct task_struct, thread.keys_kernel)); #endif #ifdef CONFIG_ARM64_MTE - DEFINE(THREAD_GCR_EL1_USER, offsetof(struct task_struct, thread.gcr_user_excl)); + DEFINE(THREAD_MTE_CTRL, offsetof(struct task_struct, thread.mte_ctrl)); #endif BLANK(); DEFINE(S_X0, offsetof(struct pt_regs, regs[0])); diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 863d44f73028..40c3543de3bc 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -167,7 +167,7 @@ alternative_else_nop_endif * the RRND (bit[16]) setting. */ mrs_s \tmp2, SYS_GCR_EL1 - bfi \tmp2, \tmp, #0, #16 + bfxil \tmp2, \tmp, #MTE_CTRL_GCR_USER_EXCL_SHIFT, #16 msr_s SYS_GCR_EL1, \tmp2 #endif .endm @@ -190,7 +190,7 @@ alternative_else_nop_endif alternative_if_not ARM64_MTE b 1f alternative_else_nop_endif - ldr \tmp, [\tsk, #THREAD_GCR_EL1_USER] + ldr \tmp, [\tsk, #THREAD_MTE_CTRL] mte_set_gcr \tmp, \tmp2 1: diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 36f51b0e438a..a5269558210c 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -195,7 +195,7 @@ void mte_check_tfsr_el1(void) static void set_gcr_el1_excl(u64 excl) { - current->thread.gcr_user_excl = excl; + current->thread.mte_ctrl = excl; /* * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value @@ -260,8 +260,8 @@ void mte_suspend_exit(void) long set_mte_ctrl(struct task_struct *task, unsigned long arg) { u64 sctlr = task->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK; - u64 gcr_excl = ~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & - SYS_GCR_EL1_EXCL_MASK; + u64 mte_ctrl = (~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & + SYS_GCR_EL1_EXCL_MASK) << MTE_CTRL_GCR_USER_EXCL_SHIFT; if (!system_supports_mte()) return 0; @@ -282,10 +282,10 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg) if (task != current) { task->thread.sctlr_user = sctlr; - task->thread.gcr_user_excl = gcr_excl; + task->thread.mte_ctrl = mte_ctrl; } else { set_task_sctlr_el1(sctlr); - set_gcr_el1_excl(gcr_excl); + set_gcr_el1_excl(mte_ctrl); } return 0; @@ -294,7 +294,9 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg) long get_mte_ctrl(struct task_struct *task) { unsigned long ret; - u64 incl = ~task->thread.gcr_user_excl & SYS_GCR_EL1_EXCL_MASK; + u64 mte_ctrl = task->thread.mte_ctrl; + u64 incl = (~mte_ctrl >> MTE_CTRL_GCR_USER_EXCL_SHIFT) & + SYS_GCR_EL1_EXCL_MASK; if (!system_supports_mte()) return 0; From patchwork Tue Jul 27 20:52:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12403605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 644D8C4338F for ; Tue, 27 Jul 2021 20:55:52 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2B3DF60F58 for ; Tue, 27 Jul 2021 20:55:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2B3DF60F58 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=2tWx0fLO6DWCMzjqRB4QXtDAZ7DIheU+HJHj8GpFhfo=; b=OZuq1W0CB2L67y83w1/2lTLVjy +pcOQHIDSm4uNT2zEw0mTbUjUftiUJuRSES+sp6bctcIJ4l8CcM0g5LzzbkQy0aOc5w+r2c8rqwwD 4O7PDrwudTi4OQaRFHk7MGi6Yj2EhdrSjXEOo/bYnBvOFHEiXcCZBDoleErpRX+o9f/tQlfpY1O9A 09mTXvBSbn3R86qNyF2HZLVSppvSqisNIwI7Xyb6AGw6x9ajrLjVYR2R99zkG+0MT7eFwRYgJRef6 VdlDaY/rHYM3XMnzhWfPgzZgSeB4ToaZBKopLrVIuz3W48i/LnzYEUpB9dcvmtI1J24n/mP0+S2dk PBaqks0A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U53-00GHU0-7U; Tue, 27 Jul 2021 20:53:37 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U4d-00GHNs-69 for linux-arm-kernel@lists.infradead.org; Tue, 27 Jul 2021 20:53:13 +0000 Received: by mail-qt1-x849.google.com with SMTP id f9-20020a05622a1a09b02902615523e725so7025989qtb.21 for ; Tue, 27 Jul 2021 13:53:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=qu+LsgaEBL/StVjS/BKSrk9Fn9pqlWBDGtSwYs/UX0Y=; b=Mjqb1ic6gsBXSbgdBSK61gdsNz0Q1B7GnJeNJ2a1F17GfUk5EP7rz4CN9f/vi9ijxX 0NapFxLflyBnE/Sg7hLDUCyjsaG8bg7/Wxu4saytk4qKb1jrLep3RhWMBF3zbKjFPAiW vSrGavdGDgNkXjAPncl3YYGRGryH/iM6JVrdcyfLl0O+bzawvcrb4Jg+WUIAQbFMSBJf bSzoBFnRV9TLp4iFCtvqq8Z/5v6WPEwuOPaM4PMDa3zbOzpiPjsTgV89dXUhZBalYF7k 27G1bbgeVyDfE5zBK6ACaSxMp+dcxaaM0I47J51Z0sho5zwt6AQeZFUPIQOcCGugMNm+ uAiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=qu+LsgaEBL/StVjS/BKSrk9Fn9pqlWBDGtSwYs/UX0Y=; b=XOXmwZF4z22yAEdIuFApypidb7L8Jdg3AmPIRkKWsvnqbM6VCDVomWYOVKYjHEZAzz PblLMWevT9TIL+yZaVVbBGY13JXEIxpjm5lGRsC+7H6hTbQUkRommLTcHddn3n/53XKE ioLJOefnaz82SQnW8Klh7LISF3Dg0leqWAAyWWWacRhuDsGn4v8Hx/ujJ5I45PND8v1D RN9ATOg2QSAfc6G1X7qPT92Qxkyp89vUngqiRBgZbevBcVfLNtSdFcvJ+1Kd8NvX3/tV px1U+g8FMRcF0IDqtFm9gaz0A0hZZWEj4N39kGUFNDLP+wiw9Q5hNF7U2m1wMj6jQPQS vWzA== X-Gm-Message-State: AOAM531uCcOexIhKgddPp4zUhPk0LujMdbZnxGMU3NHQBG2uVHk1a2dA n+poyl2VlQdm5tCgx9WJHkBNXOA= X-Google-Smtp-Source: ABdhPJxpTZ9feyZbBqzvjWKMnFK3ulazMyFdlDhaTA+nsK3zlAZts/j1mVyczPYTqw0HtFmXtqZbr4k= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:538:aa52:9b59:413b]) (user=pcc job=sendgmr) by 2002:a05:6214:9cf:: with SMTP id dp15mr6503465qvb.3.1627419189312; Tue, 27 Jul 2021 13:53:09 -0700 (PDT) Date: Tue, 27 Jul 2021 13:52:56 -0700 In-Reply-To: <20210727205300.2554659-1-pcc@google.com> Message-Id: <20210727205300.2554659-3-pcc@google.com> Mime-Version: 1.0 References: <20210727205300.2554659-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v11 2/5] arm64: mte: change ASYNC and SYNC TCF settings into bitfields From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210727_135311_309405_4F7B2D6E X-CRM114-Status: GOOD ( 16.21 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow the user program to specify both ASYNC and SYNC TCF modes by repurposing the existing constants as bitfields. This will allow the kernel to select one of the modes on behalf of the user program. With this patch the kernel will always select async mode, but a subsequent patch will make this configurable. Link: https://linux-review.googlesource.com/id/Icc5923c85a8ea284588cc399ae74fd19ec291230 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas --- v11: - rebased to 5.14rc3 v9: - make mte_update_sctlr_user static arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/kernel/mte.c | 70 ++++++++++++------------------ include/uapi/linux/prctl.h | 11 ++--- 3 files changed, 37 insertions(+), 47 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 54d34276fa91..ee82ebbb5e5a 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -19,6 +19,9 @@ #define MTE_CTRL_GCR_USER_EXCL_SHIFT 0 #define MTE_CTRL_GCR_USER_EXCL_MASK 0xffff +#define MTE_CTRL_TCF_SYNC (1UL << 16) +#define MTE_CTRL_TCF_ASYNC (1UL << 17) + #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index a5269558210c..3b6b68518003 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -193,14 +193,19 @@ void mte_check_tfsr_el1(void) } #endif -static void set_gcr_el1_excl(u64 excl) +static void mte_update_sctlr_user(struct task_struct *task) { - current->thread.mte_ctrl = excl; + unsigned long sctlr = task->thread.sctlr_user; + unsigned long pref = MTE_CTRL_TCF_ASYNC; + unsigned long mte_ctrl = task->thread.mte_ctrl; + unsigned long resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; - /* - * SYS_GCR_EL1 will be set to current->thread.gcr_user_excl value - * by mte_set_user_gcr() in kernel_exit, - */ + sctlr &= ~SCTLR_EL1_TCF0_MASK; + if (resolved_mte_tcf & MTE_CTRL_TCF_ASYNC) + sctlr |= SCTLR_EL1_TCF0_ASYNC; + else if (resolved_mte_tcf & MTE_CTRL_TCF_SYNC) + sctlr |= SCTLR_EL1_TCF0_SYNC; + task->thread.sctlr_user = sctlr; } void mte_thread_init_user(void) @@ -212,15 +217,16 @@ void mte_thread_init_user(void) dsb(ish); write_sysreg_s(0, SYS_TFSRE0_EL1); clear_thread_flag(TIF_MTE_ASYNC_FAULT); - /* disable tag checking */ - set_task_sctlr_el1((current->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK) | - SCTLR_EL1_TCF0_NONE); - /* reset tag generation mask */ - set_gcr_el1_excl(SYS_GCR_EL1_EXCL_MASK); + /* disable tag checking and reset tag generation mask */ + current->thread.mte_ctrl = MTE_CTRL_GCR_USER_EXCL_MASK; + mte_update_sctlr_user(current); + set_task_sctlr_el1(current->thread.sctlr_user); } void mte_thread_switch(struct task_struct *next) { + mte_update_sctlr_user(next); + /* * Check if an async tag exception occurred at EL1. * @@ -259,33 +265,21 @@ void mte_suspend_exit(void) long set_mte_ctrl(struct task_struct *task, unsigned long arg) { - u64 sctlr = task->thread.sctlr_user & ~SCTLR_EL1_TCF0_MASK; u64 mte_ctrl = (~((arg & PR_MTE_TAG_MASK) >> PR_MTE_TAG_SHIFT) & SYS_GCR_EL1_EXCL_MASK) << MTE_CTRL_GCR_USER_EXCL_SHIFT; if (!system_supports_mte()) return 0; - switch (arg & PR_MTE_TCF_MASK) { - case PR_MTE_TCF_NONE: - sctlr |= SCTLR_EL1_TCF0_NONE; - break; - case PR_MTE_TCF_SYNC: - sctlr |= SCTLR_EL1_TCF0_SYNC; - break; - case PR_MTE_TCF_ASYNC: - sctlr |= SCTLR_EL1_TCF0_ASYNC; - break; - default: - return -EINVAL; - } + if (arg & PR_MTE_TCF_ASYNC) + mte_ctrl |= MTE_CTRL_TCF_ASYNC; + if (arg & PR_MTE_TCF_SYNC) + mte_ctrl |= MTE_CTRL_TCF_SYNC; - if (task != current) { - task->thread.sctlr_user = sctlr; - task->thread.mte_ctrl = mte_ctrl; - } else { - set_task_sctlr_el1(sctlr); - set_gcr_el1_excl(mte_ctrl); + task->thread.mte_ctrl = mte_ctrl; + if (task == current) { + mte_update_sctlr_user(task); + set_task_sctlr_el1(task->thread.sctlr_user); } return 0; @@ -302,18 +296,10 @@ long get_mte_ctrl(struct task_struct *task) return 0; ret = incl << PR_MTE_TAG_SHIFT; - - switch (task->thread.sctlr_user & SCTLR_EL1_TCF0_MASK) { - case SCTLR_EL1_TCF0_NONE: - ret |= PR_MTE_TCF_NONE; - break; - case SCTLR_EL1_TCF0_SYNC: - ret |= PR_MTE_TCF_SYNC; - break; - case SCTLR_EL1_TCF0_ASYNC: + if (mte_ctrl & MTE_CTRL_TCF_ASYNC) ret |= PR_MTE_TCF_ASYNC; - break; - } + if (mte_ctrl & MTE_CTRL_TCF_SYNC) + ret |= PR_MTE_TCF_SYNC; return ret; } diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h index 967d9c55323d..89de78a14b9b 100644 --- a/include/uapi/linux/prctl.h +++ b/include/uapi/linux/prctl.h @@ -234,14 +234,15 @@ struct prctl_mm_map { #define PR_GET_TAGGED_ADDR_CTRL 56 # define PR_TAGGED_ADDR_ENABLE (1UL << 0) /* MTE tag check fault modes */ -# define PR_MTE_TCF_SHIFT 1 -# define PR_MTE_TCF_NONE (0UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_SYNC (1UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_ASYNC (2UL << PR_MTE_TCF_SHIFT) -# define PR_MTE_TCF_MASK (3UL << PR_MTE_TCF_SHIFT) +# define PR_MTE_TCF_NONE 0 +# define PR_MTE_TCF_SYNC (1UL << 1) +# define PR_MTE_TCF_ASYNC (1UL << 2) +# define PR_MTE_TCF_MASK (PR_MTE_TCF_SYNC | PR_MTE_TCF_ASYNC) /* MTE tag inclusion mask */ # define PR_MTE_TAG_SHIFT 3 # define PR_MTE_TAG_MASK (0xffffUL << PR_MTE_TAG_SHIFT) +/* Unused; kept only for source compatibility */ +# define PR_MTE_TCF_SHIFT 1 /* Control reclaim behavior when allocating memory */ #define PR_SET_IO_FLUSHER 57 From patchwork Tue Jul 27 20:52:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12403607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 609B1C4338F for ; Tue, 27 Jul 2021 20:55:56 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2BC1560560 for ; Tue, 27 Jul 2021 20:55:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2BC1560560 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=Oh6B/g5dXL3h7xTCTF2j66rjPPG1eeMFx/f3iav+YzQ=; b=TBm3t4rtlj8dgwk8jTLMMeyxR6 xAg7PTtnYzsr4dAsLvSpMk0jfD8rHM8vCarwuwZnMjUy+W+J2uiowvGSk1BQUnEyCEfM+ij/nT6+e Qxoc88PjvESmRvNwhqeqWp8fa5E2CHkWoNrGMKLJQiCipkH+kS3taEIg5WTnurcljkRuuXndfeoLV LI0jAp8osSfolnCa9Y+us4ayGGhcn/llUFKpxmKOy2VD90fZDEqW0qHYL1qChwO6VxZrQNtjQP97d /EnDx2IhZgIwta1AUxf+EzxPdzLO/u/Ls/yV6Z4/DFTrQiT+RRBVaQtvWSXpwImpnCBViuZavkd0r Zx6AVECQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U5H-00GHXO-40; Tue, 27 Jul 2021 20:53:51 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U4f-00GHOU-2z for linux-arm-kernel@lists.infradead.org; Tue, 27 Jul 2021 20:53:15 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 16-20020a250b100000b029055791ebe1e6so80095ybl.20 for ; Tue, 27 Jul 2021 13:53:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Z+0smVhm008lpRiabvaPvv6cJ4DYUaPSjkdR8POxKrg=; b=MKaj1lp2G2FuRq4OgEmdUn1Gi9vmtLuDK5Priw8ZFGv9eiCJV07hsLXisxoQBogYtx AnLweqvvudQnK5FdLYd7gLpMA4cEFWmJHlf4jnKq93izOGU57y5d8bJsPjjjIDRiI8Sf IvgmFJbw6sZ4HwnKTTceZ1nuXT9JoI8Nhz49uip8pjuCtCy2QlLTDrC9VS0Z5YnV4Dl1 XJUpmNBmwOZqGmJYmKSf2YCP6uLIuQ0hi9fankBfIafSiVAW1CJMdhwoSPhuSqYZ8EEl sJEOEONJnuqF51Te5qq7cbRmmdVIahNwViRJUeTGXLbna38FWTvK/a2f9N+sAln7oI5p Tjuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Z+0smVhm008lpRiabvaPvv6cJ4DYUaPSjkdR8POxKrg=; b=NAYc+sC1tCgDURA6X6+kVclimGRn8qXVpL5pQPuwkEOy/y0rqd7o8KpfcfVlaxxIN+ 3Epjnp2oxPxf7+R+yZ8NDtgnyNOcxVQtplc7dFBC5iwZZb8N5o0kVQdD6xboqe8lo3N5 yIprvWgAu8fNsi9un9YINGoW054CFEyOBMQeT4XKuF0dHtPIpHBMmcq3UITgjttgWoFu iXywhdu+YxVkypGqMQuHLk3l4YrPgY1lsikdmG04HxUH2TAEyNuuQzL8ZcAMGv5CrlSj YmXC03fyyIBXP/qQRnX1dxzoFlWKK/rWcdP5Q/JhXKGizNgpYFMVzakU4wiDX7h12k9v 0Yzg== X-Gm-Message-State: AOAM533VAJi5Lty6wzzF3khKPKsr/G1AR+wCWcld1UZ0bAimSvarz4dn MOYSZvpWKHNUm2+H1eUXGyuL7O0= X-Google-Smtp-Source: ABdhPJyKD0IsOt9AdshSEGNyASvjFwSBmq4v4aUE28EGH0HVfEIspRNJjTFRI3WN5k3Oj2179UkrL9c= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:538:aa52:9b59:413b]) (user=pcc job=sendgmr) by 2002:a25:3103:: with SMTP id x3mr33527373ybx.8.1627419191685; Tue, 27 Jul 2021 13:53:11 -0700 (PDT) Date: Tue, 27 Jul 2021 13:52:57 -0700 In-Reply-To: <20210727205300.2554659-1-pcc@google.com> Message-Id: <20210727205300.2554659-4-pcc@google.com> Mime-Version: 1.0 References: <20210727205300.2554659-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v11 3/5] arm64: move preemption disablement to prctl handlers From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210727_135313_248349_F6393E03 X-CRM114-Status: GOOD ( 22.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In the next patch, we will start reading sctlr_user from mte_update_sctlr_user and subsequently writing a new value based on the task's TCF setting and potentially the per-CPU TCF preference. This means that we need to be careful to disable preemption around any code sequences that read from sctlr_user and subsequently write to sctlr_user and/or SCTLR_EL1, so that we don't end up writing a stale value (based on the previous CPU's TCF preference) to either of them. We currently have four such sequences, in the prctl handlers for PR_SET_TAGGED_ADDR_CTRL and PR_PAC_SET_ENABLED_KEYS, as well as in the task initialization code that resets the prctl settings. Change the prctl handlers to disable preemption in the handlers themselves rather than the functions that they call, and change the task initialization code to call the respective prctl handlers instead of setting sctlr_user directly. As a result of this change, we no longer need the helper function set_task_sctlr_el1, nor does its behavior make sense any more, so remove it. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ic0e8a0c00bb47d786c1e8011df0b7fe99bee4bb5 --- arch/arm64/include/asm/pointer_auth.h | 12 ++++++------ arch/arm64/include/asm/processor.h | 2 +- arch/arm64/kernel/mte.c | 8 ++++---- arch/arm64/kernel/pointer_auth.c | 10 ++++++---- arch/arm64/kernel/process.c | 21 +++++++-------------- 5 files changed, 24 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/pointer_auth.h b/arch/arm64/include/asm/pointer_auth.h index 28a78b67d9b4..efb098de3a84 100644 --- a/arch/arm64/include/asm/pointer_auth.h +++ b/arch/arm64/include/asm/pointer_auth.h @@ -10,6 +10,9 @@ #include #include +#define PR_PAC_ENABLED_KEYS_MASK \ + (PR_PAC_APIAKEY | PR_PAC_APIBKEY | PR_PAC_APDAKEY | PR_PAC_APDBKEY) + #ifdef CONFIG_ARM64_PTR_AUTH /* * Each key is a 128-bit quantity which is split across a pair of 64-bit @@ -117,9 +120,9 @@ static __always_inline void ptrauth_enable(void) \ /* enable all keys */ \ if (system_supports_address_auth()) \ - set_task_sctlr_el1(current->thread.sctlr_user | \ - SCTLR_ELx_ENIA | SCTLR_ELx_ENIB | \ - SCTLR_ELx_ENDA | SCTLR_ELx_ENDB); \ + ptrauth_set_enabled_keys(current, \ + PR_PAC_ENABLED_KEYS_MASK, \ + PR_PAC_ENABLED_KEYS_MASK); \ } while (0) #define ptrauth_thread_switch_user(tsk) \ @@ -146,7 +149,4 @@ static __always_inline void ptrauth_enable(void) #define ptrauth_thread_switch_kernel(tsk) #endif /* CONFIG_ARM64_PTR_AUTH_KERNEL */ -#define PR_PAC_ENABLED_KEYS_MASK \ - (PR_PAC_APIAKEY | PR_PAC_APIBKEY | PR_PAC_APDAKEY | PR_PAC_APDBKEY) - #endif /* __ASM_POINTER_AUTH_H */ diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index ee82ebbb5e5a..ee2bdc1b9f5b 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -259,7 +259,7 @@ extern void release_thread(struct task_struct *); unsigned long get_wchan(struct task_struct *p); -void set_task_sctlr_el1(u64 sctlr); +void update_sctlr_el1(u64 sctlr); /* Thread switching */ extern struct task_struct *cpu_switch_to(struct task_struct *prev, diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 3b6b68518003..737503a148bb 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -218,9 +218,7 @@ void mte_thread_init_user(void) write_sysreg_s(0, SYS_TFSRE0_EL1); clear_thread_flag(TIF_MTE_ASYNC_FAULT); /* disable tag checking and reset tag generation mask */ - current->thread.mte_ctrl = MTE_CTRL_GCR_USER_EXCL_MASK; - mte_update_sctlr_user(current); - set_task_sctlr_el1(current->thread.sctlr_user); + set_mte_ctrl(current, 0); } void mte_thread_switch(struct task_struct *next) @@ -278,8 +276,10 @@ long set_mte_ctrl(struct task_struct *task, unsigned long arg) task->thread.mte_ctrl = mte_ctrl; if (task == current) { + preempt_disable(); mte_update_sctlr_user(task); - set_task_sctlr_el1(task->thread.sctlr_user); + update_sctlr_el1(task->thread.sctlr_user); + preempt_enable(); } return 0; diff --git a/arch/arm64/kernel/pointer_auth.c b/arch/arm64/kernel/pointer_auth.c index 60901ab0a7fe..2708b620b4ae 100644 --- a/arch/arm64/kernel/pointer_auth.c +++ b/arch/arm64/kernel/pointer_auth.c @@ -67,7 +67,7 @@ static u64 arg_to_enxx_mask(unsigned long arg) int ptrauth_set_enabled_keys(struct task_struct *tsk, unsigned long keys, unsigned long enabled) { - u64 sctlr = tsk->thread.sctlr_user; + u64 sctlr; if (!system_supports_address_auth()) return -EINVAL; @@ -78,12 +78,14 @@ int ptrauth_set_enabled_keys(struct task_struct *tsk, unsigned long keys, if ((keys & ~PR_PAC_ENABLED_KEYS_MASK) || (enabled & ~keys)) return -EINVAL; + preempt_disable(); + sctlr = tsk->thread.sctlr_user; sctlr &= ~arg_to_enxx_mask(keys); sctlr |= arg_to_enxx_mask(enabled); + tsk->thread.sctlr_user = sctlr; if (tsk == current) - set_task_sctlr_el1(sctlr); - else - tsk->thread.sctlr_user = sctlr; + update_sctlr_el1(sctlr); + preempt_enable(); return 0; } diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index c8989b999250..8811c303608b 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -477,7 +477,13 @@ static void compat_thread_switch(struct task_struct *next) set_tsk_thread_flag(next, TIF_NOTIFY_RESUME); } -static void update_sctlr_el1(u64 sctlr) +/* + * __switch_to() checks current->thread.sctlr_user as an optimisation. Therefore + * this function must be called with preemption disabled and the update to + * sctlr_user must be made in the same preemption disabled block so that + * __switch_to() does not see the variable update before the SCTLR_EL1 one. + */ +void update_sctlr_el1(u64 sctlr) { /* * EnIA must not be cleared while in the kernel as this is necessary for @@ -489,19 +495,6 @@ static void update_sctlr_el1(u64 sctlr) isb(); } -void set_task_sctlr_el1(u64 sctlr) -{ - /* - * __switch_to() checks current->thread.sctlr as an - * optimisation. Disable preemption so that it does not see - * the variable update before the SCTLR_EL1 one. - */ - preempt_disable(); - current->thread.sctlr_user = sctlr; - update_sctlr_el1(sctlr); - preempt_enable(); -} - /* * Thread switching. */ From patchwork Tue Jul 27 20:52:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12403609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88530C4338F for ; Tue, 27 Jul 2021 20:56:02 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 503AC60F5E for ; Tue, 27 Jul 2021 20:56:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 503AC60F5E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qRBNXD9EP8jHYmFIVQ2aHgbWeNN8zaHWz3Skc+HV9V0=; b=XzKoHsWvMYQmHb3LetmaQcLksE d2PY9Ea9E8Zvqwc6u0b59OrBePUwXGBKgBdhLZCRJxYRdd/91SS9Ro+Kb5evGTUFnO2NODMwKEI4t Dy4Q4WcwiHMvJN/YaYqjMegdE+g1bCWLdDy1V/540ygLr1Hfguj494nG8hrLU0T6CM9v0PIFDuN0K XJK4CURKhYogb0JW7h+ncx6WKpYV5aspDuuOc0bR9rLCrU0/T98hl5R0sBAyLSVhgqXwnmVlpVI4q l8pGgZfOit9xHtuPYJE6GSutuIZQc+r2tLt3loPkamlVq+4112kSqvBd0RxLeqSwquItA4sYUGNK7 +DO0pmow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U5Y-00GHdn-Fo; Tue, 27 Jul 2021 20:54:08 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U4i-00GHP1-3Z for linux-arm-kernel@lists.infradead.org; Tue, 27 Jul 2021 20:53:17 +0000 Received: by mail-qv1-xf49.google.com with SMTP id b6-20020a0cbf460000b02902dbb4e0a8f2so471760qvj.6 for ; Tue, 27 Jul 2021 13:53:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=566AaQu3N8mRJNv52SrYwj6schntyDwnGY9YvqGFQzA=; b=M/gbFRQHbMINPLpi6SnM8hBI5JlqLq16u+EhMhT0DtG60pJCm4Nv63aar8kObCZKRB bPV8La0TJoHe9lAYnmUwsr3w7kpNxHUooBIddqUQiCBRx3dXjAJ8GR0q44c3bkkJMuLp dJTfdLFw1RqVtIpq2LZzn/0ojjds0xGsCY4887U08UYqR8Ae/oD4D5nXRxA2ky6XUCnw ZlAnG15Qu98UgqXbYSv9feT19FDcBMBwTb0B79g9ApavoXR5fzoGDi4dO+cQLtpjT6js bSux9mxXBfx5MC8He2JH6nbnUhWODuWW30Ip7LCP1jkU8iNhKILPfjb9DJ0n1y8roX3g o8bA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=566AaQu3N8mRJNv52SrYwj6schntyDwnGY9YvqGFQzA=; b=Q1BXgr5HtSdk+AzGn3Nudl8dX5fmMF+0R7yP8PwxgyoJrw/7IJ77v47P5fkL/UBoI6 EGGQVqc/jpHU+wRdulYCgWeU42+ofsOB1SinMhvxHq1oYR3NUGSDeJSuqRIa81vsWbLJ e/SpLQ59luJjByiGl/hALRRALmUpbZMxV8wext+vjvrL9TQnBF3URhmakIbTbqSPMHCz 4v55FhBXv+nIFgSVPvBL25Ug2m5UC+4m4wORjHnYVJ/OzyiJnkcA4pF9rhmGfW4uI29Q HuSAP+uH9e+Xl5DhkylhmFbrOzXiR71B/uKt2TxEuLPcyB+CwNlUpyKeBOxv5lkYR9/G tCzA== X-Gm-Message-State: AOAM5321vbj1OTZPqSM6MTXqJ3yWCVOD/XlHvSC7VcoQBYqNEGSrxleY 3uMqs879jNryPQPFHe3EVuT//6s= X-Google-Smtp-Source: ABdhPJxwc3E/g+pb2PdxFhFyh/9Jwx1bbvTksAScFS0jXW2WLeyljjvZ1xx2OYUE+uS2tlfd1igxQ54= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:538:aa52:9b59:413b]) (user=pcc job=sendgmr) by 2002:a05:6214:262f:: with SMTP id gv15mr24566037qvb.36.1627419194117; Tue, 27 Jul 2021 13:53:14 -0700 (PDT) Date: Tue, 27 Jul 2021 13:52:58 -0700 In-Reply-To: <20210727205300.2554659-1-pcc@google.com> Message-Id: <20210727205300.2554659-5-pcc@google.com> Mime-Version: 1.0 References: <20210727205300.2554659-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v11 4/5] arm64: mte: introduce a per-CPU tag checking mode preference From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210727_135316_192894_9A18D73F X-CRM114-Status: GOOD ( 16.01 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a per-CPU sysfs node, mte_tcf_preferred, that allows the preferred tag checking mode to be configured. The current possible values are async and sync. Link: https://linux-review.googlesource.com/id/I7493dcd533a2785a1437b16c3f6b50919f840854 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas --- v10: - stop calling smp_call_function_single arch/arm64/kernel/mte.c | 65 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 63 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/mte.c b/arch/arm64/kernel/mte.c index 737503a148bb..20a40e5a14fa 100644 --- a/arch/arm64/kernel/mte.c +++ b/arch/arm64/kernel/mte.c @@ -4,6 +4,7 @@ */ #include +#include #include #include #include @@ -26,6 +27,8 @@ u64 gcr_kernel_excl __ro_after_init; static bool report_fault_once = true; +static DEFINE_PER_CPU_READ_MOSTLY(u64, mte_tcf_preferred); + #ifdef CONFIG_KASAN_HW_TAGS /* Whether the MTE asynchronous mode is enabled. */ DEFINE_STATIC_KEY_FALSE(mte_async_mode); @@ -195,11 +198,18 @@ void mte_check_tfsr_el1(void) static void mte_update_sctlr_user(struct task_struct *task) { + /* + * This must be called with preemption disabled and can only be called + * on the current or next task since the CPU must match where the thread + * is going to run. The caller is responsible for calling + * update_sctlr_el1() later in the same preemption disabled block. + */ unsigned long sctlr = task->thread.sctlr_user; - unsigned long pref = MTE_CTRL_TCF_ASYNC; unsigned long mte_ctrl = task->thread.mte_ctrl; - unsigned long resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; + unsigned long pref, resolved_mte_tcf; + pref = __this_cpu_read(mte_tcf_preferred); + resolved_mte_tcf = (mte_ctrl & pref) ? pref : mte_ctrl; sctlr &= ~SCTLR_EL1_TCF0_MASK; if (resolved_mte_tcf & MTE_CTRL_TCF_ASYNC) sctlr |= SCTLR_EL1_TCF0_ASYNC; @@ -438,3 +448,54 @@ int mte_ptrace_copy_tags(struct task_struct *child, long request, return ret; } + +static ssize_t mte_tcf_preferred_show(struct device *dev, + struct device_attribute *attr, char *buf) +{ + switch (per_cpu(mte_tcf_preferred, dev->id)) { + case MTE_CTRL_TCF_ASYNC: + return sysfs_emit(buf, "async\n"); + case MTE_CTRL_TCF_SYNC: + return sysfs_emit(buf, "sync\n"); + default: + return sysfs_emit(buf, "???\n"); + } +} + +static ssize_t mte_tcf_preferred_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + u64 tcf; + + if (sysfs_streq(buf, "async")) + tcf = MTE_CTRL_TCF_ASYNC; + else if (sysfs_streq(buf, "sync")) + tcf = MTE_CTRL_TCF_SYNC; + else + return -EINVAL; + + device_lock(dev); + per_cpu(mte_tcf_preferred, dev->id) = tcf; + device_unlock(dev); + + return count; +} +static DEVICE_ATTR_RW(mte_tcf_preferred); + +static int register_mte_tcf_preferred_sysctl(void) +{ + unsigned int cpu; + + if (!system_supports_mte()) + return 0; + + for_each_possible_cpu(cpu) { + per_cpu(mte_tcf_preferred, cpu) = MTE_CTRL_TCF_ASYNC; + device_create_file(get_cpu_device(cpu), + &dev_attr_mte_tcf_preferred); + } + + return 0; +} +subsys_initcall(register_mte_tcf_preferred_sysctl); From patchwork Tue Jul 27 20:52:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 12403611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D64CC432BE for ; Tue, 27 Jul 2021 20:56:49 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2D10560F6D for ; Tue, 27 Jul 2021 20:56:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2D10560F6D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=DQIeECtDia5eof29KwC9Bn+9QXFf/0Sd0OhIuYNverI=; b=qWIyW8zH2CMGt9tWrpqG/E1mpe MiHekkWfNOEFSNCKssPbVp+wz4o0MrsI7T+od79bX97zxITGHbjnr/h9dtH2XRpadySbXeNaLnKK+ G9zfEaXOYxEI+2BpatQR2D7/2baEXJiHRy0RyQ+6uyZVpTRXx61bQ5LRd2ztNKD0Vn5KnwRbPsD7g +qzDyBVGseqh26H4KqLNJKasHr2Eehbg+Y0pPNZXM6OcOBDChRBKWtojG9dq187onZHFi8rtWCj/1 TUMXogtRKdPEWr6FIr4LwaFSqizd4S6igXVixKtoP5q5xnQkXNdkYKYWBL/8acss7l5+5wruMTUPC bzMND8zw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U5t-00GHnC-Ub; Tue, 27 Jul 2021 20:54:30 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1m8U4k-00GHPU-Ib for linux-arm-kernel@lists.infradead.org; Tue, 27 Jul 2021 20:53:20 +0000 Received: by mail-qt1-x84a.google.com with SMTP id t35-20020a05622a1823b02902647b518455so7113848qtc.3 for ; Tue, 27 Jul 2021 13:53:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=h+YPtVrbGHql7p3uOrRuuEPAAUJCjempTXBjVwsVAyE=; b=HKq8umQs3qGLSDY9AGscR7rKHLGDCFh0WEKUWhf7+nNw+1nZXoDHxFVVwjRrMvw8z5 BBTPDiVq1kn5CvZtDH22DqMoZRPvmO7l0xxb5yzSupE5Ro5/OvvTbMZhprxHAsX1dnCM HZOgJBqdD+KQiiU4hnbCs69tcJAfnEH0kQLZsr+exh8i4dOQfXxgMHnY52tyrMSDi900 jOl1pxgHKOmVURsU4epvom7iCNQDPCObO52XE0UNq7uGACKvWxrfFE2I/4SvTxvAeOXU k9ONuAft5W9EXBAc+ht71k8kPCJ2SMJ5pNDcihxt1lKUMTpDMbub07bZoYADjVX4+/cs YklQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=h+YPtVrbGHql7p3uOrRuuEPAAUJCjempTXBjVwsVAyE=; b=rnZpvXdCCi8WRlH29mGlIifCwchYA8PZMdQ+aluF+3UUtZwClvqZindiA0Kx4yOYLw 7jQWDkGPfV5/XgBMocRdd/mamVOnMSwKhIdt0IZCj0nUFJYhoSycXK9ADXR2mqJ9dTRK Z5tSgyUWxOjgKEd+iAISaig8Ft9Xm/zpaRIuCm2URgaYtYHi8kkrpFT8KgxT3XClTLm1 7x8+BKleyZRoCRpAL5yMTQN4U+H9orbf3zqE3y+lPz9/vzlMSt8pVF2kx2ktghLsK4EL UOEg3olF7VBLYZGkKw/oh+OYl07d9BgqxEKQHMTzTJxTShOJ2YvE/6Q04rVifDc0rx7n 6cWA== X-Gm-Message-State: AOAM5335diJVG2oXTF/Avjyxz3G3aQmm9m/+3nvVRAc0x6d2xqLNtaf2 Nm2vKKwQ+AAkxngY/4EgbDmHIWI= X-Google-Smtp-Source: ABdhPJxbkbCu6E3bdAuu8WzkN4wKBZNR5QAMXmkbKe8ywRsgXXWbXvvwd5sDEcLuTUv2ATZGzJ3fI9c= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:200:538:aa52:9b59:413b]) (user=pcc job=sendgmr) by 2002:a05:6214:1cb:: with SMTP id c11mr25033908qvt.47.1627419196576; Tue, 27 Jul 2021 13:53:16 -0700 (PDT) Date: Tue, 27 Jul 2021 13:52:59 -0700 In-Reply-To: <20210727205300.2554659-1-pcc@google.com> Message-Id: <20210727205300.2554659-6-pcc@google.com> Mime-Version: 1.0 References: <20210727205300.2554659-1-pcc@google.com> X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH v11 5/5] Documentation: document the preferred tag checking mode feature From: Peter Collingbourne To: Catalin Marinas , Vincenzo Frascino , Will Deacon Cc: Peter Collingbourne , Evgenii Stepanov , Szabolcs Nagy , Tejas Belagod , linux-arm-kernel@lists.infradead.org, Greg Kroah-Hartman X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210727_135318_669499_BFD888BC X-CRM114-Status: GOOD ( 20.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Document the functionality added in the previous patches. Link: https://linux-review.googlesource.com/id/I48217cc3e8b8da33abc08cbaddc11cf4360a1b86 Signed-off-by: Peter Collingbourne Reviewed-by: Catalin Marinas Acked-by: Will Deacon --- v10: - document that setting the sysfs node may not take effect immediately - unabbreviate month name v9: - add documentation for sysfs node under Documentation/ABI .../ABI/testing/sysfs-devices-system-cpu | 18 +++++++ .../arm64/memory-tagging-extension.rst | 48 ++++++++++++++++--- 2 files changed, 59 insertions(+), 7 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu index 160b10c029c0..5f87b146deb9 100644 --- a/Documentation/ABI/testing/sysfs-devices-system-cpu +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu @@ -640,3 +640,21 @@ Description: SPURR ticks for cpuX when it was idle. This sysfs interface exposes the number of SPURR ticks for cpuX when it was idle. + +What: /sys/devices/system/cpu/cpuX/mte_tcf_preferred +Date: July 2021 +Contact: Linux ARM Kernel Mailing list +Description: Preferred MTE tag checking mode + + When a user program specifies more than one MTE tag checking + mode, this sysfs node is used to specify which mode should + be preferred when running on that CPU. Possible values: + + ================ ============================================== + "sync" Prefer synchronous mode + "async" Prefer asynchronous mode + ================ ============================================== + + Changes to this sysfs node may not take effect immediately. + + See also: Documentation/arm64/memory-tagging-extension.rst diff --git a/Documentation/arm64/memory-tagging-extension.rst b/Documentation/arm64/memory-tagging-extension.rst index b540178a93f8..7b99c8f428eb 100644 --- a/Documentation/arm64/memory-tagging-extension.rst +++ b/Documentation/arm64/memory-tagging-extension.rst @@ -77,14 +77,20 @@ configurable behaviours: address is unknown). The user can select the above modes, per thread, using the -``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where -``flags`` contain one of the following values in the ``PR_MTE_TCF_MASK`` +``prctl(PR_SET_TAGGED_ADDR_CTRL, flags, 0, 0, 0)`` system call where ``flags`` +contains any number of the following values in the ``PR_MTE_TCF_MASK`` bit-field: -- ``PR_MTE_TCF_NONE`` - *Ignore* tag check faults +- ``PR_MTE_TCF_NONE``  - *Ignore* tag check faults + (ignored if combined with other options) - ``PR_MTE_TCF_SYNC`` - *Synchronous* tag check fault mode - ``PR_MTE_TCF_ASYNC`` - *Asynchronous* tag check fault mode +If no modes are specified, tag check faults are ignored. If a single +mode is specified, the program will run in that mode. If multiple +modes are specified, the mode is selected as described in the "Per-CPU +preferred tag checking modes" section below. + The current tag check fault mode can be read using the ``prctl(PR_GET_TAGGED_ADDR_CTRL, 0, 0, 0, 0)`` system call. @@ -120,13 +126,39 @@ in the ``PR_MTE_TAG_MASK`` bit-field. interface provides an include mask. An include mask of ``0`` (exclusion mask ``0xffff``) results in the CPU always generating tag ``0``. +Per-CPU preferred tag checking mode +----------------------------------- + +On some CPUs the performance of MTE in stricter tag checking modes +is similar to that of less strict tag checking modes. This makes it +worthwhile to enable stricter checks on those CPUs when a less strict +checking mode is requested, in order to gain the error detection +benefits of the stricter checks without the performance downsides. To +support this scenario, a privileged user may configure a stricter +tag checking mode as the CPU's preferred tag checking mode. + +The preferred tag checking mode for each CPU is controlled by +``/sys/devices/system/cpu/cpu/mte_tcf_preferred``, to which a +privileged user may write the value ``async`` or ``sync``. The default +preferred mode for each CPU is ``async``. + +To allow a program to potentially run in the CPU's preferred tag +checking mode, the user program may set multiple tag check fault mode +bits in the ``flags`` argument to the ``prctl(PR_SET_TAGGED_ADDR_CTRL, +flags, 0, 0, 0)`` system call. If the CPU's preferred tag checking +mode is in the task's set of provided tag checking modes (this will +always be the case at present because the kernel only supports two +tag checking modes, but future kernels may support more modes), that +mode will be selected. Otherwise, one of the modes in the task's mode +set will be selected in a currently unspecified manner. + Initial process state --------------------- On ``execve()``, the new process has the following configuration: - ``PR_TAGGED_ADDR_ENABLE`` set to 0 (disabled) -- Tag checking mode set to ``PR_MTE_TCF_NONE`` +- No tag checking modes are selected (tag check faults ignored) - ``PR_MTE_TAG_MASK`` set to 0 (all tags excluded) - ``PSTATE.TCO`` set to 0 - ``PROT_MTE`` not set on any of the initial memory maps @@ -251,11 +283,13 @@ Example of correct usage return EXIT_FAILURE; /* - * Enable the tagged address ABI, synchronous MTE tag check faults and - * allow all non-zero tags in the randomly generated set. + * Enable the tagged address ABI, synchronous or asynchronous MTE + * tag check faults (based on per-CPU preference) and allow all + * non-zero tags in the randomly generated set. */ if (prctl(PR_SET_TAGGED_ADDR_CTRL, - PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | (0xfffe << PR_MTE_TAG_SHIFT), + PR_TAGGED_ADDR_ENABLE | PR_MTE_TCF_SYNC | PR_MTE_TCF_ASYNC | + (0xfffe << PR_MTE_TAG_SHIFT), 0, 0, 0)) { perror("prctl() failed"); return EXIT_FAILURE;