From patchwork Sat Nov 21 09:59:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 11923703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DATE_IN_PAST_12_24, DKIMWL_WL_HIGH,DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B58D3C388F9 for ; Sun, 22 Nov 2020 00:03:23 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 53BCD20691 for ; Sun, 22 Nov 2020 00:03:23 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="rqJEZ8wd"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="s3QDryrb" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 53BCD20691 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:Mime-Version:Message-Id:Date: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=6QEWhJ/8exGNE++7XgQLxHf3fNkMl0uaEc6IbwARhrM=; b=rqJEZ8wdC610/z+yOcfinq8iuz rrG1UEBppzeWMcsSJvMSv0o+XI6gJfwWcsam4G/sQ25wq4FqpTJaJFcpKcByEc5RWayNmTA6NLmbN Lvjd3ijHMw4jAA6eGtrqkx4L1BYHMdT93q6ooht3Ri1Xke79bRBo1wDrk4RD9DTReCy2olLyJgu5B zZ7gZdRmdlZGoKnBAgrzqjyRC1iFaanbAsbqdVQ4d+M++fLB2tu+NbBqHyMo3U612DAS3bocWWGz0 MKrMuOtmOA//lZ44kxYCyygayuUnwlkXqS/lkC5XLhEDjycpxbhpdOB8PqCC7RhuCGVn/xGrB9T+1 OIHM95/A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kgcpP-00014f-9i; Sun, 22 Nov 2020 00:02:03 +0000 Received: from mail-ot1-x349.google.com ([2607:f8b0:4864:20::349]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kgcpM-00014H-St for linux-arm-kernel@lists.infradead.org; Sun, 22 Nov 2020 00:02:01 +0000 Received: by mail-ot1-x349.google.com with SMTP id i11so5829162otr.8 for ; Sat, 21 Nov 2020 16:01:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=WsUEi0z7Dz1bXAOmQFYUbblVTp6z5WKIjT4H2OhRZ68=; b=s3QDryrbltRJk8zKcIElUF32zdgNc8BkPT6ab64oRstGNPHQ9R/E7+KGLUQKVHM95/ tWE8mquVCh87Nvk5zBgvLs9cJx15NLV7ORuiIgrkwYoe96Xuoz3MZjHFiGJ/fZ5sKNWk 0doNapBW8uAZoM2pYgyZOqmQGmpbG1ADbuz9Mlkz7x8lZDzCIGHRYpwkYL6Qt8DTAvEC 7mEl6uJxpSu+If1fgSLJlA/CTYTB+ioiZ+CJ/Dc5nGG8/aSozMx1sPdkoF+QK9158srZ dfDuzdqlPkvicRzBsWMtTJ7g80uOOEeDjsHVrz5TusTR6t8Tk6WGS7lGMLrcx+3yIB7o PUkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=WsUEi0z7Dz1bXAOmQFYUbblVTp6z5WKIjT4H2OhRZ68=; b=IBtpCd5orQSj3iIFAfa966ADfmtBTPpPDOKa7hpwOWJsDIHCJOl0aJh+6k+rdvhHQ7 Cvc7cpI2U+m7uvB5tKBh2z3canXUSdEtbIGmKaMq2fbhsuMgFFNtJznbLQDP3XQywA9y L5vhrcjNdZia2te0z8lPufJM/klMDXR2HgAZJNLBjX6ckrELnXHg2yMhdlJlQpMHxf7k wYfnNEysaUoD1L381r9s32sCYpojmjWhLEU+dy9HlgdIXAK20JRdTKee+2kCsTtcMjl5 2ipit3ViUTxP26kOhZV71kzzT8TDbLI2mig3wd627Ks3ekKWWTiLoki+MBOVn+QQCJ/7 ofpw== X-Gm-Message-State: AOAM530OxmC7ZoFkxfsIkPGiRzm7t/B68px2/bnllGOapw3nD7ZKzpev 1sbmwsmfT4BVw7Z0CwB/ZzEKjiY= X-Google-Smtp-Source: ABdhPJwpMi8YpdlPVDbGMRvK6mmXDOOcxdGseF1dACwdw8lxl7n8LLdTIpImechooRrzXDxVZYzz0kY= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:0:7220:84ff:fe09:385a]) (user=pcc job=sendgmr) by 2002:a25:cb55:: with SMTP id b82mr36207818ybg.334.1605952754258; Sat, 21 Nov 2020 01:59:14 -0800 (PST) Date: Sat, 21 Nov 2020 01:59:02 -0800 Message-Id: <20f64e26fc8a1309caa446fffcb1b4e2fe9e229f.1605952129.git.pcc@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.29.2.454.gaff20da3a2-goog Subject: [PATCH 1/2] kasan: arm64: set TCR_EL1.TBID1 when enabled From: Peter Collingbourne To: Catalin Marinas , Evgenii Stepanov , Kostya Serebryany , Vincenzo Frascino , Dave Martin , Will Deacon X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201121_190200_976804_5A004C66 X-CRM114-Status: GOOD ( 12.08 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrey Konovalov , Peter Collingbourne , Linux ARM , linux-api@vger.kernel.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On hardware supporting pointer authentication, we previously ended up enabling TBI on instruction accesses when tag-based ASAN was enabled, but this was costing us 8 bits of PAC entropy, which was unnecessary since tag-based ASAN does not require TBI on instruction accesses. Get them back by setting TCR_EL1.TBID1. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/I3dded7824be2e70ea64df0aabab9598d5aebfcc4 Reviewed-by: Andrey Konovalov --- arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/mm/proc.S | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 01a96d07ae74..42442a0ae2ab 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -260,6 +260,7 @@ #define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) +#define TCR_TBID1 (UL(1) << 52) #define TCR_NFD0 (UL(1) << 53) #define TCR_NFD1 (UL(1) << 54) #define TCR_E0PD0 (UL(1) << 55) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 23c326a06b2d..97a97a61a8dc 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -40,7 +40,7 @@ #define TCR_CACHE_FLAGS TCR_IRGN_WBWA | TCR_ORGN_WBWA #ifdef CONFIG_KASAN_SW_TAGS -#define TCR_KASAN_FLAGS TCR_TBI1 +#define TCR_KASAN_FLAGS TCR_TBI1 | TCR_TBID1 #else #define TCR_KASAN_FLAGS 0 #endif From patchwork Sat Nov 21 09:59:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Collingbourne X-Patchwork-Id: 11923669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DATE_IN_PAST_12_24, DKIMWL_WL_HIGH,DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A28E9C388F9 for ; Sat, 21 Nov 2020 23:14:39 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4A58F217A0 for ; Sat, 21 Nov 2020 23:14:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="DhE6PI9t"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="avqNQpzD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4A58F217A0 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version:Message-Id: In-Reply-To:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=QcS9wi+cTpXEqAtlxVfgDJFq5tatMIwnFy7C6QeViuo=; b=DhE6PI9t7NYZ/CArGmDhXUIwp XdV/i72t6pODDk6+S2+Jq6+HofRMmese85a3AwT9TXTIJmolppvqarL8b5PGCC2CKmFkTY0U3b4qj 9MxCKaFCz9zfbMebK0OdNod4HdgXAnqvivZ55RXI5IIL8xUegSEntOkYxW0HZwaaL2lrGhT3cCPR4 FofjLozGkSJet4VC4zWCzK0FYgDz2AYqlc3jYqyS/HN7VDjVkrLg8ZuRlyMrSHZarbQ0Glw//y1jc xNe2wqUfrSxRAy/HDyjtHO9oASt+pMxrxYnoKKYJxSTVb6phXmq4STxKKY0xMdxuzrKS52/Z5nEkz OSU0c42YQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kgc4H-0008QS-Jg; Sat, 21 Nov 2020 23:13:22 +0000 Received: from mail-vk1-xa4a.google.com ([2607:f8b0:4864:20::a4a]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kgc3x-0008Hk-NT for linux-arm-kernel@lists.infradead.org; Sat, 21 Nov 2020 23:13:03 +0000 Received: by mail-vk1-xa4a.google.com with SMTP id h193so1160296vka.4 for ; Sat, 21 Nov 2020 15:12:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:in-reply-to:message-id:mime-version:references:subject :from:to:cc; bh=meClIThUUCGmqBFGTKtEb888kGSslOj9VbNWIQpUrzM=; b=avqNQpzDHDwm2BZA3GuzxPalcyCOMf69cJWcy+em3kOeqOLZz43EmmRVcguzVJPy+u 15PzDkaGXQjiEmIotQHEEOG3jffeLLr2l80D7YJkCkAKTgn7vmvt6l4x7q0lHwn1bicG cbw+26RLSA7qQO/5w1V3qDXwqvE0ScfbesfM1+xglDtNzh6PHw66pS3gJ+GQtFNxPGbp 8RITOUB93/NIt96rBkz0zpasCwWRRDl6nsvizhu/Ckpxt8+pzc+NZQXUkqr8JlBS9h1w iS9X4zqV+GOC/3FJ+fH/VvODQvIxhJYyUJZX9qg4r3BFa+H2iECswmMBH7rCw7XsXHqy S3Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=meClIThUUCGmqBFGTKtEb888kGSslOj9VbNWIQpUrzM=; b=UxiTsiVHIII4za1MtKlePs0XDCg4HSKsnyLTHw0dWxJ0NH9V0iFpryNcuQ+RaTcqZa 1bC2cLhu0IlMypuexkeiUXMFk+kLYQlXD+N4wjHBVssYNYYplJBZoiqhCi/im9Ugryh2 6NPhSj44EZTPQyO+tSFRZrFYoYhZYjOS452y/YSRnmuDj0beHVW16DTq0t+D3JannvAe H1mO0f2CixMfNNr8V+mtQrlbL56wg/IB9PBhXpxvb/NbkgPxA0u9gWjFJWeRsGRyy/w2 YenHL3knKQMeExfBOI7xDgF3n4IHSYXSXKXAUuEjAdUjAebOgwqeM/Ylbl/TeYihBxkQ nZWg== X-Gm-Message-State: AOAM532Z9SOr18AAOtV0jxFlhO6UisqGBZWAukp40SuXuH/m085rS912 Y///p//FDzJ2uyQYj6j5raRbN+o= X-Google-Smtp-Source: ABdhPJyspOhT2hOJtPmGesSa9WQR6w7gGjgrk+NwpxIQwaGPzftFL2NSScJ1hg60po0dC4itGDyUdEA= X-Received: from pcc-desktop.svl.corp.google.com ([2620:15c:2ce:0:7220:84ff:fe09:385a]) (user=pcc job=sendgmr) by 2002:a17:902:ed45:b029:d9:ea8e:cee4 with SMTP id y5-20020a170902ed45b02900d9ea8ecee4mr5435709plb.82.1605952756404; Sat, 21 Nov 2020 01:59:16 -0800 (PST) Date: Sat, 21 Nov 2020 01:59:03 -0800 In-Reply-To: <20f64e26fc8a1309caa446fffcb1b4e2fe9e229f.1605952129.git.pcc@google.com> Message-Id: <64c0fa360333fd5275582d25019614156a8302bc.1605952129.git.pcc@google.com> Mime-Version: 1.0 References: <20f64e26fc8a1309caa446fffcb1b4e2fe9e229f.1605952129.git.pcc@google.com> X-Mailer: git-send-email 2.29.2.454.gaff20da3a2-goog Subject: [PATCH 2/2] arm64: allow TCR_EL1.TBID0 to be configured From: Peter Collingbourne To: Catalin Marinas , Evgenii Stepanov , Kostya Serebryany , Vincenzo Frascino , Dave Martin , Will Deacon X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201121_181301_842896_2E988817 X-CRM114-Status: GOOD ( 27.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Andrey Konovalov , Peter Collingbourne , Linux ARM , linux-api@vger.kernel.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Introduce a Kconfig option that controls whether TCR_EL1.TBID0 is set at boot time. Setting TCR_EL1.TBID0 increases the number of signature bits used by the pointer authentication instructions for instruction addresses by 8, which improves the security of pointer authentication, but it also has the consequence of changing the operation of the branch instructions so that they no longer ignore the top byte of the target address but instead fault if they are non-zero. Since this is a change to the userspace ABI the option defaults to off. Signed-off-by: Peter Collingbourne Link: https://linux-review.googlesource.com/id/Ife724ad708142bc475f42e8c1d9609124994bbbd --- This is more of an RFC. An open question is how to expose this. Having it be a build-time flag is probably the simplest option but I guess it could also be a boot flag. Since it involves an ABI change we may also want a prctl() so that userspace can figure out which mode it is in. I think we should try to avoid it being a per-task property so that we don't need to swap yet another system register on task switch. This goes on top of my FAR_EL1 series because it involves a change to how FAR_EL1 is handled on instruction aborts. arch/arm64/Kconfig | 18 ++++++++++++++++++ arch/arm64/include/asm/compiler.h | 18 ++++++++++++------ arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/include/asm/pointer_auth.h | 2 +- arch/arm64/kernel/ptrace.c | 8 +++----- arch/arm64/mm/fault.c | 14 +++++++++++++- arch/arm64/mm/proc.S | 8 +++++++- 7 files changed, 55 insertions(+), 14 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1515f6f153a0..6ea17249f33f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1532,6 +1532,24 @@ config ARM64_PTR_AUTH This feature works with FUNCTION_GRAPH_TRACER option only if DYNAMIC_FTRACE_WITH_REGS is enabled. +config ARM64_TBI_DATA + bool "Restrict top-byte ignore to data accesses" + help + Normally, the kernel will enable top-byte ignore for instruction + accesses as well as data accesses. With this configuration option + enabled, on hardware supporting pointer authentication top-byte + ignore will only be enabled for data accesses. + + The most important consequence of this is that it increases + the number of signature bits used by the pointer authentication + instructions for instruction addresses by 8, which improves the + security of pointer authentication, but it also has the consequence + of changing the operation of the branch instructions so that they + no longer ignore the top byte of the target address but instead + fault if they are non-zero. If your userspace does not depend on + branch instructions ignoring the top byte it is recommended to + select this option. + config CC_HAS_BRANCH_PROT_PAC_RET # GCC 9 or later, clang 8 or later def_bool $(cc-option,-mbranch-protection=pac-ret+leaf) diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h index 6fb2e6bcc392..7332fd35bf6f 100644 --- a/arch/arm64/include/asm/compiler.h +++ b/arch/arm64/include/asm/compiler.h @@ -12,15 +12,21 @@ * The EL0/EL1 pointer bits used by a pointer authentication code. * This is dependent on TBI0/TBI1 being enabled, or bits 63:56 would also apply. */ -#define ptrauth_user_pac_mask() GENMASK_ULL(54, vabits_actual) +#ifdef CONFIG_ARM64_TBI_DATA +#define ptrauth_user_insn_pac_mask() GENMASK_ULL(63, vabits_actual) +#else +#define ptrauth_user_insn_pac_mask() GENMASK_ULL(54, vabits_actual) +#endif +#define ptrauth_user_data_pac_mask() GENMASK_ULL(54, vabits_actual) #define ptrauth_kernel_pac_mask() GENMASK_ULL(63, vabits_actual) /* Valid for EL0 TTBR0 and EL1 TTBR1 instruction pointers */ -#define ptrauth_clear_pac(ptr) \ - ((ptr & BIT_ULL(55)) ? (ptr | ptrauth_kernel_pac_mask()) : \ - (ptr & ~ptrauth_user_pac_mask())) +#define ptrauth_clear_insn_pac(ptr) \ + ((ptr & BIT_ULL(55)) ? (ptr | ptrauth_kernel_pac_mask()) : \ + (ptr & ~ptrauth_user_insn_pac_mask())) -#define __builtin_return_address(val) \ - (void *)(ptrauth_clear_pac((unsigned long)__builtin_return_address(val))) +#define __builtin_return_address(val) \ + ((void *)(ptrauth_clear_insn_pac( \ + (unsigned long)__builtin_return_address(val)))) #endif /* __ASM_COMPILER_H */ diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index 42442a0ae2ab..90e69048442d 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -260,6 +260,7 @@ #define TCR_TBI1 (UL(1) << 38) #define TCR_HA (UL(1) << 39) #define TCR_HD (UL(1) << 40) +#define TCR_TBID0 (UL(1) << 51) #define TCR_TBID1 (UL(1) << 52) #define TCR_NFD0 (UL(1) << 53) #define TCR_NFD1 (UL(1) << 54) diff --git a/arch/arm64/include/asm/pointer_auth.h b/arch/arm64/include/asm/pointer_auth.h index c6b4f0603024..a0022867b8ed 100644 --- a/arch/arm64/include/asm/pointer_auth.h +++ b/arch/arm64/include/asm/pointer_auth.h @@ -73,7 +73,7 @@ extern int ptrauth_prctl_reset_keys(struct task_struct *tsk, unsigned long arg); static inline unsigned long ptrauth_strip_insn_pac(unsigned long ptr) { - return ptrauth_clear_pac(ptr); + return ptrauth_clear_insn_pac(ptr); } #define ptrauth_thread_init_user(tsk) \ diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 8ac487c84e37..44afc5c3427e 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -893,13 +893,11 @@ static int pac_mask_get(struct task_struct *target, { /* * The PAC bits can differ across data and instruction pointers - * depending on TCR_EL1.TBID*, which we may make use of in future, so - * we expose separate masks. + * depending on TCR_EL1.TBID0, so we expose separate masks. */ - unsigned long mask = ptrauth_user_pac_mask(); struct user_pac_mask uregs = { - .data_mask = mask, - .insn_mask = mask, + .data_mask = ptrauth_user_data_pac_mask(), + .insn_mask = ptrauth_user_insn_pac_mask(), }; if (!system_supports_address_auth()) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 29a6b8c9e830..617f9f43f528 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -458,11 +458,23 @@ static int __kprobes do_page_fault(unsigned long far, unsigned int esr, vm_fault_t fault; unsigned long vm_flags = VM_ACCESS_FLAGS; unsigned int mm_flags = FAULT_FLAG_DEFAULT; - unsigned long addr = untagged_addr(far); + unsigned long addr; if (kprobe_page_fault(regs, esr)) return 0; + /* + * If TBID0 is set then we may get an IABT with a tagged address here as + * a result of branching to a tagged address. In this case we want to + * avoid untagging the address, let the VMA lookup fail and get a + * SIGSEGV. Leaving the address as is will also work if TBID0 is clear + * or unsupported because the tag bits of FAR_EL1 will be clear. + */ + if (is_el0_instruction_abort(esr)) + addr = far; + else + addr = untagged_addr(far); + /* * If we're in an interrupt or have no user context, we must not take * the fault. diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 97a97a61a8dc..0e715b9604a1 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -45,6 +45,12 @@ #define TCR_KASAN_FLAGS 0 #endif +#ifdef CONFIG_ARM64_TBI_DATA +#define TCR_TBI_DATA_FLAGS TCR_TBID0 +#else +#define TCR_TBI_DATA_FLAGS 0 +#endif + /* * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory and * changed during __cpu_setup to Normal Tagged if the system supports MTE. @@ -456,7 +462,7 @@ SYM_FUNC_START(__cpu_setup) */ mov_q x10, TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ TCR_TG_FLAGS | TCR_KASLR_FLAGS | TCR_ASID16 | \ - TCR_TBI0 | TCR_A1 | TCR_KASAN_FLAGS + TCR_TBI0 | TCR_TBI_DATA_FLAGS | TCR_A1 | TCR_KASAN_FLAGS tcr_clear_errata_bits x10, x9, x5 #ifdef CONFIG_ARM64_VA_BITS_52