From patchwork Fri Sep 25 14:57:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800017 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E7D4112C for ; Fri, 25 Sep 2020 14:58:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E467720715 for ; Fri, 25 Sep 2020 14:58:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E467720715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5E11B6B009E; Fri, 25 Sep 2020 10:58:17 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 595706B009F; Fri, 25 Sep 2020 10:58:17 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A8CE6B00A0; Fri, 25 Sep 2020 10:58:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 35D526B009E for ; Fri, 25 Sep 2020 10:58:17 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D26E35DE0 for ; Fri, 25 Sep 2020 14:58:16 +0000 (UTC) X-FDA: 77301889392.21.drum14_430a40827168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id AE959180442C0 for ; Fri, 25 Sep 2020 14:58:16 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30054:30055:30056:30064:30070,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95;04yfioy7icet9zg4t9wrydc9k8oj7ochgwzfjg98pcbta7cg9e9r1by5mxscqss.439un1weewap1wc3cpts967uuc75qir74pmehh6npyxyykxj7ng1qhkekugyn3p.4-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: drum14_430a40827168 X-Filterd-Recvd-Size: 4389 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:15 +0000 (UTC) IronPort-SDR: CWW/ONfAPbIoMPTRvdButMeH678vnzQCv4N/JlndVh/gzT+CrQeSglBVwprvWP9NTPWDZZU10m Gu7JjGURgF5A== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704475" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704475" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:13 -0700 IronPort-SDR: dZdetEM4DRg4Hr3m0uJ4dfSizhMqgJIu94nv7NFPov8SDaS6Butg10uJnfPMdKSCpqlxx8PNcf FomTkBPzAl2g== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916957" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:13 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 1/8] x86/cet/ibt: Add Kconfig option for user-mode Indirect Branch Tracking Date: Fri, 25 Sep 2020 07:57:57 -0700 Message-Id: <20200925145804.5821-2-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce Kconfig option X86_BRANCH_TRACKING_USER. Indirect Branch Tracking (IBT) provides protection against CALL-/JMP- oriented programming attacks. It is active when the kernel has this feature enabled, and the processor and the application support it. When this feature is enabled, legacy non-IBT applications continue to work, but without IBT protection. Signed-off-by: Yu-cheng Yu --- v13: - Update help text, and change default to N. - Change X86_INTEL_* to X86_*. v10: - Change build-time CET check to config depends on. arch/x86/Kconfig | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 4b28a0ce4594..15c7f2606c9d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1966,6 +1966,25 @@ config X86_SHADOW_STACK_USER If unsure, say N. +config X86_BRANCH_TRACKING_USER + prompt "Intel Indirect Branch Tracking for user-mode" + def_bool n + depends on CPU_SUP_INTEL && X86_64 + depends on $(cc-option,-fcf-protection) + select X86_CET + help + Indirect Branch Tracking (IBT) provides protection against + CALL-/JMP-oriented programming attacks. It is active when + the kernel has this feature enabled, and the processor and + the application support it. When this feature is enabled, + legacy non-IBT applications continue to work, but without + IBT protection. + Support for this feature is only known to be present on + processors released in 2020 or later. CET features are also + known to increase kernel text size by 3.7 KB. + + If unsure, say N. + config EFI bool "EFI runtime service support" depends on ACPI From patchwork Fri Sep 25 14:57:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800019 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43AF2112C for ; Fri, 25 Sep 2020 14:58:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ECCA720715 for ; Fri, 25 Sep 2020 14:58:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ECCA720715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2ADC36B00A0; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 263E96B00A2; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2F9E6B00A1; Fri, 25 Sep 2020 10:58:17 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id DBDBE6B009F for ; Fri, 25 Sep 2020 10:58:17 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 91E298249980 for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) X-FDA: 77301889434.21.seed37_080acb827168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 60A33180442C0 for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30051:30054:30056:30064:30070:30090,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100;04y8aijz541m9kc9ainzz69dky181yc7kcyb9tir3cccgqjab4699bsrftfizkr.gmch134ahq3iop7jhmypzisjwrncgqkaz71j3ta4xgd56owuoa5ngxho6jww53t.w-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: seed37_080acb827168 X-Filterd-Recvd-Size: 8006 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:16 +0000 (UTC) IronPort-SDR: /PiV4D6kb/ZNRl9yfT8RnLneV09UcQoRPsmk5+G+ezoIde+vyMVvbl7LRFlrD5TkqPvC1f4Rr9 wyPUF8qQ0R4w== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704478" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704478" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:13 -0700 IronPort-SDR: dfbruIsKSdLkIzJd2sNrV54uUbD038MUuLRSp0Cl1fv5L55B1OtnAuoPQEM4ZkLFx7Fcd8WGtj 9ug7hOcugZ5g== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916962" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:13 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 2/8] x86/cet/ibt: User-mode Indirect Branch Tracking support Date: Fri, 25 Sep 2020 07:57:58 -0700 Message-Id: <20200925145804.5821-3-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Introduce user-mode Indirect Branch Tracking (IBT) support. Update setup routines to include IBT. Signed-off-by: Yu-cheng Yu --- v10: - Change no_cet_ibt to no_user_ibt. v9: - Change cpu_feature_enabled() to static_cpu_has(). v2: - Change noibt to no_cet_ibt. arch/x86/include/asm/cet.h | 3 ++ arch/x86/include/asm/disabled-features.h | 8 ++++- arch/x86/kernel/cet.c | 33 +++++++++++++++++++ arch/x86/kernel/cpu/common.c | 17 ++++++++++ .../arch/x86/include/asm/disabled-features.h | 8 ++++- 5 files changed, 67 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/cet.h b/arch/x86/include/asm/cet.h index 16870e5bc8eb..3a1cba579cb2 100644 --- a/arch/x86/include/asm/cet.h +++ b/arch/x86/include/asm/cet.h @@ -15,6 +15,7 @@ struct cet_status { unsigned long shstk_base; unsigned long shstk_size; unsigned int locked:1; + unsigned int ibt_enabled:1; }; #ifdef CONFIG_X86_CET @@ -26,6 +27,8 @@ void cet_free_shstk(struct task_struct *p); int cet_verify_rstor_token(bool ia32, unsigned long ssp, unsigned long *new_ssp); void cet_restore_signal(struct sc_ext *sc); int cet_setup_signal(bool ia32, unsigned long rstor, struct sc_ext *sc); +int cet_setup_ibt(void); +void cet_disable_ibt(void); #else static inline int prctl_cet(int option, u64 arg2) { return -EINVAL; } static inline int cet_setup_thread_shstk(struct task_struct *p, diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index edac76ed75e7..e7096a1e2698 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -62,6 +62,12 @@ #define DISABLE_SHSTK (1<<(X86_FEATURE_SHSTK & 31)) #endif +#ifdef CONFIG_X86_BRANCH_TRACKING_USER +#define DISABLE_IBT 0 +#else +#define DISABLE_IBT (1<<(X86_FEATURE_IBT & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -83,7 +89,7 @@ #define DISABLED_MASK15 0 #define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP|DISABLE_SHSTK) #define DISABLED_MASK17 0 -#define DISABLED_MASK18 0 +#define DISABLED_MASK18 (DISABLE_IBT) #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19) #endif /* _ASM_X86_DISABLED_FEATURES_H */ diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c index b285c726bb88..e95fadb264f7 100644 --- a/arch/x86/kernel/cet.c +++ b/arch/x86/kernel/cet.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include #include #include @@ -341,3 +343,34 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr, struct sc_ext *sc_ext) return 0; } + +int cet_setup_ibt(void) +{ + u64 msr_val; + + if (!static_cpu_has(X86_FEATURE_IBT)) + return -EOPNOTSUPP; + + start_update_msrs(); + rdmsrl(MSR_IA32_U_CET, msr_val); + msr_val |= (CET_ENDBR_EN | CET_NO_TRACK_EN); + wrmsrl(MSR_IA32_U_CET, msr_val); + end_update_msrs(); + current->thread.cet.ibt_enabled = 1; + return 0; +} + +void cet_disable_ibt(void) +{ + u64 msr_val; + + if (!static_cpu_has(X86_FEATURE_IBT)) + return; + + start_update_msrs(); + rdmsrl(MSR_IA32_U_CET, msr_val); + msr_val &= CET_SHSTK_EN; + wrmsrl(MSR_IA32_U_CET, msr_val); + end_update_msrs(); + current->thread.cet.ibt_enabled = 0; +} diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 084480f975aa..909b4160a2d2 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -536,6 +536,23 @@ static __init int setup_disable_shstk(char *s) __setup("no_user_shstk", setup_disable_shstk); #endif +#ifdef CONFIG_X86_BRANCH_TRACKING_USER +static __init int setup_disable_ibt(char *s) +{ + /* require an exact match without trailing characters */ + if (s[0] != '\0') + return 0; + + if (!boot_cpu_has(X86_FEATURE_IBT)) + return 1; + + setup_clear_cpu_cap(X86_FEATURE_IBT); + pr_info("x86: 'no_user_ibt' specified, disabling user Branch Tracking\n"); + return 1; +} +__setup("no_user_ibt", setup_disable_ibt); +#endif + /* * Some CPU features depend on higher CPUID levels, which may not always * be available due to CPUID level capping or broken virtualization diff --git a/tools/arch/x86/include/asm/disabled-features.h b/tools/arch/x86/include/asm/disabled-features.h index edac76ed75e7..e7096a1e2698 100644 --- a/tools/arch/x86/include/asm/disabled-features.h +++ b/tools/arch/x86/include/asm/disabled-features.h @@ -62,6 +62,12 @@ #define DISABLE_SHSTK (1<<(X86_FEATURE_SHSTK & 31)) #endif +#ifdef CONFIG_X86_BRANCH_TRACKING_USER +#define DISABLE_IBT 0 +#else +#define DISABLE_IBT (1<<(X86_FEATURE_IBT & 31)) +#endif + /* * Make sure to add features to the correct mask */ @@ -83,7 +89,7 @@ #define DISABLED_MASK15 0 #define DISABLED_MASK16 (DISABLE_PKU|DISABLE_OSPKE|DISABLE_LA57|DISABLE_UMIP|DISABLE_SHSTK) #define DISABLED_MASK17 0 -#define DISABLED_MASK18 0 +#define DISABLED_MASK18 (DISABLE_IBT) #define DISABLED_MASK_CHECK BUILD_BUG_ON_ZERO(NCAPINTS != 19) #endif /* _ASM_X86_DISABLED_FEATURES_H */ From patchwork Fri Sep 25 14:57:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800021 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6A19C92C for ; Fri, 25 Sep 2020 14:58:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2E85120715 for ; Fri, 25 Sep 2020 14:58:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E85120715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A27A38E0003; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9D9208E0001; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 398106B00A4; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id 1969A6B009F for ; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CE58F4DA2 for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) X-FDA: 77301889434.09.band26_471195527168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id AEA32180AD804 for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30045:30046:30051:30054:30055:30056:30064,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100;04y866cr33weyscmkhskjrpqhsxztyc4wstfs5c8stfi8s6a1upebddhy96h35d.stbjzzuk1wi4icjyw35fuea1mxjzu6rhch8n8tiuppkj7jg9ra8rabrosyoo9tz.q-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: band26_471195527168 X-Filterd-Recvd-Size: 5509 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:16 +0000 (UTC) IronPort-SDR: RKdaS379CS29rwwHejYojAdnhAU2V6uXKgFwV21wOFbTwkZNdjXxNhaOmGNV8gfFTOlHxA6K05 HA0xZ1piYyJA== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704483" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704483" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 IronPort-SDR: vbxmrk/2vgkev8nhj22uATZJCJ9qznrWAemWkziCn88O1cAblaFXd96zFg0I479YiFlk0lqMM/ L1FR1R7AxAyQ== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916965" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:13 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 3/8] x86/cet/ibt: Handle signals for Indirect Branch Tracking Date: Fri, 25 Sep 2020 07:57:59 -0700 Message-Id: <20200925145804.5821-4-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: An indirect CALL/JMP moves the indirect branch tracking (IBT) state machine to WAIT_ENDBR status until the instruction reaches an ENDBR opcode. If the CALL/JMP does not reach an ENDBR opcode, the processor raises a control- protection fault. WAIT_ENDBR status can be read from MSR_IA32_U_CET. WAIT_ENDBR is cleared for signal handling, and restored for sigreturn. IBT state machine is described in Intel SDM Vol. 1, Sec. 18.3. Signed-off-by: Yu-cheng Yu --- v9: - Fix missing WAIT_ENDBR in signal handling. arch/x86/kernel/cet.c | 27 +++++++++++++++++++++++++-- arch/x86/kernel/fpu/signal.c | 8 +++++--- 2 files changed, 30 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/cet.c b/arch/x86/kernel/cet.c index e95fadb264f7..1f8b72269166 100644 --- a/arch/x86/kernel/cet.c +++ b/arch/x86/kernel/cet.c @@ -295,6 +295,13 @@ void cet_restore_signal(struct sc_ext *sc_ext) msr_val |= CET_SHSTK_EN; } + if (cet->ibt_enabled) { + msr_val |= (CET_ENDBR_EN | CET_NO_TRACK_EN); + + if (sc_ext->wait_endbr) + msr_val |= CET_WAIT_ENDBR; + } + if (test_thread_flag(TIF_NEED_FPU_LOAD)) cet_user_state->user_cet = msr_val; else @@ -335,9 +342,25 @@ int cet_setup_signal(bool ia32, unsigned long rstor_addr, struct sc_ext *sc_ext) sc_ext->ssp = new_ssp; } - if (ssp) { + if (ssp || cet->ibt_enabled) { + start_update_msrs(); - wrmsrl(MSR_IA32_PL3_SSP, ssp); + + if (ssp) + wrmsrl(MSR_IA32_PL3_SSP, ssp); + + if (cet->ibt_enabled) { + u64 r; + + rdmsrl(MSR_IA32_U_CET, r); + + if (r & CET_WAIT_ENDBR) { + sc_ext->wait_endbr = 1; + r &= ~CET_WAIT_ENDBR; + wrmsrl(MSR_IA32_U_CET, r); + } + } + end_update_msrs(); } diff --git a/arch/x86/kernel/fpu/signal.c b/arch/x86/kernel/fpu/signal.c index c0c2141cb4b3..077853ef6f48 100644 --- a/arch/x86/kernel/fpu/signal.c +++ b/arch/x86/kernel/fpu/signal.c @@ -57,7 +57,8 @@ int save_cet_to_sigframe(int ia32, void __user *fp, unsigned long restorer) { int err = 0; - if (!current->thread.cet.shstk_size) + if (!current->thread.cet.shstk_size && + !current->thread.cet.ibt_enabled) return 0; if (fp) { @@ -89,7 +90,8 @@ static int get_cet_from_sigframe(int ia32, void __user *fp, struct sc_ext *ext) memset(ext, 0, sizeof(*ext)); - if (!current->thread.cet.shstk_size) + if (!current->thread.cet.shstk_size && + !current->thread.cet.ibt_enabled) return 0; if (fp) { @@ -577,7 +579,7 @@ static unsigned long fpu__alloc_sigcontext_ext(unsigned long sp) * sigcontext_ext is at: fpu + fpu_user_xstate_size + * FP_XSTATE_MAGIC2_SIZE, then aligned to 8. */ - if (cet->shstk_size) + if (cet->shstk_size || cet->ibt_enabled) sp -= (sizeof(struct sc_ext) + 8); return sp; From patchwork Fri Sep 25 14:58:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800025 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9F09D92C for ; Fri, 25 Sep 2020 14:58:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6C101221EC for ; Fri, 25 Sep 2020 14:58:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C101221EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 54488900003; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 32643900006; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 060E0900002; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id DE8FD900003 for ; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A0F5A8249980 for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) X-FDA: 77301889476.30.wood96_0a0529127168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 79293180B3C8E for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30045:30054:30056:30064:30070,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95;04y8bcj8cy9671d5o4cf1wzco1dbsypp4u4d7i7ntjme1jdcse7kt6x9ihuq1uh.s4kit4o3z53zu1rnyuow685473qrnc4db7quxdtnsbydts1mpiso4z6y1ihjim8.w-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: wood96_0a0529127168 X-Filterd-Recvd-Size: 3967 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) IronPort-SDR: 5m0FyvKCyxEzzwUYw2zr50rGIIV/mJyhol5gHuN2+ACWbqSZRQwMVqEWQZjqBMJ9BpSNLHuJ8S 5H9hqtET8rYA== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704487" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704487" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 IronPort-SDR: ByVBDmZe8R5Q3jrRfTehXGSoT6vkekq7Lm3P1KyPfW08+oqHmsDNW/FLh/n30+RzNplFg8ef9d HneBCcCXhWCw== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916971" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 4/8] x86/cet/ibt: ELF header parsing for Indirect Branch Tracking Date: Fri, 25 Sep 2020 07:58:00 -0700 Message-Id: <20200925145804.5821-5-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Update arch_setup_elf_property() for Indirect Branch Tracking. Signed-off-by: Yu-cheng Yu --- v9: - Change cpu_feature_enabled() to static_cpu_has(). arch/x86/Kconfig | 2 ++ arch/x86/kernel/process_64.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 15c7f2606c9d..cc9876f85e91 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1972,6 +1972,8 @@ config X86_BRANCH_TRACKING_USER depends on CPU_SUP_INTEL && X86_64 depends on $(cc-option,-fcf-protection) select X86_CET + select ARCH_USE_GNU_PROPERTY + select ARCH_BINFMT_ELF_STATE help Indirect Branch Tracking (IBT) provides protection against CALL-/JMP-oriented programming attacks. It is active when diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index 8725e67bcd44..1147a1052a07 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -866,6 +866,14 @@ int arch_setup_elf_property(struct arch_elf_state *state) r = cet_setup_shstk(); } + if (r < 0) + return r; + + if (static_cpu_has(X86_FEATURE_IBT)) { + if (state->gnu_property & GNU_PROPERTY_X86_FEATURE_1_IBT) + r = cet_setup_ibt(); + } + return r; } #endif From patchwork Fri Sep 25 14:58:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800023 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7807A92C for ; Fri, 25 Sep 2020 14:58:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3B4D420715 for ; Fri, 25 Sep 2020 14:58:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B4D420715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1C64A900004; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0FBDB900003; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F2CE8900004; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CBDBA900002 for ; Fri, 25 Sep 2020 10:58:18 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 8AB98180AD802 for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) X-FDA: 77301889476.01.cats84_44084d627168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 6E4D61004075D for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30051:30054:30056:30064:30070,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95;04y8z3n5sg46omgdoq9i3e9me69mqochfxxznjjiunbmqn8a3t8zxtszp5jejrr.iuauikoizd9obonsjuohgocycc698mej45rf9ca4k48tcmeyicwhru9ss17pp1k.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: cats84_44084d627168 X-Filterd-Recvd-Size: 4049 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) IronPort-SDR: GbaWWSb1egUJHMlMpa3kKoQKgT4HMrsL1mjlJAwjGUAoRNY5cdkotwpcdr/qQsUfulKDFJQL3q Ik3w2I3oSLiw== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704490" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704490" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 IronPort-SDR: 7xOXXexJwcxygu2Xzf9kweeyko7guPmI7zayEU+/z/sFyyGLF5H8l7arDdhOPWAfCnfgh1GqEA cAz+Z87qiLRg== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916975" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 5/8] x86/cet/ibt: Update arch_prctl functions for Indirect Branch Tracking Date: Fri, 25 Sep 2020 07:58:01 -0700 Message-Id: <20200925145804.5821-6-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "H.J. Lu" Update ARCH_X86_CET_STATUS and ARCH_X86_CET_DISABLE for Indirect Branch Tracking. Signed-off-by: H.J. Lu Signed-off-by: Yu-cheng Yu --- arch/x86/kernel/cet_prctl.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cet_prctl.c b/arch/x86/kernel/cet_prctl.c index bd5ad11763e4..0af1ec5d028f 100644 --- a/arch/x86/kernel/cet_prctl.c +++ b/arch/x86/kernel/cet_prctl.c @@ -22,6 +22,9 @@ static int copy_status_to_user(struct cet_status *cet, u64 arg2) buf[2] = (u64)cet->shstk_size; } + if (cet->ibt_enabled) + buf[0] |= GNU_PROPERTY_X86_FEATURE_1_IBT; + return copy_to_user((u64 __user *)arg2, buf, sizeof(buf)); } @@ -42,7 +45,8 @@ int prctl_cet(int option, u64 arg2) if (option == ARCH_X86_CET_STATUS) return copy_status_to_user(cet, arg2); - if (!static_cpu_has(X86_FEATURE_SHSTK)) + if (!static_cpu_has(X86_FEATURE_SHSTK) && + !static_cpu_has(X86_FEATURE_IBT)) return -EOPNOTSUPP; switch (option) { @@ -56,6 +60,8 @@ int prctl_cet(int option, u64 arg2) return -EINVAL; if (features & GNU_PROPERTY_X86_FEATURE_1_SHSTK) cet_disable_shstk(); + if (features & GNU_PROPERTY_X86_FEATURE_1_IBT) + cet_disable_ibt(); return 0; case ARCH_X86_CET_LOCK: From patchwork Fri Sep 25 14:58:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D875E92C for ; Fri, 25 Sep 2020 14:58:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A1DEC21D42 for ; Fri, 25 Sep 2020 14:58:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A1DEC21D42 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 81F91900002; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6B981900005; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40D96900002; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0037.hostedemail.com [216.40.44.37]) by kanga.kvack.org (Postfix) with ESMTP id 21C28900005 for ; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D41875DED for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) X-FDA: 77301889476.04.brick65_4a0793c27168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin04.hostedemail.com (Postfix) with ESMTP id A538A80127A8 for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30045:30054:30056:30064,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95;04yrrw5faacnguabb5wm4q7ptntwkoceeehbf857bxgqh6nrydsuifgjs1ujiy7.kjssa35bpatopyb5jkrcqwpobqkswh11mxnnpp7n49ahmrxicauk6qxqyedbxwp.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: brick65_4a0793c27168 X-Filterd-Recvd-Size: 3463 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:17 +0000 (UTC) IronPort-SDR: eMMcZVa/BbNXcA5psr7r8na3Gmq8Fw4wpnSvqYCBG5G7EI07RNPNXP3jmUCeC9zLk5oq/pLc23 L84uOAyKK2hg== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704493" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704493" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 IronPort-SDR: /xYfSw8Dn/zExQtH9j5R+QYIsdZssY8Ygp2yq1GuThrHK2HC0qe9nJKAoN1NtCPf2HKXE/Q/e5 En2YMISETexA== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916979" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 6/8] x86/vdso/32: Add ENDBR32 to __kernel_vsyscall entry point Date: Fri, 25 Sep 2020 07:58:02 -0700 Message-Id: <20200925145804.5821-7-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "H.J. Lu" Add ENDBR32 to __kernel_vsyscall entry point. Signed-off-by: H.J. Lu Signed-off-by: Yu-cheng Yu Acked-by: Andy Lutomirski --- arch/x86/entry/vdso/vdso32/system_call.S | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/entry/vdso/vdso32/system_call.S b/arch/x86/entry/vdso/vdso32/system_call.S index de1fff7188aa..e331fcdebd95 100644 --- a/arch/x86/entry/vdso/vdso32/system_call.S +++ b/arch/x86/entry/vdso/vdso32/system_call.S @@ -14,6 +14,9 @@ ALIGN __kernel_vsyscall: CFI_STARTPROC +#ifdef CONFIG_X86_BRANCH_TRACKING_USER + endbr32 +#endif /* * Reshuffle regs so that all of any of the entry instructions * will preserve enough state. From patchwork Fri Sep 25 14:58:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5E6DE92C for ; Fri, 25 Sep 2020 14:58:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2350620715 for ; Fri, 25 Sep 2020 14:58:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2350620715 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F0C04900007; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EE202900005; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC00C900007; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id 9B027900006 for ; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5E3F9180AD802 for ; Fri, 25 Sep 2020 14:58:19 +0000 (UTC) X-FDA: 77301889518.16.park55_47015ba27168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 3668C101E36E7 for ; Fri, 25 Sep 2020 14:58:19 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30012:30054:30056:30064:30090,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100;04yruhpba3kfppthxbeq4kq8qzwtqocg4m9ixo53u9c5cgjre3b44q3bymxsut6.usun3pgm7hs3hxhzdboiz8iw85xre6e8t7m35fyw3nb19jeg5x6knkhk4wahbjx.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: park55_47015ba27168 X-Filterd-Recvd-Size: 3705 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) IronPort-SDR: 3L7phU2pRKS8f5E1OPA8Na8pXqvBBv+HHnrfF/cpvYuX2IL5O9jVcJcpS9dlwqXxRlq+hQgUq1 +UbR3XZHGCLQ== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704495" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704495" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:15 -0700 IronPort-SDR: N81qlOa5yWAsiE4nfkkfP4uC7zCc1FR1TwBbPQlYHohaMv2pz2oZBlVa6TP74zYAR+Wksfn2eX 2rTryfN1e6Tw== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916982" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:14 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 7/8] x86/vdso: Insert endbr32/endbr64 to vDSO Date: Fri, 25 Sep 2020 07:58:03 -0700 Message-Id: <20200925145804.5821-8-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "H.J. Lu" When Indirect Branch Tracking (IBT) is enabled, vDSO functions may be called indirectly, and must have ENDBR32 or ENDBR64 as the first instruction. The compiler must support -fcf-protection=branch so that it can be used to compile vDSO. Signed-off-by: H.J. Lu Signed-off-by: Yu-cheng Yu Acked-by: Andy Lutomirski --- v12: - Replace object file list with $(vobjs) $(vobjs32). arch/x86/entry/vdso/Makefile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile index 215376d975a2..1f1b6893068a 100644 --- a/arch/x86/entry/vdso/Makefile +++ b/arch/x86/entry/vdso/Makefile @@ -94,6 +94,10 @@ endif $(vobjs): KBUILD_CFLAGS := $(filter-out $(GCC_PLUGINS_CFLAGS) $(RETPOLINE_CFLAGS),$(KBUILD_CFLAGS)) $(CFL) +ifdef CONFIG_X86_BRANCH_TRACKING_USER +$(vobjs) $(vobjs32): KBUILD_CFLAGS += -fcf-protection=branch +endif + # # vDSO code runs in userspace and -pg doesn't help with profiling anyway. # From patchwork Fri Sep 25 14:58:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu-cheng Yu X-Patchwork-Id: 11800031 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5FB4C92C for ; Fri, 25 Sep 2020 14:59:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 27534235F9 for ; Fri, 25 Sep 2020 14:59:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27534235F9 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 394F1900005; Fri, 25 Sep 2020 10:58:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 12D99900008; Fri, 25 Sep 2020 10:58:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF56C900006; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id C8689900005 for ; Fri, 25 Sep 2020 10:58:19 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 842B9180AD806 for ; Fri, 25 Sep 2020 14:58:19 +0000 (UTC) X-FDA: 77301889518.03.order96_101828127168 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 5822E28A4E8 for ; Fri, 25 Sep 2020 14:58:19 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,yu-cheng.yu@intel.com,,RULES_HIT:30003:30045:30054:30056:30064:30070,0,RBL:134.134.136.100:@intel.com:.lbl8.mailshell.net-64.95.201.95 62.18.0.100;04yggzwbt1hf5j3y4nen4ftau16jiypo9jsy76pmjpif45r4kfxb5szegedm7o7.x6g3u79tb8cjszq4gabm38a94x13gyupedkjfce4o7cgmjtj7dwoif9r4zaneea.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:22,LUA_SUMMARY:none X-HE-Tag: order96_101828127168 X-Filterd-Recvd-Size: 5821 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Fri, 25 Sep 2020 14:58:18 +0000 (UTC) IronPort-SDR: wGlb3evJqqpmVIRo47mI5wlT67GMgXCPR0FIC3+FaOWhfjM9qKWpeZ6XZDWRL+iPPpS/kOa+xf +fwpoHnO79Wg== X-IronPort-AV: E=McAfee;i="6000,8403,9755"; a="225704496" X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="225704496" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:15 -0700 IronPort-SDR: mk4f5HgZi9QgZiPRFPoL0+ZOP49MveK1M7h2fTdfm5ELo0AwQdsVLEWDy06xhC/j8QvOhtdJMe 58vuIGKuUIlA== X-IronPort-AV: E=Sophos;i="5.77,302,1596524400"; d="scan'208";a="512916985" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 25 Sep 2020 07:58:15 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu Cc: Yu-cheng Yu Subject: [PATCH v13 8/8] x86/vsyscall/64: Fixup Shadow Stack and Indirect Branch Tracking for vsyscall emulation Date: Fri, 25 Sep 2020 07:58:04 -0700 Message-Id: <20200925145804.5821-9-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20200925145804.5821-1-yu-cheng.yu@intel.com> References: <20200925145804.5821-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Vsyscall entry points are effectively branch targets. Mark them with ENDBR64 opcodes. When emulating the RET instruction, unwind shadow stack and reset IBT state machine. Signed-off-by: Yu-cheng Yu Signed-off-by: Yu-cheng Yu Signed-off-by: Yu-cheng Yu Signed-off-by: Yu-cheng Yu --- v13: - Check shadow stack address is canonical. - Change from writing to MSRs to writing to CET xstate. arch/x86/entry/vsyscall/vsyscall_64.c | 34 +++++++++++++++++++++++ arch/x86/entry/vsyscall/vsyscall_emu_64.S | 9 ++++++ arch/x86/entry/vsyscall/vsyscall_trace.h | 1 + 3 files changed, 44 insertions(+) diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c index 44c33103a955..315ee3572664 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -38,6 +38,9 @@ #include #include #include +#include +#include +#include #define CREATE_TRACE_POINTS #include "vsyscall_trace.h" @@ -286,6 +289,37 @@ bool emulate_vsyscall(unsigned long error_code, /* Emulate a ret instruction. */ regs->ip = caller; regs->sp += 8; + +#ifdef CONFIG_X86_CET + if (tsk->thread.cet.shstk_size || tsk->thread.cet.ibt_enabled) { + struct cet_user_state *cet; + struct fpu *fpu; + + fpu = &tsk->thread.fpu; + fpregs_lock(); + + if (!test_thread_flag(TIF_NEED_FPU_LOAD)) { + copy_fpregs_to_fpstate(fpu); + set_thread_flag(TIF_NEED_FPU_LOAD); + } + + cet = get_xsave_addr(&fpu->state.xsave, XFEATURE_CET_USER); + if (!cet) { + fpregs_unlock(); + goto sigsegv; + } + + if (cet->user_ssp && ((cet->user_ssp + 8) < TASK_SIZE_MAX)) + cet->user_ssp += 8; + + if (cet->user_cet & CET_ENDBR_EN) + cet->user_cet &= ~CET_WAIT_ENDBR; + + __fpu_invalidate_fpregs_state(fpu); + fpregs_unlock(); + } +#endif + return true; sigsegv: diff --git a/arch/x86/entry/vsyscall/vsyscall_emu_64.S b/arch/x86/entry/vsyscall/vsyscall_emu_64.S index 2e203f3a25a7..b2fa92104cdb 100644 --- a/arch/x86/entry/vsyscall/vsyscall_emu_64.S +++ b/arch/x86/entry/vsyscall/vsyscall_emu_64.S @@ -17,16 +17,25 @@ __PAGE_ALIGNED_DATA .type __vsyscall_page, @object __vsyscall_page: +#ifdef CONFIG_X86_BRANCH_TRACKING_USER + endbr64 +#endif mov $__NR_gettimeofday, %rax syscall ret .balign 1024, 0xcc +#ifdef CONFIG_X86_BRANCH_TRACKING_USER + endbr64 +#endif mov $__NR_time, %rax syscall ret .balign 1024, 0xcc +#ifdef CONFIG_X86_BRANCH_TRACKING_USER + endbr64 +#endif mov $__NR_getcpu, %rax syscall ret diff --git a/arch/x86/entry/vsyscall/vsyscall_trace.h b/arch/x86/entry/vsyscall/vsyscall_trace.h index 3c3f9765a85c..7aa2101ada44 100644 --- a/arch/x86/entry/vsyscall/vsyscall_trace.h +++ b/arch/x86/entry/vsyscall/vsyscall_trace.h @@ -25,6 +25,7 @@ TRACE_EVENT(emulate_vsyscall, #endif #undef TRACE_INCLUDE_PATH +#undef TRACE_INCLUDE_FILE #define TRACE_INCLUDE_PATH ../../arch/x86/entry/vsyscall/ #define TRACE_INCLUDE_FILE vsyscall_trace #include