From patchwork Fri Jul 1 15:27:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12903529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 21237C43334 for ; Fri, 1 Jul 2022 15:29:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sToPQ7Hr+POSNN3lKQ90xKTYMfkkHOnSy1x2vKUq9v8=; b=X+FNUkpU/S3wlT FYfZbrlymHZuPgsz99QhWg8eQ9+bir/QvA5ufeSNhPGKCPaiSkLldOTn5l/OtMHQOHxTwX6vowl1D Qq0+pr2+g/xCPpu7DX50CpQNBhC9Ws1ATssm6sl8Ohd/WAz4wW53jvck6Yg0Jjnlt40g8XlJrdqLe lLCn6KrdnjeoJvWRpZhgUT/HTpuwT8CN8EgCraybfHDtX+S+mWR8TUDmKVWHooTdcNXPvsS8ALoOr Qlpugt6aQ1AgLWGlrR6sLfVvAAwbzfuEICm71MEmbLVLuN2vXefkysbBSsqrUdqFiv+asVC3gT2gX 2RvZ476emrLcN3QkWDxQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o7IYp-005fiT-If; Fri, 01 Jul 2022 15:27:59 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o7IYY-005fdV-8w for linux-arm-kernel@lists.infradead.org; Fri, 01 Jul 2022 15:27:44 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id DA56FB83094; Fri, 1 Jul 2022 15:27:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97D1BC341C8; Fri, 1 Jul 2022 15:27:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656689259; bh=qeesVo6we35BNGXxfrEUZZWjSLKCrGagKTqd5Oi6Vvo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S7C5nNyCDuM8jEfH44yVsm7v+fxm3o5r8h1Qu+5LIyfyBAlsmyHcYtyj6VHJ27knk L+JXOcRhUr3gT1LR3wSoRLgllmpHF1QafIIrl5wTGRRFHToQ8IMk6nXu7FAPON1xpS beal5oTJFzZgzXzGvtgs7R1f/BghGGZENupInF56/9UbUd1fa6xEVt+23uH2A3ztJT xEb2zJ3teP3mMX5JFvRsXmJrbT6MDDVsqqI9IfU1GgUittLEpkACpZKcagA2UhwIOD KqYeWt6claZL3+W9WmZk6rcrYxnZdi0AY1zgqp4h6FJMv6zFQkmuMXpVX78SbbT7jc XaB/BTs17owuw== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, maz@kernel.org, Ard Biesheuvel , Nick Desaulniers , Kees Cook , Sami Tolvanen Subject: [PATCH v4 2/3] scs: add support for dynamic shadow call stacks Date: Fri, 1 Jul 2022 17:27:23 +0200 Message-Id: <20220701152724.3343599-3-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220701152724.3343599-1-ardb@kernel.org> References: <20220701152724.3343599-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4187; h=from:subject; bh=qeesVo6we35BNGXxfrEUZZWjSLKCrGagKTqd5Oi6Vvo=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBivxJapv4gubDUgbEPHLzErUUnYKD3RjyWL3BqO/wX 7oNoKVeJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYr8SWgAKCRDDTyI5ktmPJA+KC/ sERBvmXaGenwSVKsxpGRxgRacZO9+LijWm8hURODlR+VnSYinxb3dywNESnRrTRKlhYGzdiOLuqR7M 1I63FJliRdUnGGDm1f9ClOPguM6hQ72WxiJn4+5XMWwpYSEgiIxSddHDNnrW/xR3XAW/4zxf6dZu5O kXAMVOiIl0lirDt04bDbnKTSwILOKSeCLuvrmrq32RBkgcgk9YgFR4VKNMVoBfgN4JBjRoYucwKFT7 ZEhxPvn6QL1mFOGaAxPqqxVluHmyoLXa/Oc6D2J7w6YSs7QMC60kVsFBjnvZ4RZA/QCty9m3xC7Nai QWbpJxiOnrdDL3qaasmltv0FYyK8EzJWu3u5ei66oTBeUwt/YV5QDsVgLMvp39S1tOallrcHeeCTS+ s8nCBITj4TOIXOx6XAUrm1M3NhUMN1P8V9h6FkOXtTYXKe/7iY61KbWAF2fegxqtmhg7HncUoet2rb xRp7+hYt3a8liwuymIkr8FGdIuviuCi0ia3kH11irgFMM= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220701_082742_664123_6C665EBF X-CRM114-Status: GOOD ( 20.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to allow arches to use code patching to conditionally emit the shadow stack pushes and pops, rather than always taking the performance hit even on CPUs that implement alternatives such as stack pointer authentication on arm64, add a Kconfig symbol that can be set by the arch to omit the SCS codegen itself, without otherwise affecting how support code for SCS and compiler options (for register reservation, for instance) are emitted. Also, add a static key and some plumbing to omit the allocation of shadow call stack for dynamic SCS configurations if SCS is disabled at runtime. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers Reviewed-by: Kees Cook Reviewed-by: Sami Tolvanen --- Makefile | 2 ++ arch/Kconfig | 7 +++++++ include/linux/scs.h | 18 ++++++++++++++++++ kernel/scs.c | 14 ++++++++++++-- 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 513c1fbf7888..98f5c4da93b7 100644 --- a/Makefile +++ b/Makefile @@ -887,8 +887,10 @@ LDFLAGS_vmlinux += --gc-sections endif ifdef CONFIG_SHADOW_CALL_STACK +ifndef CONFIG_DYNAMIC_SCS CC_FLAGS_SCS := -fsanitize=shadow-call-stack KBUILD_CFLAGS += $(CC_FLAGS_SCS) +endif export CC_FLAGS_SCS endif diff --git a/arch/Kconfig b/arch/Kconfig index 52eaa16d9b79..3d9042893359 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -639,6 +639,13 @@ config SHADOW_CALL_STACK reading and writing arbitrary memory may be able to locate them and hijack control flow by modifying the stacks. +config DYNAMIC_SCS + bool + help + Set by the arch code if it relies on code patching to insert the + shadow call stack push and pop instructions rather than on the + compiler. + config LTO bool help diff --git a/include/linux/scs.h b/include/linux/scs.h index 18122d9e17ff..4ab5bdc898cf 100644 --- a/include/linux/scs.h +++ b/include/linux/scs.h @@ -53,6 +53,22 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk) return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC; } +DECLARE_STATIC_KEY_FALSE(dynamic_scs_enabled); + +static inline bool scs_is_dynamic(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return false; + return static_branch_likely(&dynamic_scs_enabled); +} + +static inline bool scs_is_enabled(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return true; + return scs_is_dynamic(); +} + #else /* CONFIG_SHADOW_CALL_STACK */ static inline void *scs_alloc(int node) { return NULL; } @@ -62,6 +78,8 @@ static inline void scs_task_reset(struct task_struct *tsk) {} static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; } static inline void scs_release(struct task_struct *tsk) {} static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; } +static inline bool scs_is_enabled(void) { return false; } +static inline bool scs_is_dynamic(void) { return false; } #endif /* CONFIG_SHADOW_CALL_STACK */ diff --git a/kernel/scs.c b/kernel/scs.c index b7e1b096d906..d7809affe740 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -12,6 +12,10 @@ #include #include +#ifdef CONFIG_DYNAMIC_SCS +DEFINE_STATIC_KEY_FALSE(dynamic_scs_enabled); +#endif + static void __scs_account(void *s, int account) { struct page *scs_page = vmalloc_to_page(s); @@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu) void __init scs_init(void) { + if (!scs_is_enabled()) + return; cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL, scs_cleanup); } int scs_prepare(struct task_struct *tsk, int node) { - void *s = scs_alloc(node); + void *s; + if (!scs_is_enabled()) + return 0; + + s = scs_alloc(node); if (!s) return -ENOMEM; @@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk) { void *s = task_scs(tsk); - if (!s) + if (!scs_is_enabled() || !s) return; WARN(task_scs_end_corrupted(tsk),