From patchwork Thu Oct 27 15:59:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13022343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 01CC5FA3740 for ; Thu, 27 Oct 2022 16:00:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Pjwy6R8wpMNIr3TsN/AlRFThFTISQ79dEwOTvCkOSis=; b=KkSfofX/8dayVQ 8AgbCSd5JNNdTOb3PXUh8kojxdUcOmBlvuwv4Jx13QCCx+1goxHexnTa1O9FcpupdEIsaHRKFTzQo 9S4lXZNuSLANJQeuj5v4zfQqywRLIrWjx7lg58QmoKen+ZvZ53S+zRq52vnWt3FLgLPK9KkYMZfGZ lARHTUkwB8u3VPKDrXY/80p/rj3WZrIi6omVO0+FMUrFHAKZLTKsg5yIPWIpIZ5SSUvwGTkOoYNUp 2ao0OoMcb12YwYQpEws15VvI2EpUYA8s9WdQosQ17nT4w2CjQv8Htb7o+K+CIpiKp+sI5zmErKd0r lhoUMtMHTJtnwYPuOD1w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oo5IC-00E91K-4x; Thu, 27 Oct 2022 15:59:40 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oo5Hw-00E8yZ-L3 for linux-arm-kernel@lists.infradead.org; Thu, 27 Oct 2022 15:59:26 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 11969B826FB; Thu, 27 Oct 2022 15:59:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9C565C433D7; Thu, 27 Oct 2022 15:59:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1666886361; bh=wT+tw5Q4vteWoPSx5XbApVgN2IINrJNuZDv+NurMYYY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TVpPMmK4WpF0zaR0fCv86aQs5SfQMi0BTP1klu8ikUulvt5W99m3giLv9VQm5is0L BKXVk4qgsKpR/IsCNAhkA2PORml7tSowTXKXDZVT8FrZss7gkIaaL++RqA55Zdiw5w cCuUVT71aslD01xWo2bRTB0V/gFb5BgYG1cO45PbTO9NoZB5nZUmu8cITKW90ZKRts uWmWtJJc231NVckRuofytVq0cS74OfNi6ANAE12etu3x3Clw/JvRR+YXFg16ajZpSA Z79RWcZtEVCEox7pLsxf4Br5B0jfM7HOF1UEVK5fwnDiOmWAlXLndG1do3IPSHA+ej IZN9Pl44IhANg== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Catalin Marinas , Marc Zyngier , Mark Rutland , Mark Brown , Sami Tolvanen , Nick Desaulniers , Kees Cook Subject: [PATCH v6 2/3] scs: add support for dynamic shadow call stacks Date: Thu, 27 Oct 2022 17:59:07 +0200 Message-Id: <20221027155908.1940624-3-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20221027155908.1940624-1-ardb@kernel.org> References: <20221027155908.1940624-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4239; i=ardb@kernel.org; h=from:subject; bh=wT+tw5Q4vteWoPSx5XbApVgN2IINrJNuZDv+NurMYYY=; b=owEB7AET/pANAwAKAcNPIjmS2Y8kAcsmYgBjWqrJg/b8LGuiygg0nqQScmXE0DYGLgVKBHdsjo9n zR525yKJAbIEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCY1qqyQAKCRDDTyI5ktmPJOTfC/ Yi45DkMOU0dlUpPrZ8LEbpnYKujsukxwL1NBNc+dC67ymdzwIt2mTj3W+GoldOmND15dMWYSbKHhXl 9P5AU9Ysu0dsY93UgU+RR+tSlDPAtWrDRj0A6ZjMfO29VD3gAtS6RrT+uHgW0lI1rTky3hIk24kD53 hR+axl4WcmfvyVOJdoJnQElf4CdPcgSI+NrzFALaK/2q95o5qpdpjzNeyuD9GUYvvPFzUo3S7ccxwJ a1FMR/me+JovKhXtIHHG0d0AR/gY0VR+Gukhu9ujNrcZYdOCqKzowfxMwhXjfQjW5D1BlcprSoB7wV LDcOXTLOfDJWt2bT8HFeAoXxKLm87HBuvzSNv9gTyiY7lZYJnX9vFLQ0TFB9TD7TyQ+YyJZKOkeWsU os53Vn4wmpwO3px2JBktyFzQLLoUy0e5HAWkuYwuna3XbFU0vsAghdJqdvS/78YAYVde5Z1lM4jY1D CtWr6WBag2ffus//Sl1usm+jMKZMCyn2tQajFj6UpN8g== X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221027_085925_001966_005AA8C6 X-CRM114-Status: GOOD ( 21.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to allow arches to use code patching to conditionally emit the shadow stack pushes and pops, rather than always taking the performance hit even on CPUs that implement alternatives such as stack pointer authentication on arm64, add a Kconfig symbol that can be set by the arch to omit the SCS codegen itself, without otherwise affecting how support code for SCS and compiler options (for register reservation, for instance) are emitted. Also, add a static key and some plumbing to omit the allocation of shadow call stack for dynamic SCS configurations if SCS is disabled at runtime. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers Reviewed-by: Kees Cook Reviewed-by: Sami Tolvanen Tested-by: Sami Tolvanen --- Makefile | 2 ++ arch/Kconfig | 7 +++++++ include/linux/scs.h | 18 ++++++++++++++++++ kernel/scs.c | 14 ++++++++++++-- 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index f41ec8c8426b..53f52cf3ea3a 100644 --- a/Makefile +++ b/Makefile @@ -966,8 +966,10 @@ LDFLAGS_vmlinux += --gc-sections endif ifdef CONFIG_SHADOW_CALL_STACK +ifndef CONFIG_DYNAMIC_SCS CC_FLAGS_SCS := -fsanitize=shadow-call-stack KBUILD_CFLAGS += $(CC_FLAGS_SCS) +endif export CC_FLAGS_SCS endif diff --git a/arch/Kconfig b/arch/Kconfig index 8f138e580d1a..072a1b39e3af 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -651,6 +651,13 @@ config SHADOW_CALL_STACK reading and writing arbitrary memory may be able to locate them and hijack control flow by modifying the stacks. +config DYNAMIC_SCS + bool + help + Set by the arch code if it relies on code patching to insert the + shadow call stack push and pop instructions rather than on the + compiler. + config LTO bool help diff --git a/include/linux/scs.h b/include/linux/scs.h index 18122d9e17ff..4ab5bdc898cf 100644 --- a/include/linux/scs.h +++ b/include/linux/scs.h @@ -53,6 +53,22 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk) return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC; } +DECLARE_STATIC_KEY_FALSE(dynamic_scs_enabled); + +static inline bool scs_is_dynamic(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return false; + return static_branch_likely(&dynamic_scs_enabled); +} + +static inline bool scs_is_enabled(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return true; + return scs_is_dynamic(); +} + #else /* CONFIG_SHADOW_CALL_STACK */ static inline void *scs_alloc(int node) { return NULL; } @@ -62,6 +78,8 @@ static inline void scs_task_reset(struct task_struct *tsk) {} static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; } static inline void scs_release(struct task_struct *tsk) {} static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; } +static inline bool scs_is_enabled(void) { return false; } +static inline bool scs_is_dynamic(void) { return false; } #endif /* CONFIG_SHADOW_CALL_STACK */ diff --git a/kernel/scs.c b/kernel/scs.c index b7e1b096d906..d7809affe740 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -12,6 +12,10 @@ #include #include +#ifdef CONFIG_DYNAMIC_SCS +DEFINE_STATIC_KEY_FALSE(dynamic_scs_enabled); +#endif + static void __scs_account(void *s, int account) { struct page *scs_page = vmalloc_to_page(s); @@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu) void __init scs_init(void) { + if (!scs_is_enabled()) + return; cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL, scs_cleanup); } int scs_prepare(struct task_struct *tsk, int node) { - void *s = scs_alloc(node); + void *s; + if (!scs_is_enabled()) + return 0; + + s = scs_alloc(node); if (!s) return -ENOMEM; @@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk) { void *s = task_scs(tsk); - if (!s) + if (!scs_is_enabled() || !s) return; WARN(task_scs_end_corrupted(tsk),