From patchwork Mon Jun 13 13:40:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12879570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 26B24C43334 for ; Mon, 13 Jun 2022 13:43:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8JppIXBoh63IPWe4LqENWGjaK2RnBQmAcnYqcOlgZqg=; b=MaFiQnPISmokTT pXzSCn8SooDbGH05hb0mIb+M7pwm12f/trw59BXa1WfOeMVyBgjndKGfQVzcD4MtCRbQt2lrtmGFE GqNAh04FjCZikGVbNsrMP1QPeLefx1DCXgCIxfh81BVfH5vkHtti8JWCggQtUVx9NI0XSkpDvLL6u bDIAIH8dnVrIyjbyk759uknovxqg8OCr/cL9zWD4KhQ1rFnCz118hNB+VQGoZ1YnesolCgvIv4MBg v2Vs91/fZdvPsUw9oCT/herkThlzsD+4un/LLDFJBl23QIIXLfaNFSiEsb/uxSUnKkv4jwMOciNk+ 5k41xLGjJvgyOPhQE3/g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o0kKW-003vr0-08; Mon, 13 Jun 2022 13:42:08 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o0kJ0-003ukL-2B for linux-arm-kernel@lists.infradead.org; Mon, 13 Jun 2022 13:40:36 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 58165B80FF2; Mon, 13 Jun 2022 13:40:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7677C341C5; Mon, 13 Jun 2022 13:40:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1655127627; bh=gFTpl/lSD8aRvKzmss0H1hIs+Fw6dAo3BaCeU1U32qY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ml9Vh1SDIrEBt06BNmO/WS846tR8edxyvlIgUIDpkWJIglraGe3tfEJppbIwzYSAU lB/zc4tmLkcPBIqOMuMO1/BhcPDRqQK2W2djYxMpPJgSnKMJTBgylu2bpRXsGw+Zqo YQ43HWNJQfBDUcCZsEf8D8AWaD4z0FtP03qPew6D2LTpUBWw48EV8Q/rCY2Eht1xWA vc9vEH9WWOlhgMe5jQTDoP+WkNWYAkl+T3tsf0j9kFJ4HEJ3l4b/24FkUWu9pEq2fQ /S3MpoX++ppPopnFnHeVsO5WFjWxy6DGvpHc0JiJ8Ui3bqfMWCMiM37/NasDzNluxC Z3oko8lHgpM5g== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, maz@kernel.org, Ard Biesheuvel , Kees Cook , Sami Tolvanen , Fangrui Song , Nick Desaulniers , Dan Li , Kees Cook Subject: [PATCH v3 2/3] scs: add support for dynamic shadow call stacks Date: Mon, 13 Jun 2022 15:40:07 +0200 Message-Id: <20220613134008.3760481-3-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220613134008.3760481-1-ardb@kernel.org> References: <20220613134008.3760481-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4071; h=from:subject; bh=gFTpl/lSD8aRvKzmss0H1hIs+Fw6dAo3BaCeU1U32qY=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBipz41TL6xg2J+zemcOPL6dgAz4R+eT5PXYGpKUca8 XzxPRUyJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYqc+NQAKCRDDTyI5ktmPJLU/C/ 9IcP8SXLHxTEl66ilBmNAVvPZ6rxVkxCrPhwb52Uhm+U1dvSNLv+g5MAl8KyR58Ad2uxMDvxs8/2g8 UsEaznZIKkknQ/f3GZadHDTn63lwLywjG0ZuMy2IT1I2wOP2kLH5fDic/0k5AnE79SOYkHkh62HFsU zIDiLpwmUROxuOrPYqXjclP1QS6onQesF/e3AZYilWmCJn0DM+JhMVZFILe8+j4rCGc/93hs3ixZiZ t6dXoeKgw/hIqQ/kans//TQTnseO/FTEVoQ5iuUEGAInj31+uMxSFuUiIKCQ+sQoQeQZz03Wn5qkvb V8XjjON1uY9rvuA+FaZpwzGK+rvFf9yYN+bmh37bZUdMAUfLi9K6G1b6bO3zIXUDiI/MtpHUrHFS3B Mp3QfXurBL7XLIiMuK9FU7BzDdWBy5E+0/oEH7fwYs8PNHB+1r5IujjSaGFnFmQ2/sEiLCNdanfVWB mCjJAFwAGFgZa1aROr/vDIje30YmWMArN40HK4f3ScVMs= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220613_064034_439200_535FC6C3 X-CRM114-Status: GOOD ( 20.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to allow arches to use code patching to conditionally emit the shadow stack pushes and pops, rather than always taking the performance hit even on CPUs that implement alternatives such as stack pointer authentication on arm64, add a Kconfig symbol that can be set by the arch to omit the SCS codegen itself, without otherwise affecting how support code for SCS and compiler options (for register reservation, for instance) are emitted. Also, add a static key and some plumbing to omit the allocation of shadow call stack for dynamic SCS configurations if SCS is disabled at runtime. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers Reviewed-by: Kees Cook Reviewed-by: Sami Tolvanen --- Makefile | 2 ++ arch/Kconfig | 7 +++++++ include/linux/scs.h | 16 ++++++++++++++++ kernel/scs.c | 14 ++++++++++++-- 4 files changed, 37 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index c43d825a3c4c..806b1dea1218 100644 --- a/Makefile +++ b/Makefile @@ -883,8 +883,10 @@ LDFLAGS_vmlinux += --gc-sections endif ifdef CONFIG_SHADOW_CALL_STACK +ifndef CONFIG_DYNAMIC_SCS CC_FLAGS_SCS := -fsanitize=shadow-call-stack KBUILD_CFLAGS += $(CC_FLAGS_SCS) +endif export CC_FLAGS_SCS endif diff --git a/arch/Kconfig b/arch/Kconfig index fcf9a41a4ef5..a6048d78f05d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -636,6 +636,13 @@ config SHADOW_CALL_STACK reading and writing arbitrary memory may be able to locate them and hijack control flow by modifying the stacks. +config DYNAMIC_SCS + bool + help + Set by the arch code if it relies on code patching to insert the + shadow call stack push and pop instructions rather than on the + compiler. + config LTO bool help diff --git a/include/linux/scs.h b/include/linux/scs.h index 18122d9e17ff..c62134d89c7b 100644 --- a/include/linux/scs.h +++ b/include/linux/scs.h @@ -53,6 +53,20 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk) return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC; } +DECLARE_STATIC_KEY_TRUE(dynamic_scs_enabled); + +static inline bool scs_is_dynamic(void) +{ + return static_branch_likely(&dynamic_scs_enabled); +} + +static inline bool scs_is_enabled(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return true; + return scs_is_dynamic(); +} + #else /* CONFIG_SHADOW_CALL_STACK */ static inline void *scs_alloc(int node) { return NULL; } @@ -62,6 +76,8 @@ static inline void scs_task_reset(struct task_struct *tsk) {} static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; } static inline void scs_release(struct task_struct *tsk) {} static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; } +static inline bool scs_is_enabled(void) { return false; } +static inline bool scs_is_dynamic(void) { return false; } #endif /* CONFIG_SHADOW_CALL_STACK */ diff --git a/kernel/scs.c b/kernel/scs.c index b7e1b096d906..8826794d2645 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -12,6 +12,10 @@ #include #include +#ifdef CONFIG_DYNAMIC_SCS +DEFINE_STATIC_KEY_TRUE(dynamic_scs_enabled); +#endif + static void __scs_account(void *s, int account) { struct page *scs_page = vmalloc_to_page(s); @@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu) void __init scs_init(void) { + if (!scs_is_enabled()) + return; cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL, scs_cleanup); } int scs_prepare(struct task_struct *tsk, int node) { - void *s = scs_alloc(node); + void *s; + if (!scs_is_enabled()) + return 0; + + s = scs_alloc(node); if (!s) return -ENOMEM; @@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk) { void *s = task_scs(tsk); - if (!s) + if (!scs_is_enabled() || !s) return; WARN(task_scs_end_corrupted(tsk),