From patchwork Mon Aug 22 09:50:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12950444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A97CCC32789 for ; Mon, 22 Aug 2022 09:52:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jaPX5lStH4v2Qz6/8XVjXJXNCfO6to3HAgbDTePABkA=; b=VzmOIS8ymokwzS kuMsyE2/vqfun9b6J2Ohh4/RXwUupYg5ZH0ZonxYxHkJd/XBsPXsDs0RZLAf1pxFgzH/2hcMqZX5o kXOyrlz5pg7/FrqcnUmpQK5HpHxsyMZ8ZjokrfUAKlcQ3uzY9qIrk9YM5iQhUQrNQx7neg5lLSKej Q6LWc3fCX1jE2d6KTHev3jwDIgMpx9OmtbxcNEVjBfCQpYx1nKO1p70cg8zxcGfJpIvXDPcHcsJSe XHWC8DeJkgsFjJq1ncy6OeAyIqUmXkOH/0cVmcWL/fk7LX35qxeLoRYKdPcS5osXlsiQXuIFONkV/ 0+6sIXyfWIe6tNv/904g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQ460-007CQK-Js; Mon, 22 Aug 2022 09:51:48 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oQ45j-007CHB-FI for linux-arm-kernel@lists.infradead.org; Mon, 22 Aug 2022 09:51:33 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 15C99B80F1A; Mon, 22 Aug 2022 09:51:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B40CBC43470; Mon, 22 Aug 2022 09:51:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1661161888; bh=QcEqjjv33fGOj9DczjqQoSgwlFVbpip/u6RMi36/Yro=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NJDITOpBG8j5kplvsnqWBiVu2ZACtqudCqAj0m549AFBhoSsJ0rFLcbUBObH23wf0 pPZu0aPuHGYDCjnbuJWR9visy7ACh/rSVHGrqv4ROOUdsaS/mDNhZYtXBIo98nECta o5bPP1UP10cw7ZUWIb7+aipQs0Gn5M/tYO1L8/4o2aChXbhM0xQPsS9Lvd8zirAO48 tzj+FvocO/19BTCZWa24o/IEathM5004zVkAbLGWZHpEXSJScnxgYzY2pm2KDhfNwC 6h5p2LQadA6GTgaRDvfQijOEOX2/N7cdLWsoiBf8yN1T/g0DXYv1YxlqgVmyWOwXCs Qr2+qRznWNpRQ== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Will Deacon , Catalin Marinas , Marc Zyngier , Mark Rutland , Mark Brown , Sami Tolvanen , Nick Desaulniers , Kees Cook Subject: [PATCH v5 2/3] scs: add support for dynamic shadow call stacks Date: Mon, 22 Aug 2022 11:50:57 +0200 Message-Id: <20220822095058.2912704-3-ardb@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220822095058.2912704-1-ardb@kernel.org> References: <20220822095058.2912704-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4239; i=ardb@kernel.org; h=from:subject; bh=QcEqjjv33fGOj9DczjqQoSgwlFVbpip/u6RMi36/Yro=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBjA1F/iA69Va1cZ7IjRS6s5SdyOUVbfPbMZ9v6uNNT 2WymJAOJAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYwNRfwAKCRDDTyI5ktmPJCRrC/ 9jl1fNpniZOHxP5EP8TIM7rMAaGr9KFjXfwa+kVnHmQxAM0qJW/ZV1WycmDz8O2Cl4+yP7fT1Vl+ri +ESC/vZI7BO9cV6B2Tlh1rSVYIbu12iDs5sZVq/yHUsBGcAp52N/cQm7bjNrw3CvNElRE/BLcgL2mB Yw/BzOx/fOR8ZLncTaTSDxCS1g/uK0lQQb7yzcHt5Y4OF4VZYbwatyLSWiGkeIn+grELyz/hnUmpSV JR5cFoUUjczdAItfAVwJ9sHXonApq34eMOcytbfUOGGgZANG7UEQjRxfKjLJPpo6ao6ogg7g1cgQVy 7Q57rrknl8T4KrykewuhcWVB8pPlU0qTS9cQMYkKInp745z9SLNwgkTF5kFF7cO5pP95/HrYsTr5g9 nJReVnytpxN4esYiKvdYAriRkCcEQSlBsykh2JB+JuuBJEZ53Sb18r6poiWSd/0it3Zt8xPDMmaVof NVJMT546959F6k7wYvXICANgo1dUH9WUXnrvpoj+l5xP0= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220822_025131_841340_2FE81998 X-CRM114-Status: GOOD ( 21.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to allow arches to use code patching to conditionally emit the shadow stack pushes and pops, rather than always taking the performance hit even on CPUs that implement alternatives such as stack pointer authentication on arm64, add a Kconfig symbol that can be set by the arch to omit the SCS codegen itself, without otherwise affecting how support code for SCS and compiler options (for register reservation, for instance) are emitted. Also, add a static key and some plumbing to omit the allocation of shadow call stack for dynamic SCS configurations if SCS is disabled at runtime. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers Reviewed-by: Kees Cook Reviewed-by: Sami Tolvanen Tested-by: Sami Tolvanen --- Makefile | 2 ++ arch/Kconfig | 7 +++++++ include/linux/scs.h | 18 ++++++++++++++++++ kernel/scs.c | 14 ++++++++++++-- 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index c7705f749601..089d372a6f6d 100644 --- a/Makefile +++ b/Makefile @@ -888,8 +888,10 @@ LDFLAGS_vmlinux += --gc-sections endif ifdef CONFIG_SHADOW_CALL_STACK +ifndef CONFIG_DYNAMIC_SCS CC_FLAGS_SCS := -fsanitize=shadow-call-stack KBUILD_CFLAGS += $(CC_FLAGS_SCS) +endif export CC_FLAGS_SCS endif diff --git a/arch/Kconfig b/arch/Kconfig index 5dbf11a5ba4e..f090cdca7508 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -645,6 +645,13 @@ config SHADOW_CALL_STACK reading and writing arbitrary memory may be able to locate them and hijack control flow by modifying the stacks. +config DYNAMIC_SCS + bool + help + Set by the arch code if it relies on code patching to insert the + shadow call stack push and pop instructions rather than on the + compiler. + config LTO bool help diff --git a/include/linux/scs.h b/include/linux/scs.h index 18122d9e17ff..4ab5bdc898cf 100644 --- a/include/linux/scs.h +++ b/include/linux/scs.h @@ -53,6 +53,22 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk) return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC; } +DECLARE_STATIC_KEY_FALSE(dynamic_scs_enabled); + +static inline bool scs_is_dynamic(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return false; + return static_branch_likely(&dynamic_scs_enabled); +} + +static inline bool scs_is_enabled(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return true; + return scs_is_dynamic(); +} + #else /* CONFIG_SHADOW_CALL_STACK */ static inline void *scs_alloc(int node) { return NULL; } @@ -62,6 +78,8 @@ static inline void scs_task_reset(struct task_struct *tsk) {} static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; } static inline void scs_release(struct task_struct *tsk) {} static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; } +static inline bool scs_is_enabled(void) { return false; } +static inline bool scs_is_dynamic(void) { return false; } #endif /* CONFIG_SHADOW_CALL_STACK */ diff --git a/kernel/scs.c b/kernel/scs.c index b7e1b096d906..d7809affe740 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -12,6 +12,10 @@ #include #include +#ifdef CONFIG_DYNAMIC_SCS +DEFINE_STATIC_KEY_FALSE(dynamic_scs_enabled); +#endif + static void __scs_account(void *s, int account) { struct page *scs_page = vmalloc_to_page(s); @@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu) void __init scs_init(void) { + if (!scs_is_enabled()) + return; cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL, scs_cleanup); } int scs_prepare(struct task_struct *tsk, int node) { - void *s = scs_alloc(node); + void *s; + if (!scs_is_enabled()) + return 0; + + s = scs_alloc(node); if (!s) return -ENOMEM; @@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk) { void *s = task_scs(tsk); - if (!s) + if (!scs_is_enabled() || !s) return; WARN(task_scs_end_corrupted(tsk),