From patchwork Thu May 5 16:10:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 12839763 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60690C433EF for ; Thu, 5 May 2022 16:12:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=t8JuulrmBWlfHxP6omloRLTa6TlSBkFN+qDSC0L3Y/c=; b=wRkGfKwdArm7BU F59jxdZBpZ3B9D169c6Z/BCVpUOZ9gsUiwmqqk7YWuS4yTflufBSP5Oljm2drJpyYQo2uPYyMEnJK jEpH6UGaeMys/k7HXsjpX15Qink2r8K2e2y2GNpN9+IBYGiTDrgO0AY1E+jaHOoOB9LVBl6zCGeA5 4MC6ZfH3eGhVMPOiiDS2TESggMK1FBG63GmIhAH87K0pCv4bpFIGOja2t9qEiGa34zjqGkXmXUNqP 8448RJ4PVz6v3HgbtWQrJnbCmjoaPFR+D/EQ/TWsh/ZZGQ6Vb4uSYbFRZOr1wxAQbsSEe21FsKlP7 +AhV7Eg0zdqI94x1R2RA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nme4E-00GnAB-SF; Thu, 05 May 2022 16:11:03 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nme3p-00Gn1Q-2i for linux-arm-kernel@lists.infradead.org; Thu, 05 May 2022 16:10:38 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 6A4F4B82DF4; Thu, 5 May 2022 16:10:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C0574C385A4; Thu, 5 May 2022 16:10:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651767034; bh=Clcqu/Ezs830DckhrTOlJvTLynFQaPtrk4ln1+Vrkjw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F0V3de2zt1aBnajXGAOxT+pooeY4eJElF/h14JMSfWcbjJExxAjJHoro4332KjENe N/4jrAA+510O8FuVq7QDf+k6lJbd+xy2oCNmtr8rC3ROFGXBX6g/HSFA9lNoYlYRo3 IHiku7QHarFnCt1UGTLeEpVrKkqTsdaaQ4Rj2RzWH4RhBi7xzbDei0OgWNzCryqSjx 42TpZV9fsINDyFU0nF7jIHpPcOovWpxxT+B5RwvW9UXfwr36MI+we/7yBY9UTN1STF 22tJEpkg2382UkfxlPlV0yUpRuEQgNsDpNbjFQZuZEZfV2VWWm/woy9z1/1QGlpn8y /f5lU49citx8g== From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: will@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, maz@kernel.org, Ard Biesheuvel , Kees Cook , Sami Tolvanen , Fangrui Song , Nick Desaulniers , Dan Li Subject: [RFC PATCH v2 2/3] scs: add support for dynamic shadow call stacks Date: Thu, 5 May 2022 18:10:10 +0200 Message-Id: <20220505161011.1801596-3-ardb@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220505161011.1801596-1-ardb@kernel.org> References: <20220505161011.1801596-1-ardb@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3819; h=from:subject; bh=Clcqu/Ezs830DckhrTOlJvTLynFQaPtrk4ln1+Vrkjw=; b=owEB7QES/pANAwAKAcNPIjmS2Y8kAcsmYgBic/bhmx8IcuNnb+l+fT9BVoswvmVT3K0ESJpl6iVv NE73DF2JAbMEAAEKAB0WIQT72WJ8QGnJQhU3VynDTyI5ktmPJAUCYnP24QAKCRDDTyI5ktmPJHfHC/ 93vk8iwRxhBKQG0oavOnrTrk6lthUVcHDFBrb7Dzxyj/WN0zIobXin99IOynHqvaOifmcEEn77COzF x9MkJro/uL9/KUup64+W0vh4gOWicQ0ej6VoJGhft8TFpzJw3T0rT7SQMcpgdpq3sG3Gs/1kPdqICT lmaLUSptKhkdaB0lnb6F1mzU1iXE8xMLXy8kIHMKUYWhyjb5ncrz3M8/GqMJ4Fkoh3LPrJGoAsDQJp mK119DW/9/hgnC8S6iCbtEuK6epoM3CUbMwe5A+A/yHo9ZHuw3FarWIrFkV1xtZ3jG8R9fJ2i+9s9s M4/XubLMLaaWu51uboPHGY8wYvYP207s+vVtLinllL5yFsDbH9vj1m16lBVD1pPFybJSkuAaoVvIi/ ViecxUrC1x2aMJnhIjjef/JhI0pKKU3aBOgYwnQjJbrzXF/Tjs8tNjfitE7B8jxhyefB0pvJnv5+me pMiu8MxLEjUwZyxYoJ4E7yTorYjzXXW8Cj9fUj8KCcjZs= X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220505_091037_470798_5D3490C8 X-CRM114-Status: GOOD ( 21.19 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In order to allow arches to use code patching to conditionally emit the shadow stack pushes and pops, rather than always taking the performance hit even on CPUs that implement alternatives such as stack pointer authentication on arm64, add a Kconfig symbol that can be set by the arch to omit the SCS codegen itself, without otherwise affecting how support code for SCS and compiler options (for register reservation, for instance) are emitted. Also, add a static key and some plumbing to omit the allocation of shadow call stack for dynamic SCS configurations if SCS is disabled at runtime. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers Reviewed-by: Kees Cook --- Makefile | 2 ++ arch/Kconfig | 7 +++++++ include/linux/scs.h | 10 ++++++++++ kernel/scs.c | 14 ++++++++++++-- 4 files changed, 31 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index fa5112a0ec1b..a578fffc0337 100644 --- a/Makefile +++ b/Makefile @@ -882,8 +882,10 @@ LDFLAGS_vmlinux += --gc-sections endif ifdef CONFIG_SHADOW_CALL_STACK +ifndef CONFIG_DYNAMIC_SCS CC_FLAGS_SCS := -fsanitize=shadow-call-stack KBUILD_CFLAGS += $(CC_FLAGS_SCS) +endif export CC_FLAGS_SCS endif diff --git a/arch/Kconfig b/arch/Kconfig index 29b0167c088b..126caa75969a 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -627,6 +627,13 @@ config SHADOW_CALL_STACK reading and writing arbitrary memory may be able to locate them and hijack control flow by modifying the stacks. +config DYNAMIC_SCS + bool + help + Set by the arch code if it relies on code patching to insert the + shadow call stack push and pop instructions rather than on the + compiler. + config LTO bool help diff --git a/include/linux/scs.h b/include/linux/scs.h index 18122d9e17ff..4cc01f21b17a 100644 --- a/include/linux/scs.h +++ b/include/linux/scs.h @@ -53,6 +53,15 @@ static inline bool task_scs_end_corrupted(struct task_struct *tsk) return sz >= SCS_SIZE - 1 || READ_ONCE_NOCHECK(*magic) != SCS_END_MAGIC; } +DECLARE_STATIC_KEY_TRUE(dynamic_scs_enabled); + +static inline bool scs_is_enabled(void) +{ + if (!IS_ENABLED(CONFIG_DYNAMIC_SCS)) + return true; + return static_branch_likely(&dynamic_scs_enabled); +} + #else /* CONFIG_SHADOW_CALL_STACK */ static inline void *scs_alloc(int node) { return NULL; } @@ -62,6 +71,7 @@ static inline void scs_task_reset(struct task_struct *tsk) {} static inline int scs_prepare(struct task_struct *tsk, int node) { return 0; } static inline void scs_release(struct task_struct *tsk) {} static inline bool task_scs_end_corrupted(struct task_struct *tsk) { return false; } +static inline bool scs_is_enabled(void) { return false; } #endif /* CONFIG_SHADOW_CALL_STACK */ diff --git a/kernel/scs.c b/kernel/scs.c index b7e1b096d906..8826794d2645 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -12,6 +12,10 @@ #include #include +#ifdef CONFIG_DYNAMIC_SCS +DEFINE_STATIC_KEY_TRUE(dynamic_scs_enabled); +#endif + static void __scs_account(void *s, int account) { struct page *scs_page = vmalloc_to_page(s); @@ -101,14 +105,20 @@ static int scs_cleanup(unsigned int cpu) void __init scs_init(void) { + if (!scs_is_enabled()) + return; cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL, scs_cleanup); } int scs_prepare(struct task_struct *tsk, int node) { - void *s = scs_alloc(node); + void *s; + if (!scs_is_enabled()) + return 0; + + s = scs_alloc(node); if (!s) return -ENOMEM; @@ -148,7 +158,7 @@ void scs_release(struct task_struct *tsk) { void *s = task_scs(tsk); - if (!s) + if (!scs_is_enabled() || !s) return; WARN(task_scs_end_corrupted(tsk),