From patchwork Tue Oct 1 22:59:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13818922 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31BF6CF318A for ; Tue, 1 Oct 2024 23:04:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B6AA5440161; Tue, 1 Oct 2024 19:04:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF47368002B; Tue, 1 Oct 2024 19:04:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96E3F440161; Tue, 1 Oct 2024 19:04:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7649D68002B for ; Tue, 1 Oct 2024 19:04:06 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 43B4CA0DFF for ; Tue, 1 Oct 2024 23:04:06 +0000 (UTC) X-FDA: 82626563292.20.BB57B62 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id 7526CC0011 for ; Tue, 1 Oct 2024 23:04:04 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=s0KZ+4Dm; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of broonie@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=broonie@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727823703; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DSikLik/+a9IhlW2veiqjeaWxkjCeW45LgExnC0lSw8=; b=A8rwLF1h9sHbqDr+WHfkGAs6vl8ZeO5jaj+HhoPs4koyml7p4Q9Y/kC9WPJwLYVjY5cpjR e5s64a7uehZRtgJQ3iP3ZT3HeyX9oC8eWOBe8M2Y3ujzPEp7Tb8zeay7R4RBjLAjGVR3yj /nzeOqBHSI+9l3VhouvyOpb/8Lk93ko= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727823703; a=rsa-sha256; cv=none; b=hqItDZNv4uR4lK+YMuyuuGdZL8JduRZ3iRCxD6bYmkrjM8a3YNjG8nPH2Wtb3iyEwPO4pY 577ljln1Owi7PKjMlhu+B/aA7KBQPSTlOufUv3ylPR97zSC7V7f3wfNYmJEnC8Ai2iEk8O BAFAXWgN9oqUCGxCrnDKNhJer3G1kuY= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=s0KZ+4Dm; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of broonie@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=broonie@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id A8A445C4C62; Tue, 1 Oct 2024 23:03:59 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30237C4CEC6; Tue, 1 Oct 2024 23:03:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727823843; bh=Us0URBRhSa1wij7R5AMlAWepe3NAZ1hCwtEWfYpyTZE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=s0KZ+4DmO57LtfWSEGq6YAnDihkmJYoPIG37qRni6HuPyMr3zWQ6hRuGS9/aTk65Q 1ssmYu48gex0m25/V7FJkX7yht1tdIZ9pLXNuqu09xMODWyC2cfR1yl81ePxYlLEVt s+NCCZbo5pHSrFUKyXzU0w0HFRdvw94utNVwKST1QyV8CrdIPXjXPpFWm/EwxGscL5 QMTIdumdfZ10g9iUwI/rrX3YhyVNs50mpmVRCR/98UlDvcCWH0lDyGt7Y8exwNFUIu /6UrglK/eqmhjCvlw7OcX986EexRyIR+sybhoMV7Dp/eg5+gIDhMInI70t3MmkPxhP nSOgci1rnSpZg== From: Mark Brown Date: Tue, 01 Oct 2024 23:59:01 +0100 Subject: [PATCH v13 22/40] arm64/gcs: Ensure that new threads have a GCS MIME-Version: 1.0 Message-Id: <20241001-arm64-gcs-v13-22-222b78d87eee@kernel.org> References: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> In-Reply-To: <20241001-arm64-gcs-v13-0-222b78d87eee@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , David Spickett , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=7454; i=broonie@kernel.org; h=from:subject:message-id; bh=Us0URBRhSa1wij7R5AMlAWepe3NAZ1hCwtEWfYpyTZE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm/H7Vlw2a91awf5dQ51yn2oKKo++ah0gmIhormFHO IFzKoGuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvx+1QAKCRAk1otyXVSH0GXiB/ 4sukR3e9ueVGNs3Q1p22exSd9lbXbDllUPyFJRC1zfyQjKx2EzGvmcpMdDVmk5/goay67QMxQjKDJd WntsAEwdsM3HHSD8pAZHPYIaWcfhsMw8mU1tFplNjSWdG+PH5bOQ8dOtN8qxNrMsz+YdzlcudTCoKR UgDf5yrtZkS9eH06xyw8KTSHt6uWM6MUGZ83TexJbc/KJGL4TbD03CYvzG0PE7oAvngEQ4kavKV1QH QSdtefxEFMfWOajyTpqhateKmpXWdS8jlaoqcW3pxVRbiUKRU0e5aMcS2rO+8KTK+gXtnpUR0m+OA4 VFyI+/KUpM2Sy1v7HCZ6SdXOYftgXV X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 7526CC0011 X-Stat-Signature: 74srxxj8ikkqbmwthqc5kghc6c1wh17w X-Rspam-User: X-HE-Tag: 1727823844-627474 X-HE-Meta: U2FsdGVkX1/sehUSikKdAVeatL+9QBvtUnX8CkWOO6Y8IJAOCv4NBmcfIzudwaFQqFkGUrB0NFo27zJwRqJNs1c1lAbU1nsS0yQIqaNtbceZDLbBbAckQ/yqcBGpWLP9ori4RpIBWGw5nAz+i/+LyADj0F5NW0Ims+AV8t7p9U5EJGx9Lwr24jsNlfLXSMIjp8Ds3tw7/kujI692bi235FHoGzp6T6EO8gG0MvtKl2AW0eSgHuI9O8PVtUIVUADj/YvBa/pvUUZh2130Iy9kKZrI7yvRWbn1pbsg67PntdiHfaGyznkpQVaa8f2Lf7OoC6Fqs3i6BOYSfsOO1JzwWB7Ny/JL1NcArsBNLa4T4Rkv04379z4RdUPJIy7Qpc6S8D6VyK6gk/+d/9gkwX3ZUF0LYVPgBJmYLVhZ9COm+T5pomt39haE+IqbaM50054+adlhKgpQsZBD1OJyQttx74Cgfj/rZw5vkuSdLD0/Lt9yF6UhUfLMpbrrLX+zLQhzlSbxZLWzbDq336tDO02lkf1dtlmzTSP0qeRNyjc46EhF9gjXs80TCtJgcqXqaLQxnNXUIvuTMa64HBkLM5kjusjWKGsmyFB6QcnE+CKZe0wvyaPSh6w/y2YfqoeQ3SEi1B+B7Lq5RfgKTo9Lh3YUjszN0IuOy8A07BcUH8ytrPlK1UirbUHwv99kyDjF5swuT9Li2Nk3gQhtUohRimPz4HXKzrBwXr82EDy+P6SKwKDvSpzbPZG4wY5gg/UtV+0QruZ89Qh9+n2PSfxo4ERP/Rqs/cwQ/jbzQP42A2UihNWMSrDfOHhPxjg+z4ilvsrN9XNs1B9wyQVeW3fBvpDS6JUHJtjwsujWo/vFqCK9+FWAS9IVzfadEMr2bNDYXtYg53RPL4l3SIItIMH7B52Vw83OgW8wRq2fOuyU/ErJS4WT4O3lJDKotPcIvZXw6IJKcxjXf4vezEVov5riL4J qwnSYwS3 xECN7ckHIbjBj91ks3xhvxGyD9fuKJiJke4Hfhzg1pp0c+LeW6EJAAwyTqiimmk/oLISDFW5orkoQTyFHk+WxgDeWKOZqNgyjvAq0M/Ey0vqo1NA84NnGFhSWi18QPfY1Plgjtv1Qu9fhPswQgYv8YpRYzgaKDqtwGuz2cAOiH9OVhISxMiUBdsBwusrx5n4TSD67aXCqQHPiqqxeuvINpphw652jX0bLxgkyQcDYvgRaP0TSP/mGbrdxnnNeOkWYHzXmy7FGc6QwHKG8KBP2o1u4HOEm9R8wRtckZMy7AkjgfITedYW9+Ckd504VAaOMPFOMhzNiNijr574rwfcwRFxI+kHPntTF0WfVJBrFmY9Uw0oOBPGnjOtKRM9RH/ZmbfGRw0XxxtX119pzDC+Zs30I/45NWorE4qHgakNL/YE+wfBDo2Pdt8zmqg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When a new thread is created by a thread with GCS enabled the GCS needs to be specified along with the regular stack. Unfortunately plain clone() is not extensible and existing clone3() users will not specify a stack so all existing code would be broken if we mandated specifying the stack explicitly. For compatibility with these cases and also x86 (which did not initially implement clone3() support for shadow stacks) if no GCS is specified we will allocate one so when a thread is created which has GCS enabled allocate one for it. We follow the extensively discussed x86 implementation and allocate min(RLIMIT_STACK/2, 2G). Since the GCS only stores the call stack and not any variables this should be more than sufficient for most applications. GCSs allocated via this mechanism will be freed when the thread exits. Reviewed-by: Thiago Jung Bauermann Acked-by: Yury Khrustalev Signed-off-by: Mark Brown --- arch/arm64/include/asm/gcs.h | 9 +++++ arch/arm64/include/asm/mmu_context.h | 9 +++++ arch/arm64/kernel/process.c | 32 +++++++++++++++++ arch/arm64/mm/gcs.c | 69 ++++++++++++++++++++++++++++++++++++ 4 files changed, 119 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 04594ef59dad..c1f274fdb9c0 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -8,6 +8,8 @@ #include #include +struct kernel_clone_args; + static inline void gcsb_dsync(void) { asm volatile(".inst 0xd503227f" : : : "memory"); @@ -58,6 +60,8 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) void gcs_set_el0_mode(struct task_struct *task); void gcs_free(struct task_struct *task); void gcs_preserve_current_state(void); +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); #else @@ -69,6 +73,11 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) static inline void gcs_set_el0_mode(struct task_struct *task) { } static inline void gcs_free(struct task_struct *task) { } static inline void gcs_preserve_current_state(void) { } +static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + return -ENOTSUPP; +} #endif diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 7c09d47e09cb..48b3d9553b67 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include @@ -311,6 +312,14 @@ static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, return por_el0_allows_pkey(vma_pkey(vma), write, execute); } +#define deactivate_mm deactivate_mm +static inline void deactivate_mm(struct task_struct *tsk, + struct mm_struct *mm) +{ + gcs_free(tsk); +} + + #include #endif /* !__ASSEMBLY__ */ diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index aedcf332f422..fdd095480c3f 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -294,9 +294,35 @@ static void flush_gcs(void) write_sysreg_s(0, SYS_GCSPR_EL0); } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + unsigned long gcs; + + if (!system_supports_gcs()) + return 0; + + p->thread.gcs_base = 0; + p->thread.gcs_size = 0; + + gcs = gcs_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(gcs)) + return PTR_ERR((void *)gcs); + + p->thread.gcs_el0_mode = current->thread.gcs_el0_mode; + p->thread.gcs_el0_locked = current->thread.gcs_el0_locked; + + return 0; +} + #else static void flush_gcs(void) { } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + return 0; +} #endif @@ -313,6 +339,7 @@ void flush_thread(void) void arch_release_task_struct(struct task_struct *tsk) { fpsimd_release_task(tsk); + gcs_free(tsk); } int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) @@ -376,6 +403,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long stack_start = args->stack; unsigned long tls = args->tls; struct pt_regs *childregs = task_pt_regs(p); + int ret; memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context)); @@ -420,6 +448,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.uw.tp_value = tls; p->thread.tpidr2_el0 = 0; } + + ret = copy_thread_gcs(p, args); + if (ret != 0) + return ret; } else { /* * A kthread has no context to ERET to, so ensure any buggy diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index f8f4f984a247..3c7a18f57ea9 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -5,9 +5,69 @@ #include #include +#include #include +#include #include +static unsigned long alloc_gcs(unsigned long addr, unsigned long size) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + mmap_write_unlock(mm); + + return mapped_addr; +} + +static unsigned long gcs_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + /* Allocate RLIMIT_STACK/2 with limits of PAGE_SIZE..2G */ + size = PAGE_ALIGN(min_t(unsigned long long, + rlimit(RLIMIT_STACK) / 2, SZ_2G)); + return max(PAGE_SIZE, size); +} + +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + if (!system_supports_gcs()) + return 0; + + if (!task_gcs_el0_enabled(tsk)) + return 0; + + if ((args->flags & (CLONE_VFORK | CLONE_VM)) != CLONE_VM) { + tsk->thread.gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0); + return 0; + } + + size = args->stack_size / 2; + + size = gcs_size(size); + addr = alloc_gcs(0, size); + if (IS_ERR_VALUE(addr)) + return addr; + + tsk->thread.gcs_base = addr; + tsk->thread.gcs_size = size; + tsk->thread.gcspr_el0 = addr + size - sizeof(u64); + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. @@ -33,6 +93,15 @@ void gcs_free(struct task_struct *task) if (!system_supports_gcs()) return; + /* + * When fork() with CLONE_VM fails, the child (tsk) already + * has a GCS allocated, and exit_thread() calls this function + * to free it. In this case the parent (current) and the + * child share the same mm struct. + */ + if (!task->mm || task->mm != current->mm) + return; + if (task->thread.gcs_base) vm_munmap(task->thread.gcs_base, task->thread.gcs_size);