From patchwork Wed Aug 28 23:27:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13782183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7351BC7114C for ; Wed, 28 Aug 2024 23:31:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0397F6B00B9; Wed, 28 Aug 2024 19:31:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F2C8E6B00BA; Wed, 28 Aug 2024 19:31:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA6636B00BB; Wed, 28 Aug 2024 19:31:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B95D26B00B9 for ; Wed, 28 Aug 2024 19:31:10 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7C85780210 for ; Wed, 28 Aug 2024 23:31:10 +0000 (UTC) X-FDA: 82503252300.07.1C9A0CB Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf22.hostedemail.com (Postfix) with ESMTP id 8B250C0009 for ; Wed, 28 Aug 2024 23:31:08 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mP13yPt9; spf=pass (imf22.hostedemail.com: domain of broonie@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724887751; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a60SLytywbIllR3a2exYWzAT6UZq1WIqVq2AukFTKeA=; b=Y/ht3ROL8jUXxKEoDuS9JMo9cCqIGfOA5NZ0kKJM0qT/7hhuUl7xjyt93iiLXH/HHPK7du 8RmVBk5gKR7UL5kyYTMRwA0pa17rqDLFFrCzL1vIRTzVtDVuUjoWhHm43cL0CGi3p6/+yH 3iOeLI3Afp9uMRr0PD9Sdo7L9eVQDh8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mP13yPt9; spf=pass (imf22.hostedemail.com: domain of broonie@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724887751; a=rsa-sha256; cv=none; b=QRYAhAcgsizf2zORdYqkVp4++IgJHpohBuNymm8J7H7T7YfVB01ZZ5UKPEnq7KB9iS2Ecp J83U5TZPhR4Boc9TFDFIXQ6chjIOqUke2Vs5dhRVy8uG1qOoMjKcj4dGyq6Qj4z3QM0fP1 fvl+SjcUuSPAuu4AQJJv20s/1tYoEn4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 749F1AE3F03; Wed, 28 Aug 2024 23:31:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 55C61C4CEC9; Wed, 28 Aug 2024 23:30:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724887866; bh=r1EPj2HNqmDTMAGJCm8ZuYXl6ItJtVM0L7jZMyZ+dfk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mP13yPt96U0NJ8JNGBGVVgP2AVcIQPF58gZXM8j0Qy0kQa8jYYDC5VYSzfyDFEl5g 61Veao72PGikqGnoud+HUsiYZTclcPIDUbZo+UCKp3elNPD+z1QjmxY4p5I3v7VFkY T1EOk1nJd5zpitaaSNM+Xw7iszKf/gxjxoc9ZwZx7DSjRCZAPxRNgMQgXn2um572RJ 8PeCXYX3WCB10UyDizQx5bBu3cjCPmKJ1/RWippRiQMyBX5kQfT2hqVKmqiYeQVF2o OxA17g1emtKnHYIekmQp1NlcayCS7Sk13SsgrYqwCd5ERaDnDwJYwG3gno0nfT4zVt oZBqTUqd2wVew== From: Mark Brown Date: Thu, 29 Aug 2024 00:27:37 +0100 Subject: [PATCH v12 21/39] arm64/gcs: Ensure that new threads have a GCS MIME-Version: 1.0 Message-Id: <20240829-arm64-gcs-v12-21-42fec947436a@kernel.org> References: <20240829-arm64-gcs-v12-0-42fec947436a@kernel.org> In-Reply-To: <20240829-arm64-gcs-v12-0-42fec947436a@kernel.org> To: Catalin Marinas , Will Deacon , Jonathan Corbet , Andrew Morton , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Arnd Bergmann , Oleg Nesterov , Eric Biederman , Shuah Khan , "Rick P. Edgecombe" , Deepak Gupta , Ard Biesheuvel , Szabolcs Nagy , Kees Cook Cc: "H.J. Lu" , Paul Walmsley , Palmer Dabbelt , Albert Ou , Florian Weimer , Christian Brauner , Thiago Jung Bauermann , Ross Burton , Yury Khrustalev , Wilco Dijkstra , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Mark Brown X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=6471; i=broonie@kernel.org; h=from:subject:message-id; bh=r1EPj2HNqmDTMAGJCm8ZuYXl6ItJtVM0L7jZMyZ+dfk=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmz7KI/5/EZA8otPXJTG4eAk+IZZZZk3RvvbHfAmvN zK4Jf5iJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZs+yiAAKCRAk1otyXVSH0PymB/ 4klnjT1lRlENsooIrfLx8Z6FCqQScC0UWLZeDIMhUJOIuVgoKoHlu6qSUErmQ/0oVXkQQ+742PWItM 5rXuixBT7PpQtlBRhFbhu1vh5dh1xlFFr9WAhxGBo5AS9Pc+K9ban4ZPFw1tXViKYBaG/QWHXqS9ZK OohbPPA12F58N+WsDHitR9kMB+hBKQzgTwBVWVFHLLaVCXYfFvXqDY4C/+sRMp7Ht+Qzvcv+NzgPn7 oMpHxAwV9HJXFzJ7k8VtlUhmVf2cDWq8mEY/yrMcMA4kE6HC40emJJm0H+L/1osaoRJvmoAEb6ajCi BQB9EXlV4kZhXUZqH/RlyGEXCfMSP5 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 8B250C0009 X-Stat-Signature: ow9w9th1dwaikxuk4g8ztgsf1agde3rn X-Rspam-User: X-HE-Tag: 1724887868-488149 X-HE-Meta: U2FsdGVkX19g6p2ymm5VRrAlShcax0B793Do45c5j5h8LZfIBL8BVmqy/R3dWP116M0vuZza/jA2aZKyx8NA9nHEmTJXHQHXGnYlRi1nSYPOQ8wd5duMY2EbqSrQgi2Y+XPJMQGtFbB8J9qColzDTmP4F4tgUGSgKcr3+5dszunfdueWPVGD0W8hEGPf7TvtQrMaL+KlZaK7/osNbEEak8ZAgOP7rTnuLhtz3szDHjp/Q0QzrXrrGtgLvmEtAVvrKMtXdRMncRHb6g8uk9hBThtHAbRsWt7KKSCr53Z16/owDh024jLZgDx+GDpKHTNmtZ7MSURz/ZnmuCL4NJaWJwdiHH+A7UlBSXeohRexMtfiV7NUOWjABefW0kcMWOiOtl7ghd01ZwhOKoaVYvTpIEYsBt0lhr8Ra8K0oGtYflmR+daUF9Ere0igBdL5cub08G7oJlQEpNKRdK36UEk27o8EBTLcA/i0+GVeHOMJvBj39be5XFLW2DqaG43m22gdxLeQnD6yuLsuoHn2cvwsqAPvCPLBZwsIvb5hw3+nLZ0Ckpgfzx80g0x9J7QX0t0ZRi/i69h5V/kYRLSkeEgruH7JWLcCHhJR7VfTs0R99s/d4B9kobwNqzwBmmnXarB0e6xGaGXeS1XhSijh7M1RdWvLOd2Pd6E0cN4oKlUWvec2ThPYscAnk7yi7HymhElpHFd80w2tZMAS7kykwQvWrwA50HVC1T7APprySdjzTtSdPVovZmLPRVwAsbrZcznaALBDcddsj+BpYAHkfzHODa0VjmvHYWFEMMSZMEvW8ZoxyFKjqmNgzlYsflbz5gy+grYw96WRgeHjpdWDxnCh/yof908NmDkpxhgnidkOl/wB1o2CR2SWPlYH1kJehJ0L42iU/gqMT7nWpBODLe44FP54OHmHBmAeM0kMNHUWOE0i+zrLxrAo5UB2KIcI6x+dpj54R8pkOXR4/f+9PbZ 2vNuOalX Vxxc3mxFU713oOmtmfibb7nyeqJAtkp3S1a9QdYanh1/mRZiA2Zxvv0OMiPfJI45OIrpjP5W8gPGLiQkrncBJ2xli+G4XRkQdfZdhSV6m+hYEvqI6c6Zh/0bVHGUzYCXQIJpM7EvPD/3T0x9865skHcCZ+YgEHinAiz43RKOrmQbg9vLPoUEgQHfIlw4BJUzqYi1ipFgUrkCcgxlzTNkdlFt+x65u1rIMm96AmRfmMekreXBIBdF3x4qfEuLhA4aUWtmZHWEd+cBtY8rmISqPaLJ2EYmCh5+Acaj/w//lg36ZirVQiIxYca5EmrhdkFp9gwfnPByN5BSxuncF8YBV//PIyVZDy/bomWDKjJt0xBmv4f66BdnGWxuwIKLAVCG4ipTh8BNTLUfMIEz+fq+hQn8k61SZYT54fOc3tGQvZSWgszvf4T+XWZFqwQKCil0Zg0DH0G5D+tZSnOFSQ5yZv7Og+siLueSD4Ghv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When a new thread is created by a thread with GCS enabled the GCS needs to be specified along with the regular stack. Unfortunately plain clone() is not extensible and existing clone3() users will not specify a stack so all existing code would be broken if we mandated specifying the stack explicitly. For compatibility with these cases and also x86 (which did not initially implement clone3() support for shadow stacks) if no GCS is specified we will allocate one so when a thread is created which has GCS enabled allocate one for it. We follow the extensively discussed x86 implementation and allocate min(RLIMIT_STACK, 2G). Since the GCS only stores the call stack and not any variables this should be more than sufficient for most applications. GCSs allocated via this mechanism will be freed when the thread exits. Reviewed-by: Catalin Marinas Reviewed-by: Thiago Jung Bauermann Signed-off-by: Mark Brown Acked-by: Yury Khrustalev --- arch/arm64/include/asm/gcs.h | 9 ++++++ arch/arm64/kernel/process.c | 26 ++++++++++++++++ arch/arm64/mm/gcs.c | 70 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 105 insertions(+) diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h index 04594ef59dad..c1f274fdb9c0 100644 --- a/arch/arm64/include/asm/gcs.h +++ b/arch/arm64/include/asm/gcs.h @@ -8,6 +8,8 @@ #include #include +struct kernel_clone_args; + static inline void gcsb_dsync(void) { asm volatile(".inst 0xd503227f" : : : "memory"); @@ -58,6 +60,8 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) void gcs_set_el0_mode(struct task_struct *task); void gcs_free(struct task_struct *task); void gcs_preserve_current_state(void); +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); #else @@ -69,6 +73,11 @@ static inline bool task_gcs_el0_enabled(struct task_struct *task) static inline void gcs_set_el0_mode(struct task_struct *task) { } static inline void gcs_free(struct task_struct *task) { } static inline void gcs_preserve_current_state(void) { } +static inline unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + return -ENOTSUPP; +} #endif diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 3622956b6515..de59aa16919c 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -285,9 +285,29 @@ static void flush_gcs(void) write_sysreg_s(0, SYS_GCSPR_EL0); } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + unsigned long gcs; + + gcs = gcs_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(gcs)) + return PTR_ERR((void *)gcs); + + p->thread.gcs_el0_mode = current->thread.gcs_el0_mode; + p->thread.gcs_el0_locked = current->thread.gcs_el0_locked; + + return 0; +} + #else static void flush_gcs(void) { } +static int copy_thread_gcs(struct task_struct *p, + const struct kernel_clone_args *args) +{ + return 0; +} #endif @@ -303,6 +323,7 @@ void flush_thread(void) void arch_release_task_struct(struct task_struct *tsk) { fpsimd_release_task(tsk); + gcs_free(tsk); } int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) @@ -366,6 +387,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long stack_start = args->stack; unsigned long tls = args->tls; struct pt_regs *childregs = task_pt_regs(p); + int ret; memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context)); @@ -407,6 +429,10 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.uw.tp_value = tls; p->thread.tpidr2_el0 = 0; } + + ret = copy_thread_gcs(p, args); + if (ret != 0) + return ret; } else { /* * A kthread has no context to ERET to, so ensure any buggy diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c index b0a67efc522b..6e8a5e14fff1 100644 --- a/arch/arm64/mm/gcs.c +++ b/arch/arm64/mm/gcs.c @@ -5,9 +5,69 @@ #include #include +#include #include +#include #include +static unsigned long alloc_gcs(unsigned long addr, unsigned long size) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + mmap_write_unlock(mm); + + return mapped_addr; +} + +static unsigned long gcs_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + /* Allocate RLIMIT_STACK/2 with limits of PAGE_SIZE..2G */ + size = PAGE_ALIGN(min_t(unsigned long long, + rlimit(RLIMIT_STACK) / 2, SZ_2G)); + return max(PAGE_SIZE, size); +} + +unsigned long gcs_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + if (!system_supports_gcs()) + return 0; + + if (!task_gcs_el0_enabled(tsk)) + return 0; + + if ((args->flags & (CLONE_VFORK | CLONE_VM)) != CLONE_VM) { + tsk->thread.gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0); + return 0; + } + + size = args->stack_size; + + size = gcs_size(size); + addr = alloc_gcs(0, size); + if (IS_ERR_VALUE(addr)) + return addr; + + tsk->thread.gcs_base = addr; + tsk->thread.gcs_size = size; + tsk->thread.gcspr_el0 = addr + size - sizeof(u64); + + return addr; +} + /* * Apply the GCS mode configured for the specified task to the * hardware. @@ -30,6 +90,16 @@ void gcs_set_el0_mode(struct task_struct *task) void gcs_free(struct task_struct *task) { + + /* + * When fork() with CLONE_VM fails, the child (tsk) already + * has a GCS allocated, and exit_thread() calls this function + * to free it. In this case the parent (current) and the + * child share the same mm struct. + */ + if (!task->mm || task->mm != current->mm) + return; + if (task->thread.gcs_base) vm_munmap(task->thread.gcs_base, task->thread.gcs_size);