From patchwork Mon Mar 10 14:52:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 14010335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5C52AC35FF1 for ; Mon, 10 Mar 2025 15:42:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=js2Y6rKRWbZN2EBupnuiQxmO53R9yz977a6iMfeU+dg=; b=A5WPqqZG6Pn5ys CShwOepNTSr3YSXmZEv2bA78OUkOYu9oZCp4ckT2aXpKB+bMd8Kj/q4jDmKh6OxqVK2oA89nif80z OLLk0BwEI5KkvuLUp2tgX92gZigbUrh/ibjLk3IJBH2Rnv6Y+WqXhcLHAMDGUZ/hSURrpj2TGVMlo /4uhJwIs6yIP/mlspvUu5omwxcI2K58sJdSHjbG9dD5HcsxDLWtNoxpAXvFpgF8jCAo3nrqQWKm3J 9lhHGfahV2Th+I6EoL+qg/61rdp1EpBPnGMC1jlaGee0hRHBNnMkzyR8LlyvluPaZVqe2ntjDlACp OQ8R9iK7mYxAUcsTgaow==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1trfGy-00000003DNj-2yFY; Mon, 10 Mar 2025 15:42:32 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1treV5-000000031gD-3b5z for linux-riscv@bombadil.infradead.org; Mon, 10 Mar 2025 14:53:04 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Cc:To:In-Reply-To:References: Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Sender:Reply-To:Content-ID:Content-Description; bh=42Qh4XiAuZulv75xzl4kByDL6UOmUjop2oq2YsBLVIA=; b=KD7GTEa/7OorxI/o7K+/l4bTTr H+amttTkQwfVQBroy2NRMryki3dYfZMrlEUGK8Jyth9iV5SYd8qoTiISoozhzVJkpZGa4OW/l/azd p3xZ20z65pPT8bvAxFBP2y8TK2rcJqWbPOeQoRaKlYRnlAiw/unZXQYxTdvpJ+y1WbvO7MT6d4yok CFqnukf0f5fumIwjXfOf5qUsKaUETQS1Wq6qKPz4X39SplOxLDHTM0JNyHQmtoiyKyizdfJbwpYZE OF1ti2rGhWnUEN6X0D5WSO6Lb1PglJiEfZEFN6dWo4b+hRTZ1R7+dO8t1md4ahAHX+AB3EWSelcwx Bx69s3ag==; Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1treV2-00000001sNQ-3u5k for linux-riscv@lists.infradead.org; Mon, 10 Mar 2025 14:53:02 +0000 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-2235189adaeso72294115ad.0 for ; Mon, 10 Mar 2025 07:53:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1741618379; x=1742223179; darn=lists.infradead.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=42Qh4XiAuZulv75xzl4kByDL6UOmUjop2oq2YsBLVIA=; b=fm9bfYGUWhq6oWuR8w5V7M07MZqd89O0OntA9ZTjabddxn14iOXVRn7q/xVGWQf+72 Gnwq+agDO5nP4GF5RNpghro7TOlwXcHi/gIYpZITxMYQcv+LCm858T0ek4ZPXvHfxIIR b3UYVznNYvrxSp1ftipFkthlPEPRc7eM3Dv7GpKJhr6L/24Nv1wCeN19e2pI1EvgwWed mHCsL8vk5JY4OGxMqsY+q7F2w0KD0c2D2hUOvAMbD3/FWeh4/tVlzmQd57OEG9fWPsDX 1ADv8uxaUeBET7EWOUXGNZPK/BlGj6epZUv5eiQBDIvWvmnaYcxhdg18u3Tz/sz7TJcH 6V4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741618379; x=1742223179; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=42Qh4XiAuZulv75xzl4kByDL6UOmUjop2oq2YsBLVIA=; b=lWSCQS19bEyvJ9m2B8p5l4YeHj/iN7yMk4wX3I0Yo1psQMLWwPfriNactwPTiL7XBv 4QKblICyxc+xSVLPasMrQZk/gIASs1oUayHH+JjYT8RfcRnX2EYQXcWuvlfRedIJxs2q fU93vzmJ+/2aOnO0ydah2vWcHc/j+buc9FMbEIhtlWlwuF8qytEWgiLNcc1Htriak5Be chOoto0lA4C3iF+13/MkqTaClIlmH+ZNR6PYsE3Yh7VC+Vg6n6C/zCiC00wHsPWnJrO3 Lq6BZw/wuIootD/oyWW9ys/lU0aDHfylXSKEzl8FoziP3EKV44TyED/kYoQuKxlvpJDf hWQg== X-Forwarded-Encrypted: i=1; AJvYcCWlb2T2Ag62tZgity5AY91Ru8XRPhHKdMoKPELJAuMV2wYJ3j4qJX/AsseWQUuB2jcRALG6j6Ag5R9RRA==@lists.infradead.org X-Gm-Message-State: AOJu0Ywlxt3UsK+pJpQ0SKjDTuXeQL4rAO3k01iYkXEAqLM2EGEkUU2t edNr8pZyomhdtkIVx8Zuq6SkWUCvW9k4pIwmEcaK3e1R9NOn+WhMbA8+kLP+HjY= X-Gm-Gg: ASbGncvpXSOjgY6HBY52gLEiJUU3sM2FOavyTLjHp61y71gcb23JAbQx45N78kE+VLu f2K+O2kATd8onItf8S8X1jyeu7NSajEwh87oJo4IfQptPV6D2wPy4pKYbeglXu1LygAUl6djmoz N5dcPM4MA+mapVy3kjIy/D/FT5tg5deN0M5jd/AU2wp7LRbditTYVAayVZZSiEMyE5the4DrRhU 3W5hbmEotFgnemZeIwXyhD6X84PgUyJvIMClRfckCzMhu/F4PlOdtLrsCqWe5TfsvzJCVyGTi4K 8pJnoPJpEENb4d9JxPPM0UD9kJ41drdI6g0coLSt3moWfaP222mk71o= X-Google-Smtp-Source: AGHT+IHEXGQJboz6H1D26OUIQwYNlDx0HzkIlJbeLtq0a2EzFTAzmNT4PJCmia/gb9gcmwxF4dQMsA== X-Received: by 2002:a17:902:e74d:b0:223:517a:d5a9 with SMTP id d9443c01a7336-22464503452mr163981475ad.15.1741618379437; Mon, 10 Mar 2025 07:52:59 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-736d11d4600sm2890275b3a.116.2025.03.10.07.52.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 10 Mar 2025 07:52:59 -0700 (PDT) From: Deepak Gupta Date: Mon, 10 Mar 2025 07:52:33 -0700 Subject: [PATCH v11 11/27] riscv/shstk: If needed allocate a new shadow stack on clone MIME-Version: 1.0 Message-Id: <20250310-v5_user_cfi_series-v11-11-86b36cbfb910@rivosinc.com> References: <20250310-v5_user_cfi_series-v11-0-86b36cbfb910@rivosinc.com> In-Reply-To: <20250310-v5_user_cfi_series-v11-0-86b36cbfb910@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , Arnd Bergmann , Christian Brauner , Peter Zijlstra , Oleg Nesterov , Eric Biederman , Kees Cook , Jonathan Corbet , Shuah Khan , Jann Horn , Conor Dooley Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, devicetree@vger.kernel.org, linux-arch@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, alistair.francis@wdc.com, richard.henderson@linaro.org, jim.shu@sifive.com, andybnac@gmail.com, kito.cheng@sifive.com, charlie@rivosinc.com, atishp@rivosinc.com, evan@rivosinc.com, cleger@rivosinc.com, alexghiti@rivosinc.com, samitolvanen@google.com, broonie@kernel.org, rick.p.edgecombe@intel.com, Deepak Gupta X-Mailer: b4 0.14.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250310_145301_212617_AD1D9EE5 X-CRM114-Status: GOOD ( 29.79 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Userspace specifies CLONE_VM to share address space and spawn new thread. `clone` allow userspace to specify a new stack for new thread. However there is no way to specify new shadow stack base address without changing API. This patch allocates a new shadow stack whenever CLONE_VM is given. In case of CLONE_VFORK, parent is suspended until child finishes and thus can child use parent shadow stack. In case of !CLONE_VM, COW kicks in because entire address space is copied from parent to child. `clone3` is extensible and can provide mechanisms using which shadow stack as an input parameter can be provided. This is not settled yet and being extensively discussed on mailing list. Once that's settled, this commit will adapt to that. Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/mmu_context.h | 7 ++ arch/riscv/include/asm/usercfi.h | 25 ++++++++ arch/riscv/kernel/process.c | 9 +++ arch/riscv/kernel/usercfi.c | 120 +++++++++++++++++++++++++++++++++++ 4 files changed, 161 insertions(+) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/mmu_context.h index 8c4bc49a3a0f..dbf27a78df6c 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -48,6 +48,13 @@ static inline unsigned long mm_untag_mask(struct mm_struct *mm) } #endif +#define deactivate_mm deactivate_mm +static inline void deactivate_mm(struct task_struct *tsk, + struct mm_struct *mm) +{ + shstk_release(tsk); +} + #include #endif /* _ASM_RISCV_MMU_CONTEXT_H */ diff --git a/arch/riscv/include/asm/usercfi.h b/arch/riscv/include/asm/usercfi.h index 5f2027c51917..82d28ac98d76 100644 --- a/arch/riscv/include/asm/usercfi.h +++ b/arch/riscv/include/asm/usercfi.h @@ -8,6 +8,9 @@ #ifndef __ASSEMBLY__ #include +struct task_struct; +struct kernel_clone_args; + #ifdef CONFIG_RISCV_USER_CFI struct cfi_status { unsigned long ubcfi_en : 1; /* Enable for backward cfi. */ @@ -17,6 +20,28 @@ struct cfi_status { unsigned long shdw_stk_size; /* size of shadow stack */ }; +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args); +void shstk_release(struct task_struct *tsk); +void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, unsigned long size); +unsigned long get_shstk_base(struct task_struct *task, unsigned long *size); +void set_active_shstk(struct task_struct *task, unsigned long shstk_addr); +bool is_shstk_enabled(struct task_struct *task); + +#else + +#define shstk_alloc_thread_stack(tsk, args) 0 + +#define shstk_release(tsk) + +#define get_shstk_base(task, size) 0UL + +#define set_shstk_base(task, shstk_addr, size) + +#define set_active_shstk(task, shstk_addr) + +#define is_shstk_enabled(task) false + #endif /* CONFIG_RISCV_USER_CFI */ #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index 7c244de77180..99acb6342a37 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -29,6 +29,7 @@ #include #include #include +#include #if defined(CONFIG_STACKPROTECTOR) && !defined(CONFIG_STACKPROTECTOR_PER_TASK) #include @@ -211,6 +212,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) unsigned long clone_flags = args->flags; unsigned long usp = args->stack; unsigned long tls = args->tls; + unsigned long ssp = 0; struct pt_regs *childregs = task_pt_regs(p); /* Ensure all threads in this mm have the same pointer masking mode. */ @@ -229,11 +231,18 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) p->thread.s[0] = (unsigned long)args->fn; p->thread.s[1] = (unsigned long)args->fn_arg; } else { + /* allocate new shadow stack if needed. In case of CLONE_VM we have to */ + ssp = shstk_alloc_thread_stack(p, args); + if (IS_ERR_VALUE(ssp)) + return PTR_ERR((void *)ssp); + *childregs = *(current_pt_regs()); /* Turn off status.VS */ riscv_v_vstate_off(childregs); if (usp) /* User fork */ childregs->sp = usp; + /* if needed, set new ssp */ + ssp ? set_active_shstk(p, ssp) : 0; if (clone_flags & CLONE_SETTLS) childregs->tp = tls; childregs->a0 = 0; /* Return value of fork() */ diff --git a/arch/riscv/kernel/usercfi.c b/arch/riscv/kernel/usercfi.c index 24022809a7b5..73cf87dab186 100644 --- a/arch/riscv/kernel/usercfi.c +++ b/arch/riscv/kernel/usercfi.c @@ -19,6 +19,41 @@ #define SHSTK_ENTRY_SIZE sizeof(void *) +bool is_shstk_enabled(struct task_struct *task) +{ + return task->thread_info.user_cfi_state.ubcfi_en ? true : false; +} + +void set_shstk_base(struct task_struct *task, unsigned long shstk_addr, unsigned long size) +{ + task->thread_info.user_cfi_state.shdw_stk_base = shstk_addr; + task->thread_info.user_cfi_state.shdw_stk_size = size; +} + +unsigned long get_shstk_base(struct task_struct *task, unsigned long *size) +{ + if (size) + *size = task->thread_info.user_cfi_state.shdw_stk_size; + return task->thread_info.user_cfi_state.shdw_stk_base; +} + +void set_active_shstk(struct task_struct *task, unsigned long shstk_addr) +{ + task->thread_info.user_cfi_state.user_shdw_stk = shstk_addr; +} + +/* + * If size is 0, then to be compatible with regular stack we want it to be as big as + * regular stack. Else PAGE_ALIGN it and return back + */ +static unsigned long calc_shstk_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); +} + /* * Writes on shadow stack can either be `sspush` or `ssamoswap`. `sspush` can happen * implicitly on current shadow stack pointed to by CSR_SSP. `ssamoswap` takes pointer to @@ -142,3 +177,88 @@ SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsi return allocate_shadow_stack(addr, aligned_size, size, set_tok); } + +/* + * This gets called during clone/clone3/fork. And is needed to allocate a shadow stack for + * cases where CLONE_VM is specified and thus a different stack is specified by user. We + * thus need a separate shadow stack too. How does separate shadow stack is specified by + * user is still being debated. Once that's settled, remove this part of the comment. + * This function simply returns 0 if shadow stack are not supported or if separate shadow + * stack allocation is not needed (like in case of !CLONE_VM) + */ +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, + const struct kernel_clone_args *args) +{ + unsigned long addr, size; + + /* If shadow stack is not supported, return 0 */ + if (!cpu_supports_shadow_stack()) + return 0; + + /* + * If shadow stack is not enabled on the new thread, skip any + * switch to a new shadow stack. + */ + if (!is_shstk_enabled(tsk)) + return 0; + + /* + * For CLONE_VFORK the child will share the parents shadow stack. + * Set base = 0 and size = 0, this is special means to track this state + * so the freeing logic run for child knows to leave it alone. + */ + if (args->flags & CLONE_VFORK) { + set_shstk_base(tsk, 0, 0); + return 0; + } + + /* + * For !CLONE_VM the child will use a copy of the parents shadow + * stack. + */ + if (!(args->flags & CLONE_VM)) + return 0; + + /* + * reaching here means, CLONE_VM was specified and thus a separate shadow + * stack is needed for new cloned thread. Note: below allocation is happening + * using current mm. + */ + size = calc_shstk_size(args->stack_size); + addr = allocate_shadow_stack(0, size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + set_shstk_base(tsk, addr, size); + + return addr + size; +} + +void shstk_release(struct task_struct *tsk) +{ + unsigned long base = 0, size = 0; + /* If shadow stack is not supported or not enabled, nothing to release */ + if (!cpu_supports_shadow_stack() || !is_shstk_enabled(tsk)) + return; + + /* + * When fork() with CLONE_VM fails, the child (tsk) already has a + * shadow stack allocated, and exit_thread() calls this function to + * free it. In this case the parent (current) and the child share + * the same mm struct. Move forward only when they're same. + */ + if (!tsk->mm || tsk->mm != current->mm) + return; + + /* + * We know shadow stack is enabled but if base is NULL, then + * this task is not managing its own shadow stack (CLONE_VFORK). So + * skip freeing it. + */ + base = get_shstk_base(tsk, &size); + if (!base) + return; + + vm_munmap(base, size); + set_shstk_base(tsk, 0, 0); +}