From patchwork Fri Oct 11 00:32:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 13831615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33518D24462 for ; Fri, 11 Oct 2024 00:32:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8E3E6B0082; Thu, 10 Oct 2024 20:32:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A3F3D6B0083; Thu, 10 Oct 2024 20:32:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B74B6B0088; Thu, 10 Oct 2024 20:32:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 67C6B6B0082 for ; Thu, 10 Oct 2024 20:32:25 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id BB15EA0325 for ; Fri, 11 Oct 2024 00:32:17 +0000 (UTC) X-FDA: 82659445008.19.F931EB4 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf27.hostedemail.com (Postfix) with ESMTP id 4407D40012 for ; Fri, 11 Oct 2024 00:32:21 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ymUf1K42; dmarc=none; spf=pass (imf27.hostedemail.com: domain of debug@rivosinc.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728606630; a=rsa-sha256; cv=none; b=0qu4ANKgGJAL3NQ8ujsjdMMbtuFemhkX+J7DlE5UQULbUtPECaD/Iwx3Grwr1yDJZSPalX bG/TC239dCcBI/xdAGG1GH9eAc+RR9SYAWKI5qlXwarS/oNzYpPWn80sNNSSZh0T1Xtvhp //6ywse89/E518e0ov9HHauj8RB698c= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ymUf1K42; dmarc=none; spf=pass (imf27.hostedemail.com: domain of debug@rivosinc.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728606630; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pcb1JUl7nxyzCR2L+K51gN2ROJJ8XrFInAPZeivTlr8=; b=YcWMYYjb8v5I7fPB9MQRUNAqr7PHuJPdW8km272Ol3IJdAL/cTWeDsJXR3mgZDn/0EFxZ/ Fb7MA9g5Uj6AdaEeSRXtYhYDU6sdNJVo8TkOtxyQ/fm11XJknOpbKIB7+v89IVIkRSKF7o qS0xYN6YK8Xt/bsLVqxZZPN4XJdYbQg= Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-71dea49e808so1367711b3a.1 for ; Thu, 10 Oct 2024 17:32:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1728606742; x=1729211542; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=pcb1JUl7nxyzCR2L+K51gN2ROJJ8XrFInAPZeivTlr8=; b=ymUf1K424mQI++Z4ykIzajsKqHjPPimiPi1ss5CRgSTgFbTLUjloF+VCCYPrZVweaa 9s0jeoiDOs8lSwOh3U9Uj+wCZsGscwNGZE/Lz8ldciwU3DAGtwycMJXAQzaQDhws4D/D TcS6fFF/OmTdn/kGtlAheP2mqplIlch21UcklgjH8BctvD+LDo4LLGzfYROFnh1Wx6b3 jawYjHJ4sgYRQC9pE8x/Ff+wGgTZNL2vtotm60VzwgdaSn+NJD9ZcAEp7XfU8F2Im3uE iwbBllgYtdVVVZ4qutQAO0aqlDU3szPRGs+2sFsSw16DoLeNbkSXhWPhuiitoOVvXSB6 DyAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728606742; x=1729211542; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pcb1JUl7nxyzCR2L+K51gN2ROJJ8XrFInAPZeivTlr8=; b=eaE8tOhC9gM1SiKuVBj52IkwAbBNmRI/dJtzFLEGZyNtO2zbSn1NB41lWorxPmaStA Nrn4+LHpADPSYlmcJLbnA84Is0eiAL1G5PW762YZktvbzad0n8HwBHJAd9mPVLxJH1Jn cC5ZZrB3iMj8Pqjj/jJPqbaCdLbSgJRGxMMz48w3rPYqiEzfQqyWAHyUQp3FMV4jKots CB+HAS2JFRryOkJ/ACH5h2CIC6+1rL6cAXdh/kwwHhc3nbT64nrVnzE5rZ5OCEACDXw4 toMUxaZHB6g971rKtwyDMc389FG8jUpG19r0WpGU+s2eTGvXizelWRcC6AqkSs6CvqHf IJcg== X-Forwarded-Encrypted: i=1; AJvYcCVh9V+Fd35UXugTqwZgbPGcjwH/D6d7VnLvadQF8lZvxXBo7jOO+QghxIx6F52xVrZqOyQioA6ohw==@kvack.org X-Gm-Message-State: AOJu0YwD/OunEPWrfLfLLKXAwnF6Yfyw7ZRTJtjXkIgd9oEUvrgkVju2 W8ocyTnYQfmr8OGIWstSdjMxwcvqDdwEmP76eH3IhwCrF9X7stmxajC83R349Hw= X-Google-Smtp-Source: AGHT+IFSf80aIAMN4Fscyo/d2B9xB9WQkYt/Q471M36cO+ATFH/XCXiTEEuhz953CSaf5qR503Upaw== X-Received: by 2002:a05:6a00:14ce:b0:71e:cd0:cc99 with SMTP id d2e1a72fcca58-71e37e2d6admr1520196b3a.4.1728606741623; Thu, 10 Oct 2024 17:32:21 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7ea4496b1afsm1545600a12.94.2024.10.10.17.32.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Oct 2024 17:32:21 -0700 (PDT) From: Deepak Gupta Date: Thu, 10 Oct 2024 17:32:03 -0700 Subject: [PATCH RFC/RFT 1/3] mm: Introduce ARCH_HAS_USER_SHADOW_STACK MIME-Version: 1.0 Message-Id: <20241010-shstk_converge-v1-1-631beca676e7@rivosinc.com> References: <20241010-shstk_converge-v1-0-631beca676e7@rivosinc.com> In-Reply-To: <20241010-shstk_converge-v1-0-631beca676e7@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Arnd Bergmann Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, Rick Edgecombe , Mark Brown , Deepak Gupta , David Hildenbrand , Carlos Bilbao X-Mailer: b4 0.14.0 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 4407D40012 X-Stat-Signature: gta6po6iqdr6impzxf5sako4b4xrn3kc X-Rspam-User: X-HE-Tag: 1728606741-112274 X-HE-Meta: U2FsdGVkX19GpdKfmmYPz0XzjjpOTpG0LnFC1UMyiMLY5+KHauequ+sc2dD8YuydWKvQ+caILXU0E1xLbqh23CAp5WQurlEJTPWgqhPskfN7llnRv2Km/MeEar6P73gwgdcpAq5BH+Ui9+AnNVnCdoDhHqtwUdEEvQ+8HQr+l4TOznARRYS47cMINH14gKr7e502YOrccsinTNhMKPugw0dnVVrBefJLzdTWUPsqRUaLMjNO52I22TPJK49KTOjLrRRieDuO3tB2oIBY6ghIeJWk/Y3ckx21A/HU22hCyvqO8Q9Oz8tds5OqPgowEHMj5knSFvBLJICBMucO5XiM8Yt9Iwpk/6zTt+KFGUFtc/Xr/UULnk4kG1dtnpPjm5tR+0AFNUb09V+U3vFV9hM1vWhJsiBRpMdISnNQEkv7XQUpHIwEJCMgJlAgPNXTvw4crnrvkmFApOw9H6HkW/mx4HeOxnDDWlCBWWLXfackZNnfgz2HpEm6QmVNFLb7rJ53GCdqm4Rl1/adQ71yX3ZMWCsjqeg5h5q8tVNh8pAi1lh1K/muVNiNyzp7/pjqFahCCZ+bKiKheT+98ozQX77ULDmKvVka2GOkeF5Xa3U6a/CWUEBdBoZp6wTzA+JQSB33buf4MfiXV2Qrk2P4MphKjMkbz7drS4vbgn+gR83d/qM8A4Ey27+I4gL1nGT68r69WyzXWCf+plNsJ7O7gNqp7NRGShhaM6G48QV9Htvl/uFflY77CxkwHGwRh4xB1wUY8miJKINrGWyT+oVLjY75fGrmcWzXvvlXt4nLYtj5GwXFYPSjEu1F2iYjWYObaTh63oRGILhbY/KV8Jyw89uyQ7sacTXc+gpK4kkMQPamf6hmyleOzFV9zu9kWmij5BW5hqjnqZxcsmtCMg7kKnbT7VbFMFBwT+YEaEWWHQuVLYyWYASMnbLjpdLkFGm8NReVbFaoKZYs5d2iKJvOpNP 09+Eh5wL MYPpoRSCOR3XgqnnwTrCpCs7uVrur1Mi/yRxygjVh72CQ7s1q36PDQMQt/jQw0hA1SkankoDKqxVJ/9Uc8IHHt6KhQkpmlDomiAFngjJodf3bQ4/InhJCC9ZLSga0/SIfCndB8c5ugMUS5fLA3t1hyHiUPSymuQCLFj9A91/WhGRIQjh55WFIBse+GAgd4/XhST5QOz60xhf6FXLgefpFN0a0C/o6MLvvtu1fxD5a6tSvnGd+rk++qfICbsTLRigeYnPYQJRXSBoLrNFUBY9RbMqFefgWjf4qOLiCD9yNL9tSaY2nmY9sPNTC7GCwpbtuYleJBjJ/MzHLNPkkoIIDvzQE9SoYl5L6EJnmYqHq0hIFcT9ckF1jZf9ybRuq5iIcA9gR1u+VK35X4BFVJ8lN0WuSGbLE+f/bK1aIcThPdhfQxaYxc5w6XdQNSyBy21GR6ZfbPQ2SI8Wbo+xQjU1P243+wcAlor/8pWTYxxm7uECS4KLmuRy4ZZmKfyYSvPuL3O102DZD0AatTO6pfCDXHeR1W/yA5isPvgbeAHs32ZsDIWjaNCaELyjatiVRqR+SS/6j1bdU4ov9cXTmLPue0SiVIbTSl/5b3GJnugVBqbH26b29pGFHxuNlCQrqvalSTdLIloQRhoQ4eCqrkdRCCNHMcRt7bXAXIpXO+hM9k+kUiLBBebUGiinhd/OXn7Pt1ye1LZOumGAnFlMJ5TRw8u33oA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Mark Brown Since multiple architectures have support for shadow stacks and we need to select support for this feature in several places in the generic code provide a generic config option that the architectures can select. Suggested-by: David Hildenbrand Acked-by: David Hildenbrand Signed-off-by: Mark Brown Reviewed-by: Rick Edgecombe Reviewed-by: Deepak Gupta Reviewed-by: Carlos Bilbao --- arch/x86/Kconfig | 1 + fs/proc/task_mmu.c | 2 +- include/linux/mm.h | 2 +- mm/Kconfig | 6 ++++++ 4 files changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 2852fcd82cbd..8ccae77d40f7 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1954,6 +1954,7 @@ config X86_USER_SHADOW_STACK depends on AS_WRUSS depends on X86_64 select ARCH_USES_HIGH_VMA_FLAGS + select ARCH_HAS_USER_SHADOW_STACK select X86_CET help Shadow stack protection is a hardware feature that detects function diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 72f14fd59c2d..23f875e78eae 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -971,7 +971,7 @@ static void show_smap_vma_flags(struct seq_file *m, struct vm_area_struct *vma) #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR [ilog2(VM_UFFD_MINOR)] = "ui", #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */ -#ifdef CONFIG_X86_USER_SHADOW_STACK +#ifdef CONFIG_ARCH_HAS_USER_SHADOW_STACK [ilog2(VM_SHADOW_STACK)] = "ss", #endif #if defined(CONFIG_64BIT) || defined(CONFIG_PPC32) diff --git a/include/linux/mm.h b/include/linux/mm.h index ecf63d2b0582..57533b9cae95 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -354,7 +354,7 @@ extern unsigned int kobjsize(const void *objp); #endif #endif /* CONFIG_ARCH_HAS_PKEYS */ -#ifdef CONFIG_X86_USER_SHADOW_STACK +#ifdef CONFIG_ARCH_HAS_USER_SHADOW_STACK /* * VM_SHADOW_STACK should not be set with VM_SHARED because of lack of * support core mm. diff --git a/mm/Kconfig b/mm/Kconfig index 4c9f5ea13271..4b2a1ef9a161 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1296,6 +1296,12 @@ config NUMA_EMU into virtual nodes when booted with "numa=fake=N", where N is the number of nodes. This is only useful for debugging. +config ARCH_HAS_USER_SHADOW_STACK + bool + help + The architecture has hardware support for userspace shadow call + stacks (eg, x86 CET, arm64 GCS or RISC-V Zicfiss). + source "mm/damon/Kconfig" endmenu From patchwork Fri Oct 11 00:32:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 13831616 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D3FAD24463 for ; Fri, 11 Oct 2024 00:32:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E10386B0083; Thu, 10 Oct 2024 20:32:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D9CC66B0088; Thu, 10 Oct 2024 20:32:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEBC46B0089; Thu, 10 Oct 2024 20:32:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A4F986B0083 for ; Thu, 10 Oct 2024 20:32:26 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2D9851C4335 for ; Fri, 11 Oct 2024 00:32:22 +0000 (UTC) X-FDA: 82659445050.28.C773D4A Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) by imf04.hostedemail.com (Postfix) with ESMTP id B9A3640006 for ; Fri, 11 Oct 2024 00:32:21 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=WHKFlhjk; dmarc=none; spf=pass (imf04.hostedemail.com: domain of debug@rivosinc.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728606700; a=rsa-sha256; cv=none; b=sL0jtT/pmEwIDVH+KszOJEdP17zb4/k0w4pyh4efVA5+YoEcW/jZTeSfOAlYmvDQnqCNM1 Cri7A+syBhwH34A49kJaBI+esHB/LHkmeSi1e1Gty9ws9SMrkCeD81QqI74U4KWM15txpy TSY5X/b/Bg1eL5n5qNT8+Oxb6wegrSA= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=WHKFlhjk; dmarc=none; spf=pass (imf04.hostedemail.com: domain of debug@rivosinc.com designates 209.85.215.174 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728606700; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OwXMBbOWCv06xRC7kuAtikxfL3CuNBKfrSTtPkKqEqk=; b=6zHtNwSuXCmRgTxmh4+rI6D1YyZHCqHegmsxPyvcbTOWY7EylNiMZ9XzarkgDTHXQqZ44U iVOEUE8W9ieZyANqTh9mS8peuhR446QnYIsTf6vbx+ZMfnjjtFUer80oyg1SrF7aoOjE5O DqI4OsjdsAIjcwIqWgucpWKzOBEJwTM= Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-7e9ad969a4fso1083309a12.3 for ; Thu, 10 Oct 2024 17:32:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1728606743; x=1729211543; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=OwXMBbOWCv06xRC7kuAtikxfL3CuNBKfrSTtPkKqEqk=; b=WHKFlhjk9MCwPwEBC8szytJwplD15ORXUv4N3PfhzqPMxLEb9zfZ9POb2MX3j4oZS6 qydJTdNEcx31uJ/Ho5tCJNPgav5dQAkJUNCNTR6QtdWIQgPKIMYCN2TTTBZ/M0m+wPLf /s9IHwIqD9YeYfW1i5HrWNYZ0PlHJqzze+XL9lyFQ/DoAJvY6ZG/bll/VRO03jFdHXyd vnZcIgFxPkitmRKJkpO1eSuhsMZSYsVSqF5yy4VUlUl+VSCzzvb2OH6bBKfS6/hgI0QX IZvPMtjsS0ZEyaGnEzG81OsGhlJxPU/dE/YY4YtRbf0CJ0wNmmNeTy50Y1sL21nMmO/P uz1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728606743; x=1729211543; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OwXMBbOWCv06xRC7kuAtikxfL3CuNBKfrSTtPkKqEqk=; b=Ig1NTZTl2Pu4GE1JqxhZH9bhm0OkuJzaK8ZIYB6x9XgAmchZhllI3g0uWx7koS+uEP ESymJdhw3T9Qj0DpQntR4cbFlE893Dlinhwi3FWdEQ1hen+ptCikSDt8LxpbDLFy8nCZ NWHPtt9DRJLUNemP5gqGTSt/KSL/uf3WVXNPWCfYWl8LmKeiVr1WYMhTdeTXe8uYYDw0 yQgUGFXhzO7ne0+oHkD8J6cXU+27YWQ/oRkPsDA+3U1TQR0L/GwTJYYv5DcBctWmDKTy PZQwUAWL0VA48WvOhKBjME+zNNwpTykprhHOTimDcEDZM7Nv+H2GZz3QMnpPy3/1KRbN 7IgA== X-Forwarded-Encrypted: i=1; AJvYcCV8j4ENFixLDj19ZJ3BXTAoL9leOt3ThsBRA0ZV8mNPK2IL9haLmqApdTF0ieszicdN/FICQpZOig==@kvack.org X-Gm-Message-State: AOJu0Yzi2JM00NrBfqSoa0pnWXhAoS0VvbuIiV8dQ+qY2JnE7I+MJU/o lficKt7kxxHczQ1eske0kTEK/jodBRAKnm5T0+n4qFyLQPQCPDGP3ToblJ5WJeQ= X-Google-Smtp-Source: AGHT+IHY7zq/7mSxHwSor2mUQedwvphBbojX3qVK+YSVQyO1pCVsaX3K1rupXosm1zq+SyZ3LBJdSg== X-Received: by 2002:a05:6a20:c997:b0:1d2:e8d8:dd46 with SMTP id adf61e73a8af0-1d8bcf23701mr1509996637.15.1728606743039; Thu, 10 Oct 2024 17:32:23 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7ea4496b1afsm1545600a12.94.2024.10.10.17.32.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Oct 2024 17:32:22 -0700 (PDT) From: Deepak Gupta Date: Thu, 10 Oct 2024 17:32:04 -0700 Subject: [PATCH RFC/RFT 2/3] mm: helper `is_shadow_stack_vma` to check shadow stack vma MIME-Version: 1.0 Message-Id: <20241010-shstk_converge-v1-2-631beca676e7@rivosinc.com> References: <20241010-shstk_converge-v1-0-631beca676e7@rivosinc.com> In-Reply-To: <20241010-shstk_converge-v1-0-631beca676e7@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Arnd Bergmann Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, Rick Edgecombe , Mark Brown , Deepak Gupta X-Mailer: b4 0.14.0 X-Rspam-User: X-Rspamd-Queue-Id: B9A3640006 X-Rspamd-Server: rspam01 X-Stat-Signature: eadgmirm7ztm1t7u5d1usrpqrmy9s9gx X-HE-Tag: 1728606741-920304 X-HE-Meta: U2FsdGVkX1/uCmtXOSw5z1glhy9dwtksgY6wbKPCjU+MLPHfYB9M4zkU0HlWqtqX3PWPho2XQ6D0nYhnzYj6z6S1cTx9nABkGBOO6csqcLpi8sOIHtQe8p4wMp1WoahU5GCtEK6MvFpAY26ZDSCfyAcbxlNmkyTNjezMbFTetgeTEfTJLPjfbF2gjE1SLN6ytsJC1WVLrtVtgGG18yY6+28N9viqyCAwVxMLMjuSm1rXdk2YpzoPm/Wzoef3enrj9Vk6YNN/dheyGXzJ0m1Kv+ZFMHmMbbb5LfCwPRWPZF8/PPbMmkWQkfhUpDvVSoGdwZqgIKkJaFJQcKfr2o99qgMaY3HAvO3Pc7ZcBFFWR19/Hcfft6Qdus1AVl1DLh6NtqU1R3dQSqpwKlLoH4bH4lDd/ArH3HUhEefOCsNZe9FXamPPPtklH1Dvm1Mi/2SAibonbqEjVz5vy6KBzx6/nL5FIMKf9OUocXEVS6BUaW786TPEehl3Nczdi8IbkQ7b9PMGLMrQFEC+exr0lFXcKuGsdfqXC3m+IsrnI0oU2EnvdkLuSIDwqFfeHtXlnbqzObvEzH2yt8zNnNIHVsYAIcZSlGM3eeVWGB2g7JSTDBKfaROkY6wX/H2Uv4cVDzaWl0kORk0pB2qwS+pT1CmBKVn+0zQII9kv12dKt7ils1TH4+uYXX0VgcWrFkhhQRuLszx37cEE4CDXhAQ8WptxwKxQNbI+BSiduQAZUJlOLpiwfjVdMQesxfBy5TX/spwDMZhWpY/pdJlvvO/Z7I3kOkEpvPQ2ojTAFNcqMNHHFp5wc7O4yZ66p7SeK0lvIJd0rRsjSGkpE+fNTIO1UrzsLVW1mMVl4PLsxRnQFjjspAIr1gHogqf/EADldTPWcfko7mPuRY3RsvLBuc7Ijw2U/k2W1uH+lic8LWA6E0GR3bvOX6odcn7N207EnKWRRHX4qXXrOU3cvcbc5LNexdj XqYBYvdw VKm9UXwk5wL6XYAnvP5Us2aZeyvF4aOPv/lF3fpGN6odXQCNwq4Y+pJJichSs5B0SSOYctwzPJooL30QAELyBobOcSiueetjPHOaeW46grZiCEw4vP7parGkg5LM1SjBRl99o47P3EVIA2qghGv8yd6z1ahr+wsBGJdG1B35Nw+GdMhoXf57sqj0TShhRI2osjEAVLP1qdJwkqpPwyfqQU5yHy5uBLQaiQDJYLPUnML1nQF5drwlsImwMZAnqnuY7UWkvT5skRSjtNMVnlwhNO0f/Prqp4FXkZEYwvOIlX0iRjCld97f+PiZIeMmRw6EL9fjSHWQ2BocI9BCV1x2AmS9MDD/F0fWjaDUgBVgP+jKBC2LAftAXJieR7oor7kdiSUBQCDlwymJ880YOLqL7ojiP0xbZG9rxljX8eNyInjOWcr6DsIKhL4DSpm15/9TMQ31yipUYr4UQfAK+QVXTdXx7KFEY7X0IuEHr4DntBod5SvxHYfaIXq5GVL7pXEt82TYCdubF76AtQ5QyfqgEbzUGEpGjGRzzt2iFa4J4SlXV4Hs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: VM_SHADOW_STACK (alias to VM_HIGH_ARCH_5) is used to encode shadow stack VMA on three architectures (x86 shadow stack, arm GCS and RISC-V shadow stack). In case architecture doesn't implement shadow stack, it's VM_NONE Introducing a helper `is_shadow_stack_vma` to determine shadow stack vma or not. Signed-off-by: Deepak Gupta --- mm/gup.c | 2 +- mm/vma.h | 10 +++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index a82890b46a36..8e6e14179f6c 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1282,7 +1282,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) !writable_file_mapping_allowed(vma, gup_flags)) return -EFAULT; - if (!(vm_flags & VM_WRITE) || (vm_flags & VM_SHADOW_STACK)) { + if (!(vm_flags & VM_WRITE) || is_shadow_stack_vma(vm_flags)) { if (!(gup_flags & FOLL_FORCE)) return -EFAULT; /* hugetlb does not support FOLL_FORCE|FOLL_WRITE. */ diff --git a/mm/vma.h b/mm/vma.h index 819f994cf727..0f238dc37231 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -357,7 +357,7 @@ static inline struct vm_area_struct *vma_prev_limit(struct vma_iterator *vmi, } /* - * These three helpers classifies VMAs for virtual memory accounting. + * These four helpers classifies VMAs for virtual memory accounting. */ /* @@ -368,6 +368,11 @@ static inline bool is_exec_mapping(vm_flags_t flags) return (flags & (VM_EXEC | VM_WRITE | VM_STACK)) == VM_EXEC; } +static inline bool is_shadow_stack_vma(vm_flags_t vm_flags) +{ + return !!(vm_flags & VM_SHADOW_STACK); +} + /* * Stack area (including shadow stacks) * @@ -376,7 +381,7 @@ static inline bool is_exec_mapping(vm_flags_t flags) */ static inline bool is_stack_mapping(vm_flags_t flags) { - return ((flags & VM_STACK) == VM_STACK) || (flags & VM_SHADOW_STACK); + return ((flags & VM_STACK) == VM_STACK) || is_shadow_stack_vma(flags); } /* @@ -387,7 +392,6 @@ static inline bool is_data_mapping(vm_flags_t flags) return (flags & (VM_WRITE | VM_SHARED | VM_STACK)) == VM_WRITE; } - static inline void vma_iter_config(struct vma_iterator *vmi, unsigned long index, unsigned long last) { From patchwork Fri Oct 11 00:32:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 13831617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E26E0D24462 for ; Fri, 11 Oct 2024 00:32:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A69876B008C; Thu, 10 Oct 2024 20:32:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 97D326B008A; Thu, 10 Oct 2024 20:32:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 780C16B008C; Thu, 10 Oct 2024 20:32:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 50D816B0088 for ; Thu, 10 Oct 2024 20:32:28 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 84676120583 for ; Fri, 11 Oct 2024 00:32:24 +0000 (UTC) X-FDA: 82659445134.17.C613450 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) by imf15.hostedemail.com (Postfix) with ESMTP id 45C12A000D for ; Fri, 11 Oct 2024 00:32:24 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ytOxRMhz; spf=pass (imf15.hostedemail.com: domain of debug@rivosinc.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=debug@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728606563; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7LsajkhEjibBVidc/N9UXu/T/kPuUFNE9m11+ozaBUc=; b=jklstMY/s2Ggx08oh4SEh9LREb7hH9plwWESZrweR1iVa+FCxMI3C046WOtrt/GNL9clpT 7oUAx5ddgTtdW/FPx5hr+ObL76IAR1jxa1QYRnP5ueBNPetQHDhN76TBDXAxUXziA1lzgy vGIDonIJdBvWn41RW1iLNXaSgo8wzbc= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=ytOxRMhz; spf=pass (imf15.hostedemail.com: domain of debug@rivosinc.com designates 209.85.210.170 as permitted sender) smtp.mailfrom=debug@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728606563; a=rsa-sha256; cv=none; b=2ZBptnIJbCUqkMddvKBMYKAw6cai9qJCmeib4VKdi6FLUmIW0kFrDh3NNLOH0J550BbQTA S6Y8ymO1ohMl3QaF4K98SbiKXzlPTr91HHqH8jWvE7KlfT5Cmk5kYcN89SZgv7F0NCvHCU 0TEM3Ia5yFdzDlFYW1EANOND8RkdPVg= Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-71dfc1124cdso1177308b3a.1 for ; Thu, 10 Oct 2024 17:32:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1728606745; x=1729211545; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=7LsajkhEjibBVidc/N9UXu/T/kPuUFNE9m11+ozaBUc=; b=ytOxRMhzOwvPPNAOHYV3Nbop1G+eiXjrQwnLZyoNHB4KjR+B75kguIIQa12p8dERDl MKzcSYJxEugjU/i9ETXREosj5MtM008mul63MpGj0dMW1Ja0tTG5lhQ5Mv8th+7hb3Tu 8GCz9oHZIz5WJ1kkxv3GiLI0ShK+CKvtBQ7YieMGxMSa/WemXTYRgvgm4JebDZUul2e+ 2cAtgnX9nqSHgAfqmTiPbZSHg/h5IE+nDln8hmZldru7tGZYuDfKBITIuWt0ZzV1xzH5 w6mCLXIJzsTtB/dnVzirkRPm0XLsRRWLLumDd9XYoc0EBTxNdhsA09NTuXSlOYEX6QoN ODyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728606745; x=1729211545; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7LsajkhEjibBVidc/N9UXu/T/kPuUFNE9m11+ozaBUc=; b=Aq2H6uFRskDCHAQHdQAIOPTzMfeL6BNqCsgiL+yjHHaCGSD83+QEY1mbbVcf+/xS8X EjHgEN+vQxh130TnB53lzQ8z6ngklNloaSpR2RQO5wGmgsFOfZ23uk0DFm6QNzj45Jun 9b60NxA76mi+aLbHqPEyxn5ln4UcdRprv/Jo20+mYwKn9Ty4GuP+YgaVDUtssvjc9iuI njx+Vu90/QDAbx7XabrbnZK0UKM3dV2qGytG+azcLTYq7NpTpsa6g6fy+RwphY+lcXlI FRJPLdMsDqv7th5bkYU8kyitpN/O2m+82o4e433qpHCpGDEaSjyk8enib1x5Nrc0/7JS 195w== X-Forwarded-Encrypted: i=1; AJvYcCXwFAQLMyi/0KR+a0Kf3awAoR6o4x1vN6r2FRHHq8kqXemlaQrR97I51xUPzD7qA5s5VJVdubnRkQ==@kvack.org X-Gm-Message-State: AOJu0YxTaErSuwfrtgKQ/ox/kzU8eCV7Crf8nKVsopefy0pd50qO6p33 wIoxhm9YQJFVEz/vjpE3Cs6+abfDx+iumF76jrZjtxPC4d+1Ksumwr6kjmJF7pA= X-Google-Smtp-Source: AGHT+IFOFivpf2dfihVtShzwf9FEEOMn7Ved5zOgRCjh+YWjJ0yTlvWa3kaJvmwenVIm58jaUJ+ALA== X-Received: by 2002:a05:6a00:17a8:b0:71d:fc8c:348a with SMTP id d2e1a72fcca58-71e26eecc42mr9331967b3a.9.1728606744583; Thu, 10 Oct 2024 17:32:24 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7ea4496b1afsm1545600a12.94.2024.10.10.17.32.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Oct 2024 17:32:24 -0700 (PDT) From: Deepak Gupta Date: Thu, 10 Oct 2024 17:32:05 -0700 Subject: [PATCH RFC/RFT 3/3] kernel: converge common shadow stack flow agnostic to arch MIME-Version: 1.0 Message-Id: <20241010-shstk_converge-v1-3-631beca676e7@rivosinc.com> References: <20241010-shstk_converge-v1-0-631beca676e7@rivosinc.com> In-Reply-To: <20241010-shstk_converge-v1-0-631beca676e7@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Arnd Bergmann Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, Rick Edgecombe , Mark Brown , Deepak Gupta X-Mailer: b4 0.14.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 45C12A000D X-Stat-Signature: sbsoxi39ekpr4syo4o4k7csgb7d184sq X-Rspam-User: X-HE-Tag: 1728606744-10159 X-HE-Meta: U2FsdGVkX18IvmdLh9pmIwPLkL+8LBZvzL0Hgo/fYPbXw5MbU8u11ISDD/B1BcuIJtVXcOj7SVQeGlCV2K9KJy2E79OVAXg4D2a5D99kDliFjiJRrgLG8KtpL772ogc6CpkwkyQSDQGVV+1lD/rKI9THEDZy4ZQpLE/+FMoYoPOb/jC+ynAXPdmhoy36fkLOg9A9ou+xmkt2WGxM0FJTvHT6+JwSjP7gKpdpBNqEckyL23KyjcZtt6EyUjPbowaqTEN27+aHMJ2AI6OKJHPb4tqXWoa2AElYkJuTYwzo1Q3nSHPu3VLvL4dFwosQRNSOJWkkjeXS0DPJcpuo8G4peusIWYa6FqmfGpbIF1rEWvo5Wq+xX+SAW+X4pLoczqUNs1XFRaaB8dqfjEu0pvkAGE8fEZqHImMaVZIxi9C0yIwuws+uyb7ZRYRDHRVkE6+uz6mCF5mB6Wf12Wi34G1c+ozSOf3KYrULhREElsvx7tDrvkxU30DcguEo1mmQvIxz7g/xZJmHcaGy266UzshA38812Ztdk+xOR9b2mFfg1w5WONq1peQttu/tbG5yDwZImW+sByVy81rfLx1UW0NTHlhoPYNYyrNDKHLS0MofsM3fhGm3ywcbqELn/guBQSt9C0ZHJzxwOuYA9PghE9iLMhWKslpOXocBHBigQbMkkmu2IkFsOakBE5CBXcSFJtoGLC36SeqSYmgMFTxJhuAnqwuklGFBzBEg06DwElqg27heiflyOMmSK3RaXksRY2ECRXg/doQVI4IZfmJU7I27iwx2mhyhTGMQdgoa9uUnRbmQjL3r628wE1mSX9yhyR/VDQA7F04sc+forEkeErHlrT0imtIji3h63NmWs7jl3GypmSjAMMyiGoIAB75Nuxy9Mmcs9PBDlbhhQODp79TvyqsyI+NY8wUW3A64N63rrxkvOVWVdj7t+Gh4QpHsXMNQqrROvbB9YJOVQtOxJXg gnAVlcyY 4ZsIAjwaieWJkyBwj89BAb5blGT6HyNwvs//pO3ZSeJzo/2agffQUcd1ZVPQ5gDYcewM5vQQdcWWzXzEVhf8VT9OPCth2aHdKH5G+9ACg8VAwqU0purAjV5oX6E1xO6bk/inxm1TdcA7agTxsdGKkvHmlh9Mm0eTLaLosUHv7mhlOW332ydvWoPVCnW3MfXO+8E02mTLhzwsPk+cV4zsOa8lMUwbHUk5UtBQ0aDP6t4en3dAZcpA8Ynwzy4sbo9BRr0OgkAgPnccc39dPayjydJTvQHE+rOGV74fJ4jdIQT+OgJ06v9yWgrrjb/P5c8wbW2Fzk1Nb3yDlxrESXHiUgAZe0x1ZoaWuLjkeKxf9hO6FsvLAa4I8TkSdFfYwILVD2hAsFICXAcOnb9WH8UeGkpzMf/L4+BIb1blr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: CPU assisted shadow stack are supported by x86, arm64 and risc-v. In terms of enabling shadow stack feature for usermode code in kernel, they have following commonalities - Expose a user ABI (via a prctl) to allow user mode to explicitly ask for enabling shadow stack instead of by default enabling it. x86 series pre-dates arm64 or risc-v announcment of support, so it ended up doing a arch specific prctl instead of generic one. arm64 and risc-v have converged on using generic prctl and each of them can handle it appropriatley. - On fork or clone, shadow stack has to be COWed or not COWed depending on CLONE_VM was passed or not. Additionally if CLONE_VFORK was passed then same (parent one) shadow stack should be used. - To create shadow stack mappings, implement `map_shadow_stack` system call. This patch picks up Mark Brown's `ARCH_HAS_USER_SHADOW_STACK` config introduction and incorproate most of the common flows between different architectures. On a high level, shadow stack allocation and shadow stack de-allocation are base operations on virtual memory and common between architectures. Similarly shadow stack setup on prctl (arch specific or otherwise) is a common flow. Treatment of shadow stack virtual memory on `clone/fork` and implementaiton of `map_shadow_stack` is also converged into common flow. To implement these common flows, each architecture have arch-specific enabling mechanism as well as arch-specific data structures in task/ thread struct. So additionally this patch tries to abstract certain operation/helpers and allowing each architecture to have their arch_* implementation to implement the abstractions. Signed-off-by: Deepak Gupta --- arch/x86/include/asm/shstk.h | 9 + arch/x86/include/uapi/asm/mman.h | 3 - arch/x86/kernel/shstk.c | 270 ++++++------------------------ include/linux/usershstk.h | 25 +++ include/uapi/asm-generic/mman-common.h | 3 + kernel/Makefile | 2 + kernel/usershstk.c | 289 +++++++++++++++++++++++++++++++++ 7 files changed, 375 insertions(+), 226 deletions(-) diff --git a/arch/x86/include/asm/shstk.h b/arch/x86/include/asm/shstk.h index 4cb77e004615..4bb20af6cf7b 100644 --- a/arch/x86/include/asm/shstk.h +++ b/arch/x86/include/asm/shstk.h @@ -37,6 +37,15 @@ static inline int shstk_update_last_frame(unsigned long val) { return 0; } static inline bool shstk_is_enabled(void) { return false; } #endif /* CONFIG_X86_USER_SHADOW_STACK */ +int arch_create_rstor_token(unsigned long ssp, unsigned long *token_addr); +bool arch_cpu_supports_shadow_stack(void); +bool arch_is_shstk_enabled(struct task_struct *task); +void arch_set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size); +void arch_get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size); +void arch_set_shstk_ptr_and_enable(unsigned long ssp); +void arch_set_thread_shstk_status(bool enable); #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_SHSTK_H */ diff --git a/arch/x86/include/uapi/asm/mman.h b/arch/x86/include/uapi/asm/mman.h index 46cdc941f958..ac1e6277212b 100644 --- a/arch/x86/include/uapi/asm/mman.h +++ b/arch/x86/include/uapi/asm/mman.h @@ -5,9 +5,6 @@ #define MAP_32BIT 0x40 /* only give out 32bit addresses */ #define MAP_ABOVE4G 0x80 /* only map above 4GB */ -/* Flags for map_shadow_stack(2) */ -#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ - #include #endif /* _ASM_X86_MMAN_H */ diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 059685612362..512339b271e1 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -25,6 +25,7 @@ #include #include #include +#include #define SS_FRAME_SIZE 8 @@ -43,11 +44,56 @@ static void features_clr(unsigned long features) current->thread.features &= ~features; } +bool arch_cpu_supports_shadow_stack(void) +{ + return cpu_feature_enabled(X86_FEATURE_USER_SHSTK); +} + +bool arch_is_shstk_enabled(struct task_struct *task) +{ + return features_enabled(ARCH_SHSTK_SHSTK); +} + +void arch_set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size) +{ + struct thread_shstk *shstk = &task->thread.shstk; + + shstk->base = base; + shstk->size = size; +} + +void arch_get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size) +{ + struct thread_shstk *shstk = &task->thread.shstk; + + *base = shstk->base; + *size = shstk->size; +} + + +void arch_set_shstk_ptr_and_enable(unsigned long ssp) +{ + fpregs_lock_and_load(); + wrmsrl(MSR_IA32_PL3_SSP, ssp); + wrmsrl(MSR_IA32_U_CET, CET_SHSTK_EN); + fpregs_unlock(); +} + +void arch_set_thread_shstk_status(bool enable) +{ + if (enable) + features_set(ARCH_SHSTK_SHSTK); + else + features_clr(ARCH_SHSTK_SHSTK); +} + /* * Create a restore token on the shadow stack. A token is always 8-byte * and aligned to 8. */ -static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) +int arch_create_rstor_token(unsigned long ssp, unsigned long *token_addr) { unsigned long addr; @@ -72,118 +118,6 @@ static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) return 0; } -/* - * VM_SHADOW_STACK will have a guard page. This helps userspace protect - * itself from attacks. The reasoning is as follows: - * - * The shadow stack pointer(SSP) is moved by CALL, RET, and INCSSPQ. The - * INCSSP instruction can increment the shadow stack pointer. It is the - * shadow stack analog of an instruction like: - * - * addq $0x80, %rsp - * - * However, there is one important difference between an ADD on %rsp - * and INCSSP. In addition to modifying SSP, INCSSP also reads from the - * memory of the first and last elements that were "popped". It can be - * thought of as acting like this: - * - * READ_ONCE(ssp); // read+discard top element on stack - * ssp += nr_to_pop * 8; // move the shadow stack - * READ_ONCE(ssp-8); // read+discard last popped stack element - * - * The maximum distance INCSSP can move the SSP is 2040 bytes, before - * it would read the memory. Therefore a single page gap will be enough - * to prevent any operation from shifting the SSP to an adjacent stack, - * since it would have to land in the gap at least once, causing a - * fault. - */ -static unsigned long alloc_shstk(unsigned long addr, unsigned long size, - unsigned long token_offset, bool set_res_tok) -{ - int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_ABOVE4G; - struct mm_struct *mm = current->mm; - unsigned long mapped_addr, unused; - - if (addr) - flags |= MAP_FIXED_NOREPLACE; - - mmap_write_lock(mm); - mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, - VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); - mmap_write_unlock(mm); - - if (!set_res_tok || IS_ERR_VALUE(mapped_addr)) - goto out; - - if (create_rstor_token(mapped_addr + token_offset, NULL)) { - vm_munmap(mapped_addr, size); - return -EINVAL; - } - -out: - return mapped_addr; -} - -static unsigned long adjust_shstk_size(unsigned long size) -{ - if (size) - return PAGE_ALIGN(size); - - return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); -} - -static void unmap_shadow_stack(u64 base, u64 size) -{ - int r; - - r = vm_munmap(base, size); - - /* - * mmap_write_lock_killable() failed with -EINTR. This means - * the process is about to die and have it's MM cleaned up. - * This task shouldn't ever make it back to userspace. In this - * case it is ok to leak a shadow stack, so just exit out. - */ - if (r == -EINTR) - return; - - /* - * For all other types of vm_munmap() failure, either the - * system is out of memory or there is bug. - */ - WARN_ON_ONCE(r); -} - -static int shstk_setup(void) -{ - struct thread_shstk *shstk = ¤t->thread.shstk; - unsigned long addr, size; - - /* Already enabled */ - if (features_enabled(ARCH_SHSTK_SHSTK)) - return 0; - - /* Also not supported for 32 bit */ - if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK) || in_ia32_syscall()) - return -EOPNOTSUPP; - - size = adjust_shstk_size(0); - addr = alloc_shstk(0, size, 0, false); - if (IS_ERR_VALUE(addr)) - return PTR_ERR((void *)addr); - - fpregs_lock_and_load(); - wrmsrl(MSR_IA32_PL3_SSP, addr + size); - wrmsrl(MSR_IA32_U_CET, CET_SHSTK_EN); - fpregs_unlock(); - - shstk->base = addr; - shstk->size = size; - features_set(ARCH_SHSTK_SHSTK); - - return 0; -} - void reset_thread_features(void) { memset(¤t->thread.shstk, 0, sizeof(struct thread_shstk)); @@ -191,48 +125,6 @@ void reset_thread_features(void) current->thread.features_locked = 0; } -unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, - unsigned long stack_size) -{ - struct thread_shstk *shstk = &tsk->thread.shstk; - unsigned long addr, size; - - /* - * If shadow stack is not enabled on the new thread, skip any - * switch to a new shadow stack. - */ - if (!features_enabled(ARCH_SHSTK_SHSTK)) - return 0; - - /* - * For CLONE_VFORK the child will share the parents shadow stack. - * Make sure to clear the internal tracking of the thread shadow - * stack so the freeing logic run for child knows to leave it alone. - */ - if (clone_flags & CLONE_VFORK) { - shstk->base = 0; - shstk->size = 0; - return 0; - } - - /* - * For !CLONE_VM the child will use a copy of the parents shadow - * stack. - */ - if (!(clone_flags & CLONE_VM)) - return 0; - - size = adjust_shstk_size(stack_size); - addr = alloc_shstk(0, size, 0, false); - if (IS_ERR_VALUE(addr)) - return addr; - - shstk->base = addr; - shstk->size = size; - - return addr + size; -} - static unsigned long get_user_shstk_addr(void) { unsigned long long ssp; @@ -402,44 +294,6 @@ int restore_signal_shadow_stack(void) return 0; } -void shstk_free(struct task_struct *tsk) -{ - struct thread_shstk *shstk = &tsk->thread.shstk; - - if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK) || - !features_enabled(ARCH_SHSTK_SHSTK)) - return; - - /* - * When fork() with CLONE_VM fails, the child (tsk) already has a - * shadow stack allocated, and exit_thread() calls this function to - * free it. In this case the parent (current) and the child share - * the same mm struct. - */ - if (!tsk->mm || tsk->mm != current->mm) - return; - - /* - * If shstk->base is NULL, then this task is not managing its - * own shadow stack (CLONE_VFORK). So skip freeing it. - */ - if (!shstk->base) - return; - - /* - * shstk->base is NULL for CLONE_VFORK child tasks, and so is - * normal. But size = 0 on a shstk->base is not normal and - * indicated an attempt to free the thread shadow stack twice. - * Warn about it. - */ - if (WARN_ON(!shstk->size)) - return; - - unmap_shadow_stack(shstk->base, shstk->size); - - shstk->size = 0; -} - static int wrss_control(bool enable) { u64 msrval; @@ -502,36 +356,6 @@ static int shstk_disable(void) return 0; } -SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) -{ - bool set_tok = flags & SHADOW_STACK_SET_TOKEN; - unsigned long aligned_size; - - if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK)) - return -EOPNOTSUPP; - - if (flags & ~SHADOW_STACK_SET_TOKEN) - return -EINVAL; - - /* If there isn't space for a token */ - if (set_tok && size < 8) - return -ENOSPC; - - if (addr && addr < SZ_4G) - return -ERANGE; - - /* - * An overflow would result in attempting to write the restore token - * to the wrong location. Not catastrophic, but just return the right - * error code and block it. - */ - aligned_size = PAGE_ALIGN(size); - if (aligned_size < size) - return -EOVERFLOW; - - return alloc_shstk(addr, aligned_size, size, set_tok); -} - long shstk_prctl(struct task_struct *task, int option, unsigned long arg2) { unsigned long features = arg2; diff --git a/include/linux/usershstk.h b/include/linux/usershstk.h new file mode 100644 index 000000000000..68d751948e35 --- /dev/null +++ b/include/linux/usershstk.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _SHSTK_H +#define _SHSTK_H + +#ifndef __ASSEMBLY__ +#include + +unsigned long alloc_shstk(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok); +int shstk_setup(void); +int create_rstor_token(unsigned long ssp, unsigned long *token_addr); +bool cpu_supports_shadow_stack(void); +bool is_shstk_enabled(struct task_struct *task); +void set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size); +void get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size); +void set_shstk_ptr_and_enable(unsigned long ssp); +void set_thread_shstk_status(bool enable); +unsigned long adjust_shstk_size(unsigned long size); +void unmap_shadow_stack(u64 base, u64 size); + +#endif /* __ASSEMBLY__ */ + +#endif /* _SHSTK_H */ diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h index 6ce1f1ceb432..2c36e4c7b6ec 100644 --- a/include/uapi/asm-generic/mman-common.h +++ b/include/uapi/asm-generic/mman-common.h @@ -87,4 +87,7 @@ #define PKEY_ACCESS_MASK (PKEY_DISABLE_ACCESS |\ PKEY_DISABLE_WRITE) +/* Flags for map_shadow_stack(2) */ +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ + #endif /* __ASM_GENERIC_MMAN_COMMON_H */ diff --git a/kernel/Makefile b/kernel/Makefile index 87866b037fbe..1922c456b954 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -140,6 +140,8 @@ KCOV_INSTRUMENT_stackleak.o := n obj-$(CONFIG_SCF_TORTURE_TEST) += scftorture.o +obj-$(CONFIG_ARCH_HAS_USER_SHADOW_STACK) += usershstk.o + $(obj)/configs.o: $(obj)/config_data.gz targets += config_data config_data.gz diff --git a/kernel/usershstk.c b/kernel/usershstk.c new file mode 100644 index 000000000000..055d70b99893 --- /dev/null +++ b/kernel/usershstk.c @@ -0,0 +1,289 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * shstk.c - Intel shadow stack support + * + * Copyright (c) 2021, Intel Corporation. + * Yu-cheng Yu + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define SHSTK_ENTRY_SIZE sizeof(void *) + +bool cpu_supports_shadow_stack(void) +{ + return arch_cpu_supports_shadow_stack(); +} + +bool is_shstk_enabled(struct task_struct *task) +{ + return arch_is_shstk_enabled(task); +} + +void set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size) +{ + arch_set_shstk_base_size(task, base, size); +} + +void get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size) +{ + arch_get_shstk_base_size(task, base, size); +} + +void set_shstk_ptr_and_enable(unsigned long ssp) +{ + arch_set_shstk_ptr_and_enable(ssp); +} + +void set_thread_shstk_status(bool enable) +{ + arch_set_thread_shstk_status(enable); +} + +int create_rstor_token(unsigned long ssp, unsigned long *token_addr) +{ + return arch_create_rstor_token(ssp, token_addr); +} + +unsigned long adjust_shstk_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); +} + +void unmap_shadow_stack(u64 base, u64 size) +{ + int r; + + r = vm_munmap(base, size); + + /* + * mmap_write_lock_killable() failed with -EINTR. This means + * the process is about to die and have it's MM cleaned up. + * This task shouldn't ever make it back to userspace. In this + * case it is ok to leak a shadow stack, so just exit out. + */ + if (r == -EINTR) + return; + + /* + * For all other types of vm_munmap() failure, either the + * system is out of memory or there is bug. + */ + WARN_ON_ONCE(r); +} + +/* + * VM_SHADOW_STACK will have a guard page. This helps userspace protect + * itself from attacks. The reasoning is as follows: + * + * The shadow stack pointer(SSP) is moved by CALL, RET, and INCSSPQ. The + * INCSSP instruction can increment the shadow stack pointer. It is the + * shadow stack analog of an instruction like: + * + * addq $0x80, %rsp + * + * However, there is one important difference between an ADD on %rsp + * and INCSSP. In addition to modifying SSP, INCSSP also reads from the + * memory of the first and last elements that were "popped". It can be + * thought of as acting like this: + * + * READ_ONCE(ssp); // read+discard top element on stack + * ssp += nr_to_pop * 8; // move the shadow stack + * READ_ONCE(ssp-8); // read+discard last popped stack element + * + * The maximum distance INCSSP can move the SSP is 2040 bytes, before + * it would read the memory. Therefore a single page gap will be enough + * to prevent any operation from shifting the SSP to an adjacent stack, + * since it would have to land in the gap at least once, causing a + * fault. + */ +unsigned long alloc_shstk(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + + flags |= IS_ENABLED(CONFIG_X86_64) ? MAP_ABOVE4G : 0; + + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + mmap_write_unlock(mm); + + if (!set_res_tok || IS_ERR_VALUE(mapped_addr)) + goto out; + + if (create_rstor_token(mapped_addr + token_offset, NULL)) { + vm_munmap(mapped_addr, size); + return -EINVAL; + } + +out: + return mapped_addr; +} + +void shstk_free(struct task_struct *tsk) +{ + unsigned long base, size; + + if (!cpu_supports_shadow_stack() || + !is_shstk_enabled(current)) + return; + + /* + * When fork() with CLONE_VM fails, the child (tsk) already has a + * shadow stack allocated, and exit_thread() calls this function to + * free it. In this case the parent (current) and the child share + * the same mm struct. + */ + if (!tsk->mm || tsk->mm != current->mm) + return; + + get_shstk_base_size(tsk, &base, &size); + /* + * If shstk->base is NULL, then this task is not managing its + * own shadow stack (CLONE_VFORK). So skip freeing it. + */ + if (!base) + return; + + /* + * shstk->base is NULL for CLONE_VFORK child tasks, and so is + * normal. But size = 0 on a shstk->base is not normal and + * indicated an attempt to free the thread shadow stack twice. + * Warn about it. + */ + if (WARN_ON(!size)) + return; + + unmap_shadow_stack(base, size); + + set_shstk_base_size(tsk, 0, 0); +} + +SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) +{ + bool set_tok = flags & SHADOW_STACK_SET_TOKEN; + unsigned long aligned_size; + + if (!cpu_supports_shadow_stack()) + return -EOPNOTSUPP; + + if (flags & ~SHADOW_STACK_SET_TOKEN) + return -EINVAL; + + /* If there isn't space for a token */ + if (set_tok && size < SHSTK_ENTRY_SIZE) + return -ENOSPC; + + if (addr && (addr & (PAGE_SIZE - 1))) + return -EINVAL; + + if (IS_ENABLED(CONFIG_X86_64) && + addr && addr < SZ_4G) + return -ERANGE; + + /* + * An overflow would result in attempting to write the restore token + * to the wrong location. Not catastrophic, but just return the right + * error code and block it. + */ + aligned_size = PAGE_ALIGN(size); + if (aligned_size < size) + return -EOVERFLOW; + + return alloc_shstk(addr, aligned_size, size, set_tok); +} + +int shstk_setup(void) +{ + struct thread_shstk *shstk = ¤t->thread.shstk; + unsigned long addr, size; + + /* Already enabled */ + if (is_shstk_enabled(current)) + return 0; + + /* Also not supported for 32 bit */ + if (!cpu_supports_shadow_stack() || + (IS_ENABLED(CONFIG_X86_64) && in_ia32_syscall())) + return -EOPNOTSUPP; + + size = adjust_shstk_size(0); + addr = alloc_shstk(0, size, 0, false); + if (IS_ERR_VALUE(addr)) + return PTR_ERR((void *)addr); + + set_shstk_ptr_and_enable(addr + size); + set_shstk_base_size(current, addr, size); + + set_thread_shstk_status(true); + + return 0; +} + +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, + unsigned long stack_size) +{ + struct thread_shstk *shstk = &tsk->thread.shstk; + unsigned long addr, size; + + if (!cpu_supports_shadow_stack()) + return -EOPNOTSUPP; + + /* + * If shadow stack is not enabled on the new thread, skip any + * switch to a new shadow stack. + */ + if (!is_shstk_enabled(tsk)) + return 0; + + /* + * For CLONE_VFORK the child will share the parents shadow stack. + * Make sure to clear the internal tracking of the thread shadow + * stack so the freeing logic run for child knows to leave it alone. + */ + if (clone_flags & CLONE_VFORK) { + set_shstk_base_size(tsk, 0, 0); + return 0; + } + + /* + * For !CLONE_VM the child will use a copy of the parents shadow + * stack. + */ + if (!(clone_flags & CLONE_VM)) + return 0; + + size = adjust_shstk_size(stack_size); + addr = alloc_shstk(0, size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + set_shstk_base_size(tsk, addr, size); + + return addr + size; +}