From patchwork Wed Oct 16 21:57:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 13839019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F894D2F7D0 for ; Wed, 16 Oct 2024 21:57:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A62256B0082; Wed, 16 Oct 2024 17:57:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C1B36B0083; Wed, 16 Oct 2024 17:57:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 862A46B0088; Wed, 16 Oct 2024 17:57:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 69C736B0082 for ; Wed, 16 Oct 2024 17:57:42 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 304BEA0C42 for ; Wed, 16 Oct 2024 21:57:23 +0000 (UTC) X-FDA: 82680827628.25.5220F30 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf10.hostedemail.com (Postfix) with ESMTP id 26A8FC0003 for ; Wed, 16 Oct 2024 21:57:35 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=oQO0Go52; dmarc=none; spf=pass (imf10.hostedemail.com: domain of debug@rivosinc.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729115827; a=rsa-sha256; cv=none; b=ZTIMHPW2lNHH5p9fpslTiFguk7al1JWIZFIN0BvpJhR2AvzNEmgOgDd77uj1kzLumvU8dL uNt0i60SZ6UBhhVGs0hOE9Pzu+KP+uSUJJ9ztbldq6Jp9bkhULdfpXFvxq8Ijka+UMzj79 hEEqstywn++ASY6di3sPRuzS+8QPvQs= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=oQO0Go52; dmarc=none; spf=pass (imf10.hostedemail.com: domain of debug@rivosinc.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=debug@rivosinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729115827; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XdsLfzbeIJUmffXDRsTtiOytmZQkMd2fCbnSq93GpGM=; b=SyAhXgQoc3wra4ClCJ2t2VtgighBKtBwa4r85HyUibmkkkvfLSrWgHMsvdEmhBTy4mlSFp Tg7Qj2lfYYcSO1sPv40Th0ggj9lVmKU9/JNubTcEtyuB3XX/jUjWqLvXxa2zekueNjXrq9 l/cspKKj85wlJKXsZdSm6HWZcK/yHpg= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-207115e3056so2270985ad.2 for ; Wed, 16 Oct 2024 14:57:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1729115859; x=1729720659; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=XdsLfzbeIJUmffXDRsTtiOytmZQkMd2fCbnSq93GpGM=; b=oQO0Go52PSq8E52IDi+jsUQCngUMwePvOkg0fyf4sQJhMv7pV3TQHsKRFqQsaPhLDe eYwToagkVpc+8CMrZeELjj2ZeE6GCVFBfrxapwTIqLQxUsMVz/UWI9bnwFMWGUhWsPVr qsupKuRT+qkg0AwgYPsklHeeYNWgUhhr2CPdzx8hev83Sm8KOyV2JKr1q/OhNO9JGsbo MntpFxqRaP6lfDYWX1uYMP4AXrx0zZQ3AVp2AZNkHIXsTG4fY85qBSq/RIYlvoALtxUk i3pFbSZR+e+lA5I3IGp28GHrm/yRYES9MdyFJuAdIOB9+T97izYaG9MA7TXG6kbCFnrh ANDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729115859; x=1729720659; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XdsLfzbeIJUmffXDRsTtiOytmZQkMd2fCbnSq93GpGM=; b=cdAekWKs3dMYX65DE21vCEDCDrWOX8+8az6+LqWxTVlti5jUAc3738QMXniu0vtySX nsR7mQdq7PEkO2A7lvkNqX+dbefVH2IE6SFy1jNOIsvwCg11ZnKftR6JXBYHwS+agQ1P 0SZSyAMobTUx5y7azbZLjpIrZWS+O0HIwVHs1GKuOSO827z6HmOZbi2s43BJhUwYm9zy qsZB/SkXZiyf4slLUiCUgSRCEw0M8HYpe01kA5I/54e3hTxDtTY+2m86nApYB7b/NMwQ 1yTy4aE0rRjWV0FbxLRV1L4sC+HJ94ZLDmhBceBIGxCFe5PbYoT0cZWPGiqRpZk/AuCI 5VYQ== X-Forwarded-Encrypted: i=1; AJvYcCUS0yf7Uv7M9sFkxjf0qRJzNrPXQ+6ri6lgGs8S7vQaNd4Vsq2D59M75Vys2NNjB4qMW7S/PnoHyQ==@kvack.org X-Gm-Message-State: AOJu0YznuODX2FlkagSmbuvlP9BM0K8/FP7rT2m5RufYtiBLrfY8K2t5 rQAClaEFoRSpTi6E8cWTtS2gXdDXRO5G8AggTR61OBcMVCqFsPuUB+q8jqSpKCw= X-Google-Smtp-Source: AGHT+IFluSxzlxpD9Tp1FCJcZbgYHrumOfhgLbhKxxdpf7sn2E/e71vVpkROxUYqoklPv+KqnK6MeQ== X-Received: by 2002:a17:902:ea10:b0:20c:c704:629e with SMTP id d9443c01a7336-20d27f2f5eemr63371415ad.56.1729115858946; Wed, 16 Oct 2024 14:57:38 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20d17f87ccasm32973295ad.62.2024.10.16.14.57.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 14:57:38 -0700 (PDT) From: Deepak Gupta Date: Wed, 16 Oct 2024 14:57:33 -0700 Subject: [PATCH RFC/RFT v2 1/2] mm: helper `is_shadow_stack_vma` to check shadow stack vma MIME-Version: 1.0 Message-Id: <20241016-shstk_converge-v2-1-c41536eb5c3b@rivosinc.com> References: <20241016-shstk_converge-v2-0-c41536eb5c3b@rivosinc.com> In-Reply-To: <20241016-shstk_converge-v2-0-c41536eb5c3b@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Arnd Bergmann Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, Rick Edgecombe , Mark Brown , Deepak Gupta X-Mailer: b4 0.14.0 X-Rspam-User: X-Stat-Signature: a3cj8asehdupnn73goafbguqk9qq6oqn X-Rspamd-Queue-Id: 26A8FC0003 X-Rspamd-Server: rspam02 X-HE-Tag: 1729115855-872817 X-HE-Meta: U2FsdGVkX1+WyEeAL1BOIqjQr+mPB/kHh0ms+9r0vifZBdj5nqW/evARu99x9X2XlnWrrqxl7CpwPtraOnyS3YOpscPq1SEIJMXmfAsQTPxnBlDvSjkPPjvMiPcDa7O2rp9w/QgZJMewhohzo0hI66t5zdYqo1VF8OkF37bSJHZL4iqwOrKwtrsjcgJsKQv6OoUhJ50C6dUVhijhooxONK8sdW23MkU9oR0oFAlh/5rdlOVA8S+6izB3/Sg4vTrVqbwha4dU74yZBZX0i8Hq1wNX5EgYR7hfSOkgDvu9BzRA1b/WD2tjYO++hqXrXt27kzVKoxlbbwY8vTtEHokPHwPghqBtDQeQflyKlN8qKSV+gepnvG5QVhEQZqdpRMvZzBQrctIC5PRnykapLCqux+r+ezFw+yP8Qu+Rbp/9kcTGSb7zwuuyeEYtiGyy4Vk4Awgren0M/kKLxioiz2+nY2b0XOKAC0LrjiBC4YxQDYFkfd1oUwySlkA8vektR9g3b9rmHc2iUa35i1ZNPM5XEHlIDYubxi8xPPFXMSlwediAxp6VWKzat0j9idlB+v82arIqLctUTtcY7Tsa049NLdkxI0aII8D932uKaXrQIWIUHWTplp9zsKc8G6UF8ycPQd0j631JHFn0P+PAh0K2tWmevDboKzsY+062w3a9N8qdkdRMFIaPiydb+aLiFSIW1boSXsVt6uZIbyb+6ckAowfXNrrvRPsMr3igrdsWPghO/QFOqoggvEe5n2MhQZgC0OPZ8NgfIqTVDu1Oj5FNzFqqBy++hUobKMcIyt+AQ3GFbU5eWR0gTXtfLeT/WZXPpnsGljQ1c2l1yuG7ylbOiZNJ+O7BuQBsHQX4z3dGMe2ZgOiQg7mVGuxPQRF13zZBcybEaZrfUJj7YfO3ssIFtMTeXRdUUdyJX9+Qqu0GqX/hi87eRT/m9NXjNgsA43njOUyI7gqAe8ZUt5ALzwU 4j4+reMI P2Y41MAzmrsaGaeiRj8WXelKYyxf/E8zLVNY0adu8jVDgyva+8ERPrd65Vy1aSvMLfBhqNrst2G+TfNVSJok0MfWG1wHZwRSP53vAsj00LKyjegJsOZF7S1wJZQKXcAzkTH9sR08PCI/DMg1Vznt89QmYN0o47GkYSHJVPNsuew2ouBDwB4nlCctC0vfrKtf9cFw/1HKRz/B8Im0981T6Zkj/JBXba99mRdhtqAG2AGGd3SS9gTW0CKs2u/Y4qAAoTiL3tksM49Ap/4BJnBRl2N8FiTa+GOoLwA40dwRRvEaJQi75RwTqxYyyRS1Qoc3SccVknKBu8gtC6+Vk924iT3tmGLjUJZZ75lFQV7Eim+0YraRZRugy0655RePFKRIGl2MdXV3huPq6QvLpGkbffwAxxR7/ppUDy7k2g3HouTiHLl4wNu2IAH0o/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: VM_SHADOW_STACK (alias to VM_HIGH_ARCH_5) is used to encode shadow stack VMA on three architectures (x86 shadow stack, arm GCS and RISC-V shadow stack). In case architecture doesn't implement shadow stack, it's VM_NONE Introducing a helper `is_shadow_stack_vma` to determine shadow stack vma or not. Signed-off-by: Deepak Gupta Reviewed-by: Mark Brown --- mm/gup.c | 2 +- mm/mmap.c | 2 +- mm/vma.h | 10 +++++++--- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index a82890b46a36..8e6e14179f6c 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1282,7 +1282,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) !writable_file_mapping_allowed(vma, gup_flags)) return -EFAULT; - if (!(vm_flags & VM_WRITE) || (vm_flags & VM_SHADOW_STACK)) { + if (!(vm_flags & VM_WRITE) || is_shadow_stack_vma(vm_flags)) { if (!(gup_flags & FOLL_FORCE)) return -EFAULT; /* hugetlb does not support FOLL_FORCE|FOLL_WRITE. */ diff --git a/mm/mmap.c b/mm/mmap.c index dd4b35a25aeb..0853e6784069 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -708,7 +708,7 @@ static unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info) */ static inline unsigned long stack_guard_placement(vm_flags_t vm_flags) { - if (vm_flags & VM_SHADOW_STACK) + if (is_shadow_stack_vma(vm_flags)) return PAGE_SIZE; return 0; diff --git a/mm/vma.h b/mm/vma.h index 819f994cf727..0f238dc37231 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -357,7 +357,7 @@ static inline struct vm_area_struct *vma_prev_limit(struct vma_iterator *vmi, } /* - * These three helpers classifies VMAs for virtual memory accounting. + * These four helpers classifies VMAs for virtual memory accounting. */ /* @@ -368,6 +368,11 @@ static inline bool is_exec_mapping(vm_flags_t flags) return (flags & (VM_EXEC | VM_WRITE | VM_STACK)) == VM_EXEC; } +static inline bool is_shadow_stack_vma(vm_flags_t vm_flags) +{ + return !!(vm_flags & VM_SHADOW_STACK); +} + /* * Stack area (including shadow stacks) * @@ -376,7 +381,7 @@ static inline bool is_exec_mapping(vm_flags_t flags) */ static inline bool is_stack_mapping(vm_flags_t flags) { - return ((flags & VM_STACK) == VM_STACK) || (flags & VM_SHADOW_STACK); + return ((flags & VM_STACK) == VM_STACK) || is_shadow_stack_vma(flags); } /* @@ -387,7 +392,6 @@ static inline bool is_data_mapping(vm_flags_t flags) return (flags & (VM_WRITE | VM_SHARED | VM_STACK)) == VM_WRITE; } - static inline void vma_iter_config(struct vma_iterator *vmi, unsigned long index, unsigned long last) { From patchwork Wed Oct 16 21:57:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Deepak Gupta X-Patchwork-Id: 13839020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61423D2F7CF for ; Wed, 16 Oct 2024 21:57:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E2726B0083; Wed, 16 Oct 2024 17:57:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 493086B0088; Wed, 16 Oct 2024 17:57:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26F806B0089; Wed, 16 Oct 2024 17:57:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 050836B0083 for ; Wed, 16 Oct 2024 17:57:43 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5A9811A0BE4 for ; Wed, 16 Oct 2024 21:57:25 +0000 (UTC) X-FDA: 82680827796.15.61CDE78 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf10.hostedemail.com (Postfix) with ESMTP id AC594C0018 for ; Wed, 16 Oct 2024 21:57:37 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=o4sULL7m; spf=pass (imf10.hostedemail.com: domain of debug@rivosinc.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=debug@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729115787; a=rsa-sha256; cv=none; b=OtAOtNaBq9Be0pTVZjiSJs1Lnb4uW91A7ETEgcRzEKou19FVdOWM6qbB4NoP6dsAnymQmI /oIu/LEnxG6fEFUHTm29nUM47koWasQcQX8NFUaqcwnvUVvVf68xmm9WRWgX67tSZwWa++ GBzwnbhD2bUfyaQQ70PTEB688Jj52+M= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=o4sULL7m; spf=pass (imf10.hostedemail.com: domain of debug@rivosinc.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=debug@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729115787; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3Cz5vF6A8ORZeY1cEQGXtdF7pJ+XZREd9sGOCjruplk=; b=l2R/l/ns6Cpv8kkFfpYstbbV2LAaIcYlPYoryEPMTNE1Z6WfGcGZK8bCPJCDSNOQC0zDZ2 XAQiHYVg/FlVX//hB3gL6tYvLcHRpLRmd/hu+ubITPjIGitRKNk4uTdMMTWgFx5BOwMFSg TRvujmFLUOAO4iiJG2LhFIyd+cymqSU= Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-20ca1b6a80aso2568305ad.2 for ; Wed, 16 Oct 2024 14:57:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1729115860; x=1729720660; darn=kvack.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=3Cz5vF6A8ORZeY1cEQGXtdF7pJ+XZREd9sGOCjruplk=; b=o4sULL7m5xy2hs2yaimEWpACY7vq+0wzHexoLSHj48aofOekXO9Kn3zafEbH2BYS3H fIUswY7pga3IyRHkApceufhA7hvMuCRG0bN+HuOnNY2Imt6uB2UX1y1GgELK9na+EibS VDzi2Adq8arv7H5NyxwY1GsSktj+AtWDGfUfv5TK2AXGUJZqziXPn6LtG6WIUO20VibN 7bJE9B6/LRSq+g7cR8Qz1hQkuBVqbZFtC//5K846NR64SkZtEj4/NmYSm4d7E62XSyll 7PnB7dc36/xzOjjk7+iV/r+GGfX1JjpFEZWBTC2O8l9D9XKPzl43+tfeo/plAP/57nRx eJfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729115860; x=1729720660; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3Cz5vF6A8ORZeY1cEQGXtdF7pJ+XZREd9sGOCjruplk=; b=M7o0Q8Iy4m3BGwjtsBSMu2SNC+PtOH0wGOVFsKymSEWouj8Exka5KfNrTDW2/AckZl 9MrJvJN7hqkMNCxktV0nI4nSz6mm+58quRN4BBRnf7gq9skAAYk0E4Er59zRSPD+tMxB rMuNbJxseXAnqarfhRqkzD8HaLRABQK1efGxD3ygIUk/9sskKdlzSFc4DwIMIZfjFY9W UFANwFGK2DBqELb6N1UGoWccwcN0bgRteXwRRbT19H9UN8ai31rzI3cA/pLQEeMbcUmG zO4+EA5pMkV5CnKs+NvkI/+vXcRNJyKPAJVhUBMq78MaKidAIRPnxWafjvypW+ZDTFR7 OFXQ== X-Forwarded-Encrypted: i=1; AJvYcCXs6HyrQTDWIZOYAnubUckhcxKiy4nYqkq6usCgA8PzLNr6jybVXEY9V/zjZreH580aocAi3K3LYg==@kvack.org X-Gm-Message-State: AOJu0YzjCAUNSBUhkyju6DuWY3zSEwbFI7oNrHk7egMdvqJU8kRX/wfW X1ugnVHB99wKG04uqEsQTANOInZnjk3IzwlvJcHeg45WLiWQT0oyKl4PNLeHn8g= X-Google-Smtp-Source: AGHT+IGZ01+86e/vK/iXrBpqVEYHxeNuio5G/NuosXloiX7aqgv/9qs580q/7Mz0aTsAfHOZaGn29g== X-Received: by 2002:a17:903:41c2:b0:20b:920e:8fd3 with SMTP id d9443c01a7336-20cbb1c407emr250356465ad.35.1729115860394; Wed, 16 Oct 2024 14:57:40 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20d17f87ccasm32973295ad.62.2024.10.16.14.57.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Oct 2024 14:57:39 -0700 (PDT) From: Deepak Gupta Date: Wed, 16 Oct 2024 14:57:34 -0700 Subject: [PATCH RFC/RFT v2 2/2] kernel: converge common shadow stack flow agnostic to arch MIME-Version: 1.0 Message-Id: <20241016-shstk_converge-v2-2-c41536eb5c3b@rivosinc.com> References: <20241016-shstk_converge-v2-0-c41536eb5c3b@rivosinc.com> In-Reply-To: <20241016-shstk_converge-v2-0-c41536eb5c3b@rivosinc.com> To: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Andrew Morton , "Liam R. Howlett" , Vlastimil Babka , Lorenzo Stoakes , Arnd Bergmann Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, Rick Edgecombe , Mark Brown , Deepak Gupta X-Mailer: b4 0.14.0 X-Stat-Signature: wc3a47pyfhuapxpcghxikxeqt7q4eikj X-Rspamd-Queue-Id: AC594C0018 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1729115857-69918 X-HE-Meta: U2FsdGVkX195u78Db4agdgRkgRq2wzvuKoGs1COsT9syP6+TAxdzHNM/dms+gVH0J6ahlSx9SxEhxYRzDhV34fO6byEICw5GjLZJ0lektv2tTDJ2mq799Uv6wkqCafC5fawjRmWjtJ9KXk59llk+NXPOwljsGUsisDTztasJOkTxPfLkkIDfE0ZkwMFSX3D4cgAii8P2Q7mR0tjxQAuF67e3rzs2xTfoKxZEeYkQQe06/CogHVeIi3ZMppo8izc6WPthcXj3cU5oHsZzSps0Gp6pxhC3w0w/VlETddXRVYYRvJ65FNN20QotmCmLQjfBDO9WF+7E30hAf2bWvwHHeAOlkaBrQZ4UP+nO4OcvlnN4pmhmr8gXB0DPIAWKCsQI94/vDAkF9Gq2WG6U7l0cToayzyc+SlQACVW1/rRChVh3vKzDn6k6PsEiEg3mC8Crqncb+GC3Uqt3Z5LCIA+UaQGK8q0CWmyEU5MzpicUFpkgJt1WxUEQ/kY3LC94Iog7Js6M7bZWgz4gVZwu/KEARomO6Xqhm1fbCdZ1U0bOzWLf76vSUnbdm3/xteqskZwtqvrBAtybkqIM0E9IrUgR7pcUbbfjDrg3AGq37HUR5WtIti8IbYzRB3ZgFcZIrGllUFL32sOtBex54snqUuOX9myAMzx1GeCmnU+BUlH2j0LwVgQK1FhdHIhxN1xPjzbxz7Kx1O5jUM2hrMQjry9Tjh1Cn5TGmzsOhLcCWDjzlAelRuj7ZD7KWqNign9e1UbXoRsktZ25ziSfrm+hlKYTHIbyombGrfLl4+JmYHBHgWfVbPFeV++rPge2HevhV2/hkznzpMcWxqIBh/HELSogRIQppZyDlaiwTUpYvZ374io/Q/DNmgeZiB6rM8A0Y2dzlEwzjl1fQfqmllU6f/tsuL07FNyRgI1OTXX9wcHLsIlAULxezVTHwFqzghMjqV7c0nDhyE6oORFEZHgLmLv 4xqlhfHP SJyvc2unu7WrOjSjaRqpkgbkaDfeEUb7W0w58N1Th7nVkdEw4TGeLgZtOlEKnhzZOlDZCmITgF1/gNiurPAJ9IlW2jajxB1T2+q2sh5hc42KkWGVSNJAOgOdHF3A03idqfIN0BctGcU3eGkUGhlV9OM6FonduDKz7MHciomA1lUUnpPl8wiOI+rIYoOzFNmb+KGeV0DrauZPos3DYPFV2tdLs/ztghoJtk5Tc2Me8G7ncX9wwMZr+gX6qaz6zmujkdbfZT2zDAxnI7xD+EX4wsQoqa+cootFEQb32ObvnXIJ2xa7xuOyLdtxz4+FRZ1cJosdDl6EwwZ6Nk4QsCv0P6LFEDySe+HKFOxxQXekYoGVa3sz2Z96yF6BdFYvDBwS0Hkpe1IfLrw5ihEiZKyi40zkFDxKbCVq688Z1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: CPU assisted shadow stack are supported by x86, arm64 and risc-v. In terms of enabling shadow stack feature for usermode code in kernel, they have following commonalities - Expose a user ABI (via a prctl) to allow user mode to explicitly ask for enabling shadow stack instead of by default enabling it. x86 series pre-dates arm64 or risc-v announcment of support, so it ended up doing a arch specific prctl instead of generic one. arm64 and risc-v have converged on using generic prctl and each of them can handle it appropriatley. - On fork or clone, shadow stack has to be COWed or not COWed depending on CLONE_VM was passed or not. Additionally if CLONE_VFORK was passed then same (parent one) shadow stack should be used. - To create shadow stack mappings, implement `map_shadow_stack` system call. This patch picks up Mark Brown's `ARCH_HAS_USER_SHADOW_STACK` config introduction and incorproate most of the common flows between different architectures. On a high level, shadow stack allocation and shadow stack de-allocation are base operations on virtual memory and common between architectures. Similarly shadow stack setup on prctl (arch specific or otherwise) is a common flow. Treatment of shadow stack virtual memory on `clone/fork` and implementaiton of `map_shadow_stack` is also converged into common flow. To implement these common flows, each architecture have arch-specific enabling mechanism as well as arch-specific data structures in task/ thread struct. So additionally this patch tries to abstract certain operation/helpers and allowing each architecture to have their arch_* implementation to implement the abstractions. Signed-off-by: Deepak Gupta --- arch/x86/include/asm/shstk.h | 7 + arch/x86/include/uapi/asm/mman.h | 3 - arch/x86/kernel/shstk.c | 223 +++++--------------------------- include/linux/usershstk.h | 22 ++++ include/uapi/asm-generic/mman-common.h | 5 + kernel/Makefile | 2 + kernel/usershstk.c | 230 +++++++++++++++++++++++++++++++++ 7 files changed, 296 insertions(+), 196 deletions(-) diff --git a/arch/x86/include/asm/shstk.h b/arch/x86/include/asm/shstk.h index 4cb77e004615..b40c3d91538b 100644 --- a/arch/x86/include/asm/shstk.h +++ b/arch/x86/include/asm/shstk.h @@ -37,6 +37,13 @@ static inline int shstk_update_last_frame(unsigned long val) { return 0; } static inline bool shstk_is_enabled(void) { return false; } #endif /* CONFIG_X86_USER_SHADOW_STACK */ +int arch_create_shstk_token(unsigned long ssp, unsigned long *token_addr); +bool arch_user_shstk_supported(void); +bool arch_is_shstk_enabled(struct task_struct *task); +void arch_set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size); +void arch_get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size); #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_SHSTK_H */ diff --git a/arch/x86/include/uapi/asm/mman.h b/arch/x86/include/uapi/asm/mman.h index 46cdc941f958..ac1e6277212b 100644 --- a/arch/x86/include/uapi/asm/mman.h +++ b/arch/x86/include/uapi/asm/mman.h @@ -5,9 +5,6 @@ #define MAP_32BIT 0x40 /* only give out 32bit addresses */ #define MAP_ABOVE4G 0x80 /* only map above 4GB */ -/* Flags for map_shadow_stack(2) */ -#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ - #include #endif /* _ASM_X86_MMAN_H */ diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 059685612362..d53a7efd70b5 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -25,6 +25,7 @@ #include #include #include +#include #define SS_FRAME_SIZE 8 @@ -43,11 +44,39 @@ static void features_clr(unsigned long features) current->thread.features &= ~features; } +bool arch_user_shstk_supported(void) +{ + return cpu_feature_enabled(X86_FEATURE_USER_SHSTK); +} + +bool arch_is_shstk_enabled(struct task_struct *task) +{ + return features_enabled(ARCH_SHSTK_SHSTK); +} + +void arch_set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size) +{ + struct thread_shstk *shstk = &task->thread.shstk; + + shstk->base = base; + shstk->size = size; +} + +void arch_get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size) +{ + struct thread_shstk *shstk = &task->thread.shstk; + + *base = shstk->base; + *size = shstk->size; +} + /* * Create a restore token on the shadow stack. A token is always 8-byte * and aligned to 8. */ -static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) +int arch_create_shstk_token(unsigned long ssp, unsigned long *token_addr) { unsigned long addr; @@ -72,88 +101,6 @@ static int create_rstor_token(unsigned long ssp, unsigned long *token_addr) return 0; } -/* - * VM_SHADOW_STACK will have a guard page. This helps userspace protect - * itself from attacks. The reasoning is as follows: - * - * The shadow stack pointer(SSP) is moved by CALL, RET, and INCSSPQ. The - * INCSSP instruction can increment the shadow stack pointer. It is the - * shadow stack analog of an instruction like: - * - * addq $0x80, %rsp - * - * However, there is one important difference between an ADD on %rsp - * and INCSSP. In addition to modifying SSP, INCSSP also reads from the - * memory of the first and last elements that were "popped". It can be - * thought of as acting like this: - * - * READ_ONCE(ssp); // read+discard top element on stack - * ssp += nr_to_pop * 8; // move the shadow stack - * READ_ONCE(ssp-8); // read+discard last popped stack element - * - * The maximum distance INCSSP can move the SSP is 2040 bytes, before - * it would read the memory. Therefore a single page gap will be enough - * to prevent any operation from shifting the SSP to an adjacent stack, - * since it would have to land in the gap at least once, causing a - * fault. - */ -static unsigned long alloc_shstk(unsigned long addr, unsigned long size, - unsigned long token_offset, bool set_res_tok) -{ - int flags = MAP_ANONYMOUS | MAP_PRIVATE | MAP_ABOVE4G; - struct mm_struct *mm = current->mm; - unsigned long mapped_addr, unused; - - if (addr) - flags |= MAP_FIXED_NOREPLACE; - - mmap_write_lock(mm); - mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, - VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); - mmap_write_unlock(mm); - - if (!set_res_tok || IS_ERR_VALUE(mapped_addr)) - goto out; - - if (create_rstor_token(mapped_addr + token_offset, NULL)) { - vm_munmap(mapped_addr, size); - return -EINVAL; - } - -out: - return mapped_addr; -} - -static unsigned long adjust_shstk_size(unsigned long size) -{ - if (size) - return PAGE_ALIGN(size); - - return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); -} - -static void unmap_shadow_stack(u64 base, u64 size) -{ - int r; - - r = vm_munmap(base, size); - - /* - * mmap_write_lock_killable() failed with -EINTR. This means - * the process is about to die and have it's MM cleaned up. - * This task shouldn't ever make it back to userspace. In this - * case it is ok to leak a shadow stack, so just exit out. - */ - if (r == -EINTR) - return; - - /* - * For all other types of vm_munmap() failure, either the - * system is out of memory or there is bug. - */ - WARN_ON_ONCE(r); -} - static int shstk_setup(void) { struct thread_shstk *shstk = ¤t->thread.shstk; @@ -191,48 +138,6 @@ void reset_thread_features(void) current->thread.features_locked = 0; } -unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, - unsigned long stack_size) -{ - struct thread_shstk *shstk = &tsk->thread.shstk; - unsigned long addr, size; - - /* - * If shadow stack is not enabled on the new thread, skip any - * switch to a new shadow stack. - */ - if (!features_enabled(ARCH_SHSTK_SHSTK)) - return 0; - - /* - * For CLONE_VFORK the child will share the parents shadow stack. - * Make sure to clear the internal tracking of the thread shadow - * stack so the freeing logic run for child knows to leave it alone. - */ - if (clone_flags & CLONE_VFORK) { - shstk->base = 0; - shstk->size = 0; - return 0; - } - - /* - * For !CLONE_VM the child will use a copy of the parents shadow - * stack. - */ - if (!(clone_flags & CLONE_VM)) - return 0; - - size = adjust_shstk_size(stack_size); - addr = alloc_shstk(0, size, 0, false); - if (IS_ERR_VALUE(addr)) - return addr; - - shstk->base = addr; - shstk->size = size; - - return addr + size; -} - static unsigned long get_user_shstk_addr(void) { unsigned long long ssp; @@ -402,44 +307,6 @@ int restore_signal_shadow_stack(void) return 0; } -void shstk_free(struct task_struct *tsk) -{ - struct thread_shstk *shstk = &tsk->thread.shstk; - - if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK) || - !features_enabled(ARCH_SHSTK_SHSTK)) - return; - - /* - * When fork() with CLONE_VM fails, the child (tsk) already has a - * shadow stack allocated, and exit_thread() calls this function to - * free it. In this case the parent (current) and the child share - * the same mm struct. - */ - if (!tsk->mm || tsk->mm != current->mm) - return; - - /* - * If shstk->base is NULL, then this task is not managing its - * own shadow stack (CLONE_VFORK). So skip freeing it. - */ - if (!shstk->base) - return; - - /* - * shstk->base is NULL for CLONE_VFORK child tasks, and so is - * normal. But size = 0 on a shstk->base is not normal and - * indicated an attempt to free the thread shadow stack twice. - * Warn about it. - */ - if (WARN_ON(!shstk->size)) - return; - - unmap_shadow_stack(shstk->base, shstk->size); - - shstk->size = 0; -} - static int wrss_control(bool enable) { u64 msrval; @@ -502,36 +369,6 @@ static int shstk_disable(void) return 0; } -SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) -{ - bool set_tok = flags & SHADOW_STACK_SET_TOKEN; - unsigned long aligned_size; - - if (!cpu_feature_enabled(X86_FEATURE_USER_SHSTK)) - return -EOPNOTSUPP; - - if (flags & ~SHADOW_STACK_SET_TOKEN) - return -EINVAL; - - /* If there isn't space for a token */ - if (set_tok && size < 8) - return -ENOSPC; - - if (addr && addr < SZ_4G) - return -ERANGE; - - /* - * An overflow would result in attempting to write the restore token - * to the wrong location. Not catastrophic, but just return the right - * error code and block it. - */ - aligned_size = PAGE_ALIGN(size); - if (aligned_size < size) - return -EOVERFLOW; - - return alloc_shstk(addr, aligned_size, size, set_tok); -} - long shstk_prctl(struct task_struct *task, int option, unsigned long arg2) { unsigned long features = arg2; diff --git a/include/linux/usershstk.h b/include/linux/usershstk.h new file mode 100644 index 000000000000..4ab27a1ab3f8 --- /dev/null +++ b/include/linux/usershstk.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _SHSTK_H +#define _SHSTK_H + +#ifndef __ASSEMBLY__ +#include + +unsigned long alloc_shstk(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok); +int create_shstk_token(unsigned long ssp, unsigned long *token_addr); +bool user_shstk_supported(void); +bool is_shstk_enabled(struct task_struct *task); +void set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size); +void get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size); +unsigned long adjust_shstk_size(unsigned long size); +void unmap_shadow_stack(u64 base, u64 size); + +#endif /* __ASSEMBLY__ */ + +#endif /* _SHSTK_H */ diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h index 6ce1f1ceb432..5d6fb32fda95 100644 --- a/include/uapi/asm-generic/mman-common.h +++ b/include/uapi/asm-generic/mman-common.h @@ -87,4 +87,9 @@ #define PKEY_ACCESS_MASK (PKEY_DISABLE_ACCESS |\ PKEY_DISABLE_WRITE) +/* Flags for map_shadow_stack(2) */ +#define SHADOW_STACK_SET_TOKEN (1ULL << 0) /* Set up a restore token in the shadow stack */ +#define SHADOW_STACK_SET_MARKER (1ULL << 1) /* Set up a top of stack marker in the shadow stack */ + +#define SHADOW_STACK_SET_MASK (SHADOW_STACK_SET_TOKEN | SHADOW_STACK_SET_MARKER) #endif /* __ASM_GENERIC_MMAN_COMMON_H */ diff --git a/kernel/Makefile b/kernel/Makefile index 87866b037fbe..1922c456b954 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -140,6 +140,8 @@ KCOV_INSTRUMENT_stackleak.o := n obj-$(CONFIG_SCF_TORTURE_TEST) += scftorture.o +obj-$(CONFIG_ARCH_HAS_USER_SHADOW_STACK) += usershstk.o + $(obj)/configs.o: $(obj)/config_data.gz targets += config_data config_data.gz diff --git a/kernel/usershstk.c b/kernel/usershstk.c new file mode 100644 index 000000000000..1ebce6b768aa --- /dev/null +++ b/kernel/usershstk.c @@ -0,0 +1,230 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * shstk.c - Intel shadow stack support + * + * Copyright (c) 2021, Intel Corporation. + * Yu-cheng Yu + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define SHSTK_ENTRY_SIZE sizeof(void *) + +bool user_shstk_supported(void) +{ + return arch_user_shstk_supported(); +} + +bool is_shstk_enabled(struct task_struct *task) +{ + return arch_is_shstk_enabled(task); +} + +void set_shstk_base_size(struct task_struct *task, unsigned long base, + unsigned long size) +{ + arch_set_shstk_base_size(task, base, size); +} + +void get_shstk_base_size(struct task_struct *task, unsigned long *base, + unsigned long *size) +{ + arch_get_shstk_base_size(task, base, size); +} + +int create_shstk_token(unsigned long ssp, unsigned long *token_addr) +{ + return arch_create_shstk_token(ssp, token_addr); +} + +unsigned long adjust_shstk_size(unsigned long size) +{ + if (size) + return PAGE_ALIGN(size); + + return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G)); +} + +void unmap_shadow_stack(u64 base, u64 size) +{ + int r; + + r = vm_munmap(base, size); + + /* + * mmap_write_lock_killable() failed with -EINTR. This means + * the process is about to die and have it's MM cleaned up. + * This task shouldn't ever make it back to userspace. In this + * case it is ok to leak a shadow stack, so just exit out. + */ + if (r == -EINTR) + return; + + /* + * For all other types of vm_munmap() failure, either the + * system is out of memory or there is bug. + */ + WARN_ON_ONCE(r); +} + +/* + * allocates a fresh shadow stack mapping and if required place a shadow + * stack token at base + */ +unsigned long alloc_shstk(unsigned long addr, unsigned long size, + unsigned long token_offset, bool set_res_tok) +{ + int flags = MAP_ANONYMOUS | MAP_PRIVATE; + + flags |= IS_ENABLED(CONFIG_X86_64) ? MAP_ABOVE4G : 0; + + struct mm_struct *mm = current->mm; + unsigned long mapped_addr, unused; + + if (addr) + flags |= MAP_FIXED_NOREPLACE; + + mmap_write_lock(mm); + mapped_addr = do_mmap(NULL, addr, size, PROT_READ, flags, + VM_SHADOW_STACK | VM_WRITE, 0, &unused, NULL); + mmap_write_unlock(mm); + + if (!set_res_tok || IS_ERR_VALUE(mapped_addr)) + goto out; + + if (create_shstk_token(mapped_addr + token_offset, NULL)) { + vm_munmap(mapped_addr, size); + return -EINVAL; + } + +out: + return mapped_addr; +} + +void shstk_free(struct task_struct *tsk) +{ + unsigned long base, size; + + if (!user_shstk_supported() || + !is_shstk_enabled(current)) + return; + + /* + * When fork() with CLONE_VM fails, the child (tsk) already has a + * shadow stack allocated, and exit_thread() calls this function to + * free it. In this case the parent (current) and the child share + * the same mm struct. + */ + if (!tsk->mm || tsk->mm != current->mm) + return; + + get_shstk_base_size(tsk, &base, &size); + /* + * If shstk->base is NULL, then this task is not managing its + * own shadow stack (CLONE_VFORK). So skip freeing it. + */ + if (!base) + return; + + /* + * shstk->base is NULL for CLONE_VFORK child tasks, and so is + * normal. But size = 0 on a shstk->base is not normal and + * indicated an attempt to free the thread shadow stack twice. + * Warn about it. + */ + if (WARN_ON(!size)) + return; + + unmap_shadow_stack(base, size); + + set_shstk_base_size(tsk, 0, 0); +} + +SYSCALL_DEFINE3(map_shadow_stack, unsigned long, addr, unsigned long, size, unsigned int, flags) +{ + bool set_tok = flags & SHADOW_STACK_SET_TOKEN; + unsigned long aligned_size; + + if (!user_shstk_supported()) + return -EOPNOTSUPP; + + if (flags & ~SHADOW_STACK_SET_MASK) + return -EINVAL; + + /* If there isn't space for a token */ + if (set_tok && size < SHSTK_ENTRY_SIZE) + return -ENOSPC; + + if (addr && (addr & (PAGE_SIZE - 1))) + return -EINVAL; + + if (IS_ENABLED(CONFIG_X86_64) && + addr && addr < SZ_4G) + return -ERANGE; + + /* + * An overflow would result in attempting to write the restore token + * to the wrong location. Not catastrophic, but just return the right + * error code and block it. + */ + aligned_size = PAGE_ALIGN(size); + if (aligned_size < size) + return -EOVERFLOW; + + return alloc_shstk(addr, aligned_size, size, set_tok); +} + +unsigned long shstk_alloc_thread_stack(struct task_struct *tsk, unsigned long clone_flags, + unsigned long stack_size) +{ + unsigned long addr, size; + + if (!user_shstk_supported()) + return -EOPNOTSUPP; + + /* + * If shadow stack is not enabled on the new thread, skip any + * switch to a new shadow stack. + */ + if (!is_shstk_enabled(tsk)) + return 0; + + /* + * For CLONE_VFORK the child will share the parents shadow stack. + * Make sure to clear the internal tracking of the thread shadow + * stack so the freeing logic run for child knows to leave it alone. + */ + if (clone_flags & CLONE_VFORK) { + set_shstk_base_size(tsk, 0, 0); + return 0; + } + + /* + * For !CLONE_VM the child will use a copy of the parents shadow + * stack. + */ + if (!(clone_flags & CLONE_VM)) + return 0; + + size = adjust_shstk_size(stack_size); + addr = alloc_shstk(0, size, 0, false); + if (IS_ERR_VALUE(addr)) + return addr; + + set_shstk_base_size(tsk, addr, size); + + return addr + size; +}