From patchwork Tue Jun 13 00:10:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 13277737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA24FC88CB2 for ; Tue, 13 Jun 2023 00:13:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C894F8E0017; Mon, 12 Jun 2023 20:12:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C11FA8E000B; Mon, 12 Jun 2023 20:12:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EF408E0017; Mon, 12 Jun 2023 20:12:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7D3CD8E000B for ; Mon, 12 Jun 2023 20:12:30 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 564C1AF4BD for ; Tue, 13 Jun 2023 00:12:30 +0000 (UTC) X-FDA: 80895798060.29.BEED0B6 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf01.hostedemail.com (Postfix) with ESMTP id 19A1D40014 for ; Tue, 13 Jun 2023 00:12:27 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HM6sQ3mE; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf01.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686615148; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OTL6GqoGz5oweSC99z4UnMl2uF9TwgX18y0erkZrvzI=; b=2/y7N2qvJ4R+xPvqCaSjsec860OhHg5QQlDopgNzIxm1uGLpFt+q9D/ckm17wfMBzA1HvY AEZvPi0a+aHRB9fSz8I3X+juW9aE+2QY88/c5WssCkjTaxDTtW0H56DnX7MVvI4Z1KXedh QFc3tf5x7dnv8hT8m1dtl3SJ/Us8774= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=HM6sQ3mE; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf01.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686615148; a=rsa-sha256; cv=none; b=I6evev0R0ZIOPRjXg+Iiws5+Z4Amj1bWSnWqRrP1JeKxEaJvY01fAk7cV3dDVZWKihdAQI ZblJtzbf77JncZiTDjxiKO+5RIfi1I6VJVMuURWBDdgln1IGKlTvoe0gqEQ1SIIg5KNS+6 pK5BpwByPIzR2HdeTuNMYaFMDEG5YxM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686615148; x=1718151148; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=nq8FP2aUY00edE8aipvtg7A+Bog2Zlh3+3MlR+vKuX8=; b=HM6sQ3mESelD7KURIqapsbVU0BbBVdWDWt607qArtFWA3GQwGTO6cys1 5SSKvrhfKqajuaC1v7/9weg/Sfa4oMw++CVfWenRiLkKKreD5PbuPI3wl YOB1DAHxFm0ueAMYhG2QMmozoiNCjvPl4eCJbbeo7e1Vil+u2EwivtszP UwxRt9jRae4xgZzK+29s4mlprBf3i90RdXbQGMWSjlCL6dcD1SahKeTxF n9Yy0RzYQyrlSLBZeqWP+w5v6vwwtq983RyWma3S7KDZ8NcXsZaR+r5vw f/iOfvQ60yUkRcZXsedEcRS63DlBt15b1Wvz8gsATDZdPOZt7mXO+CqoX g==; X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="361557180" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="361557180" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10739"; a="835671056" X-IronPort-AV: E=Sophos;i="6.00,238,1681196400"; d="scan'208";a="835671056" Received: from almeisch-mobl1.amr.corp.intel.com (HELO rpedgeco-desk4.amr.corp.intel.com) ([10.209.42.242]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Jun 2023 17:12:26 -0700 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com, david@redhat.com, debug@rivosinc.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, broonie@kernel.org Cc: rick.p.edgecombe@intel.com, Pengfei Xu Subject: [PATCH v9 22/42] mm: Don't allow write GUPs to shadow stack memory Date: Mon, 12 Jun 2023 17:10:48 -0700 Message-Id: <20230613001108.3040476-23-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> References: <20230613001108.3040476-1-rick.p.edgecombe@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 19A1D40014 X-Stat-Signature: bxcmewcuhewarjozm8zxwz3eahpmxyop X-Rspam-User: X-HE-Tag: 1686615147-583021 X-HE-Meta: U2FsdGVkX1+QEKEjpUzdrWd+2qpQHR43YijoYhGjQFHxrEkn402JKCxI+pUQ+THbhoYKmSL7pcweZqemetrP7f7iA4VtG8S650rlV9zwHEBxryLuuASYev+8xEuvNU38npK5DeOyG3xwLZvGZuGviy8M0numPopmdeC0zzuJnsdXDJZlLxr61PemXH71pafO/hlkneUX6hfN6Pic8lpNZMMEDxebS6/IMtsVrZHH8sVyLYQroNBMfF//KsaCzEYiOj5tg3/xLbfhJ9pPH5vCzUYDTOtU01D4Yjw4FzuT3OkofQO1kxlPnlRz6J4i/joPemmWqGI+XBDz5o9Muo2WeNSjVnErmMujPFcpYbO7PsHe44KSf2+hUabM4lBBt7KdUWoxGl8Gq04DRhlZz+jcaIbWI2n2GSgFtEcV4l7ZY/rJSQHYQbxIl1/+FNuBP5+lWXrX+qSPlRM2sabf13eWVua66E3Tf4/cDvL2AzCnimHRBHjV1Fof/uqUXOCCdJbip0bSzXwBbIhs4wp1fkyztpaKLjO4pXetOPkAdokkMH2MnTYOzbV6HPEwDvGlvB9HvQuDwgBohiVtEfZbyFrWhQJeXK9YjFfrV49UEuEZKnq63fzBCDB4pUg36QXaSABKUbYawbb0cMhV4F293/n60etvjgZGiTBMLRBBjeHZ7QagtXNk33BGeaeWawHS6okp45gEppyVY9VEB0xuOiSwxKznD9bXP1/UFjz8uwsstgCzEOHip3m347zv8NZGHzRU+4xC820Y2gt/fS3mTbAoLm2Tnfe4uHNCX89zyM/OmRz99/VCS0ABvIIrgnIY1tAysyNfJ1oIiNm6XRiVc6zyiSW5Kovo+OjUw6NF95xhPfUn5m9zOThCcUBF83qT68IJzI7QOhja1I8TZteq2AZ1laKMiWBr/Mm4rl+RhOvowLL22FpRAxNvLFEwJxjXjAbp02f3fVN88Ak3RHOmfJF vPCWdFEL dx2yyovGZqPos5/2NAV5wUnBKB9lp4e60pSBA6Rgtyz3JyE5u+z5u88UQnOyTt0HTns2HUqUGvOuWSDsQh8WWYxCSMnaL4UL5oCYmKR1LONqL++QfUOPyIYUKNewoTfS4He4Hz37zEAW5KzROd++NR+lplRKlWetEfrKRKAwT9uUrKvw22/3sKuW2uCCbERIt+MUH+osIItEU+gaCFkKVOXSi3LoYJr0v2RUk6T4ONMpcUMOwOQu/Hq4TljTcasnHWHfaUFHWICYD4HzMz6oPDgnhjhOo+jix6O1EM9665KaInbPDrfJtqVzdUYVb5rxezSkYquZQ9ITZK9+nsoB6d7OKSvz3c5BzaV9SFDHKAPZfcC2Sd7BXvlCZW+5oydxBkPjzfxIpc0NeFt4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The x86 Control-flow Enforcement Technology (CET) feature includes a new type of memory called shadow stack. This shadow stack memory has some unusual properties, which requires some core mm changes to function properly. In userspace, shadow stack memory is writable only in very specific, controlled ways. However, since userspace can, even in the limited ways, modify shadow stack contents, the kernel treats it as writable memory. As a result, without additional work there would remain many ways for userspace to trigger the kernel to write arbitrary data to shadow stacks via get_user_pages(, FOLL_WRITE) based operations. To help userspace protect their shadow stacks, make this a little less exposed by blocking writable get_user_pages() operations for shadow stack VMAs. Still allow FOLL_FORCE to write through shadow stack protections, as it does for read-only protections. This is required for debugging use cases. Signed-off-by: Rick Edgecombe Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Kees Cook Acked-by: Mike Rapoport (IBM) Acked-by: David Hildenbrand Tested-by: Pengfei Xu Tested-by: John Allen Tested-by: Kees Cook --- arch/x86/include/asm/pgtable.h | 5 +++++ mm/gup.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 5383f7282f89..fce35f5d4a4e 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1630,6 +1630,11 @@ static inline bool __pte_access_permitted(unsigned long pteval, bool write) { unsigned long need_pte_bits = _PAGE_PRESENT|_PAGE_USER; + /* + * Write=0,Dirty=1 PTEs are shadow stack, which the kernel + * shouldn't generally allow access to, but since they + * are already Write=0, the below logic covers both cases. + */ if (write) need_pte_bits |= _PAGE_RW; diff --git a/mm/gup.c b/mm/gup.c index bbe416236593..cc0dd5267509 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -978,7 +978,7 @@ static int check_vma_flags(struct vm_area_struct *vma, unsigned long gup_flags) return -EFAULT; if (write) { - if (!(vm_flags & VM_WRITE)) { + if (!(vm_flags & VM_WRITE) || (vm_flags & VM_SHADOW_STACK)) { if (!(gup_flags & FOLL_FORCE)) return -EFAULT; /* hugetlb does not support FOLL_FORCE|FOLL_WRITE. */