From patchwork Mon Apr 15 20:52:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13630716 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BFA8C4345F for ; Mon, 15 Apr 2024 20:53:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC0936B007B; Mon, 15 Apr 2024 16:53:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C70A06B0089; Mon, 15 Apr 2024 16:53:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B11126B008C; Mon, 15 Apr 2024 16:53:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9266B6B007B for ; Mon, 15 Apr 2024 16:53:10 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5B457A1CCC for ; Mon, 15 Apr 2024 20:53:10 +0000 (UTC) X-FDA: 82012966140.28.B6CA2E8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf12.hostedemail.com (Postfix) with ESMTP id A2C6E40002 for ; Mon, 15 Apr 2024 20:53:06 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SvNysyvc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713214388; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=7++5H78YlgDsEMmQcYVCUBmYO6bg5gOwdVwvkU8dfpY=; b=VOWexbelkcC/HPL0JmK13JCnIqmt+6vjtDGLeIhouslJKUACtaB7xwfTasWIsbZOXxfQUr vOL5kQ2oJHN8R1PAZ0D+jHuSRV5upMGIAdSezpHIaLJ8rGn6egwMMr2GuAun6KHgtikLbS VgZvgTd3U0NHKP8YZB7/+OvE6PZRiNw= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=SvNysyvc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of peterx@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=peterx@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713214388; a=rsa-sha256; cv=none; b=ljTL12kfXwCpcCMnz5yLGPFdCi4LFUSa1UUaSmi5a3TTB5OUO5bv+nWmVEv2mSrMT0MECP 5P3ih5jqJ2+6RytXzHbTXivLGLBAtDMb6AYPeor1sVRrJWFGxnxE1PTO/43qcmyDWM4AIx ahVA/Xc4sduyN606b/CUZ2HTsMcKOWk= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713214385; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=7++5H78YlgDsEMmQcYVCUBmYO6bg5gOwdVwvkU8dfpY=; b=SvNysyvc3Z1/ysLoE0gyIrqNp5ar+jO+f0X7uXryiTGX8pcbUtSAq3EAYKH5N7L9FnDPdx ts5T3b+/8CWJrvlyG52wqUKC0oKSYo8vzb2uYP3C2//w1+AXApb6uVqVtCBu+/cKPj5Age 4F+paI/vryhfeVR8g13dQc4Pzu3ocy4= Received: from mail-ua1-f71.google.com (mail-ua1-f71.google.com [209.85.222.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-661-pH68bc0uP9u_HlTmCVQn_Q-1; Mon, 15 Apr 2024 16:53:02 -0400 X-MC-Unique: pH68bc0uP9u_HlTmCVQn_Q-1 Received: by mail-ua1-f71.google.com with SMTP id a1e0cc1a2514c-7e822c441d5so370478241.0 for ; Mon, 15 Apr 2024 13:53:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713214382; x=1713819182; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=7++5H78YlgDsEMmQcYVCUBmYO6bg5gOwdVwvkU8dfpY=; b=Y0CGLJpS4dgf9IMqTu32oREHQ2/XEThMgDUMQICz4B1mB3VmUQFJlWIEMYz3ldrjev jmts4fH0Zn3aLCyLs2eyIK97iVSwnQG0jBgEhof06p7iOEZa7g7swkfrxtsaIcLkGtyK V2luzTb4IhQ/K+dAG8FRnBbr0Bf7ida7JIrR8cjUliTy83QNdFPKrKUdxhTpQ4bCles4 Pr2/kbEtrN14XDCi4iZtqbZHWdGp8YCKA1ZqgcwQpL3RsUyKPshiFqeTiTCGHaDv8cbU cIonP26nXxwT4hnVpLg7QPqLEDj9o7y4BinAfNrN6H2XJNnT6e/L2Pd+Gno0mgCZXbZ1 S6/g== X-Forwarded-Encrypted: i=1; AJvYcCXfRg8e1TRQ8DV2U4Fr7q6VeEsyc8d5jQ5gH2XsgVJvbQFDOaNGsOWRAb/8rnkrRh5iyo/P5h4SUQr2+1ZZL8Wabt4= X-Gm-Message-State: AOJu0YwPu/PZCirT28KLR0vtYxZB0rnI04iRs6CCKP5hM4UBjxvcRu2m xX+UEmZApGKR4FV/qN1i95jXpILWhZWaOasw4R+m36KvLGm60y9DTrtTP6iBwoGLCdr/Deggenv yGd60MoB8OYhPR6Y9CWdHhG5R0m7wB0dA6kxd6HNW40l6SGeS X-Received: by 2002:a05:6102:5493:b0:47a:145d:ff71 with SMTP id bk19-20020a056102549300b0047a145dff71mr8677876vsb.3.1713214381828; Mon, 15 Apr 2024 13:53:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEyPe5nOGtZ1+eez23xTwwWwXmpPWZh75RjHcvLbAwZQLmYYWzn5kF0KOJY8fVpQefCJixTdQ== X-Received: by 2002:a05:6102:5493:b0:47a:145d:ff71 with SMTP id bk19-20020a056102549300b0047a145dff71mr8677863vsb.3.1713214381409; Mon, 15 Apr 2024 13:53:01 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id u18-20020ae9c012000000b0078eeea25dd6sm1000911qkk.116.2024.04.15.13.53.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Apr 2024 13:53:00 -0700 (PDT) From: Peter Xu To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: David Hildenbrand , Andrew Morton , Nadav Amit , Axel Rasmussen , peterx@redhat.com, Pasha Tatashin Subject: [PATCH] mm/page_table_check: Support userfault wr-protect entries Date: Mon, 15 Apr 2024 16:52:59 -0400 Message-ID: <20240415205259.2535077-1-peterx@redhat.com> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: A2C6E40002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: a75d8nz5fa4t9fhsf5bdi6je1pf83w6a X-HE-Tag: 1713214386-102549 X-HE-Meta: U2FsdGVkX19YswiknQvnGleybU/4SIbPBMbxDB4NO2bZNTB6NkMdGnVu6+gh3G2+P8B/lC/c11O6Y7hvn3l8/jsDsJo0hFlqtPx+yn6l5PH2xmatwZ7JdfL6NuFl2lwUU93QsR/WNpnyncH9pWIJYbFXmD5M7S1lNAhKLQksQcYKc9jmXboEiA4UJkEW+cuwpI2ttT3bHqNhK8JuzcCCqIOtJYgeDxzwB3VCa9Xw/gF3bJ2iKSWDa/SrZ2hi3ZrnOh7wUmtK8mY0PL9V3IUOBBzLlEEzasi4a+iSH08TFGunHgG/OxdRYf+j0DUlOlb/olMq4eNIqqFbtpInOAtoSWX75YG6R3ZB0+UONzYcmafEH9YJ1kZwWTdrbgKo2DQQMUA9K7KUp0X36noPusrcmJ2lUIiwjdtNxmrGT8zeSfQaChslYqr2JuPaTO15nbYPaU5qBoEC2Cw92G2eC0IWrnH7qlrojTi4USeKEY+vKhjFPy7aLb6N1+XMd3NSieksPj1lml35USFDof2fMTxup9DGLLBIt2PfWSPasH7CdPavNyCa1YcoYwdy/ruZWW52Gpa6KCdz2354wZqU6KranAwy0DtWrehDE/AQ/mpYP4bdwlCYt8qkASiRYbaqZUuSpWjXfsYT92y0h2rs1rFcUEt2FcP4OrKfDr/HuZYew/A6b4ypKtBDASBhVRqFNG4fHqmYUrPTiiylbsWCpnVk55rPL5x+MukFHyHurv95ge31c9ODN+zVRnq+nh2C6LtRhq3Iq7f/erSL7cpX3KPiol7STzDMltmPn+knO4OOqsTFo0v0/LeYD3HhvQS7HXZ8hAc8miVuzFv+bfc0tf9nIcKNdCGWQSqmhbY13Kr/749EArfvAdfUGa+EXjSXmQ/Hs07r+CD7JUNZQ9wZnd2LQyaMqbTuRxokiQQQ+EBjkWHL0SyFcAjYnpeRV4p+dmyHaaYQ2aD75U6cNdI0aj8 JZvWW36l GETpzSDbt1KjBWdmxgU3o4NCuaxs58dJSQWJy9Zani73wqY21SmkuichqDR7TE6ko0U2hlWwokfmY2sF7h7BSr9p7x2i7xVqrW5QgIi2en3HVbdVmdc8O75QWhUBv6+cOB6tJFe7cC2yWIJbmaUjBlkfLLwny7vJKhRrGo9gVrPeL8zYpl/wwW2o6rJefw8upIt9Yw7C/DsNK7fdji6vAmlWdBEu9j43n92QfAkuLO1rjgTvk3IXW/Os/Zi0pfkpPUD/QHOVrbkvpiMa2DIVk+hrWcp3JE+xQH2m5F4spEbKEGUunKr7eG+zU390lGg7dCD3ut9RwnVI8bZ8vbOHQpbrztoIyrh+7xrZOsKNWDd6+fm6S5jMnp7tW2WYAJzV4g7MObA9VQkWe+Eq9OudfLeJY2HhYm8RoRAYtygJWDNFuoG1rfRUG1DMN23K9TcjCKrPVJzMZhzPBRmMaxsIi9C2nIDx9VgPfcUShUqHLYgCRNeKELpV5mNQhYPu8ShoFRKjqQq3aSWTfe/icEGlPZqlXvNhLnd7wW30zkmwTuGIeq6oxJuPYVkPC20rRO9J3Y8bXrNzan5jFlYAt7hZlktIFAdbO7TlMIh1ldDy4lbNjQUTZOrC32AVp1AuELMatEsqhju3YnlA+4FA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Allow page_table_check hooks to check over userfaultfd wr-protect criteria upon pgtable updates. The rule is no co-existance allowed for any writable flag against userfault wr-protect flag. This should be better than c2da319c2e, where we used to only sanitize such issues during a pgtable walk, but when hitting such issue we don't have a good chance to know where does that writable bit came from [1], so that even the pgtable walk exposes a kernel bug (which is still helpful on triaging) but not easy to track and debug. Now we switch to track the source. It's much easier too with the recent introduction of page table check. There are some limitations with using the page table check here for userfaultfd wr-protect purpose: - It is only enabled with explicit enablement of page table check configs and/or boot parameters, but should be good enough to track at least syzbot issues, as syzbot should enable PAGE_TABLE_CHECK[_ENFORCED] for x86 [1]. We used to have DEBUG_VM but it's now off for most distros, while distros also normally not enable PAGE_TABLE_CHECK[_ENFORCED], which is similar. - It conditionally works with the ptep_modify_prot API. It will be bypassed when e.g. XEN PV is enabled, however still work for most of the rest scenarios, which should be the common cases so should be good enough. - Hugetlb check is a bit hairy, as the page table check cannot identify hugetlb pte or normal pte via trapping at set_pte_at(), because of the current design where hugetlb maps every layers to pte_t... For example, the default set_huge_pte_at() can invoke set_pte_at() directly and lose the hugetlb context, treating it the same as a normal pte_t. So far it's fine because we have huge_pte_uffd_wp() always equals to pte_uffd_wp() as long as supported (x86 only). It'll be a bigger problem when we'll define _PAGE_UFFD_WP differently at various pgtable levels, because then one huge_pte_uffd_wp() per-arch will stop making sense first.. as of now we can leave this for later too. This patch also removes commit c2da319c2e altogether, as we have something better now. [1] https://lore.kernel.org/all/000000000000dce0530615c89210@google.com/ Cc: Pasha Tatashin Signed-off-by: Peter Xu --- arch/x86/include/asm/pgtable.h | 18 +----------------- mm/page_table_check.c | 32 +++++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 18 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 273f7557218c..65b8e5bb902c 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -388,23 +388,7 @@ static inline pte_t pte_wrprotect(pte_t pte) #ifdef CONFIG_HAVE_ARCH_USERFAULTFD_WP static inline int pte_uffd_wp(pte_t pte) { - bool wp = pte_flags(pte) & _PAGE_UFFD_WP; - -#ifdef CONFIG_DEBUG_VM - /* - * Having write bit for wr-protect-marked present ptes is fatal, - * because it means the uffd-wp bit will be ignored and write will - * just go through. - * - * Use any chance of pgtable walking to verify this (e.g., when - * page swapped out or being migrated for all purposes). It means - * something is already wrong. Tell the admin even before the - * process crashes. We also nail it with wrong pgtable setup. - */ - WARN_ON_ONCE(wp && pte_write(pte)); -#endif - - return wp; + return pte_flags(pte) & _PAGE_UFFD_WP; } static inline pte_t pte_mkuffd_wp(pte_t pte) diff --git a/mm/page_table_check.c b/mm/page_table_check.c index af69c3c8f7c2..d4eb1212f0f5 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -7,6 +7,8 @@ #include #include #include +#include +#include #undef pr_fmt #define pr_fmt(fmt) "page_table_check: " fmt @@ -182,6 +184,23 @@ void __page_table_check_pud_clear(struct mm_struct *mm, pud_t pud) } EXPORT_SYMBOL(__page_table_check_pud_clear); +/* Whether the swap entry cached writable information */ +static inline bool swap_cached_writable(swp_entry_t entry) +{ + unsigned type = swp_type(entry); + + return type == SWP_DEVICE_EXCLUSIVE_WRITE || + type == SWP_MIGRATION_WRITE; +} + +static inline void __page_table_check_pte(pte_t pte) +{ + if (pte_present(pte) && pte_uffd_wp(pte)) + WARN_ON_ONCE(pte_write(pte)); + else if (is_swap_pte(pte) && pte_swp_uffd_wp(pte)) + WARN_ON_ONCE(swap_cached_writable(pte_to_swp_entry(pte))); +} + void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte, unsigned int nr) { @@ -190,18 +209,29 @@ void __page_table_check_ptes_set(struct mm_struct *mm, pte_t *ptep, pte_t pte, if (&init_mm == mm) return; - for (i = 0; i < nr; i++) + for (i = 0; i < nr; i++) { + __page_table_check_pte(pte); __page_table_check_pte_clear(mm, ptep_get(ptep + i)); + } if (pte_user_accessible_page(pte)) page_table_check_set(pte_pfn(pte), nr, pte_write(pte)); } EXPORT_SYMBOL(__page_table_check_ptes_set); +static inline void __page_table_check_pmd(pmd_t pmd) +{ + if (pmd_present(pmd) && pmd_uffd_wp(pmd)) + WARN_ON_ONCE(pmd_write(pmd)); + else if (is_swap_pmd(pmd) && pmd_swp_uffd_wp(pmd)) + WARN_ON_ONCE(swap_cached_writable(pmd_to_swp_entry(pmd))); +} + void __page_table_check_pmd_set(struct mm_struct *mm, pmd_t *pmdp, pmd_t pmd) { if (&init_mm == mm) return; + __page_table_check_pmd(pmd); __page_table_check_pmd_clear(mm, *pmdp); if (pmd_user_accessible_page(pmd)) { page_table_check_set(pmd_pfn(pmd), PMD_SIZE >> PAGE_SHIFT,