From patchwork Fri Mar 24 14:26:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13186820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CE01C6FD1C for ; Fri, 24 Mar 2023 14:26:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A94726B0072; Fri, 24 Mar 2023 10:26:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A448D6B0074; Fri, 24 Mar 2023 10:26:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 90D1F6B0075; Fri, 24 Mar 2023 10:26:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 804766B0072 for ; Fri, 24 Mar 2023 10:26:29 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 476B2A0A55 for ; Fri, 24 Mar 2023 14:26:29 +0000 (UTC) X-FDA: 80604017298.17.5FD9B7F Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 4008A40012 for ; Fri, 24 Mar 2023 14:26:25 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eslNKaDZ; spf=pass (imf11.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679667986; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=njAUDap1YoiePqX1ep0d+XCPs1LaFXcKfsvd98HpMnw=; b=l2W9ONjaOQx/wGkJYIzKBNnAe/vcxXgK9qEgpkV+LAclMW/0OXCXdJ5c4G0FPRzsPRPwXe bD0HrljiQ1tKvmAIzRD0v83WbhR9qxeg4zt0j+k9Nt6XTQQqS7oM2KvUDavLCQ8GgvYtt/ BgfYQ7GMzBxGuyudNzmrQd0GJPYNg3Y= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=eslNKaDZ; spf=pass (imf11.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679667986; a=rsa-sha256; cv=none; b=ibxo/9NnlOsRISJaDhTSfD16O5mieND2Lu+nOJARzL1A67Om1zeG7MH6nsT2UTUp2ix/Ho vEPrnB2npx0DJc4ls/sa+QeCOpavj+w+hZNkG0OfvtP1g+ykld9EVsdT6OdVsiflWa0/z8 c0M2OpYZ1hzWiz6Ptcs72TwvzsU2Qu0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1679667985; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=njAUDap1YoiePqX1ep0d+XCPs1LaFXcKfsvd98HpMnw=; b=eslNKaDZpcz/XSFqVI4iN5KfFLcW+2ttOGCWWJxlKSmUnGgkt8ZmXdEZUOETcWTLsrYl6G 3cqaMZNerjuZRX8Va6i3sOYQN7nVxmQ+a6RXxAJ4/V3kkgPMheosek0ypVqL2A4pQkCxca SfJGKJ9Q3u2mdNDW9y1FMbFf6+k0NeY= Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-8-WuMSSWuyOeiE9pAjejDKRg-1; Fri, 24 Mar 2023 10:26:24 -0400 X-MC-Unique: WuMSSWuyOeiE9pAjejDKRg-1 Received: by mail-qk1-f199.google.com with SMTP id r70-20020a374449000000b00746c31401f0so841426qka.6 for ; Fri, 24 Mar 2023 07:26:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679667983; x=1682259983; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=njAUDap1YoiePqX1ep0d+XCPs1LaFXcKfsvd98HpMnw=; b=3x5SJC7RwIbue5I/8yLl9Ea3SPrZh9VpgaLRDc2LKsaYRIje2Nf9t/ljZ40G+5gnfd +0L202vihjYWy03ZWOKYrfgffmk9K7O2F0E0kFrIr67c1YQ0rVB+Io67Du89fN2lCeaj dtLtNH84352e9sGHP22JIVgJgKmrXDRKgAKL+EG8VhAhl/vnJI1YWbGzKr/n10wFUWVY fD1GQTBBgGCtbFFQPUhtsv6fs3b/uTXrB6VzyoCF99W7nXbX6XWAMk+o8jbIlJ5TF/Qc MfjQUxlMJQ2tvcVfYr0d5WFb9t5XUZQQxoRAeM/x/ONRuJmKYk9azp/3SuY+sQVL3b4b 14oA== X-Gm-Message-State: AO0yUKXcQfNct52ziaGUie+0K1wULNxg559vuc52uFRh1JaIYm+00Wh4 UgSnsB3cursgztiZaBKZE7EHzj4wNSkNCTwoT916seNzarqL70dwVGwLuFsc73rOBYK53CoAZR/ 2UIBYnDJd8kYfD0i/lutQnQ9lAMfot5Vux/G/PmPbexnfycnvQHMABXrMM6Cj0fHlfcq9 X-Received: by 2002:a05:622a:1a18:b0:3bf:a60d:43b9 with SMTP id f24-20020a05622a1a1800b003bfa60d43b9mr3348959qtb.4.1679667982697; Fri, 24 Mar 2023 07:26:22 -0700 (PDT) X-Google-Smtp-Source: AK7set9wCBN9QmO2rC/Ob5zXNxNLGPTDDrI6pt6+Z4sTLoNtnwJCOM1zLtDSVeejcWRSe7w/LcEIWg== X-Received: by 2002:a05:622a:1a18:b0:3bf:a60d:43b9 with SMTP id f24-20020a05622a1a1800b003bfa60d43b9mr3348920qtb.4.1679667982337; Fri, 24 Mar 2023 07:26:22 -0700 (PDT) Received: from x1n.. (bras-base-aurron9127w-grc-40-70-52-229-124.dsl.bell.ca. [70.52.229.124]) by smtp.gmail.com with ESMTPSA id 141-20020a370793000000b0074672975d5csm10024794qkh.91.2023.03.24.07.26.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 24 Mar 2023 07:26:21 -0700 (PDT) From: Peter Xu To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Andrew Morton , Mike Rapoport , Nadav Amit , Axel Rasmussen , peterx@redhat.com, David Hildenbrand , Mike Kravetz , Andrea Arcangeli , Muhammad Usama Anjum , linux-stable Subject: [PATCH v3] mm/hugetlb: Fix uffd wr-protection for CoW optimization path Date: Fri, 24 Mar 2023 10:26:20 -0400 Message-Id: <20230324142620.2344140-1-peterx@redhat.com> X-Mailer: git-send-email 2.39.1 MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspamd-Queue-Id: 4008A40012 X-Stat-Signature: ky14wh6cnzc1fwdpfnor8ykb3weqk5qo X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1679667985-489866 X-HE-Meta: U2FsdGVkX1/Y6pVe82BOEhMHeiQWai9lM1liLv+Z4aXIWdYalriKGa3BzBzRL1oGZrHKlm8FY7iLhwmSS5G8cDv8e2RPyzJrozQLdXiC1JgrfS4mF98vdUqKm51shL28Ul3NmzD17mMUrXFyNJ94KfVMoY/gSGJmwB51sp0dYYIg0F/vNS4gIedwyfGP79Jy1iBFfPfiODowPowoWacRzXB3+ut4ez8WT/Bzc1m0rTkFRkPriUarIIdnRe8UIJG6l3vFz8O2O/aLwJh/0i34X5aQ5mVnn6zY9bTGSe70ccaoHQY8s//Sd8v8EC129Xosb8TT24c4ytcU1ASaC+JU2xHKo6zqey27VoG48ji8Qv7CHFnN8M/MmJuxYqvXYN8TYMkBtnheffLyf7AzRzpyT5g2TiYKHdIO/bOS0NGNYi9p/wGteL8ppJCmHvqvYu89D6c9GKMMwiLQK2p4EIFa/EECyn0DYXXGFRFs23WSx6+FUXwyDbqFfEbXkks72JcMF4Pt/3VStIHd7ZykbonSTDrfHEzEVv5Wd/tYo+uI2jzNWvNnxIpAICMQb0a2Wxe4lre8KlfHXOpXBFwOoxrb+hvQy5vUUHlrGN7U8YEZ1vRjZOCIZJgyytQWIHU0OW1XXqIKbAgnIebK8kbZnQSCASJDCHDIj23xjAcuPBa5tcFbk3iuechqFyH4joqE/02wlMaClvy80d3FFy3VYKQmuDNZ7ARfBRJ0nu+st9W3EKPslHPEJUObnRYQJ7jrNkKxf/Dmh1TpceXdeI+3PZoO8Awyxy7XK3LW5b6shd0VKXDNQpmxeX1/ziSawomsCmnQ/GKaghPL1FFP6G4bwqP9V+U8n1z5aUc8yzSV1Ba6UU1yQhDqIE9VrvEXSozOFYYbHkAYSG+BQb7OZAhw+dfRQY3iBs7TDJSZuA8k4nIJ8NGzb3Xxlnxw/p4sxTDfna0WLBUpGwBFFLpQotoqRNX GtKxKD3c nhQkqstafg55qjtkokY4ioANysbNx54F+VVPKAIC+ZQJn5jv9XhBGHq28UNaUdWVexxAh9wcJ5xKvFWoy2PjPQ9MMaTlMSbNoFJYG09auY61p9AETLhkyxh4oe+EmW2EU2RpMHMHlq3pS+54s53C8xipP2eU0hIZznj7C6L0L+tCwUZqAu2IAtcOzwZz1gQ3a2MCo77bIWuxeJvWKwnVqHbRb0hYTlblEcjeBUMXrlv3l5VoBLRoB6Wgyd9v44FHwv3cVk99bNdHk1frTldI9pmCFaHBgNnUxoyjCW+xiIv0Qta1MZazFcD+DxZ3ruwPv/PQu5i5JbShw7NWTnPx3YYp+U+w29z7BPss0V7d3FUzbUZ8XTx5foLYPJtpvcRKfV8dxoAY7XnOznAU8VQZsTv72ZFY9hsMFBe0QCEP02brUMHP3nERu+W0iVOg/BUgh40jvSiU7kUTO4kMEoLI6zrFB3fNSL9JqDyWsYFTlZW7qpPTkJPBJdeqUdoZw7k1TIVs6CKHAEBEKHzWPwTl6UEdsG95D9VP+dFyt1h0uNxSpCV5qhAF6Y8+ATcWVxngzXsd/ISRrLlF2wCyHKbtdOs1rbhZ/HK3zdLESf33dtkq+Y2g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This patch fixes an issue that a hugetlb uffd-wr-protected mapping can be writable even with uffd-wp bit set. It only happens with hugetlb private mappings, when someone firstly wr-protects a missing pte (which will install a pte marker), then a write to the same page without any prior access to the page. Userfaultfd-wp trap for hugetlb was implemented in hugetlb_fault() before reaching hugetlb_wp() to avoid taking more locks that userfault won't need. However there's one CoW optimization path that can trigger hugetlb_wp() inside hugetlb_no_page(), which will bypass the trap. This patch skips hugetlb_wp() for CoW and retries the fault if uffd-wp bit is detected. The new path will only trigger in the CoW optimization path because generic hugetlb_fault() (e.g. when a present pte was wr-protected) will resolve the uffd-wp bit already. Also make sure anonymous UNSHARE won't be affected and can still be resolved, IOW only skip CoW not CoR. This patch will be needed for v5.19+ hence copy stable. Reported-by: Muhammad Usama Anjum Cc: linux-stable Fixes: 166f3ecc0daf ("mm/hugetlb: hook page faults for uffd write protection") Signed-off-by: Peter Xu Tested-by: Muhammad Usama Anjum Reviewed-by: Mike Kravetz Acked-by: David Hildenbrand --- Notes: v2 is not on the list but in an attachment in the reply; this v3 is mostly to make sure it's not the same as the patch used to be attached. Sorry Andrew, we need to drop the queued one as I rewrote the commit message. Muhammad, I didn't attach your T-b because of the slight functional change. Please feel free to re-attach if it still works for you (which I believe should). thanks, --- mm/hugetlb.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 8bfd07f4c143..a58b3739ed4b 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5478,7 +5478,7 @@ static vm_fault_t hugetlb_wp(struct mm_struct *mm, struct vm_area_struct *vma, struct folio *pagecache_folio, spinlock_t *ptl) { const bool unshare = flags & FAULT_FLAG_UNSHARE; - pte_t pte; + pte_t pte = huge_ptep_get(ptep); struct hstate *h = hstate_vma(vma); struct page *old_page; struct folio *new_folio; @@ -5487,6 +5487,17 @@ static vm_fault_t hugetlb_wp(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long haddr = address & huge_page_mask(h); struct mmu_notifier_range range; + /* + * Never handle CoW for uffd-wp protected pages. It should be only + * handled when the uffd-wp protection is removed. + * + * Note that only the CoW optimization path (in hugetlb_no_page()) + * can trigger this, because hugetlb_fault() will always resolve + * uffd-wp bit first. + */ + if (!unshare && huge_pte_uffd_wp(pte)) + return 0; + /* * hugetlb does not support FOLL_FORCE-style write faults that keep the * PTE mapped R/O such as maybe_mkwrite() would do. @@ -5500,7 +5511,6 @@ static vm_fault_t hugetlb_wp(struct mm_struct *mm, struct vm_area_struct *vma, return 0; } - pte = huge_ptep_get(ptep); old_page = pte_page(pte); delayacct_wpcopy_start();