From patchwork Wed Apr 2 16:07:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14036200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDB6DC36017 for ; Wed, 2 Apr 2025 16:07:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 291B5280007; Wed, 2 Apr 2025 12:07:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 241FD280001; Wed, 2 Apr 2025 12:07:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 093BF280007; Wed, 2 Apr 2025 12:07:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D969E280001 for ; Wed, 2 Apr 2025 12:07:41 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CE7B05A33A for ; Wed, 2 Apr 2025 16:07:43 +0000 (UTC) X-FDA: 83289584406.30.FE78D16 Received: from smtp-fw-52002.amazon.com (smtp-fw-52002.amazon.com [52.119.213.150]) by imf04.hostedemail.com (Postfix) with ESMTP id 738A54000E for ; Wed, 2 Apr 2025 16:07:41 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=amazon.com header.s=amazon201209 header.b=s+m1DQYw; spf=pass (imf04.hostedemail.com: domain of "prvs=1800b2f01=kalyazin@amazon.co.uk" designates 52.119.213.150 as permitted sender) smtp.mailfrom="prvs=1800b2f01=kalyazin@amazon.co.uk"; dmarc=pass (policy=quarantine) header.from=amazon.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743610061; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5WJu5FXIQjlvFzc2S3oagYQCTjwqOt8Vj/HqIU2A0PE=; b=B3pCCXAFZscWA+/BKQypS5Ab360E5Nn0RgaZ/LklRO9rZCTrYVSL+yhS0cUX5sgD9Huj9i hv3SHAd9Tfy9DnBRRueYOGbQmblpS6NOagFLFBF/0UqkRqyZWBrXG5sbjplEepkjdvwlGI Nb+93LFavzzZrehx7wV524ZtaXmjkos= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=amazon.com header.s=amazon201209 header.b=s+m1DQYw; spf=pass (imf04.hostedemail.com: domain of "prvs=1800b2f01=kalyazin@amazon.co.uk" designates 52.119.213.150 as permitted sender) smtp.mailfrom="prvs=1800b2f01=kalyazin@amazon.co.uk"; dmarc=pass (policy=quarantine) header.from=amazon.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743610061; a=rsa-sha256; cv=none; b=ec+3IldhjoMoWevqMQlsLHQdtUC8LeN5U0KvMlpUAnKzZIGdIZdKEwqpnFWdP/zUF4Rlw2 Ed6pPZbaiAQEBEiIhJpk3GG+jHHxkxlZT28GHWBAVYdDhX623uwH0D+LVcYLHt9KS92yH4 Jsl9VAiMyxdy6MTQ5mQM4jg6G2dTZfE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743610062; x=1775146062; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5WJu5FXIQjlvFzc2S3oagYQCTjwqOt8Vj/HqIU2A0PE=; b=s+m1DQYwf144iN/Fs/vO78aud7B4lwY4ofU/ZwmjlV5+XHyQjKaH1Fzh 1Ixk1+xjNeV8AT7iATHeYBJXmTSrcbxDb2h+lTVer7Eeekq/SmLkLA4wk k7xKsK/d8bln9/qRJO6+QB0HEyWMwsob6f2TXZI5ptIhDnThSRcivEdVI s=; X-IronPort-AV: E=Sophos;i="6.15,182,1739836800"; d="scan'208";a="710490762" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-52002.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Apr 2025 16:07:38 +0000 Received: from EX19MTAEUC002.ant.amazon.com [10.0.10.100:19931] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.36.151:2525] with esmtp (Farcaster) id 5d4c3ca3-5bf6-48b9-9c8f-8b474d7042db; Wed, 2 Apr 2025 16:07:37 +0000 (UTC) X-Farcaster-Flow-ID: 5d4c3ca3-5bf6-48b9-9c8f-8b474d7042db Received: from EX19D022EUA001.ant.amazon.com (10.252.50.125) by EX19MTAEUC002.ant.amazon.com (10.252.51.245) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Wed, 2 Apr 2025 16:07:36 +0000 Received: from EX19MTAUEC001.ant.amazon.com (10.252.135.222) by EX19D022EUA001.ant.amazon.com (10.252.50.125) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Wed, 2 Apr 2025 16:07:36 +0000 Received: from email-imr-corp-prod-iad-all-1b-8410187a.us-east-1.amazon.com (10.43.8.6) by mail-relay.amazon.com (10.252.135.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Wed, 2 Apr 2025 16:07:36 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-iad-all-1b-8410187a.us-east-1.amazon.com (Postfix) with ESMTPS id 706BE40536; Wed, 2 Apr 2025 16:07:34 +0000 (UTC) From: Nikita Kalyazin To: , , CC: , , , , , , , , , , , , , , , , Subject: [PATCH v2 1/5] mm: userfaultfd: generic continue for non hugetlbfs Date: Wed, 2 Apr 2025 16:07:17 +0000 Message-ID: <20250402160721.97596-2-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250402160721.97596-1-kalyazin@amazon.com> References: <20250402160721.97596-1-kalyazin@amazon.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 738A54000E X-Rspamd-Server: rspam05 X-Rspam-User: X-Stat-Signature: nprs6dn4pruusiuwgmmf6qxighwhkpik X-HE-Tag: 1743610061-835026 X-HE-Meta: U2FsdGVkX1/DOgDgv2J4FZErEHQYFXdqqKA8N/VFmue3UKX7HL5mIOkWzOccyD7oFbLy/e3PFBnL9KW56curCZ/hlV5HecbhOMOkljmVlGIMeGfol7jTTJM4PuPybfZ/6UbfOUoapMu0MCIw/3ok648LZyyWcjQ42YRiLU5zpauWT/b1qz4tYM79jmG5UsWG0w6DYW5adMUXei5DtEKgXHm+/jwjQT9A5kTSTzhS0VPjWKFBXwpDvEf71TTCuoSCabna75bR9XhA8JepY4CMW9u8CGrgV6JJkRiW9Xgr41idW5qlCR1yOwOj+XcxlVI42rJx9qCFmrspAr/ltPSmubPywE+I7C4S8ZbXFKPf32heJOPfjp/VJeKEFT8UsqjhenTM1wdb83d9OBKm6HA3Hqkj11cBTM7nKW7gfJ6n0wD2woSPyEtvjemvDUFkNqOi1oMR+8/pYuyJddA1MA3bl5QVwtuci/ULJUblj5IpAKc4YEkckw5ApLZ0dpQ/XD/hcfNbSnkBIi7Vk785vaPb9WNfdUV+l1gjaejNLg/4SWMv+WoWCqajpYWmwR6VnTVz46O9rbi9G98G/NOpPcQEyBrdneO79GFt7e4LrTjdCCYPDSsnJmjpDyJAXhHhEp2hDEiM82XqPO8SF3grrcGxrCcLYFS/rc+E117H1fiDfpWGRpYgeIRLeG8Q71y3WCVuBLZlui9siB6yhdiGWngPEOXa79aZRht7O2JucBAo1cPrjC2xcGzgIzjRfTdqz/xTQTiwGVbiQXYkNZ6a6ZVx+jPO08IcYGui3YfJh9YQkC9uHHKNyLd7YS7bMvKUge5z0LBcObeY0kb5MLVc4UqzjxA/rRLU676GtKZ+r+ZpxAh5Y6d6/uuhM+zW5I8N30c8WhES/rtr4Lth90I3Pgr0tp/uF4XyUaWrwpSxkAIknb9masSSViYFoG/kRrfPes7fzh5g3vZI9tEA9F+/Rx3 AWukShuJ n6zTxudtFvfcU1VW0FzgxywnpWcKgOFhpk5fGeQ8AIiz5zjB3olhvUCA4rTT7jPBjcnH5gEPlyEGEDcQ33YlhiDToD/youByAL5EXP2i3e9ILGJDrOzDNP4NlNCwp0trdyScsK+k4DbRJV6qtwLxjS9uupshMKm92rFtxXUlA5K6EBA7Jxfizqn07cafYnPOIKowyei410ZIbHwHf6xXvfXONH+T3dd6ysCuvwLoM8Xtsi0Jt4ZaAUFaBRXGxlvx4x3YDPTdqTdvz/40upbIdcPEzOeclXfAB5zx0339DPMyjgdRyYw4mJxB4TdCqx2NnlH0uYv1hr6DdNRIZWsgBqw/DRywq77uVpTAFzni8WlnZwf9OwJdd0x1ue8LaYKQSwZj98PgkbSn6ZyGh6ryxKRE9Yb46lzQKLx6iahWZEzwqEETd2ja6tuq7GOE3n8DxxZ4cV8QW8PajIKZ2Jorjn2iDtIqE1tx3+sjgVe1hhA3FQlwov6AVBhbxA7tlSCwsDl4L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Remove shmem-specific code from UFFDIO_CONTINUE implementation for non-huge pages by calling vm_ops->fault(). A new VMF flag, FAULT_FLAG_NO_USERFAULT_MINOR, is introduced to avoid recursive call to handle_userfault(). Signed-off-by: Nikita Kalyazin --- include/linux/mm_types.h | 3 +++ mm/hugetlb.c | 2 +- mm/shmem.c | 3 ++- mm/userfaultfd.c | 25 ++++++++++++++++++------- 4 files changed, 24 insertions(+), 9 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 0234f14f2aa6..91a00f2cd565 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1429,6 +1429,8 @@ enum tlb_flush_reason { * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached. * We should only access orig_pte if this flag set. * @FAULT_FLAG_VMA_LOCK: The fault is handled under VMA lock. + * @FAULT_FLAG_NO_USERFAULT_MINOR: The fault handler must not call userfaultfd + * minor handler. * * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify * whether we would allow page faults to retry by specifying these two @@ -1467,6 +1469,7 @@ enum fault_flag { FAULT_FLAG_UNSHARE = 1 << 10, FAULT_FLAG_ORIG_PTE_VALID = 1 << 11, FAULT_FLAG_VMA_LOCK = 1 << 12, + FAULT_FLAG_NO_USERFAULT_MINOR = 1 << 13, }; typedef unsigned int __bitwise zap_flags_t; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 97930d44d460..ba90d48144fc 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6228,7 +6228,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, } /* Check for page in userfault range. */ - if (userfaultfd_minor(vma)) { + if (userfaultfd_minor(vma) && !(vmf->flags & FAULT_FLAG_NO_USERFAULT_MINOR)) { folio_unlock(folio); folio_put(folio); /* See comment in userfaultfd_missing() block above */ diff --git a/mm/shmem.c b/mm/shmem.c index 1ede0800e846..5e1911e39dec 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2467,7 +2467,8 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index, fault_mm = vma ? vma->vm_mm : NULL; folio = filemap_get_entry(inode->i_mapping, index); - if (folio && vma && userfaultfd_minor(vma)) { + if (folio && vma && userfaultfd_minor(vma) && + !(vmf->flags & FAULT_FLAG_NO_USERFAULT_MINOR)) { if (!xa_is_value(folio)) folio_put(folio); *fault_type = handle_userfault(vmf, VM_UFFD_MINOR); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index d06453fa8aba..68a995216789 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -386,24 +386,35 @@ static int mfill_atomic_pte_continue(pmd_t *dst_pmd, unsigned long dst_addr, uffd_flags_t flags) { - struct inode *inode = file_inode(dst_vma->vm_file); - pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); struct folio *folio; struct page *page; int ret; + struct vm_fault vmf = { + .vma = dst_vma, + .address = dst_addr, + .flags = FAULT_FLAG_WRITE | FAULT_FLAG_REMOTE | + FAULT_FLAG_NO_USERFAULT_MINOR, + .pte = NULL, + .page = NULL, + .pgoff = linear_page_index(dst_vma, dst_addr), + }; + + if (!dst_vma->vm_ops || !dst_vma->vm_ops->fault) + return -EINVAL; - ret = shmem_get_folio(inode, pgoff, 0, &folio, SGP_NOALLOC); - /* Our caller expects us to return -EFAULT if we failed to find folio */ - if (ret == -ENOENT) + ret = dst_vma->vm_ops->fault(&vmf); + if (ret & VM_FAULT_ERROR) { ret = -EFAULT; - if (ret) goto out; + } + + page = vmf.page; + folio = page_folio(page); if (!folio) { ret = -EFAULT; goto out; } - page = folio_file_page(folio, pgoff); if (PageHWPoison(page)) { ret = -EIO; goto out_release;