From patchwork Fri Apr 4 15:43:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14038628 Received: from smtp-fw-9105.amazon.com (smtp-fw-9105.amazon.com [207.171.188.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DE4FC1F4E57; Fri, 4 Apr 2025 15:44:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=207.171.188.204 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781461; cv=none; b=ox4yvW77XkaBLaQrvQYnIirkDJEYV12ZTv7pYmM/P9izIhgrpGU/M0YAWaHOaL4gZMMWhu8jBLfudWcN5ClGX8bN3uMI/bt5eVZOeF22EK6xluqcZtmwey0vNAqclUi+4LPOM2/509ZY+JrlSdYpYqLhqZcmV9yucPyee2MNDic= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781461; c=relaxed/simple; bh=LHnS35U8gBv0D4O5Jd88bf0LA/0TioSxp/njvc2FFzs=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=fui0juTRDw7+SYXTejNadNX0Dj0s5dKRDHPUjY/o+cJAJmFK1p0vFzC5Qj+8dW5nx+2NQ19/5LOQjyxU0WfkTxWXRjSx6in+jBt/KKTe8E8eQRHbH7ku2mKR3XiK8cjH/c81k/Ekap4KUKLCE2Jh1oSTSedqPkfEuu/G3KaybYE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=s4+Qr7zL; arc=none smtp.client-ip=207.171.188.204 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="s4+Qr7zL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743781461; x=1775317461; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4KrCtE/sXZ3BNNe21aeAaGtv43DEVIENoUwdd2sqOFQ=; b=s4+Qr7zLGa/XSlFK3IAonFYimnAAOcXbgClwmia2hA+t6WIKaabvRcWU SPDM2dYjZbBFR/RwGEX2ft0uJmlvg96sQNcnGAqLsVSwtTS5MzYs7+uLo 8V6ltZqO8CkwlcKSxHg8/Zq57yMN4Xl5XouRYo5GRyg36R3BZD5zgPnth Q=; X-IronPort-AV: E=Sophos;i="6.15,188,1739836800"; d="scan'208";a="7643503" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.210]) by smtp-border-fw-9105.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 15:44:14 +0000 Received: from EX19MTAUWA001.ant.amazon.com [10.0.38.20:31324] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.62.147:2525] with esmtp (Farcaster) id ea24aab9-5b16-4f64-955a-2db6935ecc80; Fri, 4 Apr 2025 15:44:12 +0000 (UTC) X-Farcaster-Flow-ID: ea24aab9-5b16-4f64-955a-2db6935ecc80 Received: from EX19D020UWA004.ant.amazon.com (10.13.138.231) by EX19MTAUWA001.ant.amazon.com (10.250.64.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:08 +0000 Received: from EX19MTAUEA001.ant.amazon.com (10.252.134.203) by EX19D020UWA004.ant.amazon.com (10.13.138.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:07 +0000 Received: from email-imr-corp-prod-iad-all-1b-85daddd1.us-east-1.amazon.com (10.43.8.2) by mail-relay.amazon.com (10.252.134.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Fri, 4 Apr 2025 15:44:07 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-iad-all-1b-85daddd1.us-east-1.amazon.com (Postfix) with ESMTPS id A7CCB41FBE; Fri, 4 Apr 2025 15:44:05 +0000 (UTC) From: Nikita Kalyazin To: , , , , , , CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 1/6] mm: userfaultfd: generic continue for non hugetlbfs Date: Fri, 4 Apr 2025 15:43:47 +0000 Message-ID: <20250404154352.23078-2-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250404154352.23078-1-kalyazin@amazon.com> References: <20250404154352.23078-1-kalyazin@amazon.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Remove shmem-specific code from UFFDIO_CONTINUE implementation for non-huge pages by calling vm_ops->fault(). A new VMF flag, FAULT_FLAG_USERFAULT_CONTINUE, is introduced to avoid recursive call to handle_userfault(). Suggested-by: James Houghton Signed-off-by: Nikita Kalyazin --- include/linux/mm_types.h | 4 ++++ mm/hugetlb.c | 2 +- mm/shmem.c | 9 ++++++--- mm/userfaultfd.c | 37 +++++++++++++++++++++++++++---------- 4 files changed, 38 insertions(+), 14 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 0234f14f2aa6..2f26ee9742bf 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -1429,6 +1429,9 @@ enum tlb_flush_reason { * @FAULT_FLAG_ORIG_PTE_VALID: whether the fault has vmf->orig_pte cached. * We should only access orig_pte if this flag set. * @FAULT_FLAG_VMA_LOCK: The fault is handled under VMA lock. + * @FAULT_FLAG_USERFAULT_CONTINUE: The fault handler must not call userfaultfd + * minor handler as it is being called by the + * userfaultfd code itself. * * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify * whether we would allow page faults to retry by specifying these two @@ -1467,6 +1470,7 @@ enum fault_flag { FAULT_FLAG_UNSHARE = 1 << 10, FAULT_FLAG_ORIG_PTE_VALID = 1 << 11, FAULT_FLAG_VMA_LOCK = 1 << 12, + FAULT_FLAG_USERFAULT_CONTINUE = 1 << 13, }; typedef unsigned int __bitwise zap_flags_t; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 97930d44d460..c004cfdcd4e2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6228,7 +6228,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, } /* Check for page in userfault range. */ - if (userfaultfd_minor(vma)) { + if (userfaultfd_minor(vma) && !(vmf->flags & FAULT_FLAG_USERFAULT_CONTINUE)) { folio_unlock(folio); folio_put(folio); /* See comment in userfaultfd_missing() block above */ diff --git a/mm/shmem.c b/mm/shmem.c index 1ede0800e846..b4159303fe59 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2467,7 +2467,8 @@ static int shmem_get_folio_gfp(struct inode *inode, pgoff_t index, fault_mm = vma ? vma->vm_mm : NULL; folio = filemap_get_entry(inode->i_mapping, index); - if (folio && vma && userfaultfd_minor(vma)) { + if (folio && vma && userfaultfd_minor(vma) && + !(vmf->flags & FAULT_FLAG_USERFAULT_CONTINUE)) { if (!xa_is_value(folio)) folio_put(folio); *fault_type = handle_userfault(vmf, VM_UFFD_MINOR); @@ -2727,6 +2728,8 @@ static vm_fault_t shmem_falloc_wait(struct vm_fault *vmf, struct inode *inode) static vm_fault_t shmem_fault(struct vm_fault *vmf) { struct inode *inode = file_inode(vmf->vma->vm_file); + enum sgp_type sgp = vmf->flags & FAULT_FLAG_USERFAULT_CONTINUE ? + SGP_NOALLOC : SGP_CACHE; gfp_t gfp = mapping_gfp_mask(inode->i_mapping); struct folio *folio = NULL; vm_fault_t ret = 0; @@ -2743,8 +2746,8 @@ static vm_fault_t shmem_fault(struct vm_fault *vmf) } WARN_ON_ONCE(vmf->page != NULL); - err = shmem_get_folio_gfp(inode, vmf->pgoff, 0, &folio, SGP_CACHE, - gfp, vmf, &ret); + err = shmem_get_folio_gfp(inode, vmf->pgoff, 0, &folio, sgp, gfp, vmf, + &ret); if (err) return vmf_error(err); if (folio) { diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index d06453fa8aba..4b3dbc7dac64 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -380,30 +380,47 @@ static int mfill_atomic_pte_zeropage(pmd_t *dst_pmd, return ret; } -/* Handles UFFDIO_CONTINUE for all shmem VMAs (shared or private). */ +/* Handles UFFDIO_CONTINUE for all VMAs */ static int mfill_atomic_pte_continue(pmd_t *dst_pmd, struct vm_area_struct *dst_vma, unsigned long dst_addr, uffd_flags_t flags) { - struct inode *inode = file_inode(dst_vma->vm_file); - pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); struct folio *folio; struct page *page; int ret; + struct vm_fault vmf = { + .vma = dst_vma, + .address = dst_addr, + .flags = FAULT_FLAG_WRITE | FAULT_FLAG_REMOTE | + FAULT_FLAG_USERFAULT_CONTINUE, + .pte = NULL, + .page = NULL, + .pgoff = linear_page_index(dst_vma, dst_addr), + }; + + if (!dst_vma->vm_ops || !dst_vma->vm_ops->fault) + return -EINVAL; - ret = shmem_get_folio(inode, pgoff, 0, &folio, SGP_NOALLOC); - /* Our caller expects us to return -EFAULT if we failed to find folio */ - if (ret == -ENOENT) +retry: + ret = dst_vma->vm_ops->fault(&vmf); + if (ret & VM_FAULT_ERROR) { ret = -EFAULT; - if (ret) goto out; - if (!folio) { - ret = -EFAULT; + } + + if (ret & VM_FAULT_NOPAGE) { + ret = -EAGAIN; goto out; } - page = folio_file_page(folio, pgoff); + if (ret & VM_FAULT_RETRY) + goto retry; + + page = vmf.page; + folio = page_folio(page); + BUG_ON(!folio); + if (PageHWPoison(page)) { ret = -EIO; goto out_release; From patchwork Fri Apr 4 15:43:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14038629 Received: from smtp-fw-52005.amazon.com (smtp-fw-52005.amazon.com [52.119.213.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A66311F4E57; Fri, 4 Apr 2025 15:44:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=52.119.213.156 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781469; cv=none; b=IcFMGsj8RPXwlHt4YNJdwRFpQPpxYE25QAQa+RxvVRKKMB48bulgI3Tmp3DJsMSguZ2/9vuEOcoZOYhMEtfYiqehwRfp+R3G490TRyGpaBVp57mkzdCNoTtilZldweXh5X+/BBvnGkiXeB5AcEKzz3AyEc9KXTyvpirM88Q8XiI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781469; c=relaxed/simple; bh=SInnc81YoyJKGYedRoWj/e7x6DFjSaGbOR4nt0IIepQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=XbUYJBUfrFq374GYhmwP4mOw2bz/vR0wVAqYrxPpJhu6F7Md2F1HinlrD3XOSEZ98QhbOkifC/5IUQKHIgqXnMctRcquF6wyt/lSlJyKzljVoxMABwe3eHNF3ttYk/zH/NryZLk+TcxXdhMfJ4hUIHA5uf7+Pwd7hi0Ah+xFMWk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=QVFG3Qax; arc=none smtp.client-ip=52.119.213.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="QVFG3Qax" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743781468; x=1775317468; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dh/0PI2ns4oZ37OX+8TMvz834BKQ/4DyQ5yTHrbPiLw=; b=QVFG3QaxmCrGVKvb0SR++lXOwXXPCHhFmq3GdDqLDT7kVWJ+TY4sLZiq qe8FQqxwhaC5SZw/a2Q3u9T9pMfyUBiiInAqmyMELzjiqyRF+FuqaLUw4 eOqe81x9vX3RA9cxHshwBq0sDsaF57pwIheXdCIYYtZvYICfJpVjt9C7w 4=; X-IronPort-AV: E=Sophos;i="6.15,188,1739836800"; d="scan'208";a="732940908" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-52005.iad7.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 15:44:23 +0000 Received: from EX19MTAUWC001.ant.amazon.com [10.0.21.151:37011] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.25.120:2525] with esmtp (Farcaster) id 08c94e81-d4d9-4237-8218-603f770ff779; Fri, 4 Apr 2025 15:44:21 +0000 (UTC) X-Farcaster-Flow-ID: 08c94e81-d4d9-4237-8218-603f770ff779 Received: from EX19D003UWB003.ant.amazon.com (10.13.138.116) by EX19MTAUWC001.ant.amazon.com (10.250.64.174) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:21 +0000 Received: from EX19MTAUWA001.ant.amazon.com (10.250.64.204) by EX19D003UWB003.ant.amazon.com (10.13.138.116) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:21 +0000 Received: from email-imr-corp-prod-iad-1box-1a-6851662a.us-east-1.amazon.com (10.25.36.214) by mail-relay.amazon.com (10.250.64.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Fri, 4 Apr 2025 15:44:21 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-iad-1box-1a-6851662a.us-east-1.amazon.com (Postfix) with ESMTPS id 709954032D; Fri, 4 Apr 2025 15:44:18 +0000 (UTC) From: Nikita Kalyazin To: , , , , , , CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 2/6] mm: provide can_userfault vma operation Date: Fri, 4 Apr 2025 15:43:48 +0000 Message-ID: <20250404154352.23078-3-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250404154352.23078-1-kalyazin@amazon.com> References: <20250404154352.23078-1-kalyazin@amazon.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The new operation allows to decouple the userfaulfd code from dependencies to VMA types, specifically, shmem and hugetlb. The vm_flags bitmap argument is processed with "any" logic, meaning if the VMA type supports any of the flags set, it returns true. This is to avoid multiple calls when checking for __VM_UFFD_FLAGS. Signed-off-by: Nikita Kalyazin --- include/linux/mm.h | 5 +++++ mm/hugetlb.c | 7 +++++++ mm/shmem.c | 8 ++++++++ 3 files changed, 20 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 8483e09aeb2c..488d721d8542 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -680,6 +680,11 @@ struct vm_operations_struct { */ struct page *(*find_special_page)(struct vm_area_struct *vma, unsigned long addr); + /* + * True if the VMA supports userfault at least for one of the vm_flags + */ + bool (*can_userfault)(struct vm_area_struct *vma, + unsigned long vm_flags); }; #ifdef CONFIG_NUMA_BALANCING diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c004cfdcd4e2..f3901c11e1fd 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5143,6 +5143,12 @@ static unsigned long hugetlb_vm_op_pagesize(struct vm_area_struct *vma) return huge_page_size(hstate_vma(vma)); } +static bool hugetlb_vm_op_can_userfault(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + return true; +} + /* * We cannot handle pagefaults against hugetlb pages at all. They cause * handle_mm_fault() to try to instantiate regular-sized pages in the @@ -5168,6 +5174,7 @@ const struct vm_operations_struct hugetlb_vm_ops = { .close = hugetlb_vm_op_close, .may_split = hugetlb_vm_op_split, .pagesize = hugetlb_vm_op_pagesize, + .can_userfault = hugetlb_vm_op_can_userfault, }; static pte_t make_huge_pte(struct vm_area_struct *vma, struct page *page, diff --git a/mm/shmem.c b/mm/shmem.c index b4159303fe59..0b9e19abd1e9 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2891,6 +2891,12 @@ static struct mempolicy *shmem_get_policy(struct vm_area_struct *vma, return mpol_shared_policy_lookup(&SHMEM_I(inode)->policy, index); } +static bool shmem_can_userfault(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + return true; +} + static struct mempolicy *shmem_get_pgoff_policy(struct shmem_inode_info *info, pgoff_t index, unsigned int order, pgoff_t *ilx) { @@ -5309,6 +5315,7 @@ static const struct vm_operations_struct shmem_vm_ops = { .set_policy = shmem_set_policy, .get_policy = shmem_get_policy, #endif + .can_userfault = shmem_can_userfault, }; static const struct vm_operations_struct shmem_anon_vm_ops = { @@ -5318,6 +5325,7 @@ static const struct vm_operations_struct shmem_anon_vm_ops = { .set_policy = shmem_set_policy, .get_policy = shmem_get_policy, #endif + .can_userfault = shmem_can_userfault, }; int shmem_init_fs_context(struct fs_context *fc) From patchwork Fri Apr 4 15:43:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14038630 Received: from smtp-fw-9102.amazon.com (smtp-fw-9102.amazon.com [207.171.184.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A0A41F5619; Fri, 4 Apr 2025 15:44:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=207.171.184.29 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781482; cv=none; b=Vycmi+nl8rn/xGhb86psanK6m5rzJB4AbQa5LyihBNcSipSrB5RWcTMWOPbHjRYjzmsw+Bj2Bv2poeNaAB7A3Fqj+lz96Bew+oyAL+5fCX5Wws6sBXsUMuUrZSf9eEcX+KKTaPjgIlj/D64e4HEQjH115HzchMuKHG9HpZscF+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781482; c=relaxed/simple; bh=Zqlld3krwRFainf0w6FOLkLziDdo9PB2J5/SAXPM7D4=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=g9CZxDKVmA+kHFdvUYHWUNf+k0DZLFy4fKUG4ecGzsOA1CPgfki0QH81qS4Vwysda8redOJnpYGB10GLse5TeBvgyE6Nfi5lfU3YDpdmzflFchYIjgBtdeWlcY09f4U4aqMAfDLU+qbZX2UcFp6y/Np6LBxaD7g5gqPVM5/gGYg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=koZrHj+Z; arc=none smtp.client-ip=207.171.184.29 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="koZrHj+Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743781481; x=1775317481; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=wZsvbmbu/0jS7xgl288uAUsGSomu15ez6E6onuR11lQ=; b=koZrHj+ZUX8Tznh94BPVVGjZd/4GUX94cStFTwOwqvCgje4VFH0Wm7Hl NNDqRQ5xYOxGqFA4u/iaVY/Y0HAzMhJ6B2ibwJSxYRxQDwfbFRYJ01PQm ah32QbEDCXxGMv4JzfPzufR3As00TYFVcQfTR8dYKEj/BjQ4Jibt7DPG2 Q=; X-IronPort-AV: E=Sophos;i="6.15,188,1739836800"; d="scan'208";a="508886507" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-9102.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 15:44:36 +0000 Received: from EX19MTAUWA001.ant.amazon.com [10.0.38.20:55309] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.51.140:2525] with esmtp (Farcaster) id 2332ab5f-8f03-4ed5-a407-5ac2d770b39b; Fri, 4 Apr 2025 15:44:35 +0000 (UTC) X-Farcaster-Flow-ID: 2332ab5f-8f03-4ed5-a407-5ac2d770b39b Received: from EX19D020UWC003.ant.amazon.com (10.13.138.187) by EX19MTAUWA001.ant.amazon.com (10.250.64.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:34 +0000 Received: from EX19MTAUWC002.ant.amazon.com (10.250.64.143) by EX19D020UWC003.ant.amazon.com (10.13.138.187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:33 +0000 Received: from email-imr-corp-prod-iad-all-1b-3ae3de11.us-east-1.amazon.com (10.25.36.210) by mail-relay.amazon.com (10.250.64.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Fri, 4 Apr 2025 15:44:33 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-iad-all-1b-3ae3de11.us-east-1.amazon.com (Postfix) with ESMTPS id 3EB8EA02DE; Fri, 4 Apr 2025 15:44:31 +0000 (UTC) From: Nikita Kalyazin To: , , , , , , CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 3/6] mm: userfaultfd: use can_userfault vma operation Date: Fri, 4 Apr 2025 15:43:49 +0000 Message-ID: <20250404154352.23078-4-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250404154352.23078-1-kalyazin@amazon.com> References: <20250404154352.23078-1-kalyazin@amazon.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Nikita Kalyazin --- include/linux/userfaultfd_k.h | 13 ++++++------- mm/userfaultfd.c | 10 +++++++--- 2 files changed, 13 insertions(+), 10 deletions(-) diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 75342022d144..64551e8a55fb 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -221,8 +221,8 @@ static inline bool vma_can_userfault(struct vm_area_struct *vma, if (vm_flags & VM_DROPPABLE) return false; - if ((vm_flags & VM_UFFD_MINOR) && - (!is_vm_hugetlb_page(vma) && !vma_is_shmem(vma))) + if (!vma->vm_ops->can_userfault || + !vma->vm_ops->can_userfault(vma, VM_UFFD_MINOR)) return false; /* @@ -235,16 +235,15 @@ static inline bool vma_can_userfault(struct vm_area_struct *vma, #ifndef CONFIG_PTE_MARKER_UFFD_WP /* * If user requested uffd-wp but not enabled pte markers for - * uffd-wp, then shmem & hugetlbfs are not supported but only - * anonymous. + * uffd-wp, then only anonymous is supported. */ if ((vm_flags & VM_UFFD_WP) && !vma_is_anonymous(vma)) return false; #endif - /* By default, allow any of anon|shmem|hugetlb */ - return vma_is_anonymous(vma) || is_vm_hugetlb_page(vma) || - vma_is_shmem(vma); + return vma_is_anonymous(vma) || + (vma->vm_ops->can_userfault && + vma->vm_ops->can_userfault(vma, vm_flags)); } static inline bool vma_has_uffd_without_event_remap(struct vm_area_struct *vma) diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 4b3dbc7dac64..0aa82c968e16 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -728,6 +728,7 @@ static __always_inline ssize_t mfill_atomic(struct userfaultfd_ctx *ctx, unsigned long src_addr, dst_addr; long copied; struct folio *folio; + bool can_userfault; /* * Sanitize the command parameters: @@ -787,10 +788,13 @@ static __always_inline ssize_t mfill_atomic(struct userfaultfd_ctx *ctx, return mfill_atomic_hugetlb(ctx, dst_vma, dst_start, src_start, len, flags); - if (!vma_is_anonymous(dst_vma) && !vma_is_shmem(dst_vma)) + can_userfault = dst_vma->vm_ops->can_userfault && + dst_vma->vm_ops->can_userfault(dst_vma, __VM_UFFD_FLAGS); + + if (!vma_is_anonymous(dst_vma) && !can_userfault) goto out_unlock; - if (!vma_is_shmem(dst_vma) && - uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) + + if (!can_userfault && uffd_flags_mode_is(flags, MFILL_ATOMIC_CONTINUE)) goto out_unlock; while (src_addr < src_start + len) { From patchwork Fri Apr 4 15:43:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14038631 Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EFA171F584D; Fri, 4 Apr 2025 15:44:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=72.21.196.25 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781495; cv=none; b=DguxVIkHJ5Pjp4+VH4/UzNq4+UKEB0t/MJJYTBSTrzz2hzZ6YGfieJVIKiAzVRRzDrKq85MZ5riyYm521tUS+tZgmYor5QmsPJ2tHy+7ixQgCLoNK90A7V6Wb1TETN6GCuF9scBxxd2S46sJIGwN3P8hUfD8iBSwIgpexC+2VO4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781495; c=relaxed/simple; bh=kFNfO4+4H/Do6SIV4WALi6N3A6Tqd4BX3kF8tn9/k+Q=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=by29im79uD+Kj0obsIDHmJOfBjcgyOR4Jvg7Ep4YuiykO44ABe77zuoBW3leHkjmF2O8U0GCUf4fVm7p/B1NPYFanwoz02KIXauGT2521M+JvAdgnti1Q3OREr6J6TQlHVcJmYl9OP2zu6W5jt7vxPV/IAh/QDxyAGqyZcQg67Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=ssFO4xjU; arc=none smtp.client-ip=72.21.196.25 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="ssFO4xjU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743781495; x=1775317495; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Hs0vi6KJkVKWhslhiRZ4duAi0DhiXeUL0EGHDvOz7fs=; b=ssFO4xjUC1/W6rce5ly4qYMhUTD4zl3BTPLZQOmhrD6TCykkeuC+uNfE vcIuBLUlVdi3zuN4Mlow4QGJdstgDFYRyTqQxL55faKWn9j05nrNUI9vB B4WkbU3bTHfD3nGBzKK96DyrMvue0Uwmr+hcDb8/y4LtV9zqAATbRz+fY k=; X-IronPort-AV: E=Sophos;i="6.15,188,1739836800"; d="scan'208";a="480424509" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.43.8.6]) by smtp-border-fw-2101.iad2.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 15:44:50 +0000 Received: from EX19MTAUWB002.ant.amazon.com [10.0.21.151:10114] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.51.140:2525] with esmtp (Farcaster) id 7694f073-a459-43df-b4a1-c487f31edac7; Fri, 4 Apr 2025 15:44:48 +0000 (UTC) X-Farcaster-Flow-ID: 7694f073-a459-43df-b4a1-c487f31edac7 Received: from EX19D020UWC003.ant.amazon.com (10.13.138.187) by EX19MTAUWB002.ant.amazon.com (10.250.64.231) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:48 +0000 Received: from EX19MTAUWA001.ant.amazon.com (10.250.64.204) by EX19D020UWC003.ant.amazon.com (10.13.138.187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:44:47 +0000 Received: from email-imr-corp-prod-pdx-all-2b-5ec155c2.us-west-2.amazon.com (10.25.36.214) by mail-relay.amazon.com (10.250.64.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Fri, 4 Apr 2025 15:44:47 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-pdx-all-2b-5ec155c2.us-west-2.amazon.com (Postfix) with ESMTPS id 68987402AD; Fri, 4 Apr 2025 15:44:44 +0000 (UTC) From: Nikita Kalyazin To: , , , , , , CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 4/6] KVM: guest_memfd: add support for userfaultfd minor Date: Fri, 4 Apr 2025 15:43:50 +0000 Message-ID: <20250404154352.23078-5-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250404154352.23078-1-kalyazin@amazon.com> References: <20250404154352.23078-1-kalyazin@amazon.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add support for sending a pagefault event if userfaultfd is registered. Only page minor event is currently supported. Signed-off-by: Nikita Kalyazin --- virt/kvm/guest_memfd.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index fbf89e643add..096d89e7282d 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -4,6 +4,9 @@ #include #include #include +#ifdef CONFIG_KVM_PRIVATE_MEM +#include +#endif /* CONFIG_KVM_PRIVATE_MEM */ #include "kvm_mm.h" @@ -380,6 +383,13 @@ static vm_fault_t kvm_gmem_fault(struct vm_fault *vmf) kvm_gmem_mark_prepared(folio); } + if (userfaultfd_minor(vmf->vma) && + !(vmf->flags & FAULT_FLAG_USERFAULT_CONTINUE)) { + folio_unlock(folio); + filemap_invalidate_unlock_shared(inode->i_mapping); + return handle_userfault(vmf, VM_UFFD_MINOR); + } + vmf->page = folio_file_page(folio, vmf->pgoff); out_folio: From patchwork Fri Apr 4 15:43:51 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14038632 Received: from smtp-fw-80007.amazon.com (smtp-fw-80007.amazon.com [99.78.197.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E59961F63EA; Fri, 4 Apr 2025 15:45:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=99.78.197.218 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781513; cv=none; b=E/LHCd3IoR6p9nMrVuDYZv8LyGYvoigsuEpIuCsfgBO938hU719TvEGQZdQpy/WOUVHrPIKaoJlM4MWYX7KbGSyQubWlyPXVHK5ILt2RVwNY2N2Fu6LXklbReQPno0KUY96HEiYp9jJ4/As4YJwKbMjHCyFmkkwWU2bRuwLCKPI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781513; c=relaxed/simple; bh=KFCrx1jPJ3whNiHmZiSGu/LtWqL28KB0MKrtuiAq/94=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WUfToVC2Cc42UtwEkJsnG3g70yodLznA1/JsCUxQSGIaIqD/MbS4ofcHg9Eal2L46CRP8Z345asvqjCALN0ul3FxwLWWVu1wMZybql4HlQrmZCF3brZkL0/JUjZUrxu1gNBaPDxKRvcVcYkzf3TmnAwxyREh7kYcCOvAAU50X3g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=ltAzl5/F; arc=none smtp.client-ip=99.78.197.218 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="ltAzl5/F" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743781512; x=1775317512; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GO+fOmyXj9EkRr/gRISxH91V61Pvwt+WgpsvJXpM3FY=; b=ltAzl5/FzdcR+5YRIBDrBkkwZcuXlRpC7t69j7SEhEzu/nBALuaz5It8 fPSwAgL/qlzWNphY7JqTodRWQ/bXmAZ2eiOaBQiukDEYrvyvrzj8s7wwM QbFpAxRlIQuEklHrS7Z7cOn8SBbF1hiaSVy/XXgyOVWoHYQFFNwpMNrnC s=; X-IronPort-AV: E=Sophos;i="6.15,188,1739836800"; d="scan'208";a="393099257" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.210]) by smtp-border-fw-80007.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 15:45:08 +0000 Received: from EX19MTAUWB001.ant.amazon.com [10.0.7.35:38032] by smtpin.naws.us-west-2.prod.farcaster.email.amazon.dev [10.0.51.140:2525] with esmtp (Farcaster) id fdc68b7e-a4fc-4834-afdf-ba6792da542c; Fri, 4 Apr 2025 15:45:07 +0000 (UTC) X-Farcaster-Flow-ID: fdc68b7e-a4fc-4834-afdf-ba6792da542c Received: from EX19D003UWB002.ant.amazon.com (10.13.138.11) by EX19MTAUWB001.ant.amazon.com (10.250.64.248) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:45:01 +0000 Received: from EX19MTAUEC001.ant.amazon.com (10.252.135.222) by EX19D003UWB002.ant.amazon.com (10.13.138.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:45:00 +0000 Received: from email-imr-corp-prod-iad-all-1b-3ae3de11.us-east-1.amazon.com (10.43.8.6) by mail-relay.amazon.com (10.252.135.200) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Fri, 4 Apr 2025 15:45:00 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-iad-all-1b-3ae3de11.us-east-1.amazon.com (Postfix) with ESMTPS id A0B33A0242; Fri, 4 Apr 2025 15:44:58 +0000 (UTC) From: Nikita Kalyazin To: , , , , , , CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 5/6] mm: userfaultfd: add UFFD_FEATURE_MINOR_GUEST_MEMFD Date: Fri, 4 Apr 2025 15:43:51 +0000 Message-ID: <20250404154352.23078-6-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250404154352.23078-1-kalyazin@amazon.com> References: <20250404154352.23078-1-kalyazin@amazon.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Signed-off-by: Nikita Kalyazin --- fs/userfaultfd.c | 3 ++- include/uapi/linux/userfaultfd.h | 8 +++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 97c4d71115d8..32152bfa462a 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1954,7 +1954,8 @@ static int userfaultfd_api(struct userfaultfd_ctx *ctx, uffdio_api.features = UFFD_API_FEATURES; #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR uffdio_api.features &= - ~(UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM); + ~(UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM | + UFFD_FEATURE_MINOR_GUEST_MEMFD); #endif #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP uffdio_api.features &= ~UFFD_FEATURE_PAGEFAULT_FLAG_WP; diff --git a/include/uapi/linux/userfaultfd.h b/include/uapi/linux/userfaultfd.h index 2841e4ea8f2c..ed688797eba7 100644 --- a/include/uapi/linux/userfaultfd.h +++ b/include/uapi/linux/userfaultfd.h @@ -42,7 +42,8 @@ UFFD_FEATURE_WP_UNPOPULATED | \ UFFD_FEATURE_POISON | \ UFFD_FEATURE_WP_ASYNC | \ - UFFD_FEATURE_MOVE) + UFFD_FEATURE_MOVE | \ + UFFD_FEATURE_MINOR_GUEST_MEMFD) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -230,6 +231,10 @@ struct uffdio_api { * * UFFD_FEATURE_MOVE indicates that the kernel supports moving an * existing page contents from userspace. + * + * UFFD_FEATURE_MINOR_GUEST_MEMFD indicates the same support as + * UFFD_FEATURE_MINOR_HUGETLBFS, but for guest_memfd-backed pages + * instead. */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0) #define UFFD_FEATURE_EVENT_FORK (1<<1) @@ -248,6 +253,7 @@ struct uffdio_api { #define UFFD_FEATURE_POISON (1<<14) #define UFFD_FEATURE_WP_ASYNC (1<<15) #define UFFD_FEATURE_MOVE (1<<16) +#define UFFD_FEATURE_MINOR_GUEST_MEMFD (1<<17) __u64 features; __u64 ioctls; From patchwork Fri Apr 4 15:43:52 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nikita Kalyazin X-Patchwork-Id: 14038633 Received: from smtp-fw-80006.amazon.com (smtp-fw-80006.amazon.com [99.78.197.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1BCB7E110; Fri, 4 Apr 2025 15:45:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=99.78.197.217 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781527; cv=none; b=mQGWgxWnJm+rNvgPV82L0c8WwBz7/JYeH8Q1duXaKgGpX0vA3b2unRCvnysGs/YuAwnCvqzIPKoHkUuNxp5bJDavEuQJSC7+wAap8UabNxHULXVlWeWGddxfuwUmjj/nAItzbqVczGAQOD2tbiu7nfdIjlyQsrCFB+JTUfFl4Cc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743781527; c=relaxed/simple; bh=ihDN7/DxACvpk0yyZBzdmxHvkWRzBDnaG9K8bSUk2mQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rgVFBEpwVJee2DQQMIG1iWI1XvuTReHsRKAKIRODSFMVj5GjGw8SWnehkuDcAWv5Wh06uj31z3/xdpFoy7F+aHnSEWJ+NyIg/gb/f9v122c1B7q/nZonfR+ubKw6yJoznOBoNUU/Y3y5HbBJXSLLWdI7D0Y2hpMCgvPUgXCNqio= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com; spf=pass smtp.mailfrom=amazon.co.uk; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b=cn5LQ44B; arc=none smtp.client-ip=99.78.197.217 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amazon.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=amazon.co.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="cn5LQ44B" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1743781526; x=1775317526; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=yc6GH0/dKmQ7WSv9wb3vUThoZ6MNVvGsszSJT/iNJmY=; b=cn5LQ44BUJS+sYWWoYKtainZBWpBWaMBtYp4cSPIDLC6Hh6MZWzdLma7 0fQ6nn9WdGSwRtarmJHWCBQY1OND6Z6xnl77ulnn1NCxPe+FjmLcvO7qP R9m6nIIglBeVum0m8MrHot7WJRWQs9nydDtjETLeH/fSP+JV13dZ7PKSD Y=; X-IronPort-AV: E=Sophos;i="6.15,188,1739836800"; d="scan'208";a="37863645" Received: from pdx4-co-svc-p1-lb2-vlan3.amazon.com (HELO smtpout.prod.us-east-1.prod.farcaster.email.amazon.dev) ([10.25.36.214]) by smtp-border-fw-80006.pdx80.corp.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Apr 2025 15:45:22 +0000 Received: from EX19MTAEUB002.ant.amazon.com [10.0.43.254:25779] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.18.64:2525] with esmtp (Farcaster) id bc1466b6-dabf-4c34-ba00-853571b00927; Fri, 4 Apr 2025 15:45:20 +0000 (UTC) X-Farcaster-Flow-ID: bc1466b6-dabf-4c34-ba00-853571b00927 Received: from EX19D030EUB002.ant.amazon.com (10.252.61.16) by EX19MTAEUB002.ant.amazon.com (10.252.51.79) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:45:16 +0000 Received: from EX19MTAUEA002.ant.amazon.com (10.252.134.9) by EX19D030EUB002.ant.amazon.com (10.252.61.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14; Fri, 4 Apr 2025 15:45:16 +0000 Received: from email-imr-corp-prod-pdx-all-2b-dbd438cc.us-west-2.amazon.com (10.43.8.2) by mail-relay.amazon.com (10.252.134.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1544.14 via Frontend Transport; Fri, 4 Apr 2025 15:45:15 +0000 Received: from dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com (dev-dsk-kalyazin-1a-a12e27e2.eu-west-1.amazon.com [172.19.103.116]) by email-imr-corp-prod-pdx-all-2b-dbd438cc.us-west-2.amazon.com (Postfix) with ESMTPS id CCF7BA047C; Fri, 4 Apr 2025 15:45:11 +0000 (UTC) From: Nikita Kalyazin To: , , , , , , CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 6/6] KVM: selftests: test userfaultfd minor for guest_memfd Date: Fri, 4 Apr 2025 15:43:52 +0000 Message-ID: <20250404154352.23078-7-kalyazin@amazon.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250404154352.23078-1-kalyazin@amazon.com> References: <20250404154352.23078-1-kalyazin@amazon.com> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The test demonstrates that a minor userfaultfd event in guest_memfd can be resolved via a memcpy followed by a UFFDIO_CONTINUE ioctl. Signed-off-by: Nikita Kalyazin --- .../testing/selftests/kvm/guest_memfd_test.c | 99 +++++++++++++++++++ 1 file changed, 99 insertions(+) diff --git a/tools/testing/selftests/kvm/guest_memfd_test.c b/tools/testing/selftests/kvm/guest_memfd_test.c index 38c501e49e0e..9a06c2486218 100644 --- a/tools/testing/selftests/kvm/guest_memfd_test.c +++ b/tools/testing/selftests/kvm/guest_memfd_test.c @@ -10,12 +10,16 @@ #include #include #include +#include #include #include +#include #include #include #include +#include +#include #include "kvm_util.h" #include "test_util.h" @@ -206,6 +210,98 @@ static void test_create_guest_memfd_multiple(struct kvm_vm *vm) close(fd1); } +struct fault_args { + char *addr; + volatile char value; +}; + +static void *fault_thread_fn(void *arg) +{ + struct fault_args *args = arg; + + /* Trigger page fault */ + args->value = *args->addr; + return NULL; +} + +static void test_uffd_minor(int fd, size_t page_size, size_t total_size) +{ + struct uffdio_register uffd_reg; + struct uffdio_continue uffd_cont; + struct uffd_msg msg; + struct fault_args args; + pthread_t fault_thread; + void *mem, *mem_nofault, *buf = NULL; + int uffd, ret; + off_t offset = page_size; + void *fault_addr; + + ret = posix_memalign(&buf, page_size, total_size); + TEST_ASSERT_EQ(ret, 0); + + uffd = syscall(__NR_userfaultfd, O_CLOEXEC); + TEST_ASSERT(uffd != -1, "userfaultfd creation should succeed"); + + struct uffdio_api uffdio_api = { + .api = UFFD_API, + .features = UFFD_FEATURE_MINOR_GUEST_MEMFD, + }; + ret = ioctl(uffd, UFFDIO_API, &uffdio_api); + TEST_ASSERT(ret != -1, "ioctl(UFFDIO_API) should succeed"); + + mem = mmap(NULL, total_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + TEST_ASSERT(mem != MAP_FAILED, "mmap should succeed"); + + mem_nofault = mmap(NULL, total_size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); + TEST_ASSERT(mem_nofault != MAP_FAILED, "mmap should succeed"); + + uffd_reg.range.start = (unsigned long)mem; + uffd_reg.range.len = total_size; + uffd_reg.mode = UFFDIO_REGISTER_MODE_MINOR; + ret = ioctl(uffd, UFFDIO_REGISTER, &uffd_reg); + TEST_ASSERT(ret != -1, "ioctl(UFFDIO_REGISTER) should succeed"); + + ret = fallocate(fd, FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE, + offset, page_size); + TEST_ASSERT(!ret, "fallocate(PUNCH_HOLE) should succeed"); + + fault_addr = mem + offset; + args.addr = fault_addr; + + ret = pthread_create(&fault_thread, NULL, fault_thread_fn, &args); + TEST_ASSERT(ret == 0, "pthread_create should succeed"); + + ret = read(uffd, &msg, sizeof(msg)); + TEST_ASSERT(ret != -1, "read from userfaultfd should succeed"); + TEST_ASSERT(msg.event == UFFD_EVENT_PAGEFAULT, "event type should be pagefault"); + TEST_ASSERT((void *)(msg.arg.pagefault.address & ~(page_size - 1)) == fault_addr, + "pagefault should occur at expected address"); + + memcpy(mem_nofault + offset, buf + offset, page_size); + + uffd_cont.range.start = (unsigned long)fault_addr; + uffd_cont.range.len = page_size; + uffd_cont.mode = 0; + ret = ioctl(uffd, UFFDIO_CONTINUE, &uffd_cont); + TEST_ASSERT(ret != -1, "ioctl(UFFDIO_CONTINUE) should succeed"); + + TEST_ASSERT(args.value == *(char *)(mem_nofault + offset), + "memory should contain the value that was copied"); + TEST_ASSERT(args.value == *(char *)(mem + offset), + "no further fault is expected"); + + ret = pthread_join(fault_thread, NULL); + TEST_ASSERT(ret == 0, "pthread_join should succeed"); + + ret = munmap(mem_nofault, total_size); + TEST_ASSERT(!ret, "munmap should succeed"); + + ret = munmap(mem, total_size); + TEST_ASSERT(!ret, "munmap should succeed"); + free(buf); + close(uffd); +} + unsigned long get_shared_type(void) { #ifdef __x86_64__ @@ -244,6 +340,9 @@ void test_vm_type(unsigned long type, bool is_shared) test_fallocate(fd, page_size, total_size); test_invalid_punch_hole(fd, page_size, total_size); + if (is_shared) + test_uffd_minor(fd, page_size, total_size); + close(fd); kvm_vm_release(vm); }