From patchwork Fri Aug 16 11:13:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A59E5C531DC for ; Fri, 16 Aug 2024 11:14:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34D386B0150; Fri, 16 Aug 2024 07:14:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C8E36B0151; Fri, 16 Aug 2024 07:14:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11C208D0070; Fri, 16 Aug 2024 07:14:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D5E5D6B014D for ; Fri, 16 Aug 2024 07:14:32 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 88E221C5B76 for ; Fri, 16 Aug 2024 11:14:32 +0000 (UTC) X-FDA: 82457850384.27.1F416C4 Received: from mout.web.de (mout.web.de [212.227.17.11]) by imf01.hostedemail.com (Postfix) with ESMTP id 8F94640014 for ; Fri, 16 Aug 2024 11:14:30 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=j0eUVv6E; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf01.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806816; a=rsa-sha256; cv=none; b=Awa2QQV0bH4MVW0/6eS49ZO9RNGqRk2lEOX6LjAaPyC/SAwaqS8QXfeHzNPAk7obkEemaq nszvW8E1uLsEsqnM7V6ukjIjLGQ/61+SKDHcu96b1w7K90Aa39VeS9toAVTk7KmfaK18rH 6QC/QXu8y+cy4rwcEUOt5UzuR60uQu8= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=j0eUVv6E; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf01.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806816; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l3TXWV1Tp/aKQfD2gCIur/5ETslPLQoSLhNXx7Vp5IM=; b=gO7zPukZw0EIN3BdLjWZjpt002UXvVhyslpzkm4i39yKttqMxE0Rer3zWbBtISr6mp3ZyB qWON95BEJMWQsFEgvBqmvhmo7p2qlwicV505q1bjmqac0U54kX54fx0gmoOvxEGq+BF1jx 1UX+DzgYNU0B1MAfww54gw23tx4+IQ8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806863; x=1724411663; i=spasswolf@web.de; bh=l3TXWV1Tp/aKQfD2gCIur/5ETslPLQoSLhNXx7Vp5IM=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=j0eUVv6E/rG8Cnhu49wfDc3ZjB308tJTF6zB2+RJHsRCdCi+Zt4nyYI/ByvcxKzC nVunB31UCoLPS33iT3ORKWHQQyKlzZ/JUPT+PsSdWbVi0fWF2M/U1wFDR/Lg2bjCF Q0npjb/EFieS/J9QGRtwhXo79AwYsiCtYyIFc6uBoEiCEKaoSJSKvqHGBr9cRQMkL 2beEdqQLnqppjuv2iCHHC0+kbttT04abwSRndJRr9/uFNu+0fmqum8Y/8bauO9niL yGHoJZ0JO1zYe0Gy3e9C+zsBHQia0yMzeLrLAtdpJ67ueqEjSGlzbxG/HI+H95Tjd Molvf+0gFkyqCuX+OQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MnG2Q-1rvuzT0Z0O-00gsfG; Fri, 16 Aug 2024 13:14:23 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 04/19] mm/mmap: Extract the gathering of vmas from do_vmi_align_munmap() Date: Fri, 16 Aug 2024 13:13:45 +0200 Message-ID: <20240816111405.11793-5-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:wTQCai+xUeqxjSfq9zUCYkHRj+YFQBFLdsczal4zW75GOviwbTE x5RrT/HHxU8w23/NaMxYwThD5F3v/k6iYpTCuPKN4+IZLXDz77gixcKc5xsxLMJGJpPw33+ NxS7VuPndRr7JacDCDCtI4DAOrxeM0LcCIN+QdLH94axfZqSq1ebKnJzFtIXaDiLwYYVOWO 0QvJxWvDPIwKjLFdsaA5Q== UI-OutboundReport: notjunk:1;M01:P0:yr2ZPw4vKiI=;FDy6r4s5ooBsVf9RgTeJNvQhI8e ++0xFb6KVDhYGn7fbXxa0mFvvKvxPwVHl6FgqboR1sowyn0v9bqG/2SifVm/mLHHJ9JcWNZ7o XAt1Mz5L/MpDEOq9L6R9jMhjy+v7cwoCgYdI1gDEUfuWMPvyXik4dcyDcbKxdVLEcU4KzXAgJ SpU8X6tuTKQFY9JTKhMJ91fTXNyRE4frpvxnhIF3mrqVC221p0zePFEyLGZMksN3gAI+uHoox 844WAkat4ZyrPqFMJsYh+y+Kid3V7Ot3Yz6aeQ8J+qVZpeg771D33TDB/IJthuXI7FjwZBP0Z WlKXgTmqhjdb+to3KFRPBEzp5/IpH7SZ7JywG00FFkoov9U5h4f9WRQHgPgrZS0kPT7MOd0bH YgrvIwW4DXvvgXM72GS7se+JxLMz6IwIA/hWuf/uWGZztLfA7t+lNltlLR84XMoHkPewjbnEh j8tYgfZbANYsqMzctXp557BMtciWVS2R24ebHDno/VqDm5LgeUr80MdV8dlFr85LsB0nNa6jx tgg0NZTZqrqUeeqxdn+430+tg+3DiVzcs9B9sXW/RplcTqVE8Sc4qogZAEuGM50USDJE54ygB 6LsfRUw/ppPsf3Maa2ktIVxeF51kS0wNk+zcL/38/zlfihnMz7YAzOrTTaCP9a0O/BEIRd5ds Dx2p8YgxIXqu+ay+PSWP6M/xbCIKQr0oIdinqGnrqiTlO7tsu0fWqACKPF2RiSkzLAJw0mzq+ sSua4B97tMDhCm594KVCKAsVMfFW0a6Bo/9sYkmCakdr0ZrzW2uVt86GhDFEWdyip1eSre8FM dCyqm6BMTC6wa0h1SHKSGo9A== X-Rspamd-Queue-Id: 8F94640014 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 8zx9zcwypzje8n99w1ae1itfqqnbij1h X-HE-Tag: 1723806870-607360 X-HE-Meta: U2FsdGVkX18YUpgYjoclJl81irJz4nJ76PQ5GRGaI9DGxwklHIapCM0yAl1CNQ8Fg9LhS63sWPy9B0aiej5IbpZpkXEtOwSAzB2uwzjw1nb6xXVlibE6Pp4DlaogYd3h878BmFOVVtsZk+i0KcorskGfjUktyHyR0qf/UTSSxdHur/ABrHOJXXIH3fLcE1scb/iIOaqDEffvmJ/jW3BssLEBl9v6s5xGY8cmLui+w/3udkrII+iCD59Up0RiYhxTgZ/Fch7OyeAN930+uN2IXPWwRn0Wt8LhFJVW4yASRZglfgLThbItAFBoPcbLeTlZ3ta+ntW/QIZaTBpljzGY+CE3SMX2ysWxKJMgdgNUq5UJbkAKPesYIgrH66V3QyRcc436k1RlkqTSLrDvaMAVq3uSbonmGs4JbmOviI/lOcTB/+cTBa59HQp42/SrmsDm/RsqZH30A7RpADzdZwTfBECnKnvB7j25pe3BZBofeZtWWrMzaeoWY5jtFPNzLw67Kmu3aOmxXPuDEKu7+H1wI3kWXUtf9Vy9jqzJ1iYvUIy0+I7dRIm8gLUszUISpT7X8T3MjH3j01fvP3R8c77N7zDRMxt5/cGzmSB/fS8+RnkvmhdpP3JI7sSlkbhyE2lu5Ke0ou1VT3oDPK6WjWBNMgteIlMPhJzY/QSKFClJur2XRXgBaXlQJ2rJKGm9mLMIrAtjbxAKkCr1Dg6sMAcXs/LMkM9GbVKEB22ZFu6iKN8AZCppJcA0QzH7SvO+3UNumR1rxzhwX3FPoD2y5TQbqnmHFGTNQo0Q3o20TQv8AV+k4c4Xr9w+JUctPVQOYPwbdbPd/cKXrQxhB+ZJByjL+6pIKlJDghqKwBQkm9xhviIbwDzAdrKCp6JYdlYyUQBkQL0ByQTnSbnyMLN7XDL176mKfkbSo+eoGjBC/6uZHA72gRZKhe5NMI9IiK6ZxMveD22ZLrk6eIKvX/ViXe0 uPBOahSd +tKPMVKjO3nwQCp/QznJhGw7XrjLswXYFHMnBH557mejY1AG2Ef4K/79ZUV8v1my05aHwjDo8Zz6i+zm/YRxsFWHzHaQKvAp0GqlwqxQTstWI0HqPf1rDxlCtdO2wZoqcLfVTJ1DNE7eqldRe7IndulCsB3HAhm0fMRC5f4iM9fvYH/H1CBxFXfRpwRYysMSEClOha/TZAU7V7bOsc9HTgnH9rlBJhdLUUT1CVZI3MC8XsthDv0CkfjrVO31bhfF+B7af1+rv87BAASpfb5p52sN8hxxRHoaFfdb5pmrYeePg34LWaV4OGjDmHVX7kKu8tPD3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Create vmi_gather_munmap_vmas() to handle the gathering of vmas into a detached maple tree for removal later. Part of the gathering is the splitting of vmas that span the boundary. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/vma.c | 80 ++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 58 insertions(+), 22 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index d331ba575019..a980837eefd7 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -737,32 +737,30 @@ vmi_complete_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, } /* - * do_vmi_align_munmap() - munmap the aligned region from @start to @end. + * vmi_gather_munmap_vmas() - Put all VMAs within a range into a maple tree + * for removal at a later date. Handles splitting first and last if necessary + * and marking the vmas as isolated. + * * @vmi: The vma iterator * @vma: The starting vm_area_struct * @mm: The mm_struct * @start: The aligned start address to munmap. * @end: The aligned end address to munmap. * @uf: The userfaultfd list_head - * @unlock: Set to true to drop the mmap_lock. unlocking only happens on - * success. + * @mas_detach: The maple state tracking the detached tree + * @locked_vm: a pointer to store the VM_LOCKED pages count. * - * Return: 0 on success and drops the lock if so directed, error and leaves the - * lock held otherwise. + * Return: 0 on success */ -int -do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, +static int +vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, - unsigned long end, struct list_head *uf, bool unlock) + unsigned long end, struct list_head *uf, + struct ma_state *mas_detach, unsigned long *locked_vm) { struct vm_area_struct *next = NULL; - struct maple_tree mt_detach; int count = 0; int error = -ENOMEM; - unsigned long locked_vm = 0; - MA_STATE(mas_detach, &mt_detach, 0, 0); - mt_init_flags(&mt_detach, vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); - mt_on_stack(mt_detach); /* * If we need to split any vma, do it now to save pain later. @@ -801,13 +799,13 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, goto end_split_failed; } vma_start_write(next); - mas_set(&mas_detach, count); - error = mas_store_gfp(&mas_detach, next, GFP_KERNEL); + mas_set(mas_detach, count++); + error = mas_store_gfp(mas_detach, next, GFP_KERNEL); if (error) goto munmap_gather_failed; vma_mark_detached(next, true); if (next->vm_flags & VM_LOCKED) - locked_vm += vma_pages(next); + *locked_vm += vma_pages(next); count++; if (unlikely(uf)) { @@ -834,7 +832,7 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) /* Make sure no VMAs are about to be lost. */ { - MA_STATE(test, &mt_detach, 0, 0); + MA_STATE(test, mas_detach->tree, 0, 0); struct vm_area_struct *vma_mas, *vma_test; int test_count = 0; @@ -854,6 +852,48 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, while (vma_iter_addr(vmi) > start) vma_iter_prev_range(vmi); + return 0; + +userfaultfd_error: +munmap_gather_failed: +end_split_failed: + abort_munmap_vmas(mas_detach); +start_split_failed: +map_count_exceeded: + return error; +} + +/* + * do_vmi_align_munmap() - munmap the aligned region from @start to @end. + * @vmi: The vma iterator + * @vma: The starting vm_area_struct + * @mm: The mm_struct + * @start: The aligned start address to munmap. + * @end: The aligned end address to munmap. + * @uf: The userfaultfd list_head + * @unlock: Set to true to drop the mmap_lock. unlocking only happens on + * success. + * + * Return: 0 on success and drops the lock if so directed, error and leaves the + * lock held otherwise. + */ +int +do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, + struct mm_struct *mm, unsigned long start, + unsigned long end, struct list_head *uf, bool unlock) +{ + struct maple_tree mt_detach; + MA_STATE(mas_detach, &mt_detach, 0, 0); + mt_init_flags(&mt_detach, vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(mt_detach); + int error; + unsigned long locked_vm = 0; + + error = vmi_gather_munmap_vmas(vmi, vma, mm, start, end, uf, + &mas_detach, &locked_vm); + if (error) + goto gather_failed; + error = vma_iter_clear_gfp(vmi, start, end, GFP_KERNEL); if (error) goto clear_tree_failed; @@ -864,12 +904,8 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, return 0; clear_tree_failed: -userfaultfd_error: -munmap_gather_failed: -end_split_failed: abort_munmap_vmas(&mas_detach); -start_split_failed: -map_count_exceeded: +gather_failed: validate_mm(mm); return error; }