From patchwork Fri Aug 16 11:13:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13765997 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40B8EC3DA4A for ; Fri, 16 Aug 2024 11:14:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 38BF98D0062; Fri, 16 Aug 2024 07:14:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 361E78D0070; Fri, 16 Aug 2024 07:14:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 22B4C8D0062; Fri, 16 Aug 2024 07:14:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E0E418D0070 for ; Fri, 16 Aug 2024 07:14:27 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 89C1BA93C3 for ; Fri, 16 Aug 2024 11:14:27 +0000 (UTC) X-FDA: 82457850174.23.4BE2B8C Received: from mout.web.de (mout.web.de [217.72.192.78]) by imf23.hostedemail.com (Postfix) with ESMTP id 899F0140010 for ; Fri, 16 Aug 2024 11:14:25 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=isRfwnJm; spf=pass (imf23.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806807; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hyLy5zcuz4oq+TGeFENweUivt0Um5vv9+zRVy5K7Dk4=; b=yj6oMomoZgacpylZJYn2ti80oPpv+DnNjNmHqx/AyjNC8/lI2LGAQOt926pnKX9rZSArhq u3sjQWPI9qzDViB07X2gJckN93uFfI9nh4zurRFzrLoCZBxJbg6AA3Y7mvFGHAMO9a/NIH bOIhpYJ7CJolHkRu1FNsZswm/tWINWU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=isRfwnJm; spf=pass (imf23.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806807; a=rsa-sha256; cv=none; b=Xl1QrmdICamDPf+bBYlJ2kPR/ErdDtaH1dielvUUVYWIn8bt4kduoFDgxokvoiSNNQVDBy n81vpkEr5F6XEwWZffO2FYmSLBpVyyZNs0knej5LMfO+AbCBP3Mb0XpXd8DRpv0xobVqCZ SlS2kNZnSOUj8d+SmiPTMOPr2vrsx2I= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806858; x=1724411658; i=spasswolf@web.de; bh=hyLy5zcuz4oq+TGeFENweUivt0Um5vv9+zRVy5K7Dk4=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=isRfwnJmdoIjdoxSXjRx1ZoOuxuydYKOM53dmNbr/J8Hdsa9t64eLYIv4NGyNM5Q 2EpEg/PT1iMa85RnmPd5GWHFbYHslvIhUs9kOsxI3VWROgifb0YX+8ToDLs8t+GAP idIz8diiz6X++gTr6PYCtOoI+b7D6NJy4bs2arEh4U1PTvxyDl962zvBEqBLFJ9sz zrv6AvuF83dor92u7kJJiJZRABYiBmVvYtmQh+B7WEskiYFgyKIg1kjorIPojmgw4 gPYjjsWTdyB6mlC7000R9/24PjVIwCq/U6acU0XT41BII63oG6+DbJmSTbH5YAV20 XAMLRX6ldMimz74o/Q== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MvKXb-1sNFZA3G6b-0180Kx; Fri, 16 Aug 2024 13:14:17 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu Subject: [PATCH v5.1 01/19] mm/mmap: Correctly position vma_iterator in __split_vma() Date: Fri, 16 Aug 2024 13:13:42 +0200 Message-ID: <20240816111405.11793-2-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:+jC36m1sRkvD5Vpf0wAeR0fwYwAGuLGXzoRuZyjrkTvLj8USB2Q nwExSus+XkiTkJ34jIqqtkqkt3AO+PPmAXiqZW3RgRrAhEoDWGFU5Pu2G7RFqxKwzSAmdNJ BsjgHW2psegBvLOjhAiyuX3tDxwTnsAU3qt+LoA681DQK+0eHEkUA58ufkSG58CFxlRpR8Y f2U9vLsm0g0bfzfuBOn+w== UI-OutboundReport: notjunk:1;M01:P0:xlXs275Tsxw=;5tLYSClCHU2trUPg31ml7gJdKMK DPbAa4ogoE6zXVqtGcUjy8gOfcVfhYGhvYoxL8qXQWVxgS3gs/Uy1FsR7YuL2dLmVGoQuy9CX Q8ODAMPw0cbFMha20Xuavu8ECYNAyDIvJLXjRtWw770hccX9QnSuEJwV6aJBVyS6TCxT3lg12 dVr7lgTjsRg5i0neNZJE/unSU/4Qu5uWKPvl7xWnA3ogZWqPdWOf6lBo6+zGy0SSQ+Cch7qJq OFl8LYEDqdh/XqaRcAU0Z0rVVXqzD2A1WQK+6dpIxMhmxbrY7ZS16Vz8T/OIjvEm5X55ycjCP moNPL5LzbA0qbR6O84EglsY8ALLzkwWh2LqBNdmidfeUj7eM5FJvNCv2M6xwv9q8a+jeNc1AZ N4tCUkiPPRZWDxEAn2bZD04B9ifnzYWqB3YKPC7TkpMUm8JfPoQvNrgvfDwYb5GGzUw+roTml HBr//2ROSZAH+PaGCOgr1AJuzPNQBUbFwpGeMWOI3bMbkq+wb8hM+p3x+CXX01ezqgHg8iZBh J6V0cTIlERHDGc5WYgrC4yIbnQ0PuKGCWdAlgjmB08c8kR4DyotDWveOOyxjGayx4Kzc4Wv2z 9HQNs+z3sLhWjytwiHNt7FKrM2sUauymNz6c2n+A3ZZFuaqRYGU1txvW/nqNGFPJRNPWtnkKK GiwNeV69kIrRUsxLZztFGeRBf0Rz2Z7rBMCVGzMFdQjfLdLfbvF1lj7OhqlTt4Cz+06+rRhfE FFL+PhHlFSQRFXg35gSzL+SbbUP1wKy6RI9J7uYsBltAejcJYwIssmnWPY0eD02GRSXfUzTsu jwTuFyjVbr0i9dzTffCmFkmg== X-Stat-Signature: aof3a817j97ab5ka68so9bregirgg436 X-Rspam-User: X-Rspamd-Queue-Id: 899F0140010 X-Rspamd-Server: rspam02 X-HE-Tag: 1723806865-31468 X-HE-Meta: U2FsdGVkX19fnPV/v12hyOjl4n54nEmkD6rgEwP2bngcxgGIvvoYSsYHGb5GLn3W4CtJOjSSL2n2LMGaUV6gUX/+OWMpb60BvtRYCnL7pYYcJoMkoVr7nKODoXkSKmyYjO2A4APma8YSz9lypkyGT7oNnyWJYFrX2InFMd3wqoFtmlP9CT0Tt1qJfndDpJUQDNt7P+QGdKI93cuu1UfBHdHW/BZDfkcIyPPcJyQBOJAjNS7d+4N7oZtl1SbLsDMsqZH63YY22ufOzM6HwutXRnnIwqFELJOTuqOfKvWiYdvtDuGPppFE7u+18DuQrQKD7Gnk23tawpoQJJcd8qAs+PSFUUUuQvwNb8EA57+uO0ZSbbUEFqvkQ3XgzWpgvncmXtw/4XE53xM1r4Wbo3Yb9dOIXbLGRmjRRzC9FRk3PARSWPSUAiikebWNallpRjGNubc/wS1fRClMfLtVpgoWDuTBvTI7nRt+V1gj8giEglt7/GlfZWLIaA+Vqu1dcRDeceeLOO2JGpT5AkOg4lq58gwZpM6PLpnVqvFo6cWx9q4gtkapllTDBcDwZPSPoBF6X5MK6xWZaRP8huOjwBb2lNEugX5ln2MYg+cCMo2+migqKqNayAGbp5lKS4R7eikEviUtZndrIHy5/4VZPO0kX2LwosQUBaWOUrEyf6oBNKc3R4d9WrZR24BEVM19orUfcPNpP0OTuiwqORZXZLWXdaFQaqPkMcGyFXu2ZqSDG8gKQB9jS9UJPlmFb0Ishsxl+WE5CMjCUgOVNHstQHExncj9K/cfaE44v5+Ya7bfONbfqaBm1GCTXzwqcb6+RdukLJCP+gBAN5tpmZGx9afSqWjcqGvs6cfsw85ohS6UCB/bDkPEJHjHRIuYCji85T1/54MJKRsCJdt/lvnCnMOrpiDoCxX2VnPyMd/Ly0RqwKTpg/JHfepFGhSMXjZE94hqA9+0GSSSMxt5w62MDjP jJ36uv4e mC6j3F2URbS71eNyfMP2henlfiITOSkBBDitaVivsCZ0zL3LbfAu6pEF2UcDaoEV4j/s2NY3x9B1RgGD2IaRXfJHSfa3pyuKHIJManQZJ7mnsFENyvEwcBc7F7gDUXHDS9biUVfe6nr7fTlA0jT8ADA2ajXI7BKw6vvGDfFMORz4o67r/5yP5mBxuQjBviiklzD+uCb9Hf2enSmtGlfoDO6ou40eIUf4Rs2Kwv6MaxlH1PnHka1euusThlFPnumRjMlAdgyIS4VrknRYIrB+v95FngiVnLNQbHKZW+2xBfTg6f0tSxMQ6mXg/DqOstL9LT8L76BbMy1cjoQyhcmnD8Gd4PiinvNEz8dGBOWNOzGL5Ixo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The vma iterator may be left pointing to the newly created vma. This happens when inserting the new vma at the end of the old vma (!new_below). The incorrect position in the vma iterator is not exposed currently since the vma iterator is repositioned in the munmap path and is not reused in any of the other paths. This has limited impact in the current code, but is required for future changes. Fixes: b2b3b886738f ("mm: don't use __vma_adjust() in __split_vma()") Signed-off-by: Liam R. Howlett Reviewed-by: Suren Baghdasaryan Reviewed-by: Lorenzo Stoakes --- mm/vma.c | 3 +++ 1 file changed, 3 insertions(+) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index 84965f2cd580..02e4f7cf489f 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -246,6 +246,9 @@ static int __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, /* Success. */ if (new_below) vma_next(vmi); + else + vma_prev(vmi); + return 0; out_free_mpol: From patchwork Fri Aug 16 11:13:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13765998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59EC3C531DE for ; Fri, 16 Aug 2024 11:14:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F26E76B00FC; Fri, 16 Aug 2024 07:14:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DEBE46B00FA; Fri, 16 Aug 2024 07:14:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8BA86B00FC; Fri, 16 Aug 2024 07:14:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A9B686B009B for ; Fri, 16 Aug 2024 07:14:29 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6162B1C5B60 for ; Fri, 16 Aug 2024 11:14:29 +0000 (UTC) X-FDA: 82457850258.23.E550990 Received: from mout.web.de (mout.web.de [212.227.17.11]) by imf06.hostedemail.com (Postfix) with ESMTP id 69EA9180007 for ; Fri, 16 Aug 2024 11:14:27 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=bsK8cuuI; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf06.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806807; a=rsa-sha256; cv=none; b=xaMQLRY0TL3wt07TcGrcl0HOQkTA1VL5bcK8eAkepqoX0Fv/PSfPrQd0zuk88GT0W1UEbU 6amOrZowV08+pahQa9Eft+VzKGYZ9ZAOdVm3wU0x8qnFa/t2ghgRxvJh6Vn8paRb/RlLB2 TLhqiSMCy1JNEVS1Hmo6vYGL85GyEw0= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=bsK8cuuI; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf06.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806807; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=PD0UJnTTUnduI4jUI2AUZd3cQsVgrFRB0T+PkKYNVlE=; b=3fnw9sodlYTSjFTbNAbwTd9FbEhSZ1SsNjcdT0o2/WTUhDyaxBz/lmZ0YDawBWgTpAqxvc y+vn9IR9bhqBFfZybsmKLhwphzaV77yM1AKplMO37BsaIC0fIy/6R/20Ph2hgq/EBJmHEb /BSZtItldrzcAcjq0rm7IspuMQZAGcU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806860; x=1724411660; i=spasswolf@web.de; bh=PD0UJnTTUnduI4jUI2AUZd3cQsVgrFRB0T+PkKYNVlE=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=bsK8cuuIwKJQ7Clao6dvNXejoLvUACj62D88xFuXrHIS+SqZqZTiIkujgT3A9ogY IhQJSqIE6uwM4jJUu3Y+OOCpiLqhjeQUQBAvfUIKQ9GY/NyQncvVRmKmO9P1KznFv mICbU3RCOrx6UYEEc6F8F9j60SD2wIqZlWOrUuIbPditRuTPuMpICwfgd83XWKsJ+ NAVvDNQCmC8+8/prBKp1+f/N1mhVGUKIdWaMxOA5qW4vuiHTMby3PmANgpRWjQD+d pgBkbFv+LVhQcrkVk3+H+FMLZUjvNWBz5Fnw/R6Pf0xtu6f2i2Vi1M3bApaKoWOwZ /+jId0FSCoX7SPdC3A== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1M1JEu-1scB5F3PJa-00Efvc; Fri, 16 Aug 2024 13:14:19 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 02/19] mm/mmap: Introduce abort_munmap_vmas() Date: Fri, 16 Aug 2024 13:13:43 +0200 Message-ID: <20240816111405.11793-3-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:t7KgXoE5HB9OV298a3GOVGcOeApkwDqVfJq67tQ9GhMRkOUXHJq gQe/uyt64zxgKlEVwry9NxzofzGiM+VSusOjNCQCe9eR02VW5JQJw7hVKeLlUGcPh8uJ1jF NK9AUGmZJg3Ne3Syq+OAN1Q0jU6NHxW2SiHXhvRCPTLE/2LbW1krN1ZNb1yITe9LFSvu5af iZDSjIiY0lFBQveIY9w6A== UI-OutboundReport: notjunk:1;M01:P0:Ofwdbonbc40=;L5C5ebC4C8iyme8NqV5FdrBrKv3 jggi6yIg5irFKJZoFZ1AUmOFwm629xLS/Pq+aBHQ1Wy3oeXp9Q92h0JqJ56UUO1NyNpCdAro/ XL5wNfpHC4LTisxEPcRh5wJFlVM8ALu1v4zY5wohBc9Efw0gZB6DpzpRL4YeDejTjhCph66gq KuWEd+ATwhmmcOkKo1wZSFCFNgnL10kwVxKqMhQkdmJ9ou1RRnGCmlcI1W+TTrNqygX02k9Z2 MfDLUCL67zBTEk8rZPwhCssKbChfIKdzjewDjKwr1Dww++SgSYLo39KykJu/I8aqes2ye2QK6 0fNZxL0auXmG9hR+QrUjdZtkomvnWWlc9m139gMPOr7yxmh9YrgQfNS6ywo2Qkaqigia/Cnii kIu3YGK1N0MjK7qsl0B5R1Y2KjaLhKq4H+zIEVrukpDYJV+oNHccH/iGQRkljXKvLfRpOLS8Q sVzHXhv6pbtxExHMB+pA4yWJ6XMQx1+s+Q4OGEqv1XRqb1mARQBHVYr0H0cL8xYs2s4QDGAIG ZYr1YW1gWLMSXcfdTCuGbJ/mu/Q00bYXH3bLqPCZcOry6QyTQChYYgra9LxrK/xpOABnfUwaD iLGGXdfWuVOf8IcbrB/AX3L4Fe9NaTxnx2ojD4KatOqgfqlo4NzrH+34x3uuW9A7F4wpnBqHj qjtsp96aR0ts+wZdFiHycF3LXvVMCKUo37nX0giTE5FmtgCXj9nnFx8h78l1+oughaxjeVhl2 gOhC2mx6vm9g6OtEuTYmhYvjd168A2zjfBRzHFJ6pFnzTcdGKopgONNxv9vjTVV/qUYb33yR5 WsAqjsJ2otakBBPF2ad3jluA== X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 69EA9180007 X-Stat-Signature: nwasppmuppnrxe3rozfz3miffhsm9yd9 X-Rspam-User: X-HE-Tag: 1723806867-994204 X-HE-Meta: U2FsdGVkX1/h6hD+3/9tgFNB4pedxRcP30lpzM18Vum/SRcIh/LuBmI3eZ0pgsqnc4XRGaJ/6D5uL8J3scRHXbtJ5DjAi4OYnKCBbAWC+IwUhWNUNj+gEIgdaM7TAe4fJzJBQp6WBHRhldRKQQ/swRxn+XM7a6OOFeTsvaZD/FIHmhD5Ib3UHnLQ0ioPnI06IV/5s15LrjMuXc2lILiIBKh3K9W1/saQD4iNrY2EhyCVTN/EXjqMJKbnkv48Wu/vaA2H3Y/7P4mmXCEdm/yQ2isYpN1Bqw/RRKV5OI/dJSYCO6eg4PVkdyots3cs32ScNMaJiZR7gz3K33g7zbTEVFY4IaE8aMFoocNmINxleDkOn/0RTVNSlfC14l2wurO1sgP78CheVHqoPXma6c0YVEsPGaNT0atAfvRvJ3fpVwcxqTqXXrztsUHaqWQxr86xqYHkppnCbhhjBwb8O/EEqHyaJL+ZW9RGhG57pPBoUMW8t292x9yT32X5ohx81sw/dNulbNf/GTFbc3r6AZ8YlKkHuYn2si8E28hQ5kzbwNT+6Nq8NNEq8LFe6nW5cNummSOxHuI0GgeqKkn14VeJiWt9GRIjiJ33VKgo80eAgO6D7WpVFRgKY2dtp20JZdbJ6u0jNNbeU9Ok1vOHRzGUbymN8vMHM4Vk0cG+qPOKaTdhtySYr7pElxYo2D1xISX61+vONl8x+ds1bHLsAv/eh1zA4/QM1F8nLobpuins7ln1KnbRpFr+ayftQaG7cxjlxxf/odPPUP/HoCRPLLJqBYfsGGFPQT92a5DP6/Nfz13bsZX6xoXOPzqto63ySSoxPVGbt6M/n5Nq8k2sF5iOWk3gbsYF1MZPga4i+mvfq49QqvtK1/wnorBtucY8BBYQa0e00VwHuh7cXYNUd3IgGSjKFNP8YWSl4rUZHZp11lYMEeGQaT/9nDzMx/RMalkxBXw4EoODRySdqAkfizZ Uak1G5WW wzMPJvRIDlXhSiq1XsAmMBtw5yraezdT6zDpESBJXWzrNe6Nc92Xj70WOzYPzfn7qLhBMYjxJvqjEm5dmBAUVKNyhwaJd8mfai0DXNLO/PBNDq7J++EVsLSnZS/54qTiX+JC1zIv5SFS1kHyRVQaRoc7ldsD2GLjaH24EIAPssqcuCDWBIIJZqByhKJcCYleALvFxGavcX9uhFE0+cvJB1Y81WvSaQn7Soo05dSta1nUcj+yERMGPNY04bJ2p1dtbJdxypJpXrvq+6u+nwRb5TchOQcSmTWxXZSXYn8CsHJHug5Fz+U7g90BuPCpE3xdj2/4MO7q4TEG0T9XfR7A/eeX9ZQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Extract clean up of failed munmap() operations from do_vmi_align_munmap(). This simplifies later patches in the series. It is worth noting that the mas_for_each() loop now has a different upper limit. This should not change the number of vmas visited for reattaching to the main vma tree (mm_mt), as all vmas are reattached in both scenarios. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes Reviewed-by: Suren Baghdasaryan --- mm/vma.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index 02e4f7cf489f..83260a3142b3 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -668,6 +668,22 @@ void vma_complete(struct vma_prepare *vp, validate_mm(mm); } +/* + * abort_munmap_vmas - Undo any munmap work and free resources + * + * Reattach any detached vmas and free up the maple tree used to track the vmas. + */ +static inline void abort_munmap_vmas(struct ma_state *mas_detach) +{ + struct vm_area_struct *vma; + + mas_set(mas_detach, 0); + mas_for_each(mas_detach, vma, ULONG_MAX) + vma_mark_detached(vma, false); + + __mt_destroy(mas_detach->tree); +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator @@ -822,11 +838,7 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, userfaultfd_error: munmap_gather_failed: end_split_failed: - mas_set(&mas_detach, 0); - mas_for_each(&mas_detach, next, end) - vma_mark_detached(next, false); - - __mt_destroy(&mt_detach); + abort_munmap_vmas(&mas_detach); start_split_failed: map_count_exceeded: validate_mm(mm); From patchwork Fri Aug 16 11:13:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13765999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D932AC3DA4A for ; Fri, 16 Aug 2024 11:14:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C3766B0141; Fri, 16 Aug 2024 07:14:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 64B736B0142; Fri, 16 Aug 2024 07:14:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ED4B6B0148; Fri, 16 Aug 2024 07:14:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 30B986B0141 for ; Fri, 16 Aug 2024 07:14:31 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DFEAA1A1AF9 for ; Fri, 16 Aug 2024 11:14:30 +0000 (UTC) X-FDA: 82457850300.12.48AA5D6 Received: from mout.web.de (mout.web.de [217.72.192.78]) by imf28.hostedemail.com (Postfix) with ESMTP id E6D93C000A for ; Fri, 16 Aug 2024 11:14:28 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=HIKle6HL; spf=pass (imf28.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806832; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0obejYoa2r7hw/w5zRqv2QmgIUrn0jX3NuGgwo7FlG8=; b=W7zcaWJb4vadFqAMAOuJVLFRwFOpaXfvq+gy422YWslEvmab+N66H6t+mUswoqExeP3p16 YWouG+bLcd2KOgLAu04F9b1x2fWJD7HGvIWXzH63eF1QSMtxfeZSkkFLR0lJJC1U72M79l FQxxWUJ8WoA9M/GSt/drV6VTAIMEF80= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=HIKle6HL; spf=pass (imf28.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806832; a=rsa-sha256; cv=none; b=RDoYm0SjFN+yGB6KRIU6NA1B5rXc1x3KXRDNaUkYyLWteQpsoqL+ywHIG5F77xgCdiYRFu 2OH0W1zm1L9fn+dScz0DVAABDQbvARBcR58B7p3gZc4YpW7NHmaxOEXH9JJuSgT1HSTeat 0Ov5cZ6HYXJkqTZEXwPyeZVgulWAZvU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806861; x=1724411661; i=spasswolf@web.de; bh=0obejYoa2r7hw/w5zRqv2QmgIUrn0jX3NuGgwo7FlG8=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=HIKle6HL3gik2rTP8fMx51a1KXQCUfWcUkbj5im6liQLmMAMbETBT2P/PLQLctmI wB+KQZZnNgoMVsc1pUf96GR1kjXkPYZ9JvI18CLEWXeM+77LTAhc0BV0GMtlkl/xJ Zm8uq3dJ3tma0hkBl7mf/G06FaKbgB347mJTgVkUpGMr6cSG6u9AFKZy6QTVGLzWq WDiS0ifH6K4yAwyZGy+Y81+uO3DfQ1HJ4+SCRLxzQeFCV0vcsIQlozGMFNHEHZ+u6 PffBHx58cNAZtZavb5XQ6IO6ft8FSWLz8pYeZcUsGAnV+oI2DsYScTd0vNO0EkTww y75iYhSdrrG+Nl466w== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1My6lR-1sEzR41xWx-00zkD7; Fri, 16 Aug 2024 13:14:21 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 03/19] mm/mmap: Introduce vmi_complete_munmap_vmas() Date: Fri, 16 Aug 2024 13:13:44 +0200 Message-ID: <20240816111405.11793-4-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:eqWc1iB4g+G2VRb/rfgfTQ4sRq2PmtECXZa+ZieLZIbxA1UeXNo 0Ym4FiJ7Auovew6A0uW84ljNoqkHTaxNAX4YQto5wUKBt5IU2jTjlCP7i2csrJfTUb8dW6l WiqqSYLmF7P4CIJdkfkyCm/Fe1bLnggOsqruJeBk2lff+eAhZL2is4UG4EynqYVqTE4hNLz 3zIfEnBggLKafepPJI8xg== UI-OutboundReport: notjunk:1;M01:P0:0ZOF32seaA8=;vYnMaxAP0Qy1WSqp1i/hpOix4hI +RotyKtfiWNyobKZ9g1oSvEzRj/DTuKmFp/3GtpTLajVktOeGnEeUhqL9RetMT554UOrzAtu1 8M9+dpLjy8uPqlatJFFm7ljzWJ5rPPxMf3WfMzPk4MjWMPg4c+2DoMSyLsx6BG2uypX7Bh9Sb +K+it78vLrj8y2RzaQudTQbG4bFb5jG0CAOpRFUqpVtY7bCoB/m6+RLFhziMN0OYRQUOHkm0n O8OhBsrVo4goCDNXcl2UwNrfRf+COq6hQzp3qHermgG3JPBFObWCE74wqqyW1AgPQiYlMtxE1 U5AOHYQkOa78mj4CEuXzlxJH6fWDeqA8Dgstc5jfRIk/xYhbis+T+hLXH63lY5ICFzdVIlmGu ldLL3bjpEvfhj/qYYzXDDnYeiYhFmXFSr7itdp41w/JD26XA4MGAD6CbXBMyQ/EIupiKoerkO EWy+fIlyUucumAm4fzX1KKJzbu9BaIncMCDFDaoaD4hwT6kmNvAhAI7HnQIbSJeOTzecxFbvX 6FiG0MXdSwNAVBZHKWdEkCOBzTxqJgH11Oy0rwJKGSRQvkYF870pVhrkEYoVigGnfNhdVNvDE 1jSULfEgKpVfaWpfKZF0QsjnG0Hvpxi/tBxXMkQM1N9iRbPz3VJeYMwnipGDvKsX50t3t+6v5 mFxmysHS6s6I2Qt5Bej8admktNyFjT0AzYXKEzvWb20aX3rbV9WzaqZgQ4vL11FW6BitkzqB9 FlGBC8/cKI83JyRd7RU2Zvcv+k4RSfadHhDabsNGu5vdCQgm/MRhf9FOWha53dGJWSMPKtZbg PKWVm4bn4VtBl2RxjijWKlFg== X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: E6D93C000A X-Stat-Signature: 56bgjx8emttd75k536cz7bkk3j9zsiu7 X-HE-Tag: 1723806868-676715 X-HE-Meta: U2FsdGVkX19ursAXmEjbRisFjG5/v9UOHvP18LVStXbXsNpg4KhmIgmAuSuWoawOwtmbDnGsOwdXIpflMQXyfm+GqdNSKnssdv6JvmS4hYQd7lkZj49IsIcPPRRTGtgj4RythVl+Q3WEBiFhTGkqeeLCKLfkUA8KoouPGsUl9QUbAB4B0TAPs3gCOv9VSh2+m4hjJGOn9c04HUUQ3wOnbHgc6UmT1MgNwOUoKAlGxASqBGcw24d7hNth65xEW71EKUG9EplDkUV0OPO6+1+KSUhO/QH+VKRPJle0ls3EDU2sTAGK3AwELDSTfvvI1bDKLYRf+0Yl0yEseX0bi4i0wFTv+Lndhr4JDxdXo5wktEKiXqXbN0Es6tbFpnxBanoGDsmUDgq+dCfFhkXrK/2DqZQr6RS+aPEMmFZeFPvzqNRQdInrV7bnlNv98P1sSAjGi6DljKkU29b98HyCTPQasL8d7bJZRyZwweppNxU73vjqgsYxl/nASo5bPMtQzzeQWYBRwvLq6URg6/g/GbmL1Ps3Zic7iMZl8M40/qDGbZ2fXJyIUOcWLOfnWuyB5NeujL3G44b+/kxlATZquxb7nRkYi+sBdde2y1PAXjpcSJeBzoHY+xIvZn0EpT+Y6ceYXoy/qC8Www6eL6rl9Bog8jaktYeeqmyb8a55f05aVVuKAuI3gRCEGKxtBEny9IAqDyAPJ4x1X3MjUoCuUNwV8yH8ezEKeu9+VppGN+NXcGfDriA/dDGxcWM2DI1mqX2Y1prS9NKBvNImW677QOHqXFHCSzz7XIPW784dKrveaG0SLItXZonV3qcRKopF5psHM6u+fzr65O9ytc4zvtRUok5P6JIA/x3bTI6YC4tHCoqCHpRYUx3zxY771q9qSBSmI+N8T5SoIhbPaS41siwsIjLcPsQpgt2TMpswuxeC+B+m4mg4TPuAjpV+ZzjpQDUQDsjtP7luTjeYDGBl4Tk ho8gLGlh cfsYhYA5qVAUMw0/VF3uQftVbt+W5bUi0y5D/bsX+CARoOMGGv9oLui0RC0wuGzemg+G1BWoRVukT0jf20tD8flRPnvv+M+K3jR/E4g8KPsTiOZO4TM+0MDNO5lDK2IlYWwOnKG0SXq3duiUiMn1MQ//PiGOzD+JMAjzAJcshvvJAjP2w8VpGdmYupGrD1e2etkqkKcPwnOSwuA1v2y6Yw6nla6oliGq661bXhqjYAF/c72Cbv/uYQcUeGgrauMbtHskKXZtCQvRi86HTo7uMfqgGkzMjaBBli2mBqXpScZjtfpdFYHz96t2Y6rAxnLiRVlfjkYh73BcvCcvonnFI9PXvwQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Extract all necessary operations that need to be completed after the vma maple tree is updated from a munmap() operation. Extracting this makes the later patch in the series easier to understand. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes Reviewed-by: Suren Baghdasaryan --- mm/vma.c | 81 ++++++++++++++++++++++++++++++++++++++------------------ 1 file changed, 55 insertions(+), 26 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index 83260a3142b3..d331ba575019 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -684,6 +684,58 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) __mt_destroy(mas_detach->tree); } +/* + * vmi_complete_munmap_vmas() - Finish the munmap() operation + * @vmi: The vma iterator + * @vma: The first vma to be munmapped + * @mm: The mm struct + * @start: The start address + * @end: The end address + * @unlock: Unlock the mm or not + * @mas_detach: them maple state of the detached vma maple tree + * @locked_vm: The locked_vm count in the detached vmas + * + * This function updates the mm_struct, unmaps the region, frees the resources + * used for the munmap() and may downgrade the lock - if requested. Everything + * needed to be done once the vma maple tree is updated. + */ +static void +vmi_complete_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, + struct mm_struct *mm, unsigned long start, + unsigned long end, bool unlock, struct ma_state *mas_detach, + unsigned long locked_vm) +{ + struct vm_area_struct *prev, *next; + int count; + + count = mas_detach->index + 1; + mm->map_count -= count; + mm->locked_vm -= locked_vm; + if (unlock) + mmap_write_downgrade(mm); + + prev = vma_iter_prev_range(vmi); + next = vma_next(vmi); + if (next) + vma_iter_prev_range(vmi); + + /* + * We can free page tables without write-locking mmap_lock because VMAs + * were isolated before we downgraded mmap_lock. + */ + mas_set(mas_detach, 1); + unmap_region(mm, mas_detach, vma, prev, next, start, end, count, + !unlock); + /* Statistics and freeing VMAs */ + mas_set(mas_detach, 0); + remove_mt(mm, mas_detach); + validate_mm(mm); + if (unlock) + mmap_read_unlock(mm); + + __mt_destroy(mas_detach->tree); +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator @@ -703,7 +755,7 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, unsigned long end, struct list_head *uf, bool unlock) { - struct vm_area_struct *prev, *next = NULL; + struct vm_area_struct *next = NULL; struct maple_tree mt_detach; int count = 0; int error = -ENOMEM; @@ -807,31 +859,8 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, goto clear_tree_failed; /* Point of no return */ - mm->locked_vm -= locked_vm; - mm->map_count -= count; - if (unlock) - mmap_write_downgrade(mm); - - prev = vma_iter_prev_range(vmi); - next = vma_next(vmi); - if (next) - vma_iter_prev_range(vmi); - - /* - * We can free page tables without write-locking mmap_lock because VMAs - * were isolated before we downgraded mmap_lock. - */ - mas_set(&mas_detach, 1); - unmap_region(mm, &mas_detach, vma, prev, next, start, end, count, - !unlock); - /* Statistics and freeing VMAs */ - mas_set(&mas_detach, 0); - remove_mt(mm, &mas_detach); - validate_mm(mm); - if (unlock) - mmap_read_unlock(mm); - - __mt_destroy(&mt_detach); + vmi_complete_munmap_vmas(vmi, vma, mm, start, end, unlock, &mas_detach, + locked_vm); return 0; clear_tree_failed: From patchwork Fri Aug 16 11:13:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A59E5C531DC for ; Fri, 16 Aug 2024 11:14:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34D386B0150; Fri, 16 Aug 2024 07:14:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C8E36B0151; Fri, 16 Aug 2024 07:14:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11C208D0070; Fri, 16 Aug 2024 07:14:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D5E5D6B014D for ; Fri, 16 Aug 2024 07:14:32 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 88E221C5B76 for ; Fri, 16 Aug 2024 11:14:32 +0000 (UTC) X-FDA: 82457850384.27.1F416C4 Received: from mout.web.de (mout.web.de [212.227.17.11]) by imf01.hostedemail.com (Postfix) with ESMTP id 8F94640014 for ; Fri, 16 Aug 2024 11:14:30 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=j0eUVv6E; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf01.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806816; a=rsa-sha256; cv=none; b=Awa2QQV0bH4MVW0/6eS49ZO9RNGqRk2lEOX6LjAaPyC/SAwaqS8QXfeHzNPAk7obkEemaq nszvW8E1uLsEsqnM7V6ukjIjLGQ/61+SKDHcu96b1w7K90Aa39VeS9toAVTk7KmfaK18rH 6QC/QXu8y+cy4rwcEUOt5UzuR60uQu8= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=j0eUVv6E; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf01.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806816; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=l3TXWV1Tp/aKQfD2gCIur/5ETslPLQoSLhNXx7Vp5IM=; b=gO7zPukZw0EIN3BdLjWZjpt002UXvVhyslpzkm4i39yKttqMxE0Rer3zWbBtISr6mp3ZyB qWON95BEJMWQsFEgvBqmvhmo7p2qlwicV505q1bjmqac0U54kX54fx0gmoOvxEGq+BF1jx 1UX+DzgYNU0B1MAfww54gw23tx4+IQ8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806863; x=1724411663; i=spasswolf@web.de; bh=l3TXWV1Tp/aKQfD2gCIur/5ETslPLQoSLhNXx7Vp5IM=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=j0eUVv6E/rG8Cnhu49wfDc3ZjB308tJTF6zB2+RJHsRCdCi+Zt4nyYI/ByvcxKzC nVunB31UCoLPS33iT3ORKWHQQyKlzZ/JUPT+PsSdWbVi0fWF2M/U1wFDR/Lg2bjCF Q0npjb/EFieS/J9QGRtwhXo79AwYsiCtYyIFc6uBoEiCEKaoSJSKvqHGBr9cRQMkL 2beEdqQLnqppjuv2iCHHC0+kbttT04abwSRndJRr9/uFNu+0fmqum8Y/8bauO9niL yGHoJZ0JO1zYe0Gy3e9C+zsBHQia0yMzeLrLAtdpJ67ueqEjSGlzbxG/HI+H95Tjd Molvf+0gFkyqCuX+OQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MnG2Q-1rvuzT0Z0O-00gsfG; Fri, 16 Aug 2024 13:14:23 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 04/19] mm/mmap: Extract the gathering of vmas from do_vmi_align_munmap() Date: Fri, 16 Aug 2024 13:13:45 +0200 Message-ID: <20240816111405.11793-5-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:wTQCai+xUeqxjSfq9zUCYkHRj+YFQBFLdsczal4zW75GOviwbTE x5RrT/HHxU8w23/NaMxYwThD5F3v/k6iYpTCuPKN4+IZLXDz77gixcKc5xsxLMJGJpPw33+ NxS7VuPndRr7JacDCDCtI4DAOrxeM0LcCIN+QdLH94axfZqSq1ebKnJzFtIXaDiLwYYVOWO 0QvJxWvDPIwKjLFdsaA5Q== UI-OutboundReport: notjunk:1;M01:P0:yr2ZPw4vKiI=;FDy6r4s5ooBsVf9RgTeJNvQhI8e ++0xFb6KVDhYGn7fbXxa0mFvvKvxPwVHl6FgqboR1sowyn0v9bqG/2SifVm/mLHHJ9JcWNZ7o XAt1Mz5L/MpDEOq9L6R9jMhjy+v7cwoCgYdI1gDEUfuWMPvyXik4dcyDcbKxdVLEcU4KzXAgJ SpU8X6tuTKQFY9JTKhMJ91fTXNyRE4frpvxnhIF3mrqVC221p0zePFEyLGZMksN3gAI+uHoox 844WAkat4ZyrPqFMJsYh+y+Kid3V7Ot3Yz6aeQ8J+qVZpeg771D33TDB/IJthuXI7FjwZBP0Z WlKXgTmqhjdb+to3KFRPBEzp5/IpH7SZ7JywG00FFkoov9U5h4f9WRQHgPgrZS0kPT7MOd0bH YgrvIwW4DXvvgXM72GS7se+JxLMz6IwIA/hWuf/uWGZztLfA7t+lNltlLR84XMoHkPewjbnEh j8tYgfZbANYsqMzctXp557BMtciWVS2R24ebHDno/VqDm5LgeUr80MdV8dlFr85LsB0nNa6jx tgg0NZTZqrqUeeqxdn+430+tg+3DiVzcs9B9sXW/RplcTqVE8Sc4qogZAEuGM50USDJE54ygB 6LsfRUw/ppPsf3Maa2ktIVxeF51kS0wNk+zcL/38/zlfihnMz7YAzOrTTaCP9a0O/BEIRd5ds Dx2p8YgxIXqu+ay+PSWP6M/xbCIKQr0oIdinqGnrqiTlO7tsu0fWqACKPF2RiSkzLAJw0mzq+ sSua4B97tMDhCm594KVCKAsVMfFW0a6Bo/9sYkmCakdr0ZrzW2uVt86GhDFEWdyip1eSre8FM dCyqm6BMTC6wa0h1SHKSGo9A== X-Rspamd-Queue-Id: 8F94640014 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 8zx9zcwypzje8n99w1ae1itfqqnbij1h X-HE-Tag: 1723806870-607360 X-HE-Meta: U2FsdGVkX18YUpgYjoclJl81irJz4nJ76PQ5GRGaI9DGxwklHIapCM0yAl1CNQ8Fg9LhS63sWPy9B0aiej5IbpZpkXEtOwSAzB2uwzjw1nb6xXVlibE6Pp4DlaogYd3h878BmFOVVtsZk+i0KcorskGfjUktyHyR0qf/UTSSxdHur/ABrHOJXXIH3fLcE1scb/iIOaqDEffvmJ/jW3BssLEBl9v6s5xGY8cmLui+w/3udkrII+iCD59Up0RiYhxTgZ/Fch7OyeAN930+uN2IXPWwRn0Wt8LhFJVW4yASRZglfgLThbItAFBoPcbLeTlZ3ta+ntW/QIZaTBpljzGY+CE3SMX2ysWxKJMgdgNUq5UJbkAKPesYIgrH66V3QyRcc436k1RlkqTSLrDvaMAVq3uSbonmGs4JbmOviI/lOcTB/+cTBa59HQp42/SrmsDm/RsqZH30A7RpADzdZwTfBECnKnvB7j25pe3BZBofeZtWWrMzaeoWY5jtFPNzLw67Kmu3aOmxXPuDEKu7+H1wI3kWXUtf9Vy9jqzJ1iYvUIy0+I7dRIm8gLUszUISpT7X8T3MjH3j01fvP3R8c77N7zDRMxt5/cGzmSB/fS8+RnkvmhdpP3JI7sSlkbhyE2lu5Ke0ou1VT3oDPK6WjWBNMgteIlMPhJzY/QSKFClJur2XRXgBaXlQJ2rJKGm9mLMIrAtjbxAKkCr1Dg6sMAcXs/LMkM9GbVKEB22ZFu6iKN8AZCppJcA0QzH7SvO+3UNumR1rxzhwX3FPoD2y5TQbqnmHFGTNQo0Q3o20TQv8AV+k4c4Xr9w+JUctPVQOYPwbdbPd/cKXrQxhB+ZJByjL+6pIKlJDghqKwBQkm9xhviIbwDzAdrKCp6JYdlYyUQBkQL0ByQTnSbnyMLN7XDL176mKfkbSo+eoGjBC/6uZHA72gRZKhe5NMI9IiK6ZxMveD22ZLrk6eIKvX/ViXe0 uPBOahSd +tKPMVKjO3nwQCp/QznJhGw7XrjLswXYFHMnBH557mejY1AG2Ef4K/79ZUV8v1my05aHwjDo8Zz6i+zm/YRxsFWHzHaQKvAp0GqlwqxQTstWI0HqPf1rDxlCtdO2wZoqcLfVTJ1DNE7eqldRe7IndulCsB3HAhm0fMRC5f4iM9fvYH/H1CBxFXfRpwRYysMSEClOha/TZAU7V7bOsc9HTgnH9rlBJhdLUUT1CVZI3MC8XsthDv0CkfjrVO31bhfF+B7af1+rv87BAASpfb5p52sN8hxxRHoaFfdb5pmrYeePg34LWaV4OGjDmHVX7kKu8tPD3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Create vmi_gather_munmap_vmas() to handle the gathering of vmas into a detached maple tree for removal later. Part of the gathering is the splitting of vmas that span the boundary. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/vma.c | 80 ++++++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 58 insertions(+), 22 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index d331ba575019..a980837eefd7 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -737,32 +737,30 @@ vmi_complete_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, } /* - * do_vmi_align_munmap() - munmap the aligned region from @start to @end. + * vmi_gather_munmap_vmas() - Put all VMAs within a range into a maple tree + * for removal at a later date. Handles splitting first and last if necessary + * and marking the vmas as isolated. + * * @vmi: The vma iterator * @vma: The starting vm_area_struct * @mm: The mm_struct * @start: The aligned start address to munmap. * @end: The aligned end address to munmap. * @uf: The userfaultfd list_head - * @unlock: Set to true to drop the mmap_lock. unlocking only happens on - * success. + * @mas_detach: The maple state tracking the detached tree + * @locked_vm: a pointer to store the VM_LOCKED pages count. * - * Return: 0 on success and drops the lock if so directed, error and leaves the - * lock held otherwise. + * Return: 0 on success */ -int -do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, +static int +vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, - unsigned long end, struct list_head *uf, bool unlock) + unsigned long end, struct list_head *uf, + struct ma_state *mas_detach, unsigned long *locked_vm) { struct vm_area_struct *next = NULL; - struct maple_tree mt_detach; int count = 0; int error = -ENOMEM; - unsigned long locked_vm = 0; - MA_STATE(mas_detach, &mt_detach, 0, 0); - mt_init_flags(&mt_detach, vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); - mt_on_stack(mt_detach); /* * If we need to split any vma, do it now to save pain later. @@ -801,13 +799,13 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, goto end_split_failed; } vma_start_write(next); - mas_set(&mas_detach, count); - error = mas_store_gfp(&mas_detach, next, GFP_KERNEL); + mas_set(mas_detach, count++); + error = mas_store_gfp(mas_detach, next, GFP_KERNEL); if (error) goto munmap_gather_failed; vma_mark_detached(next, true); if (next->vm_flags & VM_LOCKED) - locked_vm += vma_pages(next); + *locked_vm += vma_pages(next); count++; if (unlikely(uf)) { @@ -834,7 +832,7 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) /* Make sure no VMAs are about to be lost. */ { - MA_STATE(test, &mt_detach, 0, 0); + MA_STATE(test, mas_detach->tree, 0, 0); struct vm_area_struct *vma_mas, *vma_test; int test_count = 0; @@ -854,6 +852,48 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, while (vma_iter_addr(vmi) > start) vma_iter_prev_range(vmi); + return 0; + +userfaultfd_error: +munmap_gather_failed: +end_split_failed: + abort_munmap_vmas(mas_detach); +start_split_failed: +map_count_exceeded: + return error; +} + +/* + * do_vmi_align_munmap() - munmap the aligned region from @start to @end. + * @vmi: The vma iterator + * @vma: The starting vm_area_struct + * @mm: The mm_struct + * @start: The aligned start address to munmap. + * @end: The aligned end address to munmap. + * @uf: The userfaultfd list_head + * @unlock: Set to true to drop the mmap_lock. unlocking only happens on + * success. + * + * Return: 0 on success and drops the lock if so directed, error and leaves the + * lock held otherwise. + */ +int +do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, + struct mm_struct *mm, unsigned long start, + unsigned long end, struct list_head *uf, bool unlock) +{ + struct maple_tree mt_detach; + MA_STATE(mas_detach, &mt_detach, 0, 0); + mt_init_flags(&mt_detach, vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(mt_detach); + int error; + unsigned long locked_vm = 0; + + error = vmi_gather_munmap_vmas(vmi, vma, mm, start, end, uf, + &mas_detach, &locked_vm); + if (error) + goto gather_failed; + error = vma_iter_clear_gfp(vmi, start, end, GFP_KERNEL); if (error) goto clear_tree_failed; @@ -864,12 +904,8 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, return 0; clear_tree_failed: -userfaultfd_error: -munmap_gather_failed: -end_split_failed: abort_munmap_vmas(&mas_detach); -start_split_failed: -map_count_exceeded: +gather_failed: validate_mm(mm); return error; } From patchwork Fri Aug 16 11:13:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA80C3DA4A for ; Fri, 16 Aug 2024 11:14:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 713506B0151; Fri, 16 Aug 2024 07:14:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6967B6B0152; Fri, 16 Aug 2024 07:14:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 402C68D0070; Fri, 16 Aug 2024 07:14:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 164F26B0151 for ; Fri, 16 Aug 2024 07:14:35 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C41C6161ACF for ; Fri, 16 Aug 2024 11:14:34 +0000 (UTC) X-FDA: 82457850468.12.495BD65 Received: from mout.web.de (mout.web.de [217.72.192.78]) by imf13.hostedemail.com (Postfix) with ESMTP id C9E9420014 for ; Fri, 16 Aug 2024 11:14:32 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="Ct60w/Lw"; spf=pass (imf13.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806836; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O94QRspLCb1Ter1jcWYZ79/yZW2oVM5cQef3hvy6a+o=; b=ttHHhyCJr4lpLYekYmm9nyK3emS71UvwW4Xc6KN92iv4K5ybWPqEXOc1qsgi4dozEMgrIw Ix/MZ08SAejtmW7HrcYbawXZTczZ4MYkLNjBbGXIl037fuPxC56f8eBATv/0LfZ+X0bps4 ggYKL1DQlz8SxSOw6G5Au0fwopZnJCQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="Ct60w/Lw"; spf=pass (imf13.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806836; a=rsa-sha256; cv=none; b=h4vkouQF5evVNmVMOFUUoLYZF3ED1H2d99EZE9gK17/nS5pM20xYwqHFyC3mU1KikX+nG/ TkntudyC0V4p2UqQNGY2MxjLQKEwHKonMLUZa2oAX25mFdEqkCec33Jgvwhx0p2isbB3Qn A2Bs76BptogVi4wJDaok12z3cpmZlRo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806865; x=1724411665; i=spasswolf@web.de; bh=O94QRspLCb1Ter1jcWYZ79/yZW2oVM5cQef3hvy6a+o=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=Ct60w/Lwqy4o/CiqVarKJpz/BZNpHxc8rnDQDS0jM2EggTw5K0icYoDzM5efaME1 PaAn0gr9OzHRMFbZKuFuKLaa/wXUPVZiUQtgVJUJ17n8LZ5kkgwpen1oBH/45R3+O k/dc0KnrP2wke/EsEH3k+GhMOgC5oNfaz5myT36Xzb/FPoLoN5DylhjH9oURL7FtW Ij13b8IO/CeO1UuhBGN+uoWOaht8dMZR9odCV021ddL25UclyPk7raZfE2MoE7jFy uDFBc8MwGcxvEskoOKhKy35ulalEsiLoXx+6MgZqGsB3pO1rCz2zclJykvNTEuRZU PIiLmAsygSzoOLxbrA== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1M2Phc-1sb4d51SAw-00DZT2; Fri, 16 Aug 2024 13:14:25 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 05/19] mm/mmap: Introduce vma_munmap_struct for use in munmap operations Date: Fri, 16 Aug 2024 13:13:46 +0200 Message-ID: <20240816111405.11793-6-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:Xow6P1CPkv9JrFgOzkYeaVlz5A5aXI9IPW6ca4u3EDYV+mRkkdZ L6Dg2AQnE6KRQjW5Yp6edvS/sQZKPYTYy8A+LxRdVZ3eJo0j6GYc/K/7V/wG4rn/RHYJkyr 36rmbdVbCKbBsbYi8YC/jcUkt4uEeB2sz04s+YlXsEji5due2wWJcWhQOE6Ui83SPjUsSk5 xchSGYI5N5yH1HrU90Ftw== UI-OutboundReport: notjunk:1;M01:P0:11uR2nMLhjc=;FW3YfgVlt+qM5+KL6eDuIuCMv3a 3u6qUP9IgPVWokBcxNBGReXjgEaP1qMLkKfKehaspiexAQCFNUIbitE6lHu41gfXWiy9vmUsV fUbwNY/8yv8exex9+JnONdYa2BhSU8cYPAZMoUs/wp9qJJP19yuva/Lz4CoYNj/8MIr9TqRmI L1n1eFHZezNWQfKy9QZNJl6qaojp3THGq6kQtPeZvMPK5M6Ae6eI56XWLclkkZ53nCl7a39DE TLDp/2ylFG31K+g6LGNEj+XdyJ8oTcBTmp6aHhyQAaTTVZ/0YHovJ+v9nn66QCaZUfBIW8ZG3 OIJ9QryAOgoFIPP+4wg3u1fu4rcGKVWOz2tY5yPvyT6Zfwlj2tZNQk69QJK31RAOmLGTTS+uz w4VtjiOIYf+SpKzHCqTYePSeNI5itKUyVAewdXGD8QNcXtOytDTjLE7qMjcFOs6+JwRmMazjH +e6uj9dBomVOGqBk4XEt/Kc2Y05i03Ba0Q80PkQIpImpqdtWvECriwpfTwYXp0Xos1GO6J/Tv GgnR9YlmdCqVwyBqafdvDTlxcyqrys4ANXnvLwJsSQ2TQm6REMe2o/iHZGqcUJUWRBKXqF2Wq P5MwlQDrl1gChorq2bGDA0BBniDejITQPvdWbAtRV3PMlJ8cCD7qEsWyUxnPOnshJmk0xmPIq VOLNDDExZz08TPGRkk/kcIDbacU/0gIjOzpknlkfe36GxtB0NeG1oWg8yQsUH0qP8MlhV/DOU dpc2+F6kcVCIfwgVuxNAHG8dmGBZqehzowfBi9iSVQMVxJbrkf6CMbC4zRZHEeWkR93rp31Sy mZjrmgoLmAMkiQ3x6J7LdTdg== X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: C9E9420014 X-Stat-Signature: jwa75qgr1po3zrw7iq41nhmw9aj16dz3 X-HE-Tag: 1723806872-891069 X-HE-Meta: U2FsdGVkX1/OxnUBB2889dEPVheuFxdMcRTqH/ssJC9pwoIqI1a8YHURarcBLYcPTogKiB8x3U9Y7j9mcs64PQQrUSh107/1UvD3a7aOZTr++AzFSo01aPczDB3H7fN6EGxx/rKX1WO/x3ZKs/MNUprxFJhiVEtnE7NKOY4HIGoOosp+9PGaJ4xPwwmKaU6VUjm7RQ1StasDMKwVSU+F+aQ17HrCUkyBiyJFoVuC1B39hEfEvL7A662VTUEY6W9eSEnUNgrS8QlV1B2BiXYjmURmajK2nmkA77+18/tHQ35DWhBLUrWUdaRykmSBjBSsKco1Jp9vTq0zJrvZ3/W+JxS6fSOt3ff2dRDoPBbGBrctIsvnSddxAxDFu26qDksOHOkL0uq62n4WArar8wa1e+ErIlV5rfiqX13pNm+BvhnnYvEb8SgMq5d7xE2zSm3lCBQw8VZ5hwFmfNasLiziKTfDf2/fornclZy46WtYlX83y0CSfglAkR5aeLroNkNKhiY7dWzLTe3yEVnTckDBLZOChLoNqANwbXOjltUFPA4w+VlWDppHxOIBdORhnjXXTGFfb09wURSnxAek9FF1/IyxeIMNWhdu05CH+fU4lzwcOdn/9G1bqLIuvFP3ECtX645fW+1lQ6vFES+6OXRoAwrKxoyZ84tBnABITNS93IU/2jLTaKlzRzPzNNn05KAwHz8M57fh5JHaU9ScJJI7CezmrOPCIJTkhkXWPcfovTE9rjwk5lPfasyGHGGfxsYFD63MtmyqHES1TCFbRkGv2QM2waxBSvzSAy7pTK2MXFT7jwcaMrFuSUttdkaPKjLfvrXZCLz+3LFTIzvHVqqcObjRoR2xOuljXneOcQ0FORRDIsOXLezumb1qEqHeMLwCBnAlAcPnfJnR5/mXSS0jS0Ga2e5cXygtm5H4fli+aqDmnxhPcPdNX3gUzZ2/SykswMn65GhlnJcrcNoPEPn YL92w4xb PVfo/7IKvyuo1BcwwK601MkQzB67Kex0rJvWbxk5DxratzR1HMsCizm+fvZrzl2tCVpXVtOJZCLoEoNs5h38Uq0wK8V7H3OnKUsPjDp7ABGiFq/T/Rxhk+/mjYeaaxYiB7njZVGgHVZ8mN8oDvdppBz6Rke4v3xOlOS+TrOBbxk6MoMZJZwsZQFsVF+EJLtBRHYIEdTPOMDcN+YXvatn1VfExGepFmb4ZuRDlxJ0zEANRQ0sZTvmZrEg4/IBrOtCvJ4gMATshfZ1CstGBKSgXesiiTroNCSiUn822Pqp1HQ/lcsWZ7CJpmmkqM82f3ZHpCPL/Cw3YoEMXWGH4L/r6j4lsjA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use a structure to pass along all the necessary information and counters involved in removing vmas from the mm_struct. Update vmi_ function names to vms_ to indicate the first argument type change. Signed-off-by: Liam R. Howlett Reviewed-by: Suren Baghdasaryan Reviewed-by: Lorenzo Stoakes --- mm/vma.c | 140 +++++++++++++++++++++++++++++-------------------------- mm/vma.h | 16 +++++++ 2 files changed, 90 insertions(+), 66 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index a980837eefd7..9495230df3c3 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -365,6 +365,32 @@ static void __vma_link_file(struct vm_area_struct *vma, flush_dcache_mmap_unlock(mapping); } +/* + * init_vma_munmap() - Initializer wrapper for vma_munmap_struct + * @vms: The vma munmap struct + * @vmi: The vma iterator + * @vma: The first vm_area_struct to munmap + * @start: The aligned start address to munmap + * @end: The aligned end address to munmap + * @uf: The userfaultfd list_head + * @unlock: Unlock after the operation. Only unlocked on success + */ +static inline void init_vma_munmap(struct vma_munmap_struct *vms, + struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long start, unsigned long end, struct list_head *uf, + bool unlock) +{ + vms->vmi = vmi; + vms->vma = vma; + vms->mm = vma->vm_mm; + vms->start = start; + vms->end = end; + vms->unlock = unlock; + vms->uf = uf; + vms->vma_count = 0; + vms->nr_pages = vms->locked_vm = 0; +} + /* * vma_prepare() - Helper function for handling locking VMAs prior to altering * @vp: The initialized vma_prepare struct @@ -685,81 +711,63 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) } /* - * vmi_complete_munmap_vmas() - Finish the munmap() operation - * @vmi: The vma iterator - * @vma: The first vma to be munmapped - * @mm: The mm struct - * @start: The start address - * @end: The end address - * @unlock: Unlock the mm or not - * @mas_detach: them maple state of the detached vma maple tree - * @locked_vm: The locked_vm count in the detached vmas + * vms_complete_munmap_vmas() - Finish the munmap() operation + * @vms: The vma munmap struct + * @mas_detach: The maple state of the detached vmas * - * This function updates the mm_struct, unmaps the region, frees the resources + * This updates the mm_struct, unmaps the region, frees the resources * used for the munmap() and may downgrade the lock - if requested. Everything * needed to be done once the vma maple tree is updated. */ -static void -vmi_complete_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, - struct mm_struct *mm, unsigned long start, - unsigned long end, bool unlock, struct ma_state *mas_detach, - unsigned long locked_vm) + +static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach) { struct vm_area_struct *prev, *next; - int count; + struct mm_struct *mm; - count = mas_detach->index + 1; - mm->map_count -= count; - mm->locked_vm -= locked_vm; - if (unlock) + mm = vms->mm; + mm->map_count -= vms->vma_count; + mm->locked_vm -= vms->locked_vm; + if (vms->unlock) mmap_write_downgrade(mm); - prev = vma_iter_prev_range(vmi); - next = vma_next(vmi); + prev = vma_iter_prev_range(vms->vmi); + next = vma_next(vms->vmi); if (next) - vma_iter_prev_range(vmi); + vma_iter_prev_range(vms->vmi); /* * We can free page tables without write-locking mmap_lock because VMAs * were isolated before we downgraded mmap_lock. */ mas_set(mas_detach, 1); - unmap_region(mm, mas_detach, vma, prev, next, start, end, count, - !unlock); + unmap_region(mm, mas_detach, vms->vma, prev, next, vms->start, vms->end, + vms->vma_count, !vms->unlock); /* Statistics and freeing VMAs */ mas_set(mas_detach, 0); remove_mt(mm, mas_detach); validate_mm(mm); - if (unlock) + if (vms->unlock) mmap_read_unlock(mm); __mt_destroy(mas_detach->tree); } /* - * vmi_gather_munmap_vmas() - Put all VMAs within a range into a maple tree + * vms_gather_munmap_vmas() - Put all VMAs within a range into a maple tree * for removal at a later date. Handles splitting first and last if necessary * and marking the vmas as isolated. * - * @vmi: The vma iterator - * @vma: The starting vm_area_struct - * @mm: The mm_struct - * @start: The aligned start address to munmap. - * @end: The aligned end address to munmap. - * @uf: The userfaultfd list_head + * @vms: The vma munmap struct * @mas_detach: The maple state tracking the detached tree - * @locked_vm: a pointer to store the VM_LOCKED pages count. * * Return: 0 on success */ -static int -vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, - struct mm_struct *mm, unsigned long start, - unsigned long end, struct list_head *uf, - struct ma_state *mas_detach, unsigned long *locked_vm) +static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach) { struct vm_area_struct *next = NULL; - int count = 0; int error = -ENOMEM; /* @@ -771,17 +779,18 @@ vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, */ /* Does it split the first one? */ - if (start > vma->vm_start) { + if (vms->start > vms->vma->vm_start) { /* * Make sure that map_count on return from munmap() will * not exceed its limit; but let map_count go just above * its limit temporarily, to help free resources as expected. */ - if (end < vma->vm_end && mm->map_count >= sysctl_max_map_count) + if (vms->end < vms->vma->vm_end && + vms->mm->map_count >= sysctl_max_map_count) goto map_count_exceeded; - error = __split_vma(vmi, vma, start, 1); + error = __split_vma(vms->vmi, vms->vma, vms->start, 1); if (error) goto start_split_failed; } @@ -790,25 +799,24 @@ vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, * Detach a range of VMAs from the mm. Using next as a temp variable as * it is always overwritten. */ - next = vma; + next = vms->vma; do { /* Does it split the end? */ - if (next->vm_end > end) { - error = __split_vma(vmi, next, end, 0); + if (next->vm_end > vms->end) { + error = __split_vma(vms->vmi, next, vms->end, 0); if (error) goto end_split_failed; } vma_start_write(next); - mas_set(mas_detach, count++); + mas_set(mas_detach, vms->vma_count++); error = mas_store_gfp(mas_detach, next, GFP_KERNEL); if (error) goto munmap_gather_failed; vma_mark_detached(next, true); if (next->vm_flags & VM_LOCKED) - *locked_vm += vma_pages(next); + vms->locked_vm += vma_pages(next); - count++; - if (unlikely(uf)) { + if (unlikely(vms->uf)) { /* * If userfaultfd_unmap_prep returns an error the vmas * will remain split, but userland will get a @@ -818,16 +826,17 @@ vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, * split, despite we could. This is unlikely enough * failure that it's not worth optimizing it for. */ - error = userfaultfd_unmap_prep(next, start, end, uf); + error = userfaultfd_unmap_prep(next, vms->start, + vms->end, vms->uf); if (error) goto userfaultfd_error; } #ifdef CONFIG_DEBUG_VM_MAPLE_TREE - BUG_ON(next->vm_start < start); - BUG_ON(next->vm_start > end); + BUG_ON(next->vm_start < vms->start); + BUG_ON(next->vm_start > vms->end); #endif - } for_each_vma_range(*vmi, next, end); + } for_each_vma_range(*(vms->vmi), next, vms->end); #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) /* Make sure no VMAs are about to be lost. */ @@ -836,21 +845,21 @@ vmi_gather_munmap_vmas(struct vma_iterator *vmi, struct vm_area_struct *vma, struct vm_area_struct *vma_mas, *vma_test; int test_count = 0; - vma_iter_set(vmi, start); + vma_iter_set(vms->vmi, vms->start); rcu_read_lock(); - vma_test = mas_find(&test, count - 1); - for_each_vma_range(*vmi, vma_mas, end) { + vma_test = mas_find(&test, vms->vma_count - 1); + for_each_vma_range(*(vms->vmi), vma_mas, vms->end) { BUG_ON(vma_mas != vma_test); test_count++; - vma_test = mas_next(&test, count - 1); + vma_test = mas_next(&test, vms->vma_count - 1); } rcu_read_unlock(); - BUG_ON(count != test_count); + BUG_ON(vms->vma_count != test_count); } #endif - while (vma_iter_addr(vmi) > start) - vma_iter_prev_range(vmi); + while (vma_iter_addr(vms->vmi) > vms->start) + vma_iter_prev_range(vms->vmi); return 0; @@ -886,11 +895,11 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, MA_STATE(mas_detach, &mt_detach, 0, 0); mt_init_flags(&mt_detach, vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); mt_on_stack(mt_detach); + struct vma_munmap_struct vms; int error; - unsigned long locked_vm = 0; - error = vmi_gather_munmap_vmas(vmi, vma, mm, start, end, uf, - &mas_detach, &locked_vm); + init_vma_munmap(&vms, vmi, vma, start, end, uf, unlock); + error = vms_gather_munmap_vmas(&vms, &mas_detach); if (error) goto gather_failed; @@ -899,8 +908,7 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, goto clear_tree_failed; /* Point of no return */ - vmi_complete_munmap_vmas(vmi, vma, mm, start, end, unlock, &mas_detach, - locked_vm); + vms_complete_munmap_vmas(&vms, &mas_detach); return 0; clear_tree_failed: diff --git a/mm/vma.h b/mm/vma.h index 6efdf1768a0a..f65c739cbd00 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -21,6 +21,22 @@ struct vma_prepare { struct vm_area_struct *remove2; }; +/* + * vma munmap operation + */ +struct vma_munmap_struct { + struct vma_iterator *vmi; + struct mm_struct *mm; + struct vm_area_struct *vma; /* The first vma to munmap */ + struct list_head *uf; /* Userfaultfd list_head */ + unsigned long start; /* Aligned start addr (inclusive) */ + unsigned long end; /* Aligned end addr (exclusive) */ + int vma_count; /* Number of vmas that will be removed */ + unsigned long nr_pages; /* Number of pages being removed */ + unsigned long locked_vm; /* Number of locked pages */ + bool unlock; /* Unlock after the munmap */ +}; + struct unlink_vma_file_batch { int count; struct vm_area_struct *vmas[8]; From patchwork Fri Aug 16 11:13:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C948BC3DA4A for ; Fri, 16 Aug 2024 11:14:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB6406B015E; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1A1B6B0163; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D1266B015E; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 423736B0161 for ; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id F08FEA9438 for ; Fri, 16 Aug 2024 11:14:36 +0000 (UTC) X-FDA: 82457850552.06.DE5B8E8 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf06.hostedemail.com (Postfix) with ESMTP id 085B0180017 for ; Fri, 16 Aug 2024 11:14:34 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=vcmAT7AZ; spf=pass (imf06.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806802; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/xNMCEy6ZMUJ5jlh63sbnY1rB8O0oQ4bXvYgIo02Des=; b=xC5Il8hAF/NP47jfPbaYFhiWmPpC2s+gKI0snPmMaHFXRrcrlvqRSjIH8uko3V+mNOHB8a lxCJg+suKqj7QjtgSWm4bizK6m+QWDL6y6ypKteMHBeKqV7EwpaGgGCZbBqNeTgwoILVu2 hAvdyqzkCfLV4Oj/crHLy0KuKI8s954= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806802; a=rsa-sha256; cv=none; b=b+uAqa0Z93UJoGClFTLw93d9vvFxB5VpC6ZJjl9VMRxRmjTpN2/hvgriH6FRn11EGD/iDV NYO/ziwS7hpLThfT2csqurXoLVs2vGBtuR+GyjWQTB+ECXzJAVMamdhubCJ+1fnLMFc2D6 9D5Rm2p13wA6/sANFeeb2ebiRVpJ1LY= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=vcmAT7AZ; spf=pass (imf06.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806866; x=1724411666; i=spasswolf@web.de; bh=/xNMCEy6ZMUJ5jlh63sbnY1rB8O0oQ4bXvYgIo02Des=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=vcmAT7AZZNkIuuRa5N9UjViGBadah3uIKAUXHavwmXLvLxIt5RP9oKIBi6JEgouB 79ipHY0KZ4DeH2SG4tapsi5mo4ofvs6GAzj/Z4rUREUzvK58A/IRu8jvBlaNB/fwK aH2awVNT4IqfkOW+HRjowuiM/WebN1g6KQA/KGlTGwOax27iQh3iAycQ9HHF3nhPU ZXlVzSQJQf4AHIrvm0hKU3px/BHuk/P54Ra/LYvVvoyl+MloYi3Lr4dSRwdLXIGfP /XNzjMu7gVIOOysIjpcLOil39xWhQtIhfsxs7lplB6J47fk8RtXovyjsZVatnoCx+ xIn8m/lt52rruwedKw== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1Mmho4-1rvN1o2hKo-00gKhQ; Fri, 16 Aug 2024 13:14:26 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 06/19] mm/mmap: Change munmap to use vma_munmap_struct() for accounting and surrounding vmas Date: Fri, 16 Aug 2024 13:13:47 +0200 Message-ID: <20240816111405.11793-7-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:y+XNaT7VlbfKyj8mm9kpB8LHAxr/8kSf1589Zgx9DxmcEkK71iE myTWMxxnM/mpnLCRk/GeFj/br64MsskK235q2J6ZoUyaiYC2qHuIzQzUYO1D6zOaXdcZ3dS Lipz4ITTAG4vU5DKHIAxvof6OfD8+Ox693Ojg+zhMXhutl08/4i2ZAefg2oenCGJoqD6khJ j4QPVPVVdTsL0WzwdqJyA== UI-OutboundReport: notjunk:1;M01:P0:eZpxWICXsUE=;Mtq0GbTcN03SPFKn5s9NjDyH1iA ADaXYb3QSoc4pphABoI9iKY4yFAuXXkNZ0ozG+WA2t+ay95D6Q++CvzBOE76buAbcnQSicTIx av7VGliloCI7810rZXT4dv25+Yi9gKIafPXQn/mQ8T9zdONhBaz8QUGmijv+QsWhqA4ZAanPq N/tw7CXvhd7L89hVPeyI86mNaRy+EmcwhYfPs2DzCQuHAsuQIDh5VcotyUlCDQgB5atPrT/ac dA7/VFBhyNop0lauGUcwbCkf6+jbR8RQ50MFRMFyxlY4Fojn+foNuNiI3GGOQK+eVb6yrjjbz Ng+UvbdlDlESoeWYxni7UEIkmKGEsoJVezrvOjcyAGaxQjqSXw2S5MyuCD/lJWG4DHRFj9VT8 wv3WYUNnaQuNCuXjeedWlVhI1MD6dcuCGn+6O1ylYKYpZFufQgJCLwGpPPUW7kmV2apOc4mBH MVxqsPQ0Q0zRezgfujDVRWz2LYCnf5gXHIImynj3IUWZoZzAnc81dbIsLHibqlMRZpGWgvJuQ PfSmqAIqe82Zpk8k/+0tGqN3cz7/R1sgkFGNrdVeIiVHDr0xFFgPExaJ2zAXaGYRGisgMBRDw XTIZLbF3hlnwORrgYO2o6kkppf117762j6uMLfQgAqrJIXCpgymizDkG1LfXcMCKSuGiwbx7h N5afIdOjD0Phvs6kgwCbOrh0knTF9LBgDu3MfkKfEo0Q7ufOgcwTN5Vd2RFkElexkRFHjbZOK AQRc8L4chhHufPE5hFpTKBOOtimLJX07uudXJ1pnPbK4L38s2jbFVVOs+HHjZn5P1RIX+HDgU E1QwuZRfA6gUsYKbucXtcFrg== X-Rspamd-Queue-Id: 085B0180017 X-Stat-Signature: aj4d9pxdo5n44wb48ra7wi9p96kuxus5 X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723806874-304101 X-HE-Meta: U2FsdGVkX18TVlvBX2AidB/8RcLCtFjAZXkbh/VLv/1xm4RUnjTOiuuDznTt0RfPvL5xKIrcRbvPvLsYYBRhjtiFVqbf/ns6c6c+Icnpo0lY0uOYEYwsT9nIyMqxkkQyAsS35+8loT7pPjHBNWHtn7Gzy8LYOYRMSbXJQ2G5M+6T3e5z8dPU+awXiS4rCXUL21MuQDugDq2YVUkbsK6vk56OmnYIMI048265Z0+vK2/GX9mytG6paoUJVIjPvv65VvUqrQ35tWOz3gtdU2taaSqLbwOYBGS9iDfwnrgpGIUAR6HcFQUEzlYUBQoN/Ww3uLd9yan/YjjC99ogdn8dEZWKJZabPcheTTp9DamfNOrFF8RdrgMfbBwSFxJiYURV1AsSPIuVNFPVZw01MKup8kFIj49uVNlFRPuLL3kaGaUMs4j9oj1+NNDPsVdENjg4f6T0QpbobpIZf/PMfIu1FwDIJmsaxxL72BU1f+SUjBHn37+nj/yGJRwThoNTXHERw3pJveZxwCsMZPiKD97CICws34udTa3VZ1DBoBTw9j5vaa4D7HQkACItNbQhLcEo+Mlo4U9ub1BZD6l4KzzGDNWzdH7hpVt3jUPCJ2ox/uj6ywfhXYCkKdP3l62NkCcWznygaxS8fBw02PFeyzEOR0odB7CPik0Gtmh1GcfXY6NvSQnUnbLBGsf3gK7JUGHEtgst+a3CS6/ohTZG4wGWdCrY0DE9agAvGtNnKse/XU6rsfJT7lWoW12DsTQvIOzRRSGShOjwLcD9ue4vUOM7QnY+uz7KVMiDfsAXkOuSMDBEK7z/plCwTZgv83EhZIWZOrmNNUdQerB5Xhl1RQYK1Qfewau4AgegOFa6yc9SIEJPijIfYT+7Ky0HVUzXWZJsD/JqC4qxn3S4vvamEVy1S8AAyl5qU43LqSRkU/du9v9Q9Aws8zLmdRY9nfT84L1RTuAtYs0Wg9GPcv0Kqgj MgyIg61D LuGdmCf5xhnkoh+wOLOWeI72rN9v88Eg+y+cB/T9Hs+zMpahiVPkJ+Wj2qRz9TFAzm0v36PomLqfCj3t3LFZSynSbbXAz3wbR7EU8RfbDpQYJl/NIdx+2J/tVQnMwqaIzbZLFjK9/1iq+ytF406PkSZcdCPT53MKa7E8xgK+r8nc3H/g/Ea8ZZtr0NE7ZhmbvEq+gRusqBAd9AfoC46rcjg0CRpWFTllbVeKlgRtgmmibgpKJi/V2LKP/uUCcmEjZU/SWEkiWSLaILPt161kmlBl0BaujTRzjpxBLw0RBod/0T3XO25yZN706AdwNmks/AQ4W2gq9b7gdxPmkJTfU2spw+A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Clean up the code by changing the munmap operation to use a structure for the accounting and munmap variables. Since remove_mt() is only called in one location and the contents will be reduced to almost nothing. The remains of the function can be added to vms_complete_munmap_vmas(). Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes Reviewed-by: Suren Baghdasaryan --- mm/vma.c | 79 ++++++++++++++++++++++++++++---------------------------- mm/vma.h | 6 +++++ 2 files changed, 46 insertions(+), 39 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index 9495230df3c3..816736c4f82e 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -273,30 +273,6 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, return __split_vma(vmi, vma, addr, new_below); } -/* - * Ok - we have the memory areas we should free on a maple tree so release them, - * and do the vma updates. - * - * Called with the mm semaphore held. - */ -static inline void remove_mt(struct mm_struct *mm, struct ma_state *mas) -{ - unsigned long nr_accounted = 0; - struct vm_area_struct *vma; - - /* Update high watermark before we lower total_vm */ - update_hiwater_vm(mm); - mas_for_each(mas, vma, ULONG_MAX) { - long nrpages = vma_pages(vma); - - if (vma->vm_flags & VM_ACCOUNT) - nr_accounted += nrpages; - vm_stat_account(mm, vma->vm_flags, -nrpages); - remove_vma(vma, false); - } - vm_unacct_memory(nr_accounted); -} - /* * init_vma_prep() - Initializer wrapper for vma_prepare struct * @vp: The vma_prepare struct @@ -388,7 +364,8 @@ static inline void init_vma_munmap(struct vma_munmap_struct *vms, vms->unlock = unlock; vms->uf = uf; vms->vma_count = 0; - vms->nr_pages = vms->locked_vm = 0; + vms->nr_pages = vms->locked_vm = vms->nr_accounted = 0; + vms->exec_vm = vms->stack_vm = vms->data_vm = 0; } /* @@ -715,7 +692,7 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) * @vms: The vma munmap struct * @mas_detach: The maple state of the detached vmas * - * This updates the mm_struct, unmaps the region, frees the resources + * This function updates the mm_struct, unmaps the region, frees the resources * used for the munmap() and may downgrade the lock - if requested. Everything * needed to be done once the vma maple tree is updated. */ @@ -723,7 +700,7 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { - struct vm_area_struct *prev, *next; + struct vm_area_struct *vma; struct mm_struct *mm; mm = vms->mm; @@ -732,21 +709,26 @@ static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, if (vms->unlock) mmap_write_downgrade(mm); - prev = vma_iter_prev_range(vms->vmi); - next = vma_next(vms->vmi); - if (next) - vma_iter_prev_range(vms->vmi); - /* * We can free page tables without write-locking mmap_lock because VMAs * were isolated before we downgraded mmap_lock. */ mas_set(mas_detach, 1); - unmap_region(mm, mas_detach, vms->vma, prev, next, vms->start, vms->end, - vms->vma_count, !vms->unlock); - /* Statistics and freeing VMAs */ + unmap_region(mm, mas_detach, vms->vma, vms->prev, vms->next, + vms->start, vms->end, vms->vma_count, !vms->unlock); + /* Update high watermark before we lower total_vm */ + update_hiwater_vm(mm); + /* Stat accounting */ + WRITE_ONCE(mm->total_vm, READ_ONCE(mm->total_vm) - vms->nr_pages); + mm->exec_vm -= vms->exec_vm; + mm->stack_vm -= vms->stack_vm; + mm->data_vm -= vms->data_vm; + /* Remove and clean up vmas */ mas_set(mas_detach, 0); - remove_mt(mm, mas_detach); + mas_for_each(mas_detach, vma, ULONG_MAX) + remove_vma(vma, false); + + vm_unacct_memory(vms->nr_accounted); validate_mm(mm); if (vms->unlock) mmap_read_unlock(mm); @@ -794,13 +776,14 @@ static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, if (error) goto start_split_failed; } + vms->prev = vma_prev(vms->vmi); /* * Detach a range of VMAs from the mm. Using next as a temp variable as * it is always overwritten. */ - next = vms->vma; - do { + for_each_vma_range(*(vms->vmi), next, vms->end) { + long nrpages; /* Does it split the end? */ if (next->vm_end > vms->end) { error = __split_vma(vms->vmi, next, vms->end, 0); @@ -813,6 +796,22 @@ static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, if (error) goto munmap_gather_failed; vma_mark_detached(next, true); + nrpages = vma_pages(next); + + vms->nr_pages += nrpages; + if (next->vm_flags & VM_LOCKED) + vms->locked_vm += nrpages; + + if (next->vm_flags & VM_ACCOUNT) + vms->nr_accounted += nrpages; + + if (is_exec_mapping(next->vm_flags)) + vms->exec_vm += nrpages; + else if (is_stack_mapping(next->vm_flags)) + vms->stack_vm += nrpages; + else if (is_data_mapping(next->vm_flags)) + vms->data_vm += nrpages; + if (next->vm_flags & VM_LOCKED) vms->locked_vm += vma_pages(next); @@ -836,7 +835,9 @@ static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, BUG_ON(next->vm_start < vms->start); BUG_ON(next->vm_start > vms->end); #endif - } for_each_vma_range(*(vms->vmi), next, vms->end); + } + + vms->next = vma_next(vms->vmi); #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) /* Make sure no VMAs are about to be lost. */ diff --git a/mm/vma.h b/mm/vma.h index f65c739cbd00..7ba0d71b50ca 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -28,12 +28,18 @@ struct vma_munmap_struct { struct vma_iterator *vmi; struct mm_struct *mm; struct vm_area_struct *vma; /* The first vma to munmap */ + struct vm_area_struct *prev; /* vma before the munmap area */ + struct vm_area_struct *next; /* vma after the munmap area */ struct list_head *uf; /* Userfaultfd list_head */ unsigned long start; /* Aligned start addr (inclusive) */ unsigned long end; /* Aligned end addr (exclusive) */ int vma_count; /* Number of vmas that will be removed */ unsigned long nr_pages; /* Number of pages being removed */ unsigned long locked_vm; /* Number of locked pages */ + unsigned long nr_accounted; /* Number of VM_ACCOUNT pages */ + unsigned long exec_vm; + unsigned long stack_vm; + unsigned long data_vm; bool unlock; /* Unlock after the munmap */ }; From patchwork Fri Aug 16 11:13:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF29CC3DA4A for ; Fri, 16 Aug 2024 11:14:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D3926B0168; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 73ADA6B0163; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EEEE8D0070; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 28E486B015E for ; Fri, 16 Aug 2024 07:14:37 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id DCD5B81AE7 for ; Fri, 16 Aug 2024 11:14:36 +0000 (UTC) X-FDA: 82457850552.10.9860558 Received: from mout.web.de (mout.web.de [212.227.17.11]) by imf19.hostedemail.com (Postfix) with ESMTP id D73E41A0002 for ; Fri, 16 Aug 2024 11:14:34 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=W7Tn0jRd; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf19.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806792; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZrQQQZT7ARQFRs0tgv1kX7XArulslu7aLmdVZtTsOAs=; b=CQ1rhis2d9vZJfNi1/Iq0VBeGQSa67ziqjlCH5RZcfEJOEFm+SQqlmSm7YgmfxIEIvbJCW WZ9u3dgLFtzklDKUY3T7t3+sS/ecE7i24jIN9RV226v5GUoASIO3zrCHID3Mpfpx0PKooe Jq5XFZp2Fnh8j2xDLELs5IJOl8rBXaI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806792; a=rsa-sha256; cv=none; b=RxVZ3odKknuy3KsbGKj7SlXLLgOfGyJ0bPuE8yi3N4Cmz4VTw3NI4WldAjIzKjmETWsB9d ApuRwCprzlv59oxA+0yXPCwkpy1xN5B+2b/mQnJc1Tl2okOr4fKfeyLIIPabQzDTR2+NTE fEUk1l4+r6anYkxKQSPb0rKBx8dgmeY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=W7Tn0jRd; dmarc=pass (policy=quarantine) header.from=web.de; spf=pass (imf19.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806867; x=1724411667; i=spasswolf@web.de; bh=ZrQQQZT7ARQFRs0tgv1kX7XArulslu7aLmdVZtTsOAs=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=W7Tn0jRd5zBYzYAbHIabP66dwywTX/boxQILpw6V8lqFe3+nJkS2Wi2xzo34Azh6 R78dHG+LAbrdfPDE9KQBWTVmpBWKV7VtbTJBsMNqbb8lbtRI1eVv9nSkhzL3aOpiN Pnj/gfq9OJkl26gulZN16B5V2S6USoSW6zmGKAJw9TXfTVGXAAXqowFi+YcNHs8a6 XISXMn6AbT/OoKEtz+f2ShBnNs1IK52zh1BTDaF2Y9Lkn9M++0GpioYDv7xmrliDC m4W7H/lTAPB8vjcGN0I14bsuNArJ81ZZZcPhaMIrWW5yNQI5ADh4jU0yGoktQvhuJ Qcs7cjh9kHclEw9Gxg== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MnpKq-1rprmT2gsE-00apSa; Fri, 16 Aug 2024 13:14:27 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , Lorenzo Stoakes Subject: [PATCH v5.1 07/19] mm/mmap: Extract validate_mm() from vma_complete() Date: Fri, 16 Aug 2024 13:13:48 +0200 Message-ID: <20240816111405.11793-8-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:PeHNp6vd5g1Yuna7MwxF2eluiE5lGDtciQJQOfuIb655M/kSqfx rpu/JA/CNPZ80YnrYGfyMRECVf0kWQAzgsq1VZMIfL7snPdj6pSTkY/z9xT9wO/mQwSfpU3 oPEVAcFZyj4JnbU+/81JPz/dpnHrr72PWZcmKYYxhk4lBORwLEP9oLHS6k3c+3OukPKCoa8 TZO6Z9/DCccnVx7rEmaKA== UI-OutboundReport: notjunk:1;M01:P0:NAqq5pnOTuo=;FJgYYPfe70gjTOnVZLDmq2sgskM eFjDI3YQ30p3NBtcBus+7MAQ110HGDd7L6JI//poo0vtKQl2j83FqbW7GBFXuxvOTQFV8YJ8Y eenAkNojKhVm1B24OYAvHYryZqP3mQ7XxCpJPaWP0E/pSRhSRng3RuYeWLYKO3WkY1+hvRCZd pPKkDhSf7UMrR79hDL0G9bOC0Aj/YTIwIC7j6Evhh1VnsOMQ2bJhDkgSBYJ7Tq29ViozX/tNT x1VWHrd0XD1mbm11Hb4cby6sCcAT7FuVOhq28+Lkk63FBz8vOev0YI47lz6xLHLhE7AttRYkw ODmEdJWyO2PiXAdTn9ue7PWpJ06SqECKj7xiaQAVyIDiJ6+xRwXn6kRW9KxhLjRlWjc4Fwkeh B/J51S+tpaqW5+bfDSFoJUBbMoWJb1FdFSp6A1opGntqHbKystc3vWxdYPaRwGgIkV37GuiYm rNMF32g6/XPM56IrcC3a7ECgsudgME9gVd9LInmnn4Q1HRJmax/wLkXObPocXqHRGRr5ObJ0A MI+srh7vlbTPsy/Weq8q8Xt9lYWHBRkc/4TP5EajiuZNsNULMdwQl4uW/RwFZ//Gfn4Xxzp7K yYuKsB0F8ra27xyFMPBBTDR18jgLkmyMX+Y9jbO7R47C12O62vsHOCgihwiIQZMELnGCgYKRi QH/zBsi1SguQb4661LrWttUmv3N9CJto0bmsBb0gxz0nMEeJ/UpBfD6t1Oi9aEq+FUFitXPwX pQMc6WfYmF7w2NB+q/4NRJbGubU6zpQl26uZjCQAz+35vHANrBy7E0eYf7lsrQsu8+cunAEBV BZmlCQ9ofOm4jQpu2sfsPSeA== X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D73E41A0002 X-Stat-Signature: kax58xngqsp6f4k3aopu7kgpjnfbxpsa X-Rspam-User: X-HE-Tag: 1723806874-825334 X-HE-Meta: U2FsdGVkX1+rNDiIA2kyVKRcVnriYm81GCzldpIXg6ESszv1pd+86IxPaA9s9SyZJijlxUsSUvkarjY0jrvH9us/shC+wOUdP1zPW5eKTVNdmE2h3H2OspfuEXyxACNTdxEfEbmzq/4a/no6y79jjPZBZWa2+FKbiIKsqmVh1vf0ETf4ckdhkZgMHbC2C9Hquq1rvDgACXGVrxxVHho2E4f3lSBbN1WwqpbMrDio4oyNjHRyrcHsEZJ5ldTvmoyhV20sqZ5/3odMqm6hs6DFbZhjdDb/BRiYNbiD6/RnQ7HrgPZejbuV6xgq8sADl+kjsC7l7vaikVUWzK6GhMCETa2pSlM+5Yt458VuKnxKDSsBk4Ql3vVrXCGhBTOnKsYBfXWc8p0mUGccjt+rTLt/GvvMolZqfIpXv92W1VqqyrsMa0ooFAHF4iIwCFTCP6T9pbLcE+6YjbypF9DpjJohlspgePKkwy6Fr5qfNKgbHPUXRrrebg5tdQWv8irDKQy7/qxxsvvKWL1Gds871xTUJVlIs3ZN3E+Q5uDirgn5TiBY8EG2XIgcpnBsjSk0WxR3pPP0ZdqX1PV/uxk2+B1pdkOBtJGZQOx0myXN2yrib4dLui+koxNsqQuCrc7krvCN9LEOtxz0Hrebff18Fr4psHwDk6IAAUz+3LXRt7eN9lrLavu9cGrfSlZeFYDxKngY20esBEtpg9AO6KHywW3FiRNFo6shgMI8BEq+C1sz/YfLjECswm9kqsztV1pqn2FpRvAaBHYpMbV2i3H2+5iNTn/gOz6qFdUKNKrVonarmFaYmNz5Jmt3LMiET5bH2dR5SJrfWN9mrhq4plI+h3KCTcWEAyFAobV+aR4wjY4362MJxHABKbwx0L6liO4KiWfPKmPUoPbgTkV24W9SBPc6sE32VBNLjiTMzWvVOoOhM5xgGrTPst18mP8sxgNjVeA2+UquYqCICZTkx030mad F+wTY5wL 295Az/H0NbZIJGz2YFgMiXpr9x2eR3BVZiiYSrV1MbtqC72TPOetcnXTkFyIk6uqk4cnzjstuaHL4ZmEDfYbDiFUpKAXvoCtcthIhjoATyR2Ub2+kOkPc1gTVqCShjLk3J1Z6SFMRMhIpQyfKuxM73cRbID3whKEHZMurmaVfj+pa7zjPY/31g5NHCrrh3VW54OMudJB0UmEX2MDC8fvY5VT+en7gIj8ENQT9SqMYYdF70iYtAS8RgqAWm+g0oEpkhBVWJfEA1L9ZTOZFRxMBWkLLCOOLcs7KghYnKDZbJB+hdIREKr4aXxzSF4bCZXjtNnbf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: vma_complete() will need to be called during an unsafe time to call validate_mm(). Extract the call in all places now so that only one location can be modified in the next change. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/mmap.c | 1 + mm/vma.c | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 1da0e8325b33..6aed0b769cc4 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1805,6 +1805,7 @@ static int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma, vma_iter_store(vmi, vma); vma_complete(&vp, vmi, mm); + validate_mm(mm); khugepaged_enter_vma(vma, flags); goto out; } diff --git a/mm/vma.c b/mm/vma.c index 816736c4f82e..e4677364082b 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -242,6 +242,7 @@ static int __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, /* vma_complete stores the new vma */ vma_complete(&vp, vmi, vma->vm_mm); + validate_mm(vma->vm_mm); /* Success. */ if (new_below) @@ -548,6 +549,7 @@ int vma_expand(struct vma_iterator *vmi, struct vm_area_struct *vma, vma_iter_store(vmi, vma); vma_complete(&vp, vmi, vma->vm_mm); + validate_mm(vma->vm_mm); return 0; nomem: @@ -589,6 +591,7 @@ int vma_shrink(struct vma_iterator *vmi, struct vm_area_struct *vma, vma_iter_clear(vmi); vma_set_range(vma, start, end, pgoff); vma_complete(&vp, vmi, vma->vm_mm); + validate_mm(vma->vm_mm); return 0; } @@ -668,7 +671,6 @@ void vma_complete(struct vma_prepare *vp, } if (vp->insert && vp->file) uprobe_mmap(vp->insert); - validate_mm(mm); } /* @@ -1196,6 +1198,7 @@ static struct vm_area_struct } vma_complete(&vp, vmi, mm); + validate_mm(mm); khugepaged_enter_vma(res, vm_flags); return res; From patchwork Fri Aug 16 11:13:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1AB2BC531DC for ; Fri, 16 Aug 2024 11:14:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE0E66B0163; Fri, 16 Aug 2024 07:14:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA3826B0167; Fri, 16 Aug 2024 07:14:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 894806B0169; Fri, 16 Aug 2024 07:14:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5F9EE6B0163 for ; Fri, 16 Aug 2024 07:14:38 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1DA6F161ADC for ; Fri, 16 Aug 2024 11:14:38 +0000 (UTC) X-FDA: 82457850636.24.741D1BB Received: from mout.web.de (mout.web.de [212.227.17.11]) by imf14.hostedemail.com (Postfix) with ESMTP id 1DCA3100013 for ; Fri, 16 Aug 2024 11:14:35 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=QP7zv1ZA; spf=pass (imf14.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806803; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N4bRnjjY8T58BB1QJ9rqpGkDs7lyqxXGca+v1fWnuPk=; b=e1dwOvZbBBBouVPv0ujw5sbY/ASyEID3hCPlBZvq8ylV1Dn8SK3yCR/yN9i/6IzYwxoOGu 1wgqsIp7I5gEAq5f4LvF9FpB7LrBJ8tw9vpFsHXTTA7iT3zmOkvyr+ge9FNmkRgNC1XiBO h3vowyzvSAzywfUZ91Say5ku/2IkFlM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806803; a=rsa-sha256; cv=none; b=XRgh9IV10iZ35JrkrnUuMA49i10HSN6hA0N+INnyT+FfeE8kLOvuqgxu0AaU8fgjzdU6+Y ZYLkdIrAhlr/By7hT6xxxLnkBg2BrqQMulHol9+Bdjtc2LqMD94JhmVL+r1/NY+yucokvc v471XE832G+JNO5dxRENduCUPE060kg= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=QP7zv1ZA; spf=pass (imf14.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806868; x=1724411668; i=spasswolf@web.de; bh=N4bRnjjY8T58BB1QJ9rqpGkDs7lyqxXGca+v1fWnuPk=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=QP7zv1ZAEuhAIjBwUYNku0TL5L13CaL0eizszYyTdLpa7TJjMDv9oQR5icA0cCvr FTyKxgOf+sU0gXe2sS3xIsMHFdXfLVc0V7bf5pHIcFdPU7MW5BsMbp+MjCIHZiHUm BiYKJGohUJPRgukF430bogFOYxojXR4WQQjdkayk3F7Rkx2FZp+wYjauZoID5BwJ5 pWL0poZl59A/3CeraUguyFtviI4Zy7J8yGtgCbi4Yjoez/n/YpMhxdUxkMPGBsPV+ KPVUXlqCYaP51P9MxXi7mV0Zq1j5QxcX1lHvpT0aUjGutRUZy6P75exwsMR62ofVX tXOZVoj/AoA0PN/5tQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1N4NDQ-1sD2R82wGk-00xraG; Fri, 16 Aug 2024 13:14:28 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 08/19] mm/mmap: Inline munmap operation in mmap_region() Date: Fri, 16 Aug 2024 13:13:49 +0200 Message-ID: <20240816111405.11793-9-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:TPOfyyR3SiBeEM2qoJMPq/2uyQCzoa7BZ7yudOR9h8xakQoYQxi nZjrtXsjc2GHiHZ/4WD6G7wkm6Oh4Svn3lFz4TwFYAfh7WjpliuGes9rLDQsoz71PC64R2M 45+06O85GCxZcVTrXS29Nonbl56Kw+G9lGARtCS5Zeu01GaVBWJDA06ODZVmloyClBQLqxI XyexqvWQjnxNzDLICGFGA== UI-OutboundReport: notjunk:1;M01:P0:TzksNjmqQ6I=;PWZMpXnb4uhMdV3IePNblIScYmc /C7XuJqhxpoi6EmQK6PVMxQbazUXZR5u25r2mxbhulOkCw5+thEi+k3vPqHr5ezGsbtxLyCWz 01+wGkW2VUmaqaRZyFeq9Z/qaXYFWRrk22dLhCJwra93eFQIrzv5l82y7azuz4kwbSXY4NLo+ CxRkjSmnSbpK8IBV6GTUfa17tplhVxSm8eWOqmLyDtjmWG/pXFfutwXPMIUsDOrWGJX5wd7LT +cSVpnsBFj8D5QFgrNyD3s+k16PF6zlK10h/F9vyxOV84YH6xC+270OUgNhLPO9odws+HbLrf Kh0jvHyOn2qh/kzGbYDn4ZAQaihRm0hA1UUZlvn4CLqJugAh5OppCTSixDEiCVq2qC+GjbUgb h2RX5ZSzprmwlFVtvX1kQl7iZrPR7bvos7xEJxOfk6ePjxiy+cpGbXAJGMqGC8TthBhctlBEV Hvx9Oi3SrRa9GunezM3cjJGlIb414bVl9lCu+LbsORasD2cVGENZjUxbrnbDu2U5KwXIN42oT S6/Utr5OJEQm4HaDDbCvwh8vuHDjjCOPucl7X4Nlufmh+aIZOP5pB+MIoQ56x0T1xFf7SX0vr eAQJeMl5e85tm+7XjbPf5blqVCP0wpD0mQBcZEeNO5YCag4AfW3NmYzUNkFHgcdMf6LJHtkZd N7EIRcnhjuDbuHJSW+yyMEdwPRGj1baR8WpBtQ3owW3EjPKx9dAV81ejB1iS3zCnsfnkvtRnd dN+t8dYWecbeuTxKC0RxnfHKQOXvKxIbMvoSai9pFNbV0rafnE45y8/yNmNM4UUVgT2u5mJuQ TZE4BqC1mbtHtISnghh3k3hg== X-Rspamd-Queue-Id: 1DCA3100013 X-Stat-Signature: sy6fk8qunqffgn1hnbuuocweggfhpjmp X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723806875-936908 X-HE-Meta: U2FsdGVkX18VWBl49sVDan17mM6t2EDJV7i7Q5mUOGGqr6MIyR+Dbgc/3wLk9/DvACha4I6WFq3seVbB7vj5y0c3czcoJloFouRPdlJ2DvpgF/UAmu7IgxVDAYs4Pi1nz4+/IRMRH+SatRzBx8syz1cx4UwEqpmHLct1pxVUZGM34TufekNEm7BS+b5M+I/ZoyVb6SZlrQoX+oZD9GskwYfuabBUoUD9wVUj74A8rBsk2Al/nKDqB7wBzPpolUnfHg42jUpemn+OBhcIYFrOxjTms5TKiMQyB+MCNM+6w9/w04Fcp1YSCVDqol30IX96s34FYvt6QL6fKiPf2XCxViWdbORxLvSFKSo+fSGcm2OfQ+X1GC9Lv/fkLag+kFFuCWnoYOhuDoOQCN9ZqiQq9QxkvceHtlqYaXNbqQcDsvLOM8QI3sYVS9wsWCLVCguGjm0yWqRxyY68BT3VIfnABOSd6qsDrMrDdBo1INmJauT2QJaVyneZ+CM90cqLrIe5rsYqDpNKI7nus5Y6kv7p4oy9pdUhG2bMU8yPYKL3QGBtjBHV5U8ZL9vuE/f5FtbUk2+cMEODRyUvl6g4gQJhGsLdU5n0qtc1GvZQlnO0Qing/DG0XQFdG4kqvbOJcc/7LFWur+S5Rps9sg6sWUN+1GSs91nqOkaET/6PRaSA7tiQ//nwDI0JrOIswa+m7gDz9L1Gz5PLmPsVqNsvfY1LACHZHw9pdIjyVMh6keMIPSiCHGwzRSOtGMnlfvDnMXtT2zEaLD+OqMUBAm0aYRij+LvjSNg5iqqx8S2vjhkKoCOrBxh/Z+ojKzKXT6hw3dtMrxu34rE/nPhYZ2OEczc9O3e4mZivJw/lcQYFSuIrw9P/EwwV5HLXMe3QmjyRSclOpyKNNW0pSGmb3ZtZD9PivFD07nWuF6aNC+OKrto/EuCCV3/vRMIQ7ZF4+YXACOCIbpYz2HtVbZlXYDMTAUO BfORgFaN WRi6mFnvM7KDSbIEjprZ1WqEZZMhuj5sJSLR2icozqZMjqgMGNojR8DPTVr8IFA+BXtv81NPuxGhJErOfPaMfolLWAR+7G17EgzmnUTAPCczme1ZjjICK6i18KCFWqijwp0PS4BHYbluRZBMbGY45XgstF+7c53N2+9Iri096fV3WGAdnta6bSbmZ/g0Hmd4Ehj/Xc3b7F+N0B4rMkfltdHHSQQWMHGePTBMUufpaVJKwCTJvf0+ohLTLhyR13YXG4NaEGpvPvv8ICoJfhFT8iguQZOfDH742QjtkwXIr9sRfSiKXb6PLBEycWHeMRCwIz0nO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: mmap_region is already passed sanitized addr and len, so change the call to do_vmi_munmap() to do_vmi_align_munmap() and inline the other checks. The inlining of the function and checks is an intermediate step in the series so future patches are easier to follow. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/mmap.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 6aed0b769cc4..6ed27cf12217 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1388,12 +1388,21 @@ unsigned long mmap_region(struct file *file, unsigned long addr, return -ENOMEM; } - /* Unmap any existing mapping in the area */ - error = do_vmi_munmap(&vmi, mm, addr, len, uf, false); - if (error == -EPERM) - return error; - else if (error) - return -ENOMEM; + + if (unlikely(!can_modify_mm(mm, addr, end))) + return -EPERM; + + /* arch_unmap() might do unmaps itself. */ + arch_unmap(mm, addr, end); + + /* Find the first overlapping VMA */ + vma = vma_find(&vmi, end); + if (vma) { + /* Unmap any existing mapping in the area */ + if (do_vmi_align_munmap(&vmi, vma, mm, addr, end, uf, false)) + return -ENOMEM; + vma = NULL; + } /* * Private writable mapping: check memory availability From patchwork Fri Aug 16 11:13:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D2BAC531DE for ; Fri, 16 Aug 2024 11:14:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C8D926B0167; Fri, 16 Aug 2024 07:14:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BEEF06B0169; Fri, 16 Aug 2024 07:14:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A42AD6B016B; Fri, 16 Aug 2024 07:14:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 77FD86B0167 for ; Fri, 16 Aug 2024 07:14:39 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3275EA9440 for ; Fri, 16 Aug 2024 11:14:39 +0000 (UTC) X-FDA: 82457850678.16.F2F9471 Received: from mout.web.de (mout.web.de [212.227.17.11]) by imf07.hostedemail.com (Postfix) with ESMTP id 426654001B for ; Fri, 16 Aug 2024 11:14:37 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=BNZQMBdl; spf=pass (imf07.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806840; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q0h6rMe/QGaT++JPp5Z3rkGRpCNRbqXYhv4xZ4c2KKA=; b=4WzoNQpI07tFYWhy6J79PJJ8PEekAS5GF5uIQi6DJMOiHPiN9NsqsaH1Q3JEVEVweGZIIV MLodzWXTAU5+oOIXLQAaEAk5NYy+AmJQaNIz/gAQI3bQn+M9fj8/k1uKfSHnmxoHNnEu1T BHLBITVZzE1s4IZYhssNohWHX3fgVYM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=BNZQMBdl; spf=pass (imf07.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.11 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806840; a=rsa-sha256; cv=none; b=nqJo2ow3PzE7m3aTZNMGl3dPG1eBh8SrfVDl3TOqb5yf74oadymG0Oe6Mdg1yLGXrVYjnR ivGqM7KlUNzHQYLVNC2qm9eEmUpxB5H1Beu6CJ2G5AvcTnSks0DtLU5J+j1mHwWDp0DeNw Ztxx5hZIUgr58wM8MTi6b61JjyrdMk0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806870; x=1724411670; i=spasswolf@web.de; bh=q0h6rMe/QGaT++JPp5Z3rkGRpCNRbqXYhv4xZ4c2KKA=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=BNZQMBdltYarlRDfr9l+xhmEOPgu9dPnz1Vy9AwBEuxr8kVyy5YMCQ/XXZ/sjkBp PL25ghqtjvKhQqHM80zU1tHv8j1h5eFoeEAByPSoy+pFfzXDQQnpVjIv7yk3d178B tnzjVme3+aY4X6TXNq6XFO4IKvTxM17KYhJnEdJU/6rJ1W8DtqVq9CErAIZUWrkY6 4qqHSmHCSds6Dv0YzEn5qN1LXOUVEA3dtdyZ8hB9A48pTtAhzvy59rB0R1K0iwpS1 KihTCbL98EqYRU3IkBUkB302Ng7HaTn378t240WUYkw2KXLII7EAI+dmiWtBr+0rA v9QJU+x3DP7ceUSeDA== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1N62yY-1s859h3aUy-00suJG; Fri, 16 Aug 2024 13:14:29 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 09/19] mm/mmap: Expand mmap_region() munmap call Date: Fri, 16 Aug 2024 13:13:50 +0200 Message-ID: <20240816111405.11793-10-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:3fFxHOvIpxuFtPf29BwBcYr+yqtYehqzJgvJRmGLNR5rOdc3yPh GnhJ5knCS7QqrGbxM4+N3sB1dK0m3tOiqQNX/oZaSkeGkyW2goQaQFwguClg7nXkljAuNrv VLOtsAsKtAbCw7uxsSOSChzIRpt92HTrS9e5onFPgvYbwKfjn19554eeeuPzWfd88tq5kSm 9afxwJHMcns5e3+Mm28rg== UI-OutboundReport: notjunk:1;M01:P0:tp495Uzsi2o=;GLD5Dcf/qUytGv/C6v/BITps8Us +roeUFaZ/z5RrHebMmZgfH+F8xHoZymY5ZqBlzcK1zjLUAuSZKY9+YVXq6QZsZpL7YisgJsEQ T5uRQaXAu3JnIdsvEx/up1cmL2khi0SOKpd+BBnXH16QOviyk91OZRIDm83WyP8kHPMZFqmmn rACVD0h7cWN4wy7NabEwIzIUI2D5J6K6ClK0ezWIlAz5UJFKDiWeYt0+YPP5ENQ8fsac9qOhj eEDiM+rmzmq24stZhvKp5hNB/iDLB/MLiYyjJRt4kcFi8FMW5eJL2nYvjs/WcyltmTIEnCKdV 1NrcqJyMkNUv1j3nXE15+6ZhrpbXpPPWRCrHxgS2l44r9EuYNCLYGj+rpOlnSBXqA1wgn63mm VIybrgzIGm1aMSgWFZ2yWTMeOznnGw2CtJC5VpJEFzX5EjEv+S1uiPlFHV+t9qURZsbtNn/kP OU+8G8zCPSNtaWzKhupwevc9+ljHktVJQG+4959tVHWgBJ6s+H3R8bDnL02KLvFTwrWQyGzAS fu5blI2j0srPMH5Hsep15VQNDxr7aCXHNOSwImB6aVusBvQRTEdsqRaAjPgONh+lshC8EmoeE Asgvcc3fn2NaLcfRr5Yzm8f4Fnha7+mUCvvqgbbCstRSCIdYSC/hEoqiCTW6uFjWgCcwirBVV nrq+BKyLIcuSiD7HLz8mZUAhJ97o2PNUJZF25BCTLdlYVN/sFWp3VGJktPvYlt/qRmoVmJo5V M7zNdEK7sBfw22z6iWfW/Ij6AwKssGFzwa3GGXM+8VA9SSGGpWk5nRl9FBVlj8YXBamJQVP5C PxfzF1Llmai/300S+RI/2QNA== X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 426654001B X-Stat-Signature: hfazscw13aow77k9a9gzaxcg4yy5swn3 X-HE-Tag: 1723806877-491331 X-HE-Meta: U2FsdGVkX1+JHruSn9tpg9ERseZxib07P4OgLUkj+vNpY6ligQouAaMJANcffNo/SgcxWBrDOzG7gmd5F9FfhVz74AY+GFjYWlkooxXKq9osN9m9uPMLdVUa/wBZ0pQFp4j0zslAbmyOrD1vzk82w9VvYMoklLrfEWlyGtVyPQYpH81K/5p6XWSTg1R6ph0TqfWlsC3CohiSsnDCLg9KnARHUHZFuoaDMpDvPz9L26s2xkVzXS0f2sZQHK95BGajDpCLZesKdjmTd83JbANGQoqndowkEAxgFG17saV+3vuSPY0GYqk0zo2aR+d2csF31rrOhYn3HOXVgicWpEkuqwdXhcxlHS9FXjmyG8vy1rw6LpxvRTP1YBWSZjhf3vhjI3a2R8CEgqnZMrvTmi82kHnQpULAZaeQjj2mzfAcx++ER0oOqCtdSO/bs+I3TxS91ZsOqqTUIucVllhJycJQ9S3CKfQVGInHUenwZBGcRZc6xAvTSz6sHiDiUdziHFj25iJrLWjy2rkAjLPRzy/G466eYpguCrTK7cOwh6EpI2dvwoG+ShX/p249d3x7CDXcbMqHU2DfkOEBRRHGrEE7nYGaahJRnVnzOxxhfAM52zUndwq5O/3vz6tYIf3UYkgACPV3LKpNppnbaM1LZXvVSgZGAJUhFV7aoaKI/w7c0itzD0MHMDAdTOStCrBjGnXd8v3H+7cwmPMnu2ulZE7euMrJZnhZe0AnvV0W0i/73eyyxHbCCyVf3g+1hOFcn5f3CwWC/u7kZ90C90cjFLSvZJpIaVsyacCQU2mKqMXB+QGXbrKLJnvETquqTzoNH00490dDimFGy3XmA3aoCtIkCdRhFBgyoA5SHO1Mcmz0CBeCG4ksVWy44i4MySJ8NWOp4ka0mf1w48kmd5nG/trHtskSmhCoNvWmBUvJyhwy5U4xhCKi5e7C4112SrZsIg3dGBnDFbKHSfGv6OcShnt 28eGVMll ogC+Yk1kcBiC28vgxQisXbLBywg03a9TB/y49QuPKdgC9Em+va6pdOxct2WTQbo7MdeKFn4Xd3aNdZeG95O+vQz6G1VGYF/uJYsH6qlJ3AVcQK6SBrGLv9NYNX9PTgn1j9xcGXpOVz/vU+ihOLPnv7dqHZPplQrRnHON2/485DY70urZ3ZNO5c1rcfB4yZqN3rlu4f0h9SBaXBYgpOpqRhUEptkwI+v3VbGRt/aDQ39c5PLa0r1e4HKNs62aETymGl7qEUe7ZZagPdGE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Open code the do_vmi_align_munmap() call so that it can be broken up later in the series. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/mmap.c | 26 ++++++++++++++++++++++---- 1 file changed, 22 insertions(+), 4 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 6ed27cf12217..56d0aaff8478 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1366,6 +1366,9 @@ unsigned long mmap_region(struct file *file, unsigned long addr, struct vm_area_struct *next, *prev, *merge; pgoff_t pglen = len >> PAGE_SHIFT; unsigned long charged = 0; + struct vma_munmap_struct vms; + struct ma_state mas_detach; + struct maple_tree mt_detach; unsigned long end = addr + len; unsigned long merge_start = addr, merge_end = end; bool writable_file_mapping = false; @@ -1398,10 +1401,27 @@ unsigned long mmap_region(struct file *file, unsigned long addr, /* Find the first overlapping VMA */ vma = vma_find(&vmi, end); if (vma) { - /* Unmap any existing mapping in the area */ - if (do_vmi_align_munmap(&vmi, vma, mm, addr, end, uf, false)) + mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(mt_detach); + mas_init(&mas_detach, &mt_detach, /* addr = */ 0); + init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); + /* Prepare to unmap any existing mapping in the area */ + if (vms_gather_munmap_vmas(&vms, &mas_detach)) + return -ENOMEM; + + /* Remove any existing mappings from the vma tree */ + if (vma_iter_clear_gfp(&vmi, addr, end, GFP_KERNEL)) return -ENOMEM; + + /* Unmap any existing mapping in the area */ + vms_complete_munmap_vmas(&vms, &mas_detach); + next = vms.next; + prev = vms.prev; + vma_prev(&vmi); vma = NULL; + } else { + next = vma_next(&vmi); + prev = vma_prev(&vmi); } /* @@ -1414,8 +1434,6 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vm_flags |= VM_ACCOUNT; } - next = vma_next(&vmi); - prev = vma_prev(&vmi); if (vm_flags & VM_SPECIAL) { if (prev) vma_iter_next_range(&vmi); From patchwork Fri Aug 16 11:13:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 830EDC531DC for ; Fri, 16 Aug 2024 11:14:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB2BF6B016B; Fri, 16 Aug 2024 07:14:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E3B7F6B016D; Fri, 16 Aug 2024 07:14:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3DB26B016E; Fri, 16 Aug 2024 07:14:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9E8D56B016B for ; Fri, 16 Aug 2024 07:14:40 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0F912161AD3 for ; Fri, 16 Aug 2024 11:14:40 +0000 (UTC) X-FDA: 82457850720.15.AB9B981 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf06.hostedemail.com (Postfix) with ESMTP id 16165180025 for ; Fri, 16 Aug 2024 11:14:37 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=KTIUKbqw; spf=pass (imf06.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806841; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HVP/JBjkTr9Kn9Gb734inJ0oeOamhPQyJTNaGFebizI=; b=4EvCGwcwqf+Cok8L/TkiE9oXEtYEyfFoxpgEJ3nr7vPxafSiiY137dmoXpmoVUthjlUV8Y LdHqmR7rAySHciDgRJk6MPjhOGie8SxC1q8UX/WUJm43lL0QdaOAra2sHiXWrSzv9iGfRP ImTNVs5EyVK1YPpWI5m9zueBmiNrfzs= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=KTIUKbqw; spf=pass (imf06.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806841; a=rsa-sha256; cv=none; b=CDmyp2B5UiiEcAjjcwaMzOrVGtBby14l1RtSP2UHxpsywcyswSoqcKnPkUjQ0/G6D6XTcB WnTXiYJ8xJhmGYE0xFKqneL4aoAevLbM93Ipv7WRWD2VpuQ1nGALoi5WPJoEBEnjquweVe mihiQO/03wc0jGQJjfjK+kGOhbgEPos= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806871; x=1724411671; i=spasswolf@web.de; bh=HVP/JBjkTr9Kn9Gb734inJ0oeOamhPQyJTNaGFebizI=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=KTIUKbqwITLr7s3ozUruWYzgNjxVdUriID/MyHfuFupfMx1YVebgTrsoY6dLeq4k OCX4U0zKfvsNLGQ2rJP6QltSstQ4NWtGZDfAKWXLD2ijVPhz8Yp2tra+fh/bcmR2n SjaW5PxmbincXKXI0CjAIu5AtU5OnD8AguuHy1Z5su4mPKm1xJhkTti85JcoImI8P OtsenG7/TEDcbiT1qZ1S3Mfnk4V5i71ho+DHjWSuihffRlXUHtAG+hb4PoXSQvw9v aCGctUAkjVwayc8ySZmsLWTfdMtNFdNrGCWX9u4BPOncDeXhRn5a8gE6lkUBdVKBE WH61SGftsH7BwAzUCQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MYcll-1siVpl3UGp-00KwGf; Fri, 16 Aug 2024 13:14:30 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" Subject: [PATCH v5.1 10/19] mm/mmap: Support vma == NULL in init_vma_munmap() Date: Fri, 16 Aug 2024 13:13:51 +0200 Message-ID: <20240816111405.11793-11-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:q6+PZLLMAQhWGIUUnFe+NK15H6pxq8ferPSFmbhVruB+Iqj34vi Vvig817bDMFiALU3jtN/o0wRK0UO1X5QqtBSqp+Iqio8LXi6y3r/oDwXXq3gx2BsHCnuh14 AH3rPst1zjnX+pOpVIkFQyAJwYAevi5S5yMmDiVojj2gHzh4HsT4BPDOqoGBXjYpzvjLcfN uEY1Eo3o4fHIWymmZivMQ== UI-OutboundReport: notjunk:1;M01:P0:tkq2ZyJLUUs=;NDKjhBHZLAb57n0MPvbpI3x+bmC GlJwSGtGlSmhH+wk8dboCb737ZESaJU11qZazsPwQL02aDo2gj6uoNqASE2fd5Q7qYBx/3fBX YYVZVL5dxrBbOTJ/4XNwWisEA7uQWpRyealJgCEabYWKQzQurC7FbEnZbebt0Kdms+zkl6Iq6 tIMVtVbvp2jweIlYej6fNdhJSRvYO4lWDsrsd11PzyqmnDj8gfG+48zGHIyf5GrI0SuCjp/Qa F4aVFJ6YderwAViV6tmOe4QyK2oNj8VD0sUZqG/d3cghCfHpWd9WZVpy2g+XiEVnyLG6AkMG/ tUGtex19WbBY4HH3Pjw3sJ3g18amYEJq3pa8X9L0z9FzIfGC6WK1WaYGSzKzcBM67lWcFcDNI UszMkBaaoIMk3gF/UEzGCa/5MTER4p6Q5O5EKovTzO2yWUz/jk4YizFGWVWmCL6elTISGSitw kijhC8P8STWE/Rq4s1ppo25ongtBW0trP+X/7fjfyhMORookvTg2j75x6F0U5ZNmxVgzctu/j YF/cmkuWSFIlxrGcmocR+F6cEDhKAdBtXJ2uY1sFCULoIxCWYHw4lrYS4vEa6hfXzBfp6AvnP lgLu4Bfzv00BwoOzmSFrotPuAvD0R2wSDBkmiN+hhdOmGdhXJU4jV+Tn0Ixw2nl4ujSDEbWxu 6TYA/k45Yzzhm/ZlTAV9D3iTZQ7UPO5NU8gplaNOMwxi049jiXn8hdgPdBU+2pjmw64od2Hb1 kOwOh1+f5JSo2mjBbkG7y5C07ptNIhZZphDvohTC3/fqdGt6GThMcac4JY/5eUyzwufcaDATp AF5Uqn0euHQj/wPWzjny+q/Q== X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 16165180025 X-Stat-Signature: 34gbw5j9eyt9r94tbrhpgb1n1dukmg71 X-HE-Tag: 1723806877-379285 X-HE-Meta: U2FsdGVkX1/Ygq9aiGF6c1A0JbwwZlqqiXIQjiExf4KIyhdtSbA5SrNNvx28MLeYrXs/0Oigfv+3Byq2lV2NF44UB2ExhGzwktqNJbaU3ddzMC4yyusTgW9ypBHIjkpQ5hbuX7eWyo8soqdfEFFQI+ScNhpEnAjQsNtmDKUJ531Kr4xhZ3KoOUco46o9BYoxeX50I3qRrOXGbEyVazHwsLfLDdJ/4VnDQL2NChBI9bKjo9ukmvWGDeQ3u69Hknd4+Ltcm/0JSbKv6L7zx1RS+MSyQm6wD48Mhq3Xa3vZE2OQ6eWJkhh3zaDVu1PZABuK6jK+Euz45al8846crw+Lr7/PTJPJ//blbqKEVv/BC5MRpLpr6Pyq99dsxbRpcrGeDR38j4EUCfS2OaBfC10XK0gtZwziHOvXFkKvEQzERC0tDhkAXeit6tgEGPi41/unyU2o/3WmWihWk9Tc4ax8e8ZNdUUdwIJd28S9uDTPmVw5QOllx37Df3TnT6yY2GygT7wX+uMCiUvqCsU30sDhHcCr0sALlZUyuDzSbVRG8SZQ1FFFxxF/E1a1idZ/MnH/q893hifRWzt9hzt10ficvXXUqDpi+woFUPsCwlWjpN+vBcS2HWWIcwXn+h0d4MxyKS5Y1fomAVI6dpjN1OSolIGUPMJTK8jzj58xZX14/RkR+zSKvpTGM1hN1GAq0Y66uSi7vuJFMWPGr5hRkVebt88FtcDUPvY0f42IWW7us9/jqHWba8mCdec2Jc/imgVC+pzvPBGRA0Thu9zPI3MpOTpt/FoUSoagO7G30nL9F7wuiUEJZNLPFQ+tY+6moJF8IaKkCqp/cH12eh/jE5VpOBj+nwROL/Aum18BCAVsLjd99zeX9bv1HRkaKRkU49T2AGJNn6C2fm9xSx+lN5b4D9RshuBmagK5HqzzDIwnLR42+98hE81FyKAH/W3tFrZM35XTGOnQR0vz7t+y045 nKvgCO+a 5RQOY32Aoip1LpyE970kSvziXpXN1d+ZBIPdbirZxBdEFs2dztQTVWU0oKunZyZE8yKZARdIh7kWbHMSrXGs8ZaWwBOuhDynOAtArnqmlPdi/8pOc45vxH5beEFGu78bdBJ7b9IX9wmg1SZbu4WyGV+xpNQrFrNzO7dbS8r4vrISdlZABAHTBztTnU1Es/jc4FOdqekxgoDZljK73qiokFpnUrR5B97NW+ix5cdmmLEm8bshYfSl7dBbI3bcQGLK4UiB1qCkr8y6wLdbyTtuJdp44PGs6udvx4AuufzzSjbFEuyRtk/srumDAiO6rFJ6qbEe0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Adding support for a NULL vma means the init_vma_munmap() can be initialized for a less error-prone process when calling vms_complete_munmap_vmas() later on. Signed-off-by: Liam R. Howlett --- mm/mmap.c | 5 +---- mm/vma.c | 26 ++++++++++++++++---------- mm/vma.h | 14 ++++++++++++++ 3 files changed, 31 insertions(+), 14 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 56d0aaff8478..16fd14b243f9 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1395,16 +1395,13 @@ unsigned long mmap_region(struct file *file, unsigned long addr, if (unlikely(!can_modify_mm(mm, addr, end))) return -EPERM; - /* arch_unmap() might do unmaps itself. */ - arch_unmap(mm, addr, end); - /* Find the first overlapping VMA */ vma = vma_find(&vmi, end); + init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); if (vma) { mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); mt_on_stack(mt_detach); mas_init(&mas_detach, &mt_detach, /* addr = */ 0); - init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); /* Prepare to unmap any existing mapping in the area */ if (vms_gather_munmap_vmas(&vms, &mas_detach)) return -ENOMEM; diff --git a/mm/vma.c b/mm/vma.c index e4677364082b..a5ca42b7161b 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -352,16 +352,22 @@ static void __vma_link_file(struct vm_area_struct *vma, * @uf: The userfaultfd list_head * @unlock: Unlock after the operation. Only unlocked on success */ -static inline void init_vma_munmap(struct vma_munmap_struct *vms, +void +init_vma_munmap(struct vma_munmap_struct *vms, struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long start, unsigned long end, struct list_head *uf, - bool unlock) + unsigned long start, unsigned long end, + struct list_head *uf, bool unlock) { vms->vmi = vmi; vms->vma = vma; - vms->mm = vma->vm_mm; - vms->start = start; - vms->end = end; + if (vma) { + vms->mm = vma->vm_mm; + vms->start = start; + vms->end = end; + } else { + vms->mm = NULL; + vms->start = vms->end = 0; + } vms->unlock = unlock; vms->uf = uf; vms->vma_count = 0; @@ -699,8 +705,8 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) * needed to be done once the vma maple tree is updated. */ -static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach) +void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach) { struct vm_area_struct *vma; struct mm_struct *mm; @@ -748,8 +754,8 @@ static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, * * Return: 0 on success */ -static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach) +int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach) { struct vm_area_struct *next = NULL; int error = -ENOMEM; diff --git a/mm/vma.h b/mm/vma.h index 7ba0d71b50ca..8b2401f93c74 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -63,6 +63,12 @@ void anon_vma_interval_tree_post_update_vma(struct vm_area_struct *vma); /* Required for do_brk_flags(). */ void vma_prepare(struct vma_prepare *vp); +/* Required for mmap_region() */ +void init_vma_munmap(struct vma_munmap_struct *vms, + struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long start, unsigned long end, + struct list_head *uf, bool unlock); + /* Required for do_brk_flags(). */ void init_vma_prep(struct vma_prepare *vp, struct vm_area_struct *vma); @@ -78,6 +84,14 @@ int vma_expand(struct vma_iterator *vmi, struct vm_area_struct *vma, int vma_shrink(struct vma_iterator *vmi, struct vm_area_struct *vma, unsigned long start, unsigned long end, pgoff_t pgoff); +/* Required for mmap_region() */ +void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach); + +/* Required for mmap_region() */ +int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach); + int do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, From patchwork Fri Aug 16 11:13:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79951C3DA4A for ; Fri, 16 Aug 2024 11:14:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3AED26B016E; Fri, 16 Aug 2024 07:14:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 122F06B016D; Fri, 16 Aug 2024 07:14:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D517A6B016F; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A3EF76B016E for ; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 58630161AE3 for ; Fri, 16 Aug 2024 11:14:41 +0000 (UTC) X-FDA: 82457850762.18.99326C5 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf04.hostedemail.com (Postfix) with ESMTP id 49DBC40011 for ; Fri, 16 Aug 2024 11:14:39 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="TuCpX/N8"; spf=pass (imf04.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806842; a=rsa-sha256; cv=none; b=bfx4LZ1IE4LeuYVPqiPYCz880KSKTT4HTF4gGvPlpOivsNxBozng2qBRhhbwuKnfYcW7bI ka6OexBlNJSSslWoqxnUbOp+sFl+kJubI/HeIQV+wxpoYtiIBtA+8DZNc/vGqUvd7gA3e0 vd68ZQtVPkS5o/z2HmhhYmey2G+5r50= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="TuCpX/N8"; spf=pass (imf04.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806842; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KqLpCASAg0v6fyS+T7RSQ7Led3ug2Wk1EwA5uf3qlqs=; b=Yh4qjLoYZFy0nSN6nLegxpnmK7wlie5q5P05tCAw54X7tXr0FDeH7TzeID0YZSr0pAQCeP y1YMlZ0U8Dtb/Vc48S4YscCBpWz5AV9oUVCt5PJTMkIo3j9yXERHiXdNp4eV5YqZlX1eMy sbq/spJ9MtcogNABfricdnMYszvybhs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806872; x=1724411672; i=spasswolf@web.de; bh=KqLpCASAg0v6fyS+T7RSQ7Led3ug2Wk1EwA5uf3qlqs=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=TuCpX/N8xsW0eKxPxeG6dJ6Y77L1OkDJFkJcUm8KItesKAZyR52AGlQEUMq5q6ys WXn7S2kGA6f8cFjk55BAjLFmHLJL1fBjRb6FMBUFDKPCOWqoGWP9ydRgzmx6dE1wG XiPtV/FFY7ha0urhLfQB3DoWxo3X9xvkb+jKLaCEaAq6iRHSRM9OuH3AIs6DdCsvX caCa6KOjN9N/VQhKMQ9ob7Qbnyx6LaiTb9bJJAtYvObM+9Z2d8hA4Rn8j63ZvFNOF W1lVB6TV5pTxL4rdwuBpzrMH6ccQzjwx982zxV+mkHPsSPtNEuRJo0Vl6DdquKywV 0U02jpPCuD23d3SXXQ== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1N4NDQ-1sD2R73bQU-00xraG; Fri, 16 Aug 2024 13:14:31 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 11/19] mm/mmap: Reposition vma iterator in mmap_region() Date: Fri, 16 Aug 2024 13:13:52 +0200 Message-ID: <20240816111405.11793-12-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:zz36G1dDzlpCZzGc4BqVfsIpgRKxbbe/K3KeQh0rlbU0Mr8YLuw KRmPeUju2SCl0xjMdC8oTSGzJLI4AyuQGa8nWVnr0TzHc7rQilOpyIbb7eeWbZiCfZPOlzY C0TXbu6GIHsqcTL3ZWoDPfsZLK1d9HLVwKlEFTn+qBGbvfFUdDGUTLHUVM11TQ5bvnqPaML my6q7cvd82jmcerhF5BMQ== UI-OutboundReport: notjunk:1;M01:P0:fpY6WEjyq+g=;SpgQJJgrtJwY8xZvCyMbgRARI35 CTPYiWacnMKnrSBypojaCzesJMxZ5Ny/w9Uq7bsQzXK2+GGz7/Z/STr/4N7M5EkxqvAaXuHvC 6agNfpdeNsNhiXJpVzbPutK/q93Ux9MjisCqkK3PyHXsmwJjr0VCmNs4DNjgdFuUilCvUAo0N fWuv17vMezhbrdcgADjCTmKNRuHHvGfhQxLNuBRqVuNi1c8fVSaDJSqVx6quaKlTq4tluaYUZ 23ynXSe3nW3GRGJfzhuc+8ae3ZfWdwlcKur5UQvL/NE+JeM34yMzfZOfAnvZt1RPtntqYsg2K k/QfUBKhxDG6kQP6U2fQ/k+Hz0KPUeXWzdpPYgjRbi033wRnxYyzYz2t0uxNRi8VWqjXjxmqo szMG7evtVEl4nYUJXK4UuTtYOcGhzfG6ovNnOEMm1vQ8DYmBaAaCFeIRIBhjdn58JzJjLfa6p IHUnR6P5VsYd2H+4gNni/v/vTwSizkH87kmeSD49YJuztSJd9GxE2SmTU0c++471qgHJ1PoLJ uJQqOncT8+h6OEOxEJvLCCeBZWeYmy5crS9coPsUQIOC0EGw6xIm9ENVl2fkBwDWbCW5dc4Gd R9lKHW2Y3wf2UtiYsiOAMFYYy9spXOayuJock2n+3XIinz16oS/C5NuIzx5418VXHapthlkeD 7foCQptAJPpEZ5ojiRjQJB4mmGN5Uh4NPNqbY/+of6WlJbSAAp7ymvETcXn4xSZyJ2GTH7ygh meQzNGeDH6a7u5DCZEOs3vLenyd9P3RbGeEkM8shaXtTrMSjajyFshiiQndHLeRUSaXP+pe3M qIGfesomTvLtCuhzCwX+9dtw== X-Stat-Signature: pj5ahmxe9mgghaxiepydkb4crcaw9f4u X-Rspamd-Queue-Id: 49DBC40011 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1723806879-263118 X-HE-Meta: U2FsdGVkX1+W5cvIPGPLAC1myPFihrcPoOaE/HkWUWmJEuWXGi1cslKQFEezIJp0ftbDa/VJbFQR7G7JcFyvyRbJwtcjEwwRj/T/kOuPLX4pyb9nmiFhYYaAtSrZ5V+z1POQpb3KhHG/F72/JVEOUtMdfFPijCQIT7UdPUB4wd9nhd7tcaWc4PnARPlEVu6i1PZD57wcfaOxX+h8rS/xkJjwkmTGm6H78KIVobhzm9W4iQGn8YqX+k5QaUH9qx6ZMeWhvNHa8bQ1gxlj9Gamwla0nV06yiNfymd2+ifw55bS34P+6ud5sEpvDf5JWhu+ojCMD8NLo+9Sf36+LXMLYiFCfkhAU3HZK/1kFKfgZrNFAwuyx3zK93ygcoe8Wxk76W6JGri3/akGq8p2hGWg8t2YOgvqjsl44hnJ/TTOJaDkvME30ObOVitKMnd+Xe88WUbvxdcFuk++2PvFsx5V8VdODWQswCVTfqN2f/O3DpDE2B995DUpEJg6RRROdvPof2k02q0LT9Hi1XxQrJrhP/kYNXYsvcu5TySSO4Vo2nQ5brwO59Mk+WXfMOYk2Uu17+SzXYnw+p+sZUooW/m1NREsQFDFBWLnckjYjrHtgGIlwH2yyiDfE0aj+h5YDeFsE8aT7pQfH2VrfJ59o/aUsfqGTC+Z+kr/1j18lO+pGiiLC3YUAi6/8wSe3N5sQRUD38iU0RgxNIHmOSB4iedV+KxwCdMig9DKjdWpwVM0zDyGCRTgRDp8r9dV0CyOFOtaBuLaiRmka7iTpII7+n1F6V4+kZC/uAGr2XMi1fWFlXBZfqULCjO82wqEMucbXHdK3A78Pp6qxahAwyVRwVbKBB4kRW6hLR2g3rCK15PFg9ohUKuzt/0giy8SUtGbHo9zE3vHaSFhBTY3qr9MkFFxtTsC0EDmYNexo9yad7HmiNWakd4dw5GssV7NrPtrG/J5O8B80MOHJ5ADgAlRxKA Mp1VTE7V tQRFNCMJJ+VWEJ6Ss4CI9Yn3rgTAjjlVX6KL6ZNrfOpV+RpaUjO1m9HDKIMQLE2wFAQnBv8/U3qv7CFZz/AfBEgDT6LStdUrys6OTrWG02ZtFSyJiczaDmcWreIXItfRrCWPx2i8CQbNPIppR1sQ7q/Uvo6bGYGuqaBlic0hOJU4jAgOsPp2T49uk2nom8NLuiE13TdZggq2/YkGvWW2qSuaW5EUun29+X9AXTE5q8vjJxVoru0Nwdm3WjzLbw8rkAtk09yHTSUtX7GlWDtW/EpwMBW4UnMsffp6BJ7/b3UQcywNuoMbrl6l7+D0k/aB7hIuL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of moving (or leaving) the vma iterator pointing at the previous vma, leave it pointing at the insert location. Pointing the vma iterator at the insert location allows for a cleaner walk of the vma tree for MAP_FIXED and the no expansion cases. The vma_prev() call in the case of merging the previous vma is equivalent to vma_iter_prev_range(), since the vma iterator will be pointing to the location just before the previous vma. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/mmap.c | 38 ++++++++++++++++++++++---------------- 1 file changed, 22 insertions(+), 16 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 16fd14b243f9..6720b55b47ed 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1404,21 +1404,22 @@ unsigned long mmap_region(struct file *file, unsigned long addr, mas_init(&mas_detach, &mt_detach, /* addr = */ 0); /* Prepare to unmap any existing mapping in the area */ if (vms_gather_munmap_vmas(&vms, &mas_detach)) - return -ENOMEM; + goto gather_failed; /* Remove any existing mappings from the vma tree */ if (vma_iter_clear_gfp(&vmi, addr, end, GFP_KERNEL)) - return -ENOMEM; + goto clear_tree_failed; /* Unmap any existing mapping in the area */ vms_complete_munmap_vmas(&vms, &mas_detach); next = vms.next; prev = vms.prev; - vma_prev(&vmi); vma = NULL; } else { next = vma_next(&vmi); prev = vma_prev(&vmi); + if (prev) + vma_iter_next_range(&vmi); } /* @@ -1431,11 +1432,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vm_flags |= VM_ACCOUNT; } - if (vm_flags & VM_SPECIAL) { - if (prev) - vma_iter_next_range(&vmi); + if (vm_flags & VM_SPECIAL) goto cannot_expand; - } /* Attempt to expand an old mapping */ /* Check next */ @@ -1456,19 +1454,21 @@ unsigned long mmap_region(struct file *file, unsigned long addr, merge_start = prev->vm_start; vma = prev; vm_pgoff = prev->vm_pgoff; - } else if (prev) { - vma_iter_next_range(&vmi); + vma_prev(&vmi); /* Equivalent to going to the previous range */ } - /* Actually expand, if possible */ - if (vma && - !vma_expand(&vmi, vma, merge_start, merge_end, vm_pgoff, next)) { - khugepaged_enter_vma(vma, vm_flags); - goto expanded; + if (vma) { + /* Actually expand, if possible */ + if (!vma_expand(&vmi, vma, merge_start, merge_end, vm_pgoff, next)) { + khugepaged_enter_vma(vma, vm_flags); + goto expanded; + } + + /* If the expand fails, then reposition the vma iterator */ + if (unlikely(vma == prev)) + vma_iter_set(&vmi, addr); } - if (vma == prev) - vma_iter_set(&vmi, addr); cannot_expand: /* @@ -1629,6 +1629,12 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vm_unacct_memory(charged); validate_mm(mm); return error; + +clear_tree_failed: + abort_munmap_vmas(&mas_detach); +gather_failed: + validate_mm(mm); + return -ENOMEM; } static int __vm_munmap(unsigned long start, size_t len, bool unlock) From patchwork Fri Aug 16 11:13:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17F38C531DE for ; Fri, 16 Aug 2024 11:14:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 124CC6B0173; Fri, 16 Aug 2024 07:14:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EFC5B6B016E; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C92626B0172; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9A29B6B016D for ; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4CA9481A7A for ; Fri, 16 Aug 2024 11:14:41 +0000 (UTC) X-FDA: 82457850762.14.AFC5489 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf28.hostedemail.com (Postfix) with ESMTP id 57A51C000A for ; Fri, 16 Aug 2024 11:14:39 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="gtijZp/r"; spf=pass (imf28.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806805; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=75LYP4e/f3KZVW22gX/HhYvcu5dNO1K9ehpMZYRgA08=; b=3cmvwHOuw4J0XEa3Urt4TbrciJEcwOnXSE4doFcSt7eyB+aOTwD7UB4XY0Jq41Q26HPPpX svZsB74vccmsP8Q0FxijvpoH/U72CGVHSt+v4wbxoCZqsCs7jfl63NS7RURVx9XzAKB6h4 A+zXaB5i9+Nr9H5AFLH8f0KJuBcVbow= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806805; a=rsa-sha256; cv=none; b=sY2B0r+k492h+0FYAP+H3tuNrjxK4fqmtB1cBwuKumKNm6YYmkEX4zmavc6/J7bIQi50n+ PHSLnP2/m5E5Infua3jkYi6cZy9ayuMJ12P2FSyIE/QeEGfK0uUxxvLuHurGSpwlpTevq2 Ng7mKIuHpFY8xC25/H4Kjx+dU2g+YfM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="gtijZp/r"; spf=pass (imf28.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806873; x=1724411673; i=spasswolf@web.de; bh=75LYP4e/f3KZVW22gX/HhYvcu5dNO1K9ehpMZYRgA08=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=gtijZp/rFtCnJ1o1PQUYebo2XlRds1eLxKf6Mx95uKNKUaLb6pOcQ6pYdti5x1WT DPnvHsrW2M6hKNk74IXRx2nEtwO8G5afY+pGqGrG2ueM/aspIFWDkhqk7s5YqTIWv MK906rspw0smKTxQG1/JtdlQhannPTRkpDzrDTTAXJPM4y84ChmQTFMt32Oqd51Q9 EdS9ihY4JknA/T1OhWa/THYCAKnKTwlArXrL517/DWJUM8F1flwyX/qp1CY/uo1Bm CyYuhuCfQoTn0aSGTmtIhOnHSIRgulY1nKtuKb7fNfmGi0+gQymX+R1oDrB4cQ/VS 43P2L6E/OIvpvEOPMA== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1M9qd5-1sZ7cL3LBl-00BcKV; Fri, 16 Aug 2024 13:14:32 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 12/19] mm/mmap: Track start and end of munmap in vma_munmap_struct Date: Fri, 16 Aug 2024 13:13:53 +0200 Message-ID: <20240816111405.11793-13-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:cbjrIoJH4StOfzxVC0jYOdFmhLzIj9iUOdmwhMv8XfbyJCJ1jZR m8w1W7pGwWq9jr8ErUhTvmF3eHtQgkmvMmMxE9tOnaY6uMCYoZ+eVq9e0myVBBYELu+09Fz dL4xEbKcgcX5oQNJgB7liSOz8RjThl8DdHr4wexsPI9oAm/+phdztagTKvXgFZKFBh6SV2c AlMz2WsyCWWssRMqUeiRA== UI-OutboundReport: notjunk:1;M01:P0:yNz7ixttSuw=;bJKS49+ZB3xXUwJ7tz9w0l1BtYu +3ktfsJIJg+2QZxau3/Tj0cgCwGX01o6/lpObmmDpPmUdczRHn3x4NwJtHh6NYDxX8zoo0zon ORvGiMg8KAy+qcTibVrXeRBamhdpN9D1BlaabB76p60Ks7K7CJ8huAwbeBR2QdwGmWYTc67iO 6BQ73NGcU4njcfbzzRup1ura5038Hmu6Bx9LPq9CYQy62EGoB1+TLQt+6LIBDNCS0J3nGsUhk M90+h+haevDMPtECgdC/Ee0kvnK6DyE2M9zenQOimKFsNiJDtiKdp35sIs7js0hshWH2/W5rX vcii3dSbmV/CEn97/EEu1hoNsRPEvyQeoQoA6EGMduW8NcsDPR1m5mA34x3YtL3KNJnnCp80g /vVxPOXxBNSBnNkXp/+EHBN3uDiuVlD9VDao7PdNeLMN0JzJWaMnwweM7NaAtiitLHdWtjICl cDpYwPs8gM2oSvkAIaJPXi70DvNXpaGuRDDduGav2TczmxYBoVMSiqXFXqw+mUtUC9exElHKq IB0wiJ5/P5kIFmyvduEDSJuw7YTy3UZVP9KXP20C9CDsR5huxG85xckAUGBQWi6bz02aq9fXu o/1XpkPgXSD80EZdHEWvnx1PpCIsSH4RxICE3vN85cOiJrijaswWhMTYvdAj0GwJAUXs9IqCe q0emqiYWkUZb1H3IMVZxw5qGeX0cSvHP9zNb5hnS7/JeMWvbwPQldlzucTbCFAfWGg2F1jGA3 nqPD3Mp4bV/PplVGv8h1sdjD6agUVzrZywmNowMQV79+Iit2QnxDKts0HddVAgZf6LDaw1MCf KCyNyerw5fYEulUEIDR9YSaQ== X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 57A51C000A X-Stat-Signature: 46jmbz6rnez5o7gczqbiwxz9j3w4gxxu X-HE-Tag: 1723806879-93242 X-HE-Meta: U2FsdGVkX1/1FetLBSMubEal8aaKs2n0nve/srhwuJB+UFxUEhZHPvF41BjIaYQ38afIvFae9g13b+014l9sUj83LJNDmALU/IRydvts6DQOY6tNhwKAVdAfk/wIfM5/5YKmBHpPMAzU4jdOWiEKweitLw5zltlgqCkRDy9SKUllbLuWvM/MeAzEMQBv1EWBXmpLqgLb45D7druIgoke0Bzz2GHsoLXbrz41+eF/20j7PbPSl2N2DvlerXPaFX70xXNL3yZv5cCaL+9qpuObKLhkc1+rn7jnrHy9dU7op+w2EDu7GAPTUic4tcv8Bzxt8x8sExmiBGOxZAk2qO/w9SxGc4/ol6DCVtUroAjn3OpH+bwZl6VhC367bBADbFKe/Bf2ylbQmMpf/H92l04nCYSw0+60TiGfsvpUgtKVdL7CcbyfhcUo+40xLYqkIJ3XU5e8NtuDrk+wpZ+0YuE5nDM0gnTqSecPWRi1ECeNyV77KH4penJWI4gQN590pWMSzMP1lhIadzjbv62pb8sARvY6BrUpVez9kznYe14d5Acpa8/phntVSTrvJcmStj8N5G4sOmXZDAhJILWHMMyIkZ8UnqH19o6c3H44Av2z1jQ4dolqZVrpzenK0lS5pbEdI7CUQsM9yRZtKRv2tYwdoScIyP/+DXLegdQIgTZvFZEC4BMPjfBFTBBttP2C01b4a42pxk1g1h0n4jc4ljsvUy6Ub4Kkts/CJN2N5ijWmttY0LKBbA8O+21r+uzwOF/yWkjSWv4+L774K9hMk6V3yNXuYzxblcvPFnej/rKZk++ACy1dQ+y94Mi4Iqc6VqteInTUCg2wNF/6Xitlnw+BWJacdRPt6s2LPfT6iq6AlvF2ntqS9WVMPFJohLUyQuZOqmdcEYDBiNTsBxx5BZzJ1j8+NuRJKA3j50qb2FesPKpL5syfRv3xSs3baFQqxxYYOJaFVPeMTx2oDKcgCVc kAcNrtU0 IVMQS656m5a11jI18K8Z38v+ne/BVKZ8zuXtAdsfXR6zttGZKlw7jpqIySR0h2FI9j6YsCJ17BnPVSiX7fMk7aGvGS35BKEdC80vs+W/4rY/KbLlybsYtrywl6SjM4LNsKHKLEnmuKDVYHUeHT4p63+/baASk2XKMmub8YoB+VKw52//tk7urZWr+28uAjkxo9fpyz89x5FCEyXPGpd7teNGl6JluIh3D8r4RWJTNVKZ+5+ank6a0oW07QU+N1/MtCOaX+J8dkqK3i0DMqflzix+4vHd97++YIxy3eFtINyCf5zAUwogaD7OSeFUXDQpgeChK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Set the start and end address for munmap when the prev and next are gathered. This is needed to avoid incorrect addresses being used during the vms_complete_munmap_vmas() function if the prev/next vma are expanded. Add a new helper vms_complete_pte_clear(), which is needed later and will avoid growing the argument list to unmap_region() beyond the 9 it already has. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/vma.c | 38 ++++++++++++++++++++++++++++++-------- mm/vma.h | 5 +++++ 2 files changed, 35 insertions(+), 8 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index a5ca42b7161b..e106d412c4c3 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -373,6 +373,8 @@ init_vma_munmap(struct vma_munmap_struct *vms, vms->vma_count = 0; vms->nr_pages = vms->locked_vm = vms->nr_accounted = 0; vms->exec_vm = vms->stack_vm = vms->data_vm = 0; + vms->unmap_start = FIRST_USER_ADDRESS; + vms->unmap_end = USER_PGTABLES_CEILING; } /* @@ -684,7 +686,7 @@ void vma_complete(struct vma_prepare *vp, * * Reattach any detached vmas and free up the maple tree used to track the vmas. */ -static inline void abort_munmap_vmas(struct ma_state *mas_detach) +void abort_munmap_vmas(struct ma_state *mas_detach) { struct vm_area_struct *vma; @@ -695,6 +697,28 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) __mt_destroy(mas_detach->tree); } +static void vms_complete_pte_clear(struct vma_munmap_struct *vms, + struct ma_state *mas_detach, bool mm_wr_locked) +{ + struct mmu_gather tlb; + + /* + * We can free page tables without write-locking mmap_lock because VMAs + * were isolated before we downgraded mmap_lock. + */ + mas_set(mas_detach, 1); + lru_add_drain(); + tlb_gather_mmu(&tlb, vms->mm); + update_hiwater_rss(vms->mm); + unmap_vmas(&tlb, mas_detach, vms->vma, vms->start, vms->end, + vms->vma_count, mm_wr_locked); + mas_set(mas_detach, 1); + /* start and end may be different if there is no prev or next vma. */ + free_pgtables(&tlb, mas_detach, vms->vma, vms->unmap_start, + vms->unmap_end, mm_wr_locked); + tlb_finish_mmu(&tlb); +} + /* * vms_complete_munmap_vmas() - Finish the munmap() operation * @vms: The vma munmap struct @@ -717,13 +741,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, if (vms->unlock) mmap_write_downgrade(mm); - /* - * We can free page tables without write-locking mmap_lock because VMAs - * were isolated before we downgraded mmap_lock. - */ - mas_set(mas_detach, 1); - unmap_region(mm, mas_detach, vms->vma, vms->prev, vms->next, - vms->start, vms->end, vms->vma_count, !vms->unlock); + vms_complete_pte_clear(vms, mas_detach, !vms->unlock); /* Update high watermark before we lower total_vm */ update_hiwater_vm(mm); /* Stat accounting */ @@ -785,6 +803,8 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, goto start_split_failed; } vms->prev = vma_prev(vms->vmi); + if (vms->prev) + vms->unmap_start = vms->prev->vm_end; /* * Detach a range of VMAs from the mm. Using next as a temp variable as @@ -846,6 +866,8 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, } vms->next = vma_next(vms->vmi); + if (vms->next) + vms->unmap_end = vms->next->vm_start; #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) /* Make sure no VMAs are about to be lost. */ diff --git a/mm/vma.h b/mm/vma.h index 8b2401f93c74..b857e7dc4bfe 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -33,6 +33,8 @@ struct vma_munmap_struct { struct list_head *uf; /* Userfaultfd list_head */ unsigned long start; /* Aligned start addr (inclusive) */ unsigned long end; /* Aligned end addr (exclusive) */ + unsigned long unmap_start; /* Unmap PTE start */ + unsigned long unmap_end; /* Unmap PTE end */ int vma_count; /* Number of vmas that will be removed */ unsigned long nr_pages; /* Number of pages being removed */ unsigned long locked_vm; /* Number of locked pages */ @@ -54,6 +56,9 @@ void validate_mm(struct mm_struct *mm); #define validate_mm(mm) do { } while (0) #endif +/* Required for mmap_region() */ +void abort_munmap_vmas(struct ma_state *mas_detach); + /* Required for expand_downwards(). */ void anon_vma_interval_tree_pre_update_vma(struct vm_area_struct *vma); From patchwork Fri Aug 16 11:13:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1DFCC531DC for ; Fri, 16 Aug 2024 11:15:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCDAC6B0177; Fri, 16 Aug 2024 07:14:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B52C78D0070; Fri, 16 Aug 2024 07:14:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 933B16B0177; Fri, 16 Aug 2024 07:14:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5F4656B0177 for ; Fri, 16 Aug 2024 07:14:43 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1DAB9C04FC for ; Fri, 16 Aug 2024 11:14:43 +0000 (UTC) X-FDA: 82457850846.05.E819C97 Received: from mout.web.de (mout.web.de [217.72.192.78]) by imf13.hostedemail.com (Postfix) with ESMTP id 30B6D20014 for ; Fri, 16 Aug 2024 11:14:40 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=PnWlvgWq; spf=pass (imf13.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806808; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/7yEM/06EBzG8T4jP9FKzjOed5lkzr+9lbfhTuSKD54=; b=UlXBYRZ+8vxuc/mtz7fnimcyHpi4XM+hnu3+ar++DewASkZq3Sg+9u5tJT6R4dMR4DqXkK eJNBZUiK3kEHphzY0KMQGpg71Ope2qQ/d3zdSVtEgG/9ESs4CKJl5NPnUlapiFa8SOEo6I fW/maiJEcc2gv7m1BvEtLWR+J+szPvo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806808; a=rsa-sha256; cv=none; b=gH8IfpdKyzOxHvoxaluwWlkEHaWvyPmOfvmXdq7L1aO9yNkLw36XUEndAE841X2y4Ikb/M vK4sqyVdtv31CSKhAZJGfYjtIQU0dZCSPK2VqMji6URBwCFiLQm2dXmMvnhWWvadNgCsih f1Un1zgAsxGilnKF8JBxrUuoalqZfls= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=PnWlvgWq; spf=pass (imf13.hostedemail.com: domain of spasswolf@web.de designates 217.72.192.78 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806874; x=1724411674; i=spasswolf@web.de; bh=/7yEM/06EBzG8T4jP9FKzjOed5lkzr+9lbfhTuSKD54=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=PnWlvgWq07rw5K2AbEW/W0YHB8+83pEqmsERFc8LTpYPSnQO8i5dBCClBupZILvI nKHQwo/AFeV2PU+bLnYTDKjxej3jlSfcDzypXbta/Gik8nBeUCaVNKEjXCBynFJOL OPI4SrrhQrhv6Frf9je2f+Bn+s/Y3aCogMQHw/+p8AfzHwHiCbSvrsb5G7aN7Y7si ein9Sy73F60/nCHqyHj27HrLYc8Cmh52Jyx0yzl4uVmnm/uYFfDD63/JlPQ/bn5Hz yJGo7Fn0OU8aUJUe1mIVTSvgRgMFFEMha4RJo0Q0IVnzArcj/PrHS+8EzwRQRNGiY Rq2szIHGOqxaInQABg== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1N0qr9-1sHjG03Qph-012U23; Fri, 16 Aug 2024 13:14:33 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 13/19] mm/mmap: Clean up unmap_region() argument list Date: Fri, 16 Aug 2024 13:13:54 +0200 Message-ID: <20240816111405.11793-14-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:jcyktmfYp2H8ZpYasihi+6y9QqrVM70RvOr4ii4E0k7G04jdQfr WRh/wc07K8NzAyi0df2eqblsJNtMNODFUvfx0e6ON3i+aUPh3lth9islkc/OC61HvgNrx1j rwhjBWEYxangRdzx7VNJCcfzL8YnfnreVppys89DZeALsqEsuEbG6hFOK20dXCS4kT+YZbz 8aOg+qnqVRU2DwJjE4l3w== UI-OutboundReport: notjunk:1;M01:P0:VHYw7/oy+O4=;fk6pGRSVvznfcKvi0qmAW8Ep6CK c5YlN2Zten+WKPKObYsf9C5Rm6XRw+TpsPsCsKoPEv0DGQK99PqaG7cSJZnaJFxEdbOsgW+D/ 1Gto6bNQtmjC2OAKM2rmGqo6eyd8T1Ik8DeSRhb6z/f/eMrddrolne26e82y6Kj56qAyOrGSc zg/Dkic8BqxoV4O67b3WNXhSO4+wHnauvRB7Zm+z1pwmGoNKlf+R9SAXI1gvYvTzb1Yhn6xjI Zm5M7s6XN0XrYMrMNcklkTWOq9Sm81li/QX0bTvZd5Y8WMdeLT7gAPPqs3EFCgtnfk1vaEQmo z41Y47J7zgqGIJeS02BHwtBDjEbpYkWtztxiYFkl1zDhyjiegvjsRuK9dq8Md3Jx6ddwXuK56 zoJb4GVg3jLHjDqA6gqqJtAQEFVLAqWyYZYJiCCGQhMRx9yTOc6vzBEZOd98Mqd8HwihboxcY uRxTnGTJBKfGG7kt12cHEAc+jSP5puYOKRNcHni3jRmRW/6NA/OOFlar/+M19A7Nx3YAMAq4j q1tAfYraN1Aogkbq7wBA+Qf9XENmJf3SHeLJCNmyfFVcZs2GPhE2aCUd14sAC11FAzIcBgFY3 wLCU5tz5+zZm+wXSpp1lx8cIYIF5a77fTqr3JZl/CHZd8+Q5MekCI9eH2Be+fYRAttIwx3UER kt5M09QDera6ZPDAt8s7SeQOqcjd7xDLdIkdjt1A9SuSFtdQmvCgn7VUhJOkiCJufl1Bs7Uhb SOmY5FmmPRsuWNyDTJeRPGioO5+17UX7WcFOhXxBgY6/XlfThiDbYqLpTEcRgpvsFlsbopcg/ sxnq39j0dwq7ATBKZUuegfKg== X-Rspamd-Queue-Id: 30B6D20014 X-Stat-Signature: fuww9zd39btn3rcq4un4e5ny7bt6r9my X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1723806880-978401 X-HE-Meta: U2FsdGVkX18phtN6y0+5M+aWjtuwPpLDikCdLLRkIsICVHN2qS1znVr+2NLqXLVy6hD2bHMc57Q5kvl/3sFwW7rPaY9N6keyPghejClQBykg7mPzr+UrAEGZRT59uoMiwkwSZrbdpgHHGbt7DwTq+l+hqNDxtdIMjjxDyP79zADANbXtWGScmyaJ8Klxvp4AKWu1oFCix0XZfPDk7AWOrAfXKDlko2Z9SCAk9Unzd0KVKBO/pMiFk2ze2pRXyC9GsKGvkZ6vUAicx2SLBHa2cL4L6Y3BDq0XWNDaAgVZE+jCVgGjnMf/oZESMihok2PeZbWYHJi8hE/CZKQnvvD3AFtvtPNRgb/BwCg7vzBxPHQGHE1qNl0YBc+nm1wUCa5stuqLsEAAu9oMdB1GftMQqCwpNbvctcrbULJIJGz0eAnfR1QOYutgN2iN/gUNXPoYe/xDLJGWoV37qa/neQDfVv/mhVqdZ0RDIN6Z+fdAIDYf5Irwm5D+3zVUV+VETu2GL3jXrL1Br1O88E/cMgEl8HX1HXmSksds8vkBg1z3XZ65MyQ9jKFo2Iiu7R8YmGl0MfPSGqUHGiQfJ57BOxBn3aX3JYF8IRPK3dZ6GphVl891IZsNANIjsJ/zAFU6ZX7CrnL6dhPfwjIQZrXa82j3Ry58f9lbuRVV3kJBXNlW+uCSO1ZjtZsCd8j51mKReaVafwQp+RFAY4KgKRDPh6HaCOgzUEXtgl2piXcutBD1d0rXRBBXiyTWl/BqmOoJaIlHgjeZ2zTKJ6ncL9RnbilFmBn4Uh9XLI2CujARfUp/Z5xt2j8Huu2huXbXX2QRfTVSt8hPwtOzYUN1YPx9UBzSvVvgjbLRxqugqnlPOnAM08/wsIw0x+haBsYe5qaFAsJmGq1BAXMIxEwiy59Sd2b/mwpkYdvtK02iFTClKtziMMIr82rddxjmXqO+aOqvMj2aEe4Ms/OkKDqrf6Kx95O YHBI06bU Bh0I9bTvXf2Jp7aLXH4bh1d341YAbAPoGf9uKeBIGaPgSvtfSmIXrYHikubX1zoLs2lcSVFazWDw4wEs+kYEG7N9grmehcDvjPrvy48WveKhY00/2AYwPneTmbOMnu79AYcgANLlTbBAvzNtqd6QZh8/TNXhDluZtvtACslKjYvjDYqLuZ8czw9n+BuL1GuY6zzDIns/eaGbPLfhv27MUNrQCHElndx0ODZdBh31zF5P5N9obV4Ml2aKPVc6W94FvinjzMpktBjafC+5ccuWiQxMhzNkF4oqQQdbgNHWOqI8TpSV68tuWkCctNlcEBmeUEZ4A X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: With the only caller to unmap_region() being the error path of mmap_region(), the argument list can be significantly reduced. There is also no need to forward declare the static function any longer. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/mmap.c | 3 +-- mm/vma.c | 17 ++++++++--------- mm/vma.h | 6 ++---- 3 files changed, 11 insertions(+), 15 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 6720b55b47ed..1f0b8dc5a089 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1617,8 +1617,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vma_iter_set(&vmi, vma->vm_end); /* Undo any partial mapping done by a device driver. */ - unmap_region(mm, &vmi.mas, vma, prev, next, vma->vm_start, - vma->vm_end, vma->vm_end, true); + unmap_region(&vmi.mas, vma, prev, next); } if (writable_file_mapping) mapping_unmap_writable(file->f_mapping); diff --git a/mm/vma.c b/mm/vma.c index e106d412c4c3..0244320d76ab 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -155,22 +155,21 @@ void remove_vma(struct vm_area_struct *vma, bool unreachable) * * Called with the mm semaphore held. */ -void unmap_region(struct mm_struct *mm, struct ma_state *mas, - struct vm_area_struct *vma, struct vm_area_struct *prev, - struct vm_area_struct *next, unsigned long start, - unsigned long end, unsigned long tree_end, bool mm_wr_locked) +void unmap_region(struct ma_state *mas, struct vm_area_struct *vma, + struct vm_area_struct *prev, struct vm_area_struct *next) { + struct mm_struct *mm = vma->vm_mm; struct mmu_gather tlb; - unsigned long mt_start = mas->index; lru_add_drain(); tlb_gather_mmu(&tlb, mm); update_hiwater_rss(mm); - unmap_vmas(&tlb, mas, vma, start, end, tree_end, mm_wr_locked); - mas_set(mas, mt_start); + unmap_vmas(&tlb, mas, vma, vma->vm_start, vma->vm_end, vma->vm_end, + /* mm_wr_locked = */ true); + mas_set(mas, vma->vm_end); free_pgtables(&tlb, mas, vma, prev ? prev->vm_end : FIRST_USER_ADDRESS, - next ? next->vm_start : USER_PGTABLES_CEILING, - mm_wr_locked); + next ? next->vm_start : USER_PGTABLES_CEILING, + /* mm_wr_locked = */ true); tlb_finish_mmu(&tlb); } diff --git a/mm/vma.h b/mm/vma.h index b857e7dc4bfe..2a2ca489e622 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -108,10 +108,8 @@ int do_vmi_munmap(struct vma_iterator *vmi, struct mm_struct *mm, void remove_vma(struct vm_area_struct *vma, bool unreachable); -void unmap_region(struct mm_struct *mm, struct ma_state *mas, - struct vm_area_struct *vma, struct vm_area_struct *prev, - struct vm_area_struct *next, unsigned long start, - unsigned long end, unsigned long tree_end, bool mm_wr_locked); +void unmap_region(struct ma_state *mas, struct vm_area_struct *vma, + struct vm_area_struct *prev, struct vm_area_struct *next); /* Required by mmap_region(). */ bool From patchwork Fri Aug 16 11:13:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766010 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F55AC531DE for ; Fri, 16 Aug 2024 11:15:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 27B246B0199; Fri, 16 Aug 2024 07:14:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 206596B01A6; Fri, 16 Aug 2024 07:14:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED7896B01A3; Fri, 16 Aug 2024 07:14:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B6FCA6B018E for ; Fri, 16 Aug 2024 07:14:44 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 67AE5141812 for ; Fri, 16 Aug 2024 11:14:44 +0000 (UTC) X-FDA: 82457850888.10.398A2E7 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf22.hostedemail.com (Postfix) with ESMTP id 58A30C0016 for ; Fri, 16 Aug 2024 11:14:42 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=CDoIzaxB; spf=pass (imf22.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806869; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k8XFZH7Bl7Qh7IxzWQ76db6VsLxfjJZK9cPC/vdDVvk=; b=s0gzzNLvXMft898MFXStkw6K6FImL6J2CU2xK7XQQu21vwCmCPQ1QlKZ1s2yVFPepFxPC8 1zc81wD0z9yn3wfJmobgnn9l1+XQfohBa4CfD7rZ4j97NRvLCpTP/7lesZ554OUdoBO7Ox cd0pHCGR+LXyjPvdtSY7e2Wmln/og2k= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=CDoIzaxB; spf=pass (imf22.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806869; a=rsa-sha256; cv=none; b=BD17ibU78f5Z/m2a954OabQZxLanbnQrbFSdjds8LMSQoVd6/vqJz8dLCDXaYQHtpUd+6Z vEmW+mF7YS5xVW3f4nS1kw057iUdexV8NA7ksRieJxLnBjWFQAT3V2WLeHko3IkTSgqvty eRfSnATe9+WNgI2JPRMK7Hfw8JcHubo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806875; x=1724411675; i=spasswolf@web.de; bh=k8XFZH7Bl7Qh7IxzWQ76db6VsLxfjJZK9cPC/vdDVvk=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=CDoIzaxBSOx+BFHAyOC09H0lfePAuo0eSTtQrpwyy/FUIlVBmq/i7Gs6doZZA+5Y rleI+UaIGj73m+8d9j+BXstt+9Qjyb5lFHBPCn6FjMYqVtp5e+/Fja0/quYlerHA1 +H9kQkuV+ABkmiOAmBi2CdjL1nfI7gDFQ4QkUvpNOqzIT0F0r70Of5h/P2ElqvWId 4HINhJ/CmpWhwVbURJed7joKmUv9g/g5kDtvayF/2EWG4e5n0+2Lw9xidcs/MvTtW AjH2LQmDpgHPUTTTiQcSISNoQsQVMk8fAOnlu9joM6Fehbs8C9BKPY0c2N2Qq5a9H n6WRQY/reqRAkApI8A== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MhWkj-1s0cLR3g03-00lVdx; Fri, 16 Aug 2024 13:14:35 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" Subject: [PATCH v5.1 14/19] mm/mmap: Avoid zeroing vma tree in mmap_region() Date: Fri, 16 Aug 2024 13:13:55 +0200 Message-ID: <20240816111405.11793-15-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:P/xRUK91XTPegJHA/ffuBXgtaC/TqoFyWMcTDzcdnF40569KBXG pQW00XWomIZvuJJKEKbn1BRQNTDlNEBXHadq3zw0nMLe/40mMQTWP4OHTajZY+2xHsxurgU gjT7AvQUG+Gd+eGpF64UV/pQKgNzb2U43GsLBPPaRXp1TVp+YPDbadmvEiwp2/Ie9Kv52KC vZCQURPUl4gOgHLPwFUCQ== UI-OutboundReport: notjunk:1;M01:P0:MqZlFogfKbk=;jqIHAe8QT0NPe3NYh5g75v/wn+u lGKAPZZzXBDvRjtZ2sVmNw7N0p0EOC+oV8EAq97hoZSWSh/XfxyA2U8OU46a36J8wQpuEzrbN dnGLDxz2l8+a3D6c1d8Y0EyVP13gohZNWS5MSa7+KmQib4+QF+12JLEagIRgNQ7nwzHcQJN9W 4Ud8zMmoXI1IGJttG1vqd7wPK2FuvuNuNdw68noIWwX6FW0fZLM4gDI/aobR4YvAQfxLRaF7B r80q6Febih540ehCEVp/O8iBqZ2j9jBp3FxW/D19zynEC44paStMuyRw7D+eK0zKfBxxg01rk Q1rBf7giwp4EV9NhoxytmeiLtVZN4WJB28epdMOAK2kFUfj6PuGIxq0i/kQ5Kc6YZOiqxaf1g CW0VrZRDPkKIWP6Oa2GFtB2ufaIZY1aeDsCCs90mv6Fu3+wzaYvsMqorY2YnYzq1jUsgJR5Ri 5q7RH0wxeQ+QkwqjNXS71X8yGACwJQ+ifhxFrrWqcJPWQH835QPow594hs6SsuedVvjcYkWQL 07lab8/EQQk8JibFRXLeCnNaKhrmTEcCUeFNlneLeMZse35IcCHrWx3vm6CqqfWZHb/gqBoKw Erp6AqiREM0cF8rqnRg4ztssr7k26YknW1MdDRi6PNFEClHxxi2I0MmDRTqKC1+Z3ypgIMDyX tyPPi2QXTUqTBSGQ/X2rE9J8P82PIpxsPM2BG5bVTIUgDFjNntwFfONLOh+nruMoLo5f+82aK 95M0lvJu7OvMKM3s9x30R2AGdi9xCWyVcg5PCk3kuOqCnYbNm6xczf2n5t1d88kPiqXP/14Nb rKKfUzMZ8pL7Q+wiHp9i4KUA== X-Rspam-User: X-Stat-Signature: qyo3b4d3o4eoy74834ihnhj99q76owis X-Rspamd-Queue-Id: 58A30C0016 X-Rspamd-Server: rspam11 X-HE-Tag: 1723806882-240608 X-HE-Meta: U2FsdGVkX181IZ1AJF+oyyujQBAx01aR4bWkVHmLo6fPM0I/RL9umF+3XgGhUpPKjgfhLobMLWbYe4IyZ/FCh5kMotDs6hq8dslIIhDzR/P5mipFHYeBlZLBKWKNMOxoTaGiFMldiAobegXlLrcwnbLgq7jQAz64Tx/e020CX441+X+dGTw4BJxQcP/If6zvm6OxkeMdT2Wpf8MGBn40vsNDITsYxl1aoxiIjWCObzu/io/wuUzO1H3boKZXp5CwQayT2IqU8cl8pOUt5gsD2rgBjhDtKVHnupm12rclZUas5XxLMKZQ7gvTT9mHDKkLkA4qCjh5LOAsYKqC1+88hB/lIxH6svaPnb39fvnw/V7nQcoDR7MV8jLRATyLLLhIUfW+aArz+/mS6kjIug2u2rQrLitxC/3EQQUjf1idivKz6WWAPdIph7LupX/WyB4llr2d+wpUhZI+stdbS3MXgTtmjXPkfbYihqAgeo9fprd3VFpLEPDaGOnC0Nqzt2Hn3uLLTz2pXbh2ejOSudKISYQ27/iiNNXvqD1jwxEGfNhgiC79+vJa2B+0cRX7FSzK37Cwj1ROe8up87UINh1DmgnkHwm+v5+nMamS37kB4PsBWYWtMOklsog/wpuVY15cXKeLJKy1VdYieeVdl0s+d9uwHMsYEFAW8X2IMBnAc+gFff8IxsvsLaYyHbWnYz0k9l3qhy7gOecy8OLIubxkFWGtLN9laCTQSaJqFpC5D2LDDIwe/amou/uO+dJOZU/dv8udOMUqjrep++CkNgZ9E1OwoY+z4PxAfRcFiaDS/313uKsE/FlT4ZgHXNQE8imTgLa4nwPFZjdg/76QJJBJ3PFtin6wBCd7qiQleOKg4JIpFumjg2uLUVV00Y9CUOAf0OdDU4cxU0dfxph7q8McLzNZkiL7g5YzszNiFv1V5GOxGQLpiFJpbJRyXodb+WyHCRweydTrzUD/N0UtQyY vFlNmmYd 0LRlbuS4bbRgQ4pyIk7bqcobCHTvAdEsKV2rqpWgCa5bWV4+bFTebd2IDe1rhHk3uHC4i/bUpusQiYU0k1zznc2NykV8uHhs0ScORIockTqRi4zjeA5BiIHpHJwBBQfmapKTguIMrPNG6+uN4WPIZajNf+nBDwapVCzt3m8O2RW6+cfsL6nunl9dR00bFIwESRChT0vyilt3z7HFoOUgV24QPqj3Cj6W4MeyXRnXs1DVaawe3tKZYTEDxrX0wguKF3XeiMvRvdPgy067m7AT1DxwZXrFrd+tMTdplzXofhlu4D8b+a51F+wWn5PFbKxF0GXDt X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of zeroing the vma tree and then overwriting the area, let the area be overwritten and then clean up the gathered vmas using vms_complete_munmap_vmas(). If a driver is mapping over an existing vma, then clear the ptes before the call_mmap() invocation. If the vma has a vm_ops->close(), then call the close() function. This is done using the vms_clear_ptes() and vms_close_vmas() helpers. This has the side effect of needing to call open() on the vmas if the mmap_region() fails later on. Temporarily keep track of the number of pages that will be removed and reduce the charged amount. This commit drops the validate_mm() call in the vma_expand() function. It is necessary to drop the validate as it would fail since the mm map_count would be incorrect during a vma expansion, prior to the cleanup from vms_complete_munmap_vmas(). Clean up the error handing of the vms_gather_munmap_vmas() by calling the verification within the function. Note that before this change, a MAP_FIXED could fail and leave a gap in the vma tree. With this change, a MAP_FIXED failure will fail without creating a gap and leave *a* vma in the area (may have been split) and attept to restore them to an operational state (re-attached and vm_ops->open()'ed if they were vm_ops->close()'d). Signed-off-by: Liam R. Howlett --- mm/mmap.c | 60 +++++++++++++++++++++++++------------------------------ mm/vma.c | 59 ++++++++++++++++++++++++++++++++++++++++++------------ mm/vma.h | 16 ++++++++++++--- 3 files changed, 86 insertions(+), 49 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index 1f0b8dc5a089..e152b6caaf9c 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1373,24 +1373,19 @@ unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long merge_start = addr, merge_end = end; bool writable_file_mapping = false; pgoff_t vm_pgoff; - int error; + int error = -ENOMEM; VMA_ITERATOR(vmi, mm, addr); + unsigned long nr_pages, nr_accounted; - /* Check against address space limit. */ - if (!may_expand_vm(mm, vm_flags, len >> PAGE_SHIFT)) { - unsigned long nr_pages; - - /* - * MAP_FIXED may remove pages of mappings that intersects with - * requested mapping. Account for the pages it would unmap. - */ - nr_pages = count_vma_pages_range(mm, addr, end); - - if (!may_expand_vm(mm, vm_flags, - (len >> PAGE_SHIFT) - nr_pages)) - return -ENOMEM; - } + nr_pages = count_vma_pages_range(mm, addr, end, &nr_accounted); + /* + * Check against address space limit. + * MAP_FIXED may remove pages of mappings that intersects with requested + * mapping. Account for the pages it would unmap. + */ + if (!may_expand_vm(mm, vm_flags, (len >> PAGE_SHIFT) - nr_pages)) + return -ENOMEM; if (unlikely(!can_modify_mm(mm, addr, end))) return -EPERM; @@ -1404,14 +1399,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr, mas_init(&mas_detach, &mt_detach, /* addr = */ 0); /* Prepare to unmap any existing mapping in the area */ if (vms_gather_munmap_vmas(&vms, &mas_detach)) - goto gather_failed; - - /* Remove any existing mappings from the vma tree */ - if (vma_iter_clear_gfp(&vmi, addr, end, GFP_KERNEL)) - goto clear_tree_failed; + return -ENOMEM; - /* Unmap any existing mapping in the area */ - vms_complete_munmap_vmas(&vms, &mas_detach); next = vms.next; prev = vms.prev; vma = NULL; @@ -1427,8 +1416,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr, */ if (accountable_mapping(file, vm_flags)) { charged = len >> PAGE_SHIFT; + charged -= nr_accounted; if (security_vm_enough_memory_mm(mm, charged)) - return -ENOMEM; + goto abort_munmap; + vms.nr_accounted = 0; vm_flags |= VM_ACCOUNT; } @@ -1477,10 +1468,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr, * not unmapped, but the maps are removed from the list. */ vma = vm_area_alloc(mm); - if (!vma) { - error = -ENOMEM; + if (!vma) goto unacct_error; - } vma_iter_config(&vmi, addr, end); vma_set_range(vma, addr, end, pgoff); @@ -1489,6 +1478,9 @@ unsigned long mmap_region(struct file *file, unsigned long addr, if (file) { vma->vm_file = get_file(file); + /* call_mmap() may map PTE, so ensure there are no existing PTEs */ + vms_clear_ptes(&vms, &mas_detach, /* mm_wr_locked = */ true); + vms_close_vmas(&vms, &mas_detach); error = call_mmap(file, vma); if (error) goto unmap_and_free_vma; @@ -1579,6 +1571,10 @@ unsigned long mmap_region(struct file *file, unsigned long addr, expanded: perf_event_mmap(vma); + /* Unmap any existing mapping in the area */ + if (vms.nr_pages) + vms_complete_munmap_vmas(&vms, &mas_detach); + vm_stat_account(mm, vm_flags, len >> PAGE_SHIFT); if (vm_flags & VM_LOCKED) { if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || @@ -1626,14 +1622,12 @@ unsigned long mmap_region(struct file *file, unsigned long addr, unacct_error: if (charged) vm_unacct_memory(charged); - validate_mm(mm); - return error; -clear_tree_failed: - abort_munmap_vmas(&mas_detach); -gather_failed: +abort_munmap: + if (vms.nr_pages) + abort_munmap_vmas(&mas_detach, vms.closed); validate_mm(mm); - return -ENOMEM; + return error; } static int __vm_munmap(unsigned long start, size_t len, bool unlock) @@ -1972,7 +1966,7 @@ void exit_mmap(struct mm_struct *mm) do { if (vma->vm_flags & VM_ACCOUNT) nr_accounted += vma_pages(vma); - remove_vma(vma, true); + remove_vma(vma, /* unreachable = */ true, /* closed = */ false); count++; cond_resched(); vma = vma_next(&vmi); diff --git a/mm/vma.c b/mm/vma.c index 0244320d76ab..86757443a7a2 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -136,10 +136,10 @@ can_vma_merge_after(struct vm_area_struct *vma, unsigned long vm_flags, /* * Close a vm structure and free it. */ -void remove_vma(struct vm_area_struct *vma, bool unreachable) +void remove_vma(struct vm_area_struct *vma, bool unreachable, bool closed) { might_sleep(); - if (vma->vm_ops && vma->vm_ops->close) + if (!closed && vma->vm_ops && vma->vm_ops->close) vma->vm_ops->close(vma); if (vma->vm_file) fput(vma->vm_file); @@ -374,6 +374,8 @@ init_vma_munmap(struct vma_munmap_struct *vms, vms->exec_vm = vms->stack_vm = vms->data_vm = 0; vms->unmap_start = FIRST_USER_ADDRESS; vms->unmap_end = USER_PGTABLES_CEILING; + vms->clear_ptes = false; /* No PTEs to clear yet */ + vms->closed = false; } /* @@ -556,7 +558,6 @@ int vma_expand(struct vma_iterator *vmi, struct vm_area_struct *vma, vma_iter_store(vmi, vma); vma_complete(&vp, vmi, vma->vm_mm); - validate_mm(vma->vm_mm); return 0; nomem: @@ -685,22 +686,30 @@ void vma_complete(struct vma_prepare *vp, * * Reattach any detached vmas and free up the maple tree used to track the vmas. */ -void abort_munmap_vmas(struct ma_state *mas_detach) +void abort_munmap_vmas(struct ma_state *mas_detach, bool closed) { struct vm_area_struct *vma; mas_set(mas_detach, 0); - mas_for_each(mas_detach, vma, ULONG_MAX) + mas_for_each(mas_detach, vma, ULONG_MAX) { + if (closed && vma->vm_ops && vma->vm_ops->close && + vma->vm_ops->open) + vma->vm_ops->open(vma); + vma_mark_detached(vma, false); + } __mt_destroy(mas_detach->tree); } -static void vms_complete_pte_clear(struct vma_munmap_struct *vms, - struct ma_state *mas_detach, bool mm_wr_locked) +void vms_clear_ptes(struct vma_munmap_struct *vms, + struct ma_state *mas_detach, bool mm_wr_locked) { struct mmu_gather tlb; + if (!vms->clear_ptes) + return; + /* * We can free page tables without write-locking mmap_lock because VMAs * were isolated before we downgraded mmap_lock. @@ -716,6 +725,22 @@ static void vms_complete_pte_clear(struct vma_munmap_struct *vms, free_pgtables(&tlb, mas_detach, vms->vma, vms->unmap_start, vms->unmap_end, mm_wr_locked); tlb_finish_mmu(&tlb); + vms->clear_ptes = false; +} + +void vms_close_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach) +{ + struct vm_area_struct *vma; + + if (!vms->vma_count) + return; + + mas_set(mas_detach, 0); + mas_for_each(mas_detach, vma, ULONG_MAX) + if (vma->vm_ops && vma->vm_ops->close) + vma->vm_ops->close(vma); + + vms->closed = true; } /* @@ -740,7 +765,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, if (vms->unlock) mmap_write_downgrade(mm); - vms_complete_pte_clear(vms, mas_detach, !vms->unlock); + vms_clear_ptes(vms, mas_detach, !vms->unlock); /* Update high watermark before we lower total_vm */ update_hiwater_vm(mm); /* Stat accounting */ @@ -751,7 +776,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, /* Remove and clean up vmas */ mas_set(mas_detach, 0); mas_for_each(mas_detach, vma, ULONG_MAX) - remove_vma(vma, false); + remove_vma(vma, /* unreachable = */ false, vms->closed); vm_unacct_memory(vms->nr_accounted); validate_mm(mm); @@ -891,14 +916,18 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, while (vma_iter_addr(vms->vmi) > vms->start) vma_iter_prev_range(vms->vmi); + /* There are now PTEs that need to be cleared */ + vms->clear_ptes = true; + return 0; userfaultfd_error: munmap_gather_failed: end_split_failed: - abort_munmap_vmas(mas_detach); + abort_munmap_vmas(mas_detach, /* closed = */ false); start_split_failed: map_count_exceeded: + validate_mm(vms->mm); return error; } @@ -942,9 +971,9 @@ do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, return 0; clear_tree_failed: - abort_munmap_vmas(&mas_detach); -gather_failed: + abort_munmap_vmas(&mas_detach, /* closed = */ false); validate_mm(mm); +gather_failed: return error; } @@ -1667,17 +1696,21 @@ bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot) } unsigned long count_vma_pages_range(struct mm_struct *mm, - unsigned long addr, unsigned long end) + unsigned long addr, unsigned long end, + unsigned long *nr_accounted) { VMA_ITERATOR(vmi, mm, addr); struct vm_area_struct *vma; unsigned long nr_pages = 0; + *nr_accounted = 0; for_each_vma_range(vmi, vma, end) { unsigned long vm_start = max(addr, vma->vm_start); unsigned long vm_end = min(end, vma->vm_end); nr_pages += PHYS_PFN(vm_end - vm_start); + if (vma->vm_flags & VM_ACCOUNT) + *nr_accounted += PHYS_PFN(vm_end - vm_start); } return nr_pages; diff --git a/mm/vma.h b/mm/vma.h index 2a2ca489e622..73297dc7fa28 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -43,6 +43,8 @@ struct vma_munmap_struct { unsigned long stack_vm; unsigned long data_vm; bool unlock; /* Unlock after the munmap */ + bool clear_ptes; /* If there are outstanding PTE to be cleared */ + bool closed; /* vma->vm_ops->close() called already */ }; struct unlink_vma_file_batch { @@ -57,7 +59,7 @@ void validate_mm(struct mm_struct *mm); #endif /* Required for mmap_region() */ -void abort_munmap_vmas(struct ma_state *mas_detach); +void abort_munmap_vmas(struct ma_state *mas_detach, bool closed); /* Required for expand_downwards(). */ void anon_vma_interval_tree_pre_update_vma(struct vm_area_struct *vma); @@ -97,6 +99,13 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach); +/* Required for mmap_region() */ +void vms_clear_ptes(struct vma_munmap_struct *vms, + struct ma_state *mas_detach, bool mm_wr_locked); + +/* Required for mmap_region() */ +void vms_close_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach); + int do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, @@ -106,7 +115,7 @@ int do_vmi_munmap(struct vma_iterator *vmi, struct mm_struct *mm, unsigned long start, size_t len, struct list_head *uf, bool unlock); -void remove_vma(struct vm_area_struct *vma, bool unreachable); +void remove_vma(struct vm_area_struct *vma, bool unreachable, bool closed); void unmap_region(struct ma_state *mas, struct vm_area_struct *vma, struct vm_area_struct *prev, struct vm_area_struct *next); @@ -220,7 +229,8 @@ bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot); int mm_take_all_locks(struct mm_struct *mm); void mm_drop_all_locks(struct mm_struct *mm); unsigned long count_vma_pages_range(struct mm_struct *mm, - unsigned long addr, unsigned long end); + unsigned long addr, unsigned long end, + unsigned long *nr_accounted); static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struct *vma) { From patchwork Fri Aug 16 11:13:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B57ECC531DC for ; Fri, 16 Aug 2024 11:15:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 407CA6B01F1; Fri, 16 Aug 2024 07:14:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 33EEA6B01F3; Fri, 16 Aug 2024 07:14:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11B106B01F4; Fri, 16 Aug 2024 07:14:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D0E676B01F1 for ; Fri, 16 Aug 2024 07:14:45 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 839DD121B18 for ; Fri, 16 Aug 2024 11:14:45 +0000 (UTC) X-FDA: 82457850930.21.27029A1 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf30.hostedemail.com (Postfix) with ESMTP id 95B1A8000B for ; Fri, 16 Aug 2024 11:14:43 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=L5L+6Zcl; spf=pass (imf30.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806785; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E0tenUw2wopYUJtJXU9/jIJXrBQpDpxL8g+F0RGALdE=; b=2DDVP7OO6i0hq6S1v+W2xx9g+xKNjESKdmdCVHiL5jLo6RkbpS6e7p9eURiPulzBPWzGL5 nvXPFTkjApq/A3+t42x/CMdwa1ljDVwi6q/9HGTJYOL3vMV1cqBoHpsRBRYgTm9o1k5VYU YEjz02jRomueNOV2iM3+GNxIcJFVTUQ= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b=L5L+6Zcl; spf=pass (imf30.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806785; a=rsa-sha256; cv=none; b=16CIPXPhl/gg65WChEhzPirIDo8iYVX+arJFv4ChTTbBIG/2lSbTQHet9WGnQ2b8Ir1+fj CvrzithM6/jPUjyGPTCPNQQf11m6u9MdQe+a/44Eb/VwKbjqOpghpoQyeYJQ4lmsIp8lxF td5xX02RJMH6clsOXNmia6E/aZseFUg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806876; x=1724411676; i=spasswolf@web.de; bh=E0tenUw2wopYUJtJXU9/jIJXrBQpDpxL8g+F0RGALdE=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=L5L+6ZclSX42QYLUW8vtiT2DDV1YDpOhnQExdYUeK+gKP70ob6BDFbrVi57Dq7OD SXzKAOx1jelINkzMw2MfwIV2Kl52MFHcPAX375TlyL2TmOyHep89DAAZ8+ngEuJ6L Ly2dssKnMC0UV2kJBrEOUMgEwAdlX241/FTZvk8JvjV5W2a8QybRKXiGem/R6kGnq hRoEN3F4MEZhJPdgQ4+r7OWE1S0mDvuxJTIQ7nbsc9aERcP758G+R+Bd3CrFg8OU+ K+fQj5pRuDpnX2m6ifzsS+kZp5egWy/gp9959iTbgEA0h7t0Yyq1xxzw88p/sNyX6 DkBQeZlouAHl4Kilpw== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1MIL4Y-1sQYuW0HC8-00380E; Fri, 16 Aug 2024 13:14:36 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 15/19] mm/mmap: Use PHYS_PFN in mmap_region() Date: Fri, 16 Aug 2024 13:13:56 +0200 Message-ID: <20240816111405.11793-16-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:eCVsU7mzHyULyhBGJDmXZIAx9FHHpxBOTVU7MpEzaMObRLktneQ WA9rkX1sAY1n5DbOrkqc9/qYe7aDDa2u7GPHUCOV8ylG9qS49ksbEXIrvUIXVxJJa6IplUS RnAiqkGx7jYtteewJE8JIdovsqUFWe1pDqORtMKUBECCsrjkaid7BH/By5+WMbpm5GbwLm3 q2AOA3OXOU82MSdb86uxw== UI-OutboundReport: notjunk:1;M01:P0:bA1vkDMWn8Q=;9XG+PHSHOtTHHkd5lMT97oXUb+9 /QaI7lZ3s0q6Hr0G6SODvGqbbwspTcKMBg/AngIZtq+pN/N/whiiXDuBZLFDdUN72p1ebRTmS ZsdDtfDYTAidoAWZ42p7wBUj45WSk7qJrtt3lj9XDyh368F7fh9yHgG1dNUQymRhefuqV4w4S yVQQeSTFRJtYslXnqWdh0mlLMCnyM/ugUFANiJrtGBnIbdjtcBbQAWvuf89NzrGm43p91VlED HqzBXet5JHKzyFrdR2rG+m5SkXaWEnCsEyTboAJ37nicTU2OT/OzB1A6jtufxkVeIVSA10H2p mzsq0D9e+vabkp/IKwwvk3VINxlSsjeMRpaRb4dtDu1uLAIlLybpIo/aC2k5pMXc185jw8Egb dwA42nv1Qi6Zavr1OUnVkO3b2gZKoC7hnZ2Fntl4yCfN/RhdzqgVmWE+klqF7tzLAUO3g8KHi KoDZABV1mJ+m93IqnMloI6nngx7qXWovS917yy+WiOGTY45HNjkWB9ZdlJztiHQTB9Ryiwoxb pU/andK5GpJoiHjyVQEuOvgnMNn5caRsVwmL/GRHdeFSFZ91/YjpjxMOdlxDUA8DAPkEF7/7S AlEtgHAIUSW5ongs+ezmE1uVlcO89p7XyqcnWZnp1sE35oID9GpvX8VVcg8G/AjIZtR28aQL0 f+ZwnUeTUegV9JblYi+o0M9Pggjke/kaWMidpvB01UaqPvA2a4tzHkYGf5FqIy3S9LhponjG0 +y3B0IY3KFIKDB5/pnF2wXzLpePCQzbGKKGWo+jS/EoDnfep7kwgESuZ2mHACN4MMtWGM1dWF T0Ra8v2JMpZ5MNkzUpMz6dzg== X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 95B1A8000B X-Stat-Signature: i4t4fygb8c1dmtr1o7mf5xwqzmu78uec X-Rspam-User: X-HE-Tag: 1723806883-477771 X-HE-Meta: U2FsdGVkX1+7DXaDzHLgp3jTW6T4PxFIIdd600sC3UyZwPQpOU/N9pXV2AElgI7FISCUGcqvOjglQLbKr4VQeyRLbMp9Cv+MuAoVfKK75F5wAN2uHgoQwV1XzEktgoFv+3bzvPxC8ZLp9lRz8xSAjWwcBBRBZya3qDvCxs+GDbIVPjSTs8/UDd9a+bTBCOg/cmigic/5cysfhNvPoIqSV4pxX2eZzYmf18qkGcCbsMsRJJBpNipqZYYLjqHb5DwzNVlQdOP5NZArGLS09AFqaT9KGIUJ8wHZnTWgQzOHtF+jOgtHDmon+bFTIvwOF6v+gzg66aB3WJAkVUdfc3VZ/2Y3TA4zjMo1uVsYC94h8kQ8OSlfb37csZqeTiJ2OXbktUhTeBsa+pfKVYTGLXtYUBMNpSTr7rOlXowENsh6A5NbCfgjWRT+WSKjpdjZTFJZjGYodBNsxrzRg+JR5wwbRtY737cpfPoyLeO4QUO3idN8BF3a4CyVqgLfZ1SsDQooVzNCF+S2d9WCqN83+H0O9NXJSXJGxta2/6qfSy2G3K70VZYY0YbrKRPkCzZ5zyavuziFz49DfiCPJE0CfofoAqj9+hxnlagMXbN0m2fx0nJCWGLcpJkgBwl19S9Nc2xpKSmXsEekNo9DlvqyKdYKFOsBeytOiyLUTJmTA8t/dG0yH00ABeutIPIQgK1UtxvSpvQeO1ycIGDdES7To4/C70/Gy37m0iRp5TLh//0iwF9ijYwGMYgtZqyxNpVpsMnr9WO/1g3NgI1P1XMgSP7F9XvVI1Jx2JVoDwxDLI4BCGaHQT9fNyNRmbFFnygem2G0dLQmnPHEgACNobuaR1JB/GU9XEHNeRmQ0PUfICQdcGJpHspWjhUsak5zG4VVBBDwGca4lThuUmh2Vc+iZHft7t9bZ2NfksjEJzFaq8dHdm6fRXwEYkNl66KWvlOHOO7inlNw/ctZ11rVVcMwmN/ Ex22LQL/ YMxK+cTpBHOIlicfPSbO8WSYTESTRobzMy3mDcHOFyIcQ+ZvR/45b5fmswpJK4gMZ2ZYGAB9VbPt2jMIB/iFl/rq3Np1eYNescYl67Hs5wvET4UmCryAjV2ax6OcCx2kEEgb/9DVgDJuA6sk7KxuZd17/DzuM9V0Bm+6xvRk7XEnSMLlJtYIyX3hseVK2cxa27XEkXQNkJ/Qn5kFJjbrYv2jnkI9wucNL6G1DjfhXxwiG1pHkTxCAWYHgZXyLyyOaP2qxf+M1FGPr3VLps90dtKHh8lvCHq6PzKneeX4zv/WryJvwAXQ0Vehf+MrAjQDESjmAvLIJT/d5uLQcenBYegIZJg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of shifting the length by PAGE_SIZE, use PHYS_PFN. Also use the existing local variable everywhere instead of some of the time. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes Reviewed-by: Suren Baghdasaryan --- mm/mmap.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) -- 2.45.2 diff --git a/mm/mmap.c b/mm/mmap.c index e152b6caaf9c..57fb1c82a852 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1364,7 +1364,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr, struct mm_struct *mm = current->mm; struct vm_area_struct *vma = NULL; struct vm_area_struct *next, *prev, *merge; - pgoff_t pglen = len >> PAGE_SHIFT; + pgoff_t pglen = PHYS_PFN(len); unsigned long charged = 0; struct vma_munmap_struct vms; struct ma_state mas_detach; @@ -1384,7 +1384,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr, * MAP_FIXED may remove pages of mappings that intersects with requested * mapping. Account for the pages it would unmap. */ - if (!may_expand_vm(mm, vm_flags, (len >> PAGE_SHIFT) - nr_pages)) + if (!may_expand_vm(mm, vm_flags, pglen - nr_pages)) return -ENOMEM; if (unlikely(!can_modify_mm(mm, addr, end))) @@ -1415,7 +1415,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr, * Private writable mapping: check memory availability */ if (accountable_mapping(file, vm_flags)) { - charged = len >> PAGE_SHIFT; + charged = pglen; charged -= nr_accounted; if (security_vm_enough_memory_mm(mm, charged)) goto abort_munmap; @@ -1575,14 +1575,14 @@ unsigned long mmap_region(struct file *file, unsigned long addr, if (vms.nr_pages) vms_complete_munmap_vmas(&vms, &mas_detach); - vm_stat_account(mm, vm_flags, len >> PAGE_SHIFT); + vm_stat_account(mm, vm_flags, pglen); if (vm_flags & VM_LOCKED) { if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || is_vm_hugetlb_page(vma) || vma == get_gate_vma(current->mm)) vm_flags_clear(vma, VM_LOCKED_MASK); else - mm->locked_vm += (len >> PAGE_SHIFT); + mm->locked_vm += pglen; } if (file)