From patchwork Fri Aug 16 11:13:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bert Karwatzki X-Patchwork-Id: 13766007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17F38C531DE for ; Fri, 16 Aug 2024 11:14:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 124CC6B0173; Fri, 16 Aug 2024 07:14:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EFC5B6B016E; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C92626B0172; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9A29B6B016D for ; Fri, 16 Aug 2024 07:14:41 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4CA9481A7A for ; Fri, 16 Aug 2024 11:14:41 +0000 (UTC) X-FDA: 82457850762.14.AFC5489 Received: from mout.web.de (mout.web.de [212.227.17.12]) by imf28.hostedemail.com (Postfix) with ESMTP id 57A51C000A for ; Fri, 16 Aug 2024 11:14:39 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="gtijZp/r"; spf=pass (imf28.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723806805; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=75LYP4e/f3KZVW22gX/HhYvcu5dNO1K9ehpMZYRgA08=; b=3cmvwHOuw4J0XEa3Urt4TbrciJEcwOnXSE4doFcSt7eyB+aOTwD7UB4XY0Jq41Q26HPPpX svZsB74vccmsP8Q0FxijvpoH/U72CGVHSt+v4wbxoCZqsCs7jfl63NS7RURVx9XzAKB6h4 A+zXaB5i9+Nr9H5AFLH8f0KJuBcVbow= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723806805; a=rsa-sha256; cv=none; b=sY2B0r+k492h+0FYAP+H3tuNrjxK4fqmtB1cBwuKumKNm6YYmkEX4zmavc6/J7bIQi50n+ PHSLnP2/m5E5Infua3jkYi6cZy9ayuMJ12P2FSyIE/QeEGfK0uUxxvLuHurGSpwlpTevq2 Ng7mKIuHpFY8xC25/H4Kjx+dU2g+YfM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=web.de header.s=s29768273 header.b="gtijZp/r"; spf=pass (imf28.hostedemail.com: domain of spasswolf@web.de designates 212.227.17.12 as permitted sender) smtp.mailfrom=spasswolf@web.de; dmarc=pass (policy=quarantine) header.from=web.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=web.de; s=s29768273; t=1723806873; x=1724411673; i=spasswolf@web.de; bh=75LYP4e/f3KZVW22gX/HhYvcu5dNO1K9ehpMZYRgA08=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:Message-ID:In-Reply-To: References:MIME-Version:Content-Transfer-Encoding:cc: content-transfer-encoding:content-type:date:from:message-id: mime-version:reply-to:subject:to; b=gtijZp/rFtCnJ1o1PQUYebo2XlRds1eLxKf6Mx95uKNKUaLb6pOcQ6pYdti5x1WT DPnvHsrW2M6hKNk74IXRx2nEtwO8G5afY+pGqGrG2ueM/aspIFWDkhqk7s5YqTIWv MK906rspw0smKTxQG1/JtdlQhannPTRkpDzrDTTAXJPM4y84ChmQTFMt32Oqd51Q9 EdS9ihY4JknA/T1OhWa/THYCAKnKTwlArXrL517/DWJUM8F1flwyX/qp1CY/uo1Bm CyYuhuCfQoTn0aSGTmtIhOnHSIRgulY1nKtuKb7fNfmGi0+gQymX+R1oDrB4cQ/VS 43P2L6E/OIvpvEOPMA== X-UI-Sender-Class: 814a7b36-bfc1-4dae-8640-3722d8ec6cd6 Received: from localhost.localdomain ([84.119.92.193]) by smtp.web.de (mrweb106 [213.165.67.124]) with ESMTPSA (Nemesis) id 1M9qd5-1sZ7cL3LBl-00BcKV; Fri, 16 Aug 2024 13:14:32 +0200 From: Bert Karwatzki To: "Liam R . Howlett" Cc: Bert Karwatzki , Suren Baghdasaryan , Vlastimil Babka , Lorenzo Stoakes , Matthew Wilcox , sidhartha.kumar@oracle.com, "Paul E . McKenney" , Jiri Olsa , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Kees Cook , Jeff Xu , "Liam R . Howlett" , Lorenzo Stoakes Subject: [PATCH v5.1 12/19] mm/mmap: Track start and end of munmap in vma_munmap_struct Date: Fri, 16 Aug 2024 13:13:53 +0200 Message-ID: <20240816111405.11793-13-spasswolf@web.de> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240816111405.11793-1-spasswolf@web.de> References: <20240816111405.11793-1-spasswolf@web.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:cbjrIoJH4StOfzxVC0jYOdFmhLzIj9iUOdmwhMv8XfbyJCJ1jZR m8w1W7pGwWq9jr8ErUhTvmF3eHtQgkmvMmMxE9tOnaY6uMCYoZ+eVq9e0myVBBYELu+09Fz dL4xEbKcgcX5oQNJgB7liSOz8RjThl8DdHr4wexsPI9oAm/+phdztagTKvXgFZKFBh6SV2c AlMz2WsyCWWssRMqUeiRA== UI-OutboundReport: notjunk:1;M01:P0:yNz7ixttSuw=;bJKS49+ZB3xXUwJ7tz9w0l1BtYu +3ktfsJIJg+2QZxau3/Tj0cgCwGX01o6/lpObmmDpPmUdczRHn3x4NwJtHh6NYDxX8zoo0zon ORvGiMg8KAy+qcTibVrXeRBamhdpN9D1BlaabB76p60Ks7K7CJ8huAwbeBR2QdwGmWYTc67iO 6BQ73NGcU4njcfbzzRup1ura5038Hmu6Bx9LPq9CYQy62EGoB1+TLQt+6LIBDNCS0J3nGsUhk M90+h+haevDMPtECgdC/Ee0kvnK6DyE2M9zenQOimKFsNiJDtiKdp35sIs7js0hshWH2/W5rX vcii3dSbmV/CEn97/EEu1hoNsRPEvyQeoQoA6EGMduW8NcsDPR1m5mA34x3YtL3KNJnnCp80g /vVxPOXxBNSBnNkXp/+EHBN3uDiuVlD9VDao7PdNeLMN0JzJWaMnwweM7NaAtiitLHdWtjICl cDpYwPs8gM2oSvkAIaJPXi70DvNXpaGuRDDduGav2TczmxYBoVMSiqXFXqw+mUtUC9exElHKq IB0wiJ5/P5kIFmyvduEDSJuw7YTy3UZVP9KXP20C9CDsR5huxG85xckAUGBQWi6bz02aq9fXu o/1XpkPgXSD80EZdHEWvnx1PpCIsSH4RxICE3vN85cOiJrijaswWhMTYvdAj0GwJAUXs9IqCe q0emqiYWkUZb1H3IMVZxw5qGeX0cSvHP9zNb5hnS7/JeMWvbwPQldlzucTbCFAfWGg2F1jGA3 nqPD3Mp4bV/PplVGv8h1sdjD6agUVzrZywmNowMQV79+Iit2QnxDKts0HddVAgZf6LDaw1MCf KCyNyerw5fYEulUEIDR9YSaQ== X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 57A51C000A X-Stat-Signature: 46jmbz6rnez5o7gczqbiwxz9j3w4gxxu X-HE-Tag: 1723806879-93242 X-HE-Meta: U2FsdGVkX1/1FetLBSMubEal8aaKs2n0nve/srhwuJB+UFxUEhZHPvF41BjIaYQ38afIvFae9g13b+014l9sUj83LJNDmALU/IRydvts6DQOY6tNhwKAVdAfk/wIfM5/5YKmBHpPMAzU4jdOWiEKweitLw5zltlgqCkRDy9SKUllbLuWvM/MeAzEMQBv1EWBXmpLqgLb45D7druIgoke0Bzz2GHsoLXbrz41+eF/20j7PbPSl2N2DvlerXPaFX70xXNL3yZv5cCaL+9qpuObKLhkc1+rn7jnrHy9dU7op+w2EDu7GAPTUic4tcv8Bzxt8x8sExmiBGOxZAk2qO/w9SxGc4/ol6DCVtUroAjn3OpH+bwZl6VhC367bBADbFKe/Bf2ylbQmMpf/H92l04nCYSw0+60TiGfsvpUgtKVdL7CcbyfhcUo+40xLYqkIJ3XU5e8NtuDrk+wpZ+0YuE5nDM0gnTqSecPWRi1ECeNyV77KH4penJWI4gQN590pWMSzMP1lhIadzjbv62pb8sARvY6BrUpVez9kznYe14d5Acpa8/phntVSTrvJcmStj8N5G4sOmXZDAhJILWHMMyIkZ8UnqH19o6c3H44Av2z1jQ4dolqZVrpzenK0lS5pbEdI7CUQsM9yRZtKRv2tYwdoScIyP/+DXLegdQIgTZvFZEC4BMPjfBFTBBttP2C01b4a42pxk1g1h0n4jc4ljsvUy6Ub4Kkts/CJN2N5ijWmttY0LKBbA8O+21r+uzwOF/yWkjSWv4+L774K9hMk6V3yNXuYzxblcvPFnej/rKZk++ACy1dQ+y94Mi4Iqc6VqteInTUCg2wNF/6Xitlnw+BWJacdRPt6s2LPfT6iq6AlvF2ntqS9WVMPFJohLUyQuZOqmdcEYDBiNTsBxx5BZzJ1j8+NuRJKA3j50qb2FesPKpL5syfRv3xSs3baFQqxxYYOJaFVPeMTx2oDKcgCVc kAcNrtU0 IVMQS656m5a11jI18K8Z38v+ne/BVKZ8zuXtAdsfXR6zttGZKlw7jpqIySR0h2FI9j6YsCJ17BnPVSiX7fMk7aGvGS35BKEdC80vs+W/4rY/KbLlybsYtrywl6SjM4LNsKHKLEnmuKDVYHUeHT4p63+/baASk2XKMmub8YoB+VKw52//tk7urZWr+28uAjkxo9fpyz89x5FCEyXPGpd7teNGl6JluIh3D8r4RWJTNVKZ+5+ank6a0oW07QU+N1/MtCOaX+J8dkqK3i0DMqflzix+4vHd97++YIxy3eFtINyCf5zAUwogaD7OSeFUXDQpgeChK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Set the start and end address for munmap when the prev and next are gathered. This is needed to avoid incorrect addresses being used during the vms_complete_munmap_vmas() function if the prev/next vma are expanded. Add a new helper vms_complete_pte_clear(), which is needed later and will avoid growing the argument list to unmap_region() beyond the 9 it already has. Signed-off-by: Liam R. Howlett Reviewed-by: Lorenzo Stoakes --- mm/vma.c | 38 ++++++++++++++++++++++++++++++-------- mm/vma.h | 5 +++++ 2 files changed, 35 insertions(+), 8 deletions(-) -- 2.45.2 diff --git a/mm/vma.c b/mm/vma.c index a5ca42b7161b..e106d412c4c3 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -373,6 +373,8 @@ init_vma_munmap(struct vma_munmap_struct *vms, vms->vma_count = 0; vms->nr_pages = vms->locked_vm = vms->nr_accounted = 0; vms->exec_vm = vms->stack_vm = vms->data_vm = 0; + vms->unmap_start = FIRST_USER_ADDRESS; + vms->unmap_end = USER_PGTABLES_CEILING; } /* @@ -684,7 +686,7 @@ void vma_complete(struct vma_prepare *vp, * * Reattach any detached vmas and free up the maple tree used to track the vmas. */ -static inline void abort_munmap_vmas(struct ma_state *mas_detach) +void abort_munmap_vmas(struct ma_state *mas_detach) { struct vm_area_struct *vma; @@ -695,6 +697,28 @@ static inline void abort_munmap_vmas(struct ma_state *mas_detach) __mt_destroy(mas_detach->tree); } +static void vms_complete_pte_clear(struct vma_munmap_struct *vms, + struct ma_state *mas_detach, bool mm_wr_locked) +{ + struct mmu_gather tlb; + + /* + * We can free page tables without write-locking mmap_lock because VMAs + * were isolated before we downgraded mmap_lock. + */ + mas_set(mas_detach, 1); + lru_add_drain(); + tlb_gather_mmu(&tlb, vms->mm); + update_hiwater_rss(vms->mm); + unmap_vmas(&tlb, mas_detach, vms->vma, vms->start, vms->end, + vms->vma_count, mm_wr_locked); + mas_set(mas_detach, 1); + /* start and end may be different if there is no prev or next vma. */ + free_pgtables(&tlb, mas_detach, vms->vma, vms->unmap_start, + vms->unmap_end, mm_wr_locked); + tlb_finish_mmu(&tlb); +} + /* * vms_complete_munmap_vmas() - Finish the munmap() operation * @vms: The vma munmap struct @@ -717,13 +741,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, if (vms->unlock) mmap_write_downgrade(mm); - /* - * We can free page tables without write-locking mmap_lock because VMAs - * were isolated before we downgraded mmap_lock. - */ - mas_set(mas_detach, 1); - unmap_region(mm, mas_detach, vms->vma, vms->prev, vms->next, - vms->start, vms->end, vms->vma_count, !vms->unlock); + vms_complete_pte_clear(vms, mas_detach, !vms->unlock); /* Update high watermark before we lower total_vm */ update_hiwater_vm(mm); /* Stat accounting */ @@ -785,6 +803,8 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, goto start_split_failed; } vms->prev = vma_prev(vms->vmi); + if (vms->prev) + vms->unmap_start = vms->prev->vm_end; /* * Detach a range of VMAs from the mm. Using next as a temp variable as @@ -846,6 +866,8 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, } vms->next = vma_next(vms->vmi); + if (vms->next) + vms->unmap_end = vms->next->vm_start; #if defined(CONFIG_DEBUG_VM_MAPLE_TREE) /* Make sure no VMAs are about to be lost. */ diff --git a/mm/vma.h b/mm/vma.h index 8b2401f93c74..b857e7dc4bfe 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -33,6 +33,8 @@ struct vma_munmap_struct { struct list_head *uf; /* Userfaultfd list_head */ unsigned long start; /* Aligned start addr (inclusive) */ unsigned long end; /* Aligned end addr (exclusive) */ + unsigned long unmap_start; /* Unmap PTE start */ + unsigned long unmap_end; /* Unmap PTE end */ int vma_count; /* Number of vmas that will be removed */ unsigned long nr_pages; /* Number of pages being removed */ unsigned long locked_vm; /* Number of locked pages */ @@ -54,6 +56,9 @@ void validate_mm(struct mm_struct *mm); #define validate_mm(mm) do { } while (0) #endif +/* Required for mmap_region() */ +void abort_munmap_vmas(struct ma_state *mas_detach); + /* Required for expand_downwards(). */ void anon_vma_interval_tree_pre_update_vma(struct vm_area_struct *vma);