From patchwork Sun Oct 8 20:23:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13412795 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2023EE95A80 for ; Sun, 8 Oct 2023 20:23:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 979896B015C; Sun, 8 Oct 2023 16:23:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92A696B015E; Sun, 8 Oct 2023 16:23:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A4136B0165; Sun, 8 Oct 2023 16:23:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 699816B015C for ; Sun, 8 Oct 2023 16:23:25 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 474771CA4FB for ; Sun, 8 Oct 2023 20:23:25 +0000 (UTC) X-FDA: 81323419170.15.CDFE963 Received: from mail-wm1-f46.google.com (mail-wm1-f46.google.com [209.85.128.46]) by imf25.hostedemail.com (Postfix) with ESMTP id 5DEE4A0013 for ; Sun, 8 Oct 2023 20:23:23 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="kOnZ5/Km"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=lstoakes@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696796603; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oh6ec6CTQesC51vfa/4MCcXzfoVwgVcGNsFkXLPVEho=; b=laYJ1pfFzs86lrxPhwwY8Pfr0G1C4YieGXyl99iPJ2NcASFTiC4+2f24TI5AzcAtTsHh5o Q61zsQZzQ2xgKoQyvXWU35dQMue/3iaugTeyO3a2fk4styHjLKLlXSOP7xyNM+YNGHbRk/ rIUshLDvYTmqgxNQh8nwOXZo6TWBwlo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="kOnZ5/Km"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.46 as permitted sender) smtp.mailfrom=lstoakes@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696796603; a=rsa-sha256; cv=none; b=ItSgr95GtjOeVToG4VJ4zsf32dKFqsFTCKHgWuPgDZRSwUVxFHuy+r2gRr1709TIJkN8nm 9T1mgqyMAh5tM7znWuJDNOFiNvguDPxQtSpiJscvladAj2NrgohfoW5s/05WgTWLP4yDdS MRTVo937EQbNO3Hlkj8209KGqRFwD6A= Received: by mail-wm1-f46.google.com with SMTP id 5b1f17b1804b1-406650da82bso35409305e9.3 for ; Sun, 08 Oct 2023 13:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696796601; x=1697401401; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oh6ec6CTQesC51vfa/4MCcXzfoVwgVcGNsFkXLPVEho=; b=kOnZ5/KmR7XzrQT/eE092t8Rv7gHCl19a5FY8HYxxHEoVrTBZRGRM1QYy3HGj32IMZ gR2SKt7LtzZwBQ3DL+mXuXwU8TKg+VKfdatNR+YToGNisNitMV3zqEhQXiGspW7EFV1E Ff1Ld2qoeIhULFO2uF2SLv2pXcGftJnuikgsijNQ5684fTJHxRpsAQQT0WzLyhPCOEqZ 7zZZU1Kxc0FNwHsqCkfAlXqlrhn/QVRp+vnh9MR3RiWJpCpeRyjwh4+Hm+V4K1JsqOQA 8JKDm6PTm+O3Ydjq4mouDEOs2S7yNRAyzg3HVIhnXBKVuTxj6zLh5+rbq0LdQauGY5HB 556g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696796601; x=1697401401; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oh6ec6CTQesC51vfa/4MCcXzfoVwgVcGNsFkXLPVEho=; b=QoLABBrkSiC1I/U2w2ePNKGOlNTtMsIrpe/CGlz0r5CaWBuP7rV/DZfp3ncXXt0buk gP2pDYkLTOnq0GdNBpannIrtdGVHQTmUmmnf4WldLZchV9uaDJCfQSV9t4d3qGcR5C1N zEbnqW4hrFDfWWZ1TTUhlskAVEB+Xs9MTfkRyentG/SgRH7Ux4f/LGoNimItumwxnoJe rH1RnxxR3qO0SxKrgJiY1GZsv5Jp1e4NLbppO9nYAsut8lxmDNLttL6DHQ4QW6UebXpJ dX+JtICFSpRl6Pi508Jt3ugMBId7UlmLSm+zDZ0lB3im0W3sd5X/yhIhyHmPSdebC3z8 rGxQ== X-Gm-Message-State: AOJu0Yy+pdIF71I5hM/oOdLEE5dVnIerarMarMYxOvyFv9y0tLuxfw7E 3PxXkqF4iF9NrSSfUMiJO4oK9LaNSH4= X-Google-Smtp-Source: AGHT+IFX6EUh5dGcLh467RUTdLkD7TfmQOxpSCn/MZRadvHnRtJtkVpbin/HqLminQaG/NJnzpjLCg== X-Received: by 2002:a1c:7419:0:b0:3fe:89be:cd3 with SMTP id p25-20020a1c7419000000b003fe89be0cd3mr11961603wmc.22.1696796601251; Sun, 08 Oct 2023 13:23:21 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id c5-20020a05600c0ac500b0040586360a36sm11474879wmr.17.2023.10.08.13.23.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 13:23:20 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , Alexander Viro , Christian Brauner Cc: "=Liam R . Howlett" , Vlastimil Babka , linux-fsdevel@vger.kernel.org, Lorenzo Stoakes Subject: [PATCH 1/4] mm: abstract the vma_merge()/split_vma() pattern for mprotect() et al. Date: Sun, 8 Oct 2023 21:23:13 +0100 Message-ID: X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 5DEE4A0013 X-Stat-Signature: 135gkq8r4hscoz65x9p6dkp8zi4iuh7a X-HE-Tag: 1696796603-905076 X-HE-Meta: U2FsdGVkX1/wuy/G2zik36bIb0aWfxq7sGNlPrfXahYb1Nfw6QGhJhhF+OGomgVRBPuDUT4XUIF/72TyrexEM3+3RBblkn1jBICz50gM2/7I/g7utyr5UBu475fbqrjpBHuJsItsiD26eQEMQJ6fZ1es9xfVp38BtejBaq09IxsDhZJMp98FDHtt4AHXFpRFq1OhoDDxQLipmiKHiIsUk7zwy2W1Mxwd/8EQ5OjIY2nf87rpv6a0BH/Vs1jLKpSDkxnqnxJsxWpuaUjxlectcv6IlMGCtlW4u6QPYDVOol9Om7iDQPvqmTBXbBeMf4mHSgq2k0jsStBXlnuuPNROQCPe9W285nxDVLCWJt4oaE7m/lxt8UQFgy+i6g1YkTZdhpsorw9GcpCy6SjOtOT+jDKgyyaJKERMfYlVbKxPT7Mnpce1p0+zK8TGKLo0Iojm1B/xwZb7WjFeeyQZa9FCk3iqJ3+/9FOQjFb6zORvd/F9Zp5c11ImXcu1A/px+KICLF3xy1n3FsNG0Sg5U4XXdRKN/1swL/RMBQ+ynFE6N3SC6LPSZ50HcDGULI1WWbPBpDusKyYbFaOzxV8Cw0RSqx62MRjHSrKkHvTu5vMZfSch9yCY67Z0nSNeqYYlby8YnjxnKPLNt9j7qzooIReAkCw+dgZfAfTJXJVPN3I/f/sHD0L9GUZCwAZVBVhrtmF5NFQVOcQY3Qm5HobHDgFg+FECTYb3HkmTmy9SD4r6PwHD9945tI2L/ZdDof4H6+Ty9pOD0dnTcB69B5oIwzkJPA8Gd6FJZwEZ9dqmGUQp4HF3kQhmXvneTB3F+KM5wIJpLelnFbdYe4e5Xuyubyiv1EnsUQ0bymTuJV0gQFLx6mjxAIXm8W9x019hmVkhQvv8qC0Lb+/JKAP+64tM/vA24kFvvOJTvRpbQ7eLLbaEZKxLQxVDPEB0/zdCa5HGX+S10TZJt9dsPaYtZYAiJLY KVpTX2EJ gA2udOBXECb7nTWq6XwnMdsJV2wCTWs+LXJ1kjbcVBXZjS5iAIiWseGIQuRM42+GuFUuCvFWzfJ+77kZzWV2ZA7+Ny/NqwVpafmz6laLiz/EPpN/rwgJSk9HpSD+TqCH3p+x033BGhtTHk8Nq5VhiEr6zp4dGFUYBFma8g4csUal9nop5J15kG9gmwYVgKsDm8u6VHyMlIG2Z6NTN/R6yU2YBe0TxQBwwXsJLG7gZ+g+dcVe12vjomNp7DT1YWUb+nkT4uBYajxwylo40YG1IAqQSw1nVRDbTZQss1g8+gaBZXfH/CyZ0trnsNlO58Aw2vUXF2lJBYCX/u44fkYuv86WvQdxpYBTOoG5KVbVcKSrUFuvViPsFEg4YnBWTuyA+fd2S7APtZmD0B9Zou9KNU9YYX2GgGsoq/BWpo4JrpPvhmgXZcwtx/0w44kQHQugiyKLj7llR0XMWAp+eXtxD5g4rkwwgyA8bC5Nr X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: mprotect() and other functions which change VMA parameters over a range each employ a pattern of:- 1. Attempt to merge the range with adjacent VMAs. 2. If this fails, and the range spans a subset of the VMA, split it accordingly. This is open-coded and duplicated in each case. Also in each case most of the parameters passed to vma_merge() remain the same. Create a new static function, vma_modify(), which abstracts this operation, accepting only those parameters which can be changed. To avoid the mess of invoking each function call with unnecessary parameters, create wrapper functions for each of the modify operations, parameterised only by what is required to perform the action. Note that the userfaultfd_release() case works even though it does not split VMAs - since start is set to vma->vm_start and end is set to vma->vm_end, the split logic does not trigger. In addition, since we calculate pgoff to be equal to vma->vm_pgoff + (start - vma->vm_start) >> PAGE_SHIFT, and start - vma->vm_start will be 0 in this instance, this invocation will remain unchanged. Signed-off-by: Lorenzo Stoakes --- fs/userfaultfd.c | 53 +++++++++----------------- include/linux/mm.h | 23 ++++++++++++ mm/madvise.c | 25 ++++--------- mm/mempolicy.c | 20 ++-------- mm/mlock.c | 24 ++++-------- mm/mmap.c | 93 ++++++++++++++++++++++++++++++++++++++++++++++ mm/mprotect.c | 27 ++++---------- 7 files changed, 157 insertions(+), 108 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index a7c6ef764e63..9e5232d23927 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -927,11 +927,10 @@ static int userfaultfd_release(struct inode *inode, struct file *file) continue; } new_flags = vma->vm_flags & ~__VM_UFFD_FLAGS; - prev = vma_merge(&vmi, mm, prev, vma->vm_start, vma->vm_end, - new_flags, vma->anon_vma, - vma->vm_file, vma->vm_pgoff, - vma_policy(vma), - NULL_VM_UFFD_CTX, anon_vma_name(vma)); + prev = vma_modify_uffd(&vmi, prev, vma, vma->vm_start, + vma->vm_end, new_flags, + NULL_VM_UFFD_CTX); + if (prev) { vma = prev; } else { @@ -1331,7 +1330,6 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx, unsigned long start, end, vma_end; struct vma_iterator vmi; bool wp_async = userfaultfd_wp_async_ctx(ctx); - pgoff_t pgoff; user_uffdio_register = (struct uffdio_register __user *) arg; @@ -1484,26 +1482,18 @@ static int userfaultfd_register(struct userfaultfd_ctx *ctx, vma_end = min(end, vma->vm_end); new_flags = (vma->vm_flags & ~__VM_UFFD_FLAGS) | vm_flags; - pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); - prev = vma_merge(&vmi, mm, prev, start, vma_end, new_flags, - vma->anon_vma, vma->vm_file, pgoff, - vma_policy(vma), - ((struct vm_userfaultfd_ctx){ ctx }), - anon_vma_name(vma)); + prev = vma_modify_uffd(&vmi, prev, vma, start, vma_end, + new_flags, + ((struct vm_userfaultfd_ctx){ ctx })); if (prev) { /* vma_merge() invalidated the mas */ vma = prev; goto next; } - if (vma->vm_start < start) { - ret = split_vma(&vmi, vma, start, 1); - if (ret) - break; - } - if (vma->vm_end > end) { - ret = split_vma(&vmi, vma, end, 0); - if (ret) - break; + + if (IS_ERR(prev)) { + ret = PTR_ERR(prev); + break; } next: /* @@ -1568,7 +1558,6 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, const void __user *buf = (void __user *)arg; struct vma_iterator vmi; bool wp_async = userfaultfd_wp_async_ctx(ctx); - pgoff_t pgoff; ret = -EFAULT; if (copy_from_user(&uffdio_unregister, buf, sizeof(uffdio_unregister))) @@ -1671,24 +1660,16 @@ static int userfaultfd_unregister(struct userfaultfd_ctx *ctx, uffd_wp_range(vma, start, vma_end - start, false); new_flags = vma->vm_flags & ~__VM_UFFD_FLAGS; - pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); - prev = vma_merge(&vmi, mm, prev, start, vma_end, new_flags, - vma->anon_vma, vma->vm_file, pgoff, - vma_policy(vma), - NULL_VM_UFFD_CTX, anon_vma_name(vma)); + prev = vma_modify_uffd(&vmi, prev, vma, start, vma_end, + new_flags, NULL_VM_UFFD_CTX); if (prev) { vma = prev; goto next; } - if (vma->vm_start < start) { - ret = split_vma(&vmi, vma, start, 1); - if (ret) - break; - } - if (vma->vm_end > end) { - ret = split_vma(&vmi, vma, end, 0); - if (ret) - break; + + if (IS_ERR(prev)) { + ret = PTR_ERR(prev); + break; } next: /* diff --git a/include/linux/mm.h b/include/linux/mm.h index a7b667786cde..c069813f215f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3253,6 +3253,29 @@ extern struct vm_area_struct *copy_vma(struct vm_area_struct **, unsigned long addr, unsigned long len, pgoff_t pgoff, bool *need_rmap_locks); extern void exit_mmap(struct mm_struct *); +struct vm_area_struct *vma_modify_flags(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + unsigned long new_flags); +struct vm_area_struct *vma_modify_flags_name(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, + unsigned long end, + unsigned long new_flags, + struct anon_vma_name *new_name); +struct vm_area_struct *vma_modify_policy(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + struct mempolicy *new_pol); +struct vm_area_struct *vma_modify_uffd(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + unsigned long new_flags, + struct vm_userfaultfd_ctx new_ctx); static inline int check_data_rlimit(unsigned long rlim, unsigned long new, diff --git a/mm/madvise.c b/mm/madvise.c index a4a20de50494..73024693d5c8 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -141,7 +141,7 @@ static int madvise_update_vma(struct vm_area_struct *vma, { struct mm_struct *mm = vma->vm_mm; int error; - pgoff_t pgoff; + struct vm_area_struct *merged; VMA_ITERATOR(vmi, mm, start); if (new_flags == vma->vm_flags && anon_vma_name_eq(anon_vma_name(vma), anon_name)) { @@ -149,28 +149,17 @@ static int madvise_update_vma(struct vm_area_struct *vma, return 0; } - pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); - *prev = vma_merge(&vmi, mm, *prev, start, end, new_flags, - vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma), - vma->vm_userfaultfd_ctx, anon_name); - if (*prev) { - vma = *prev; + merged = vma_modify_flags_name(&vmi, *prev, vma, start, end, new_flags, + anon_name); + if (merged) { + vma = *prev = merged; goto success; } *prev = vma; - if (start != vma->vm_start) { - error = split_vma(&vmi, vma, start, 1); - if (error) - return error; - } - - if (end != vma->vm_end) { - error = split_vma(&vmi, vma, end, 0); - if (error) - return error; - } + if (IS_ERR(merged)) + return PTR_ERR(merged); success: /* vm_flags is protected by the mmap_lock held in write mode. */ diff --git a/mm/mempolicy.c b/mm/mempolicy.c index b01922e88548..b608b1744197 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -786,8 +786,6 @@ static int mbind_range(struct vma_iterator *vmi, struct vm_area_struct *vma, { struct vm_area_struct *merged; unsigned long vmstart, vmend; - pgoff_t pgoff; - int err; vmend = min(end, vma->vm_end); if (start > vma->vm_start) { @@ -802,26 +800,14 @@ static int mbind_range(struct vma_iterator *vmi, struct vm_area_struct *vma, return 0; } - pgoff = vma->vm_pgoff + ((vmstart - vma->vm_start) >> PAGE_SHIFT); - merged = vma_merge(vmi, vma->vm_mm, *prev, vmstart, vmend, vma->vm_flags, - vma->anon_vma, vma->vm_file, pgoff, new_pol, - vma->vm_userfaultfd_ctx, anon_vma_name(vma)); + merged = vma_modify_policy(vmi, *prev, vma, vmstart, vmend, new_pol); if (merged) { *prev = merged; return vma_replace_policy(merged, new_pol); } - if (vma->vm_start != vmstart) { - err = split_vma(vmi, vma, vmstart, 1); - if (err) - return err; - } - - if (vma->vm_end != vmend) { - err = split_vma(vmi, vma, vmend, 0); - if (err) - return err; - } + if (IS_ERR(merged)) + return PTR_ERR(merged); *prev = vma; return vma_replace_policy(vma, new_pol); diff --git a/mm/mlock.c b/mm/mlock.c index 42b6865f8f82..50ebea3b7885 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -476,10 +476,10 @@ static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma, unsigned long end, vm_flags_t newflags) { struct mm_struct *mm = vma->vm_mm; - pgoff_t pgoff; int nr_pages; int ret = 0; vm_flags_t oldflags = vma->vm_flags; + struct vm_area_struct *merged; if (newflags == oldflags || (oldflags & VM_SPECIAL) || is_vm_hugetlb_page(vma) || vma == get_gate_vma(current->mm) || @@ -487,25 +487,15 @@ static int mlock_fixup(struct vma_iterator *vmi, struct vm_area_struct *vma, /* don't set VM_LOCKED or VM_LOCKONFAULT and don't count */ goto out; - pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); - *prev = vma_merge(vmi, mm, *prev, start, end, newflags, - vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma), - vma->vm_userfaultfd_ctx, anon_vma_name(vma)); - if (*prev) { - vma = *prev; + merged = vma_modify_flags(vmi, *prev, vma, start, end, newflags); + if (merged) { + vma = *prev = merged; goto success; } - if (start != vma->vm_start) { - ret = split_vma(vmi, vma, start, 1); - if (ret) - goto out; - } - - if (end != vma->vm_end) { - ret = split_vma(vmi, vma, end, 0); - if (ret) - goto out; + if (IS_ERR(merged)) { + ret = PTR_ERR(merged); + goto out; } success: diff --git a/mm/mmap.c b/mm/mmap.c index 673429ee8a9e..8c21171b431f 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2437,6 +2437,99 @@ int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, return __split_vma(vmi, vma, addr, new_below); } +/* + * We are about to modify one or multiple of a VMA's flags, policy, userfaultfd + * context and anonymous VMA name within the range [start, end). + * + * As a result, we might be able to merge the newly modified VMA range with an + * adjacent VMA with identical properties. + * + * If no merge is possible and the range does not span the entirety of the VMA, + * we then need to split the VMA to accommodate the change. + */ +static struct vm_area_struct *vma_modify(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + unsigned long vm_flags, + struct mempolicy *policy, + struct vm_userfaultfd_ctx uffd_ctx, + struct anon_vma_name *anon_name) +{ + pgoff_t pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); + struct vm_area_struct *merged; + + merged = vma_merge(vmi, vma->vm_mm, prev, start, end, vm_flags, + vma->anon_vma, vma->vm_file, pgoff, policy, + uffd_ctx, anon_name); + if (merged) + return merged; + + if (vma->vm_start < start) { + int err = split_vma(vmi, vma, start, 1); + + if (err) + return ERR_PTR(err); + } + + if (vma->vm_end > end) { + int err = split_vma(vmi, vma, end, 0); + + if (err) + return ERR_PTR(err); + } + + return NULL; +} + +/* We are about to modify the VMA's flags. */ +struct vm_area_struct *vma_modify_flags(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + unsigned long new_flags) +{ + return vma_modify(vmi, prev, vma, start, end, new_flags, + vma_policy(vma), vma->vm_userfaultfd_ctx, + anon_vma_name(vma)); +} + +/* We are about to modify the VMA's flags and/or anon_name. */ +struct vm_area_struct *vma_modify_flags_name(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, + unsigned long end, + unsigned long new_flags, + struct anon_vma_name *new_name) +{ + return vma_modify(vmi, prev, vma, start, end, new_flags, + vma_policy(vma), vma->vm_userfaultfd_ctx, new_name); +} + +/* We are about to modify the VMA's flags memory policy. */ +struct vm_area_struct *vma_modify_policy(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + struct mempolicy *new_pol) +{ + return vma_modify(vmi, prev, vma, start, end, vma->vm_flags, + new_pol, vma->vm_userfaultfd_ctx, anon_vma_name(vma)); +} + +/* We are about to modify the VMA's uffd context and/or flags. */ +struct vm_area_struct *vma_modify_uffd(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, unsigned long end, + unsigned long new_flags, + struct vm_userfaultfd_ctx new_ctx) +{ + return vma_modify(vmi, prev, vma, start, end, new_flags, + vma_policy(vma), new_ctx, anon_vma_name(vma)); +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator diff --git a/mm/mprotect.c b/mm/mprotect.c index b94fbb45d5c7..fdc94453bced 100644 --- a/mm/mprotect.c +++ b/mm/mprotect.c @@ -581,7 +581,7 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb, long nrpages = (end - start) >> PAGE_SHIFT; unsigned int mm_cp_flags = 0; unsigned long charged = 0; - pgoff_t pgoff; + struct vm_area_struct *merged; int error; if (newflags == oldflags) { @@ -625,31 +625,18 @@ mprotect_fixup(struct vma_iterator *vmi, struct mmu_gather *tlb, } } - /* - * First try to merge with previous and/or next vma. - */ - pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); - *pprev = vma_merge(vmi, mm, *pprev, start, end, newflags, - vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma), - vma->vm_userfaultfd_ctx, anon_vma_name(vma)); - if (*pprev) { - vma = *pprev; + merged = vma_modify_flags(vmi, *pprev, vma, start, end, newflags); + if (merged) { + vma = *pprev = merged; VM_WARN_ON((vma->vm_flags ^ newflags) & ~VM_SOFTDIRTY); goto success; } *pprev = vma; - if (start != vma->vm_start) { - error = split_vma(vmi, vma, start, 1); - if (error) - goto fail; - } - - if (end != vma->vm_end) { - error = split_vma(vmi, vma, end, 0); - if (error) - goto fail; + if (IS_ERR(merged)) { + error = PTR_ERR(merged); + goto fail; } success: From patchwork Sun Oct 8 20:23:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13412796 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB76EE95A67 for ; Sun, 8 Oct 2023 20:23:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 471186B0165; Sun, 8 Oct 2023 16:23:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 421F36B0168; Sun, 8 Oct 2023 16:23:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24C366B016B; Sun, 8 Oct 2023 16:23:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 133DC6B0165 for ; Sun, 8 Oct 2023 16:23:27 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E59111A01E6 for ; Sun, 8 Oct 2023 20:23:26 +0000 (UTC) X-FDA: 81323419212.02.87B5AD8 Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by imf01.hostedemail.com (Postfix) with ESMTP id 230AE4000B for ; Sun, 8 Oct 2023 20:23:24 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HijVL4sI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.53 as permitted sender) smtp.mailfrom=lstoakes@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696796605; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xj76wJ9Zoom8wv+//ugduIlfQcCX14rBwnZA3eUs6cY=; b=Dr68HurjR3tWi1iCcSRsK2DdwCVbXHQyeaMM4SGZlDi3AqWu+gimy04POAv/tJRsptswzb lxbeq8bZUDfu+YCuE4dFBuji5Kw2AF8UAtgBIavfCteE1WBPbwUN7gBxprJGSEpN4TgU0p MdHtB0G9ZD4oPY+EjVHuSJsAUurv86A= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=HijVL4sI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.53 as permitted sender) smtp.mailfrom=lstoakes@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696796605; a=rsa-sha256; cv=none; b=4OGkY8ly7aMm/ICWJ1zOQ3O06KjxX71Mh/yWWGBmZk9mej8KyHmzvqa9+n7T1eu+KyfPTq Jx1LAqfq46BaHnInayraMFKQjgw+Bl+9YuBcE8Z+iO3HKkxwlVlOJwePIVMb4KlZgV3e31 aoh+iQO6HiJlE0ZW55wFw/uObtMKgcM= Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-406618d0992so37875535e9.0 for ; Sun, 08 Oct 2023 13:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696796603; x=1697401403; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xj76wJ9Zoom8wv+//ugduIlfQcCX14rBwnZA3eUs6cY=; b=HijVL4sIN9WRs5aBfv8NqC8y/ZflsVc7lnge+caI2Gy1mjtZielkYm2k69fLCpQHnl JKkdgQjl/JXxPF178YdKE8qc4oNgZ5wPN7WO/tLd5tz4vpQbvO/wCWy/rZG1gr1e4MZE ehAPF8YH2IN8OjZdFjEoaJwZ2QKSUxJ2zboMZxjTvpQu67T1ee1DhzxWKZzaYJ79vfMU CIye/Dod3ZGTopBQKuIHVM5tYY25TaxBfoRGvUbKiNxcMFfIMNVE9JZASI4IWtkGt7xS ksQLnpWYc5qBD62HHcf4N44GBTMPA1JGuG3JgbeaX8pXNrEMLG+tTAEcx6EJ6JH4tk5Y geNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696796603; x=1697401403; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xj76wJ9Zoom8wv+//ugduIlfQcCX14rBwnZA3eUs6cY=; b=sdvvYDMPFB0DhM8G8sWeKW1pfpDiTgc/Xe8p8UxwDA3XE5y34MXJWig8Gf7dUFBgDZ BHeY50GVL4SlA4QdHKW+kXaWlT4b/tTCXjTS1yL/0n6LpXflqA29IvjTplV+9K4gIzrb OMfHyztTVVTHPUsYYwttZZ7pp+qt51krrDggFd2OMd3wn4N5Rx5w/LhPiYvI2wgaAG9X HuLdb6IP3NlH70TGos4E3MFch+28zqezcSr7da85jnEDN57d/7pbBd1GYhcXwLZtoFwg 0nlEdXU33qII+MW73Yk+UMQPkv/Ny+H5B9vHzJ1/19Iw7WdIyfCp0Eon22FPD5pU7OIL Lr8A== X-Gm-Message-State: AOJu0Ywq8uwjFXAPQI0cb4h1mdkpLxzYNi9GxO4qCZ/mmP14sRivR1X8 tfjv5gP+TcCnm8xVO+TlJZQBwTfv8g0= X-Google-Smtp-Source: AGHT+IG4F9Wkca1FesebOtAogwJp1fVsExkOmC7if9rq1cZTVc9/EQiXNYXRnEH6/EOgRmTVDlQ1tw== X-Received: by 2002:a05:600c:2b0e:b0:406:848f:8711 with SMTP id y14-20020a05600c2b0e00b00406848f8711mr11974165wme.21.1696796602890; Sun, 08 Oct 2023 13:23:22 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id c5-20020a05600c0ac500b0040586360a36sm11474879wmr.17.2023.10.08.13.23.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 13:23:21 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , Alexander Viro , Christian Brauner Cc: "=Liam R . Howlett" , Vlastimil Babka , linux-fsdevel@vger.kernel.org, Lorenzo Stoakes Subject: [PATCH 2/4] mm: make vma_merge() and split_vma() internal Date: Sun, 8 Oct 2023 21:23:14 +0100 Message-ID: <6237f46d751d5dca385242a92c09169ad4d277ee.1696795837.git.lstoakes@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 230AE4000B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: ppfucuqiz884cz63o59r7xhzmex5mdgd X-HE-Tag: 1696796604-499379 X-HE-Meta: U2FsdGVkX18KIzT1DbmNRVHz+y7b0+F9137j69l+SiVSwwq5H4blV3XwS7WuEXAOV4nDZ+ol7NzHL7GQn8I9POXEuDpIE9T4sce3ZW26DBEDs6Yof7RpMlo9qNTGYlhVTrdcf3Li07pVOKfwC3eP1swrI0loL69BRfqi6TLLrRcZR6D7JNLMa0Of8r4CAqInjETyvchQ4ICDG6ODZOwMzGI4FaYbnYwZXyWXKf1jp84C+kSu4ybld1iWVDNTu/C0yuZ39pbyJmCay7Ktkw4joWkMM3Wa/u/9XuXHvHSnblop/qC3DHJF2Tl5/VYuW5c8JYA9sJI9e49OIVQRqMyEMeMiPdhZrFfugGhFcCmZsrvLWsNXPG3K/Ns/Z6P70mpNXUG1Sf6yFT2nZtmrXRQYBwIn9ghWEUu5UQn2HjfhKzg0DtJnxHzUWqNt/UdQysb7jF0+o1pnJCE6TE5oFyn4h9r8w4VK3jXdLVcXxC74C08eoITYFV2qBNCFgPc2luoH5vtgrVkvKqDjl29VguASHkCu3r3xPwPCICWLcx946u7SQa4zFmGQng3Ey0irQQ+v2hD6cGQIkL4pQ3OVOdByd4mrOQCO364VbWLWrVgJxhWdVLuXT4kj/QFiojEd7z2yhEeN1pnOGLOQwIj2HJhCz4aG94/UC6hNv0RCIPSiJg8ElpJkogrNyFQYn9EoE6rZ3ko3aRSMCZgwKGQC6xDcgOwsZlYKGneyeF6ms9okCoCymk8xN0eLJG0HXYmFRCGx5EAWLtg7FCZdB/KBkDQAGh1jO7UKTMCJG/OMfQY3NG4aRZb32DyU8d432SorQcpc+1j2dOcmjpxUPT9s60CHSyA+lS2Rko5E0nUTgkWsmLFexPhU4UY7iq0/37lvhGyh84mZRdcmxNaNNZnb9cvP/e1F4DuMTj0V6F7UUJqDtFbMAs7PAIFOGwApFP93zS3AHDlXSO9dO7Edyg8by8N 2UiTt67/ DeoYqlxHqi107q6KcqN2e67WuHIkymGveIP0p9mIxdB24fJNMFl2dZKvOZYrbES3q+VSW4yiWrLU57N5c/LpfNo8bq7hzWCR8DlMkcgUmoXKx1A8NCdwibxkfh6K6VxR8LtrHNs3f7KryId16cHn32MKPSVFxplPKHOyOPnzXervO5saYzaCjLPPBZigCeIiLjdmoSvlOnY9rhcijapCn+O/oSHBraSpCPOxsU/CB0sqBiUvnQd95AWkl2uEScsybvKNAcZkyvSnAy/zUI4EYpyFr3reyhci+XSeN2thVFGSeEIpUJ3A0x7dSVGLtEOBaEqYYMqGpUOT88S7lbnrtNpcsyE4085tA4db//pawmEncAKniNc29oLLd51UxEFo0RnH+ANv3GRVoXYCZhHjv8nE2Xx8n3CEjSOT2nMjaQxjxmswGyH6uGlDOdE4nH4DbQFhFuZsJGUto3JM+q70KVEdwIJ26Z3SoumZuoSc5fq2xcLryRK61dJuO8iR1rVcEicDSW+cpGPnLkNQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now the vma_merge()/split_vma() pattern has been abstracted, we use it entirely internally within mm/mmap.c, so make the function static. We also no longer need vma_merge() anywhere else except mm/mremap.c, so make it internal. In addition, the split_vma() nommu variant also need not be exported. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 9 --------- mm/internal.h | 9 +++++++++ mm/mmap.c | 8 ++++---- mm/nommu.c | 4 ++-- 4 files changed, 15 insertions(+), 15 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index c069813f215f..6aa532682094 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3237,16 +3237,7 @@ extern int vma_expand(struct vma_iterator *vmi, struct vm_area_struct *vma, struct vm_area_struct *next); extern int vma_shrink(struct vma_iterator *vmi, struct vm_area_struct *vma, unsigned long start, unsigned long end, pgoff_t pgoff); -extern struct vm_area_struct *vma_merge(struct vma_iterator *vmi, - struct mm_struct *, struct vm_area_struct *prev, unsigned long addr, - unsigned long end, unsigned long vm_flags, struct anon_vma *, - struct file *, pgoff_t, struct mempolicy *, struct vm_userfaultfd_ctx, - struct anon_vma_name *); extern struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *); -extern int __split_vma(struct vma_iterator *vmi, struct vm_area_struct *, - unsigned long addr, int new_below); -extern int split_vma(struct vma_iterator *vmi, struct vm_area_struct *, - unsigned long addr, int new_below); extern int insert_vm_struct(struct mm_struct *, struct vm_area_struct *); extern void unlink_file_vma(struct vm_area_struct *); extern struct vm_area_struct *copy_vma(struct vm_area_struct **, diff --git a/mm/internal.h b/mm/internal.h index 3a72975425bb..ddaeb9f2d9d7 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1011,6 +1011,15 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd, unsigned int flags); +/* + * mm/mmap.c + */ +struct vm_area_struct *vma_merge(struct vma_iterator *vmi, + struct mm_struct *, struct vm_area_struct *prev, unsigned long addr, + unsigned long end, unsigned long vm_flags, struct anon_vma *, + struct file *, pgoff_t, struct mempolicy *, struct vm_userfaultfd_ctx, + struct anon_vma_name *); + enum { /* mark page accessed */ FOLL_TOUCH = 1 << 16, diff --git a/mm/mmap.c b/mm/mmap.c index 8c21171b431f..58d71f84e917 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2346,8 +2346,8 @@ static void unmap_region(struct mm_struct *mm, struct ma_state *mas, * has already been checked or doesn't make sense to fail. * VMA Iterator will point to the end VMA. */ -int __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long addr, int new_below) +static int __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long addr, int new_below) { struct vma_prepare vp; struct vm_area_struct *new; @@ -2428,8 +2428,8 @@ int __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, * Split a vma into two pieces at address 'addr', a new vma is allocated * either for the first part or the tail. */ -int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long addr, int new_below) +static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long addr, int new_below) { if (vma->vm_mm->map_count >= sysctl_max_map_count) return -ENOMEM; diff --git a/mm/nommu.c b/mm/nommu.c index f9553579389b..fc4afe924ad5 100644 --- a/mm/nommu.c +++ b/mm/nommu.c @@ -1305,8 +1305,8 @@ SYSCALL_DEFINE1(old_mmap, struct mmap_arg_struct __user *, arg) * split a vma into two pieces at address 'addr', a new vma is allocated either * for the first part or the tail. */ -int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long addr, int new_below) +static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long addr, int new_below) { struct vm_area_struct *new; struct vm_region *region; From patchwork Sun Oct 8 20:23:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13412797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54360E95A82 for ; Sun, 8 Oct 2023 20:23:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C88716B0168; Sun, 8 Oct 2023 16:23:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C3A326B016B; Sun, 8 Oct 2023 16:23:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD8E96B017C; Sun, 8 Oct 2023 16:23:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 97D956B0168 for ; Sun, 8 Oct 2023 16:23:28 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 67D68C01D0 for ; Sun, 8 Oct 2023 20:23:28 +0000 (UTC) X-FDA: 81323419296.23.C4FB5B8 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf02.hostedemail.com (Postfix) with ESMTP id 872A280004 for ; Sun, 8 Oct 2023 20:23:26 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=THgLQaTr; spf=pass (imf02.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696796606; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GphpI5imAra4MlTQiZu1du23bwIkT93dz+Vpq/IpRZ4=; b=zWytBOMtFMp+/Ls4Kv6C9LWXNN0w5vq+rKfOFHU1vZ6NxHlTKE4Xehm40tcEx69ePpmRXw Q9g5/jRkmd4+PpZTElL7fsR1kife/M3WM8MtS1xsBJ9scCQKq65RevTKut13b+iIYnk4Su 8bqo5hYzbkIZGTGvDFJSgd52YZzgDh0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696796606; a=rsa-sha256; cv=none; b=WEb6OSPwqOXoZ2TeFxWXXL+4D+rROMEBjijz+Oy1ZTxueTfdxKSIBViXZEfoNDRr1an0QC dM7Wf0vnvujInGk3OtiCJBUfe6W1KfFbudx2qpMRK9uEz4+EB7RGSDXYIPwJ2ieHk5canZ NWAj4Fg39GoG/qBn7RjVx292DtPU81A= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=THgLQaTr; spf=pass (imf02.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-406609df1a6so37137745e9.3 for ; Sun, 08 Oct 2023 13:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696796605; x=1697401405; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GphpI5imAra4MlTQiZu1du23bwIkT93dz+Vpq/IpRZ4=; b=THgLQaTrSvv9CiQ3jKT9saZYIuMhwVUj86p+mB8gfQsUk9KXId2gD2ExtWOd3jtvg+ MFiNFF3yB9QoR5v/XS3ZtYAdk387PErAF5gXiXne4o2EuGDlJpmxOz61VU8uctDDBMHx QsoeZh2j/CmvKtebHN1IqZiLNMAU3NxXEotM41g/HJNB+qq3uTeHddGNZHmMmF7M3h7d pFOfwBJTKDNXiOmKsM4tJVTPBglvZa0swqevIIJHqmpudNDKHv2qBn3gdit02mrvMrny 5MF7pBaqso4oZdIrqLZ/MWMTfL276AK9pahTwHlVNnN4fMPC9wA9IKAYx47yedWh8LBS XLSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696796605; x=1697401405; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GphpI5imAra4MlTQiZu1du23bwIkT93dz+Vpq/IpRZ4=; b=VtjJIaz8wFnQb8gAIdvlGsQcqioj14aNwrA8uZlb4WH3g6P0FiIbulUDLBGZLNcZ1S hqiRmCM/1OMxiOTp9oO58jw8a1mRjHRodOoYkkYYdSkQudXb+xXn+8OyYpI3laC4eKUh mArHrOYSZ7itiMkELbh96ueoqHVHXxHV/cQ0IfnLOeA3Ge5O7RL0V64/FS6smSmv37mU cAv5OComg22FULwvEbFhcPAPAZfPNtlLBbFSO//QsfmS9YUVMcY8NbZOIn3nWkCUqJG1 Pf2fv55nMExrsrzAGR+HRn01k39D2icL9coFGht0cQEjkVV/xlMy/gqh4its8B3FnB4w 265A== X-Gm-Message-State: AOJu0YxkNupPxWbUP4+0dfPcRnIydfg/4R0uXejuT2Yi2KGHbJ9SIExf q9zG371dT2/QfCOK4Ew6RVw7LAIEt0I= X-Google-Smtp-Source: AGHT+IFLzkXje9IEvfeVChyj/N8dc9zcG9gS5hukGXOpPqdRcGECy0kYVxH9DGQPH7cdYFKyB/CDNA== X-Received: by 2002:a05:600c:204f:b0:3fe:3004:1ffd with SMTP id p15-20020a05600c204f00b003fe30041ffdmr12585263wmg.4.1696796604593; Sun, 08 Oct 2023 13:23:24 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id c5-20020a05600c0ac500b0040586360a36sm11474879wmr.17.2023.10.08.13.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 13:23:23 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , Alexander Viro , Christian Brauner Cc: "=Liam R . Howlett" , Vlastimil Babka , linux-fsdevel@vger.kernel.org, Lorenzo Stoakes Subject: [PATCH 3/4] mm: abstract merge for new VMAs into vma_merge_new_vma() Date: Sun, 8 Oct 2023 21:23:15 +0100 Message-ID: X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 872A280004 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: fau5kzwaz7cjs81zffnqksb4ouw1yes3 X-HE-Tag: 1696796606-524270 X-HE-Meta: U2FsdGVkX194TR7daTi6PqEacrOu4M3mR37w51wVYZMpI3VFPZhKZNnun6UAjGf6l2pFTrho+I0UEPb51xtApXRSeoZkfO6rtQxyELttmmsZRJGg2gzzGs3zP04O5iuwPkzLGkV4pbS3ggzMr8nEhW6S1kUQmJ5VB+C48HY1pgjexvgtUpbIBCqPUhDoBLIvGVottD3+5fmLiYmDYqlsEDi0jZ0trYSf7VuC064EW8hSxP3ach2ufiGNrDYqi5OyHO1PMlBDhiWICOpXgtj7x2fkPdlpxVxNtaACDzdCdM1A+nf0gTyrEHuPRzth1T8BPODV70WGEsr0BJjN2vulqaWhrZqwdBkJ82Vwc7vtGJvV0yaJQBm822V8idyPkD25LVCEhZao6dF8Z6eFCNDhcJzr21dhlbSMlyqNUUGt5BS/tbd3EmN9J0HIvzRrDbkAbU4CPfzm8+4RECUNR2517nYeUNW6JRM8hWij+UdO66g0tE6a2+ky2H+W8PXkQkPGfaIqlVyqo3q/x1tR7ERoh715Vsldb1GMS1YKNIM/Jw+NWuCQmmiof10MOVJLKBX6DSN2g/KMuMatHbvbJPx6U4Vdn1wuo6kxHT+JxgOk2xqET7HyMOThF3XslMqUmzpiPyO75klFjioe1fmPYh+wUqFX09H0rKhZRqvJO6eEDTbR4MnfbYfwhfXz0x90m7blUdZcIULn3md0ZY5DggBFzpaZHwqJVpfeLEqLDCV/s3PgO5ciiJBd3wr5i0eZnNmVWipvKWACYVAc48gZz3Caq6GVP3OpvXgtvy6xgph8yWL6yoY/ymNVlE/krSCKJTFM4JLODShATfvQNg8BlLC4S7tLqaiSneQP+TubA9SUdQDSiyxlHgDiQhjPu/gtX/v1dslwdIpbf32S1UK5RzuRPdTLyI4d3gdFJ4FaezwwOQwA/6awhq5LVQXrb16wnI4jSmy9T7uFEoCSq6S3zZg ssoF9QyH 1xsnBEoftyHj1gxkNuOH1lKySPM1Fa6GAZkcBKECdZ/i/p7qjUW0X2RUvGLkEva/Nv6rrIjnAJViMU2FiumuOOEtCZF5+q4/a2pivZ59Mao/whmAzE7xeGCQKM6IyqYp2KacSceheh011UpLDfWd/3MF2IfGam4Jew6TSvgkBDuEFBwStCft63W6tvjCIM/YDwHe8kU1w9/di83E/RoTETp9sWnYfRl1F4iG0GoNsOsaA8JHA+t5FgXNUC9dwo4H85ssdoqKoAsj9gA9GYM+zNKrsFnUsYo66ruBHBJjGACYe2VJ03IZlR2eBs2PVymO2dnJhAJfl2VJ/i+GhBvp+o1feYf2yTAsoJaTyfZLfBuM3xD5skxfq6T7WP1db+ffxv31Hr98+XYxsf6cf5dYkaVCr4TE8lVEitJp0gzMFo1nX8EmRBlLAW38HSP0veSNgAa1RXeGLg3fuKV+O4lSPTxwMOxqOPemz0PpL/ozqVXaeQtsjK+h8BSU4npiYzg0w19CF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Only in mmap_region() and copy_vma() do we add VMAs which occupy entirely new regions of virtual memory. We can share the logic between these invocations and make it absolutely explici to reduce confusion around the rather inscrutible parameters possessed by vma_merge(). This also paves the way for a simplification of the core vma_merge() implementation, as we seek to make the function entirely an implementation detail. Note that on mmap_region(), vma fields are initialised to zero, so we can simply reference these rather than explicitly specifying NULL. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/mmap.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 58d71f84e917..51be864b876b 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2530,6 +2530,22 @@ struct vm_area_struct *vma_modify_uffd(struct vma_iterator *vmi, vma_policy(vma), new_ctx, anon_vma_name(vma)); } +/* + * Attempt to merge a newly mapped VMA with those adjacent to it. The caller + * must ensure that [start, end) does not overlap any existing VMA. + */ +static struct vm_area_struct *vma_merge_new_vma(struct vma_iterator *vmi, + struct vm_area_struct *prev, + struct vm_area_struct *vma, + unsigned long start, + unsigned long end, + pgoff_t pgoff) +{ + return vma_merge(vmi, vma->vm_mm, prev, start, end, vma->vm_flags, + vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma), + vma->vm_userfaultfd_ctx, anon_vma_name(vma)); +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator @@ -2885,10 +2901,9 @@ unsigned long mmap_region(struct file *file, unsigned long addr, * vma again as we may succeed this time. */ if (unlikely(vm_flags != vma->vm_flags && prev)) { - merge = vma_merge(&vmi, mm, prev, vma->vm_start, - vma->vm_end, vma->vm_flags, NULL, - vma->vm_file, vma->vm_pgoff, NULL, - NULL_VM_UFFD_CTX, NULL); + merge = vma_merge_new_vma(&vmi, prev, vma, + vma->vm_start, vma->vm_end, + pgoff); if (merge) { /* * ->mmap() can change vma->vm_file and fput @@ -3430,9 +3445,7 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, if (new_vma && new_vma->vm_start < addr + len) return NULL; /* should never get here */ - new_vma = vma_merge(&vmi, mm, prev, addr, addr + len, vma->vm_flags, - vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma), - vma->vm_userfaultfd_ctx, anon_vma_name(vma)); + new_vma = vma_merge_new_vma(&vmi, prev, vma, addr, addr + len, pgoff); if (new_vma) { /* * Source vma may have been merged into new_vma From patchwork Sun Oct 8 20:23:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13412798 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B5D3E95A90 for ; Sun, 8 Oct 2023 20:23:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EAB2E6B016B; Sun, 8 Oct 2023 16:23:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E65DA6B017C; Sun, 8 Oct 2023 16:23:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5F376B0187; Sun, 8 Oct 2023 16:23:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AFCAD6B016B for ; Sun, 8 Oct 2023 16:23:29 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8EE671601B2 for ; Sun, 8 Oct 2023 20:23:29 +0000 (UTC) X-FDA: 81323419338.12.F67FDFE Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by imf30.hostedemail.com (Postfix) with ESMTP id C21A680004 for ; Sun, 8 Oct 2023 20:23:27 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iBzQFaVf; spf=pass (imf30.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696796607; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VCWwJGD+i6SLQnaAJLGE5OXPFaOejL4paLAiGQJbd2k=; b=LJLpRVH/38rzZJs1dXJ7HYcx2QWjBWugSPs2DFRYCHDxQQrH7ECCraq0xQcXkZ9j2nsetV /HUOVJBkjXAV89xa7IDLokSowR2Sd4AwBdT/+UNjKVtirnRPnqgA2CL5ToA7as02qMhaP1 dbl2MylDZeJdBJO6wvj6uQuVC91qbuU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iBzQFaVf; spf=pass (imf30.hostedemail.com: domain of lstoakes@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=lstoakes@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696796607; a=rsa-sha256; cv=none; b=py9Qbxi5qR16qWSVgSRZX793zZ6kDPDriQ3Q7twi2Dn71wJ3ZJw58oCDS8JtjQijjSDbBX L3mam5R4eZWnYmJR2xfCKlXdHgsFM4BVtKqT+0NW1UO/580lRXNlFOxHwluDeLmDeBogEc NC9LKtsyvg3GbNdPv/LOaykBqpzRxsw= Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-405505b07dfso28738595e9.0 for ; Sun, 08 Oct 2023 13:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1696796606; x=1697401406; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VCWwJGD+i6SLQnaAJLGE5OXPFaOejL4paLAiGQJbd2k=; b=iBzQFaVfSl8nHkciUO1LbHPRLM1/O/myQlJipwcVFDSOWh/u2jr3LqAVAEq5/L4jPF I6s0/0RDJkIeGEKVOHHUemdLxlmuvC2xtnTvhLyLru3VLF4WTMG8qRB19NsJkOWBXKDV XOkNPt+Yj2xP0tQPwbRllT0CFapQCUk/yIBHJXbg1hS852g2CSzKg5ExTl4A4g3PPth0 EfesEAJrExWjeYTg+Bbx0X3q6lD9Ufl8lvUNg5Bhpb4Wbu8SlD5c8ecqXSHbcAF5VkDN aa6y76JVaeL0N73kbdpzGow62x965LkvfLSp41BXp45MlDXNg4yEGhdkz0mpo/D9/Tro qmYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696796606; x=1697401406; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VCWwJGD+i6SLQnaAJLGE5OXPFaOejL4paLAiGQJbd2k=; b=b834yVX6sNbLEgsjAXSH0i2y1xeqQMxE06sQ7gQHlXABCHCcuH1J99JxUfCbZ3CaJ6 34aLSx3D/MAWcRYpShGBdBMB0/eZYIGdmTmPx/CPU7vuTaG/d5p6kmrdwUpQ9kpk3ztZ 5cBGqdaRGUmvj6YV357cf7uvZNJyrxKRzNapfvsX3MmtBLDIeguACrlTd+fVEUKDQVj6 0VKlAUY0MJWZImKHNQzqOpihVkCL0jFjaoBtMy8LjArUbRsFwr+54l+esPRjDSsOkzLF Q+uQ46HR5INgm1Tfy3BIavfZ/o2DuCEJ9CsrZdA/kLxpIPEqG0lJzB79mhKWCz6PMPam VsWQ== X-Gm-Message-State: AOJu0YzVN5xyIbeBmgmPEdQ9NsP+I/5BzfxaxjHz0OQ5CRMHgFUeEzwg +H+pvfBxcWG84s7/IB2QSou1Dk3K2dc= X-Google-Smtp-Source: AGHT+IFCA87467gu7z5IDWce4HL8A128HwTPWTaB+9+eEVOi6NwgXaeueapZGaEywr2dFnR51gtXgg== X-Received: by 2002:a05:6000:1112:b0:317:6734:c2ae with SMTP id z18-20020a056000111200b003176734c2aemr7884139wrw.11.1696796606069; Sun, 08 Oct 2023 13:23:26 -0700 (PDT) Received: from lucifer.home ([2a00:23c5:dc8c:8701:1663:9a35:5a7b:1d76]) by smtp.googlemail.com with ESMTPSA id c5-20020a05600c0ac500b0040586360a36sm11474879wmr.17.2023.10.08.13.23.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 13:23:25 -0700 (PDT) From: Lorenzo Stoakes To: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrew Morton , Alexander Viro , Christian Brauner Cc: "=Liam R . Howlett" , Vlastimil Babka , linux-fsdevel@vger.kernel.org, Lorenzo Stoakes Subject: [PATCH 4/4] mm: abstract VMA extension and merge into vma_merge_extend() helper Date: Sun, 8 Oct 2023 21:23:16 +0100 Message-ID: <1ed3d1ba0069104e1685298aa2baf980c38a85ff.1696795837.git.lstoakes@gmail.com> X-Mailer: git-send-email 2.42.0 In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: C21A680004 X-Rspam-User: X-Stat-Signature: fqppwa7wf5tin1i1ay1ficnecbgr43xx X-Rspamd-Server: rspam01 X-HE-Tag: 1696796607-513016 X-HE-Meta: U2FsdGVkX19JA7sVKDEVnkwIVDing+dQ7tkrZ0mk7juVJ08l84TA+2G0rInejfB4awTZ9W3nminD5WAcx1WhP312/+GbipjBWs2XDgxMuvhXOSrtX/HHdZV7Nm12z9BW+mxqSgmESWgGByqtuaXlfBbYqfPU46WInZVFUYLZW/B4mWT9s+uAVY/UOqJd6l7SeqyNG61gu8Zr+kN5DBlT/R5SbQvQarpeRvBoWICMCyC7HAcsLe55IlA0hTGnHbJnAKDKYWthngA1evhGEq0nfbfsEpkzxgZIrRRTQTbvDFVTBke19LMPusAbVYwhIy1UCTL8CiBDOGZG+GsOk2Y8t9iVDk9WOpLgkIqKG9TOuOOK+gRH3gmrpsWdOd8rlfZEKstK6TIUdyF+3wGjCtLvSfwosNqnH7gSu4QvlyETgVOc8ywpgANBs2eFTENEhbNogZP8HVA4sdJMieuuF7ofd4fak9/g1OXSk7Cj6hWenVjknJ1PAMOwVzcnuSsuJlc9rHkzLclzkEZG5OpN2BtqSugYOPtoGHq6GAbrhSB19FAfyTZnZtEOv2GjpNcjS8GuYrZg1Tpk/5GqdJw0G3MBDNbDwVt4pSdOLh0PpAJKDLFUTyyrdVrAJhGgU0t/iOuUGk8T0V17xxel7apTCJRJRfL6K/NkHViO0nyBzNAxmtzzPw27t46Zfa5C2tljIYgYhgmRvc/aYeg+4609sDeKLgbecQczD0P3Iv5LGSfJejj5lxECikBP6Wq5b16j1WtB6iS30borx5mwQmuy48yMOW2FcAr/czyPrUrVabXvT4zeSuZgzsqEWQfPZHB90fA/1kndAna9k1txddo0PlIko0m/SNPWlUyvBcP8Zuzu2rAbdoUS7YcjFys5kcPzkxY5jgn6n+lOHXokdxoi0xx7pUWZxd+gUnKVsXzDSyBpUCbeatHbza8jnDwEmH6TNpQfkav/O0beD0Bhgt3VW0T yoe8Zcqs UzH12ne/RA5JdSrIGbjCdevU7LXqnUsnMPkmrsqHXQ574Z5DywogwVxse5DttfvgwyUWhQmpZX3N78HeOJRBgxbjiW1xL8qojK7oKktomKQone99KHI1HCmK/Nbh/PDToSwWSxKZ+aPNv7YIfZmvCxu4ekqnqK+LRMStKP4RJCiHfyc7vgsMh9J611OWtKmXrqlUCZTwPqY740bpCNP3kixAW9+KrTRSRdB2whowotnvLr3668owSB7o0I+yM5jDzstuJJuwT4YdbHZJUUGzLO96nBdpukf31qBSvL0mC03aseUkZQDvqqiYGz72B4qtV2350diQ+XjXR1AH7TR6yeZKRsCvzVDywVk8A355Udt1Pvq0p7Nm5tNgQ6hDs37cspYrdMe5eu2pH+yN1ivzaCYv12X9AIh32IdVjtNu35EaOl7AGY8A0Xw2KwK0B5OdM7SQAfdBxjOXh8tFLXAJAvowI5BV8EvZlIx1C X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: mremap uses vma_merge() in the case where a VMA needs to be extended. This can be significantly simplified and abstracted. This makes it far easier to understand what the actual function is doing, avoids future mistakes in use of the confusing vma_merge() function and importantly allows us to make future changes to how vma_merge() is implemented by knowing explicitly which merge cases each invocation uses. Note that in the mremap() extend case, we perform this merge only when old_len == vma->vm_end - addr. The extension_start, i.e. the start of the extended portion of the VMA is equal to addr + old_len, i.e. vma->vm_end. With this refactoring, vma_merge() is no longer required anywhere except mm/mmap.c, so mark it static. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/internal.h | 8 +++----- mm/mmap.c | 32 +++++++++++++++++++++++++------- mm/mremap.c | 30 +++++++++++++----------------- 3 files changed, 41 insertions(+), 29 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index ddaeb9f2d9d7..6fa722b07a94 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1014,11 +1014,9 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, /* * mm/mmap.c */ -struct vm_area_struct *vma_merge(struct vma_iterator *vmi, - struct mm_struct *, struct vm_area_struct *prev, unsigned long addr, - unsigned long end, unsigned long vm_flags, struct anon_vma *, - struct file *, pgoff_t, struct mempolicy *, struct vm_userfaultfd_ctx, - struct anon_vma_name *); +struct vm_area_struct *vma_merge_extend(struct vma_iterator *vmi, + struct vm_area_struct *vma, + unsigned long delta); enum { /* mark page accessed */ diff --git a/mm/mmap.c b/mm/mmap.c index 51be864b876b..5d2f2e8d7307 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -860,13 +860,13 @@ can_vma_merge_after(struct vm_area_struct *vma, unsigned long vm_flags, * **** is not represented - it will be merged and the vma containing the * area is returned, or the function will return NULL */ -struct vm_area_struct *vma_merge(struct vma_iterator *vmi, struct mm_struct *mm, - struct vm_area_struct *prev, unsigned long addr, - unsigned long end, unsigned long vm_flags, - struct anon_vma *anon_vma, struct file *file, - pgoff_t pgoff, struct mempolicy *policy, - struct vm_userfaultfd_ctx vm_userfaultfd_ctx, - struct anon_vma_name *anon_name) +static struct vm_area_struct +*vma_merge(struct vma_iterator *vmi, struct mm_struct *mm, + struct vm_area_struct *prev, unsigned long addr, unsigned long end, + unsigned long vm_flags, struct anon_vma *anon_vma, struct file *file, + pgoff_t pgoff, struct mempolicy *policy, + struct vm_userfaultfd_ctx vm_userfaultfd_ctx, + struct anon_vma_name *anon_name) { struct vm_area_struct *curr, *next, *res; struct vm_area_struct *vma, *adjust, *remove, *remove2; @@ -2546,6 +2546,24 @@ static struct vm_area_struct *vma_merge_new_vma(struct vma_iterator *vmi, vma->vm_userfaultfd_ctx, anon_vma_name(vma)); } +/* + * Expand vma by delta bytes, potentially merging with an immediately adjacent + * VMA with identical properties. + */ +struct vm_area_struct *vma_merge_extend(struct vma_iterator *vmi, + struct vm_area_struct *vma, + unsigned long delta) +{ + pgoff_t pgoff = vma->vm_pgoff + + ((vma->vm_end - vma->vm_start) >> PAGE_SHIFT); + + /* vma is specified as prev, so case 1 or 2 will apply. */ + return vma_merge(vmi, vma->vm_mm, vma, vma->vm_end, vma->vm_end + delta, + vma->vm_flags, vma->anon_vma, vma->vm_file, pgoff, + vma_policy(vma), vma->vm_userfaultfd_ctx, + anon_vma_name(vma)); +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator diff --git a/mm/mremap.c b/mm/mremap.c index ce8a23ef325a..38d98465f3d8 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -1096,14 +1096,12 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, /* old_len exactly to the end of the area.. */ if (old_len == vma->vm_end - addr) { + unsigned long delta = new_len - old_len; + /* can we just expand the current mapping? */ - if (vma_expandable(vma, new_len - old_len)) { - long pages = (new_len - old_len) >> PAGE_SHIFT; - unsigned long extension_start = addr + old_len; - unsigned long extension_end = addr + new_len; - pgoff_t extension_pgoff = vma->vm_pgoff + - ((extension_start - vma->vm_start) >> PAGE_SHIFT); - VMA_ITERATOR(vmi, mm, extension_start); + if (vma_expandable(vma, delta)) { + long pages = delta >> PAGE_SHIFT; + VMA_ITERATOR(vmi, mm, vma->vm_end); long charged = 0; if (vma->vm_flags & VM_ACCOUNT) { @@ -1115,17 +1113,15 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, } /* - * Function vma_merge() is called on the extension we - * are adding to the already existing vma, vma_merge() - * will merge this extension with the already existing - * vma (expand operation itself) and possibly also with - * the next vma if it becomes adjacent to the expanded - * vma and otherwise compatible. + * Function vma_merge_extend() is called on the + * extension we are adding to the already existing vma, + * vma_merge_extend() will merge this extension with the + * already existing vma (expand operation itself) and + * possibly also with the next vma if it becomes + * adjacent to the expanded vma and otherwise + * compatible. */ - vma = vma_merge(&vmi, mm, vma, extension_start, - extension_end, vma->vm_flags, vma->anon_vma, - vma->vm_file, extension_pgoff, vma_policy(vma), - vma->vm_userfaultfd_ctx, anon_vma_name(vma)); + vma = vma_merge_extend(&vmi, vma, delta); if (!vma) { vm_unacct_memory(charged); ret = -ENOMEM;