From patchwork Fri May 27 10:48:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jakub_Mat=C4=9Bna?= X-Patchwork-Id: 12863265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E222AC433F5 for ; Fri, 27 May 2022 10:47:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B6EE8D0006; Fri, 27 May 2022 06:47:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 294718D0001; Fri, 27 May 2022 06:47:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1573A8D0005; Fri, 27 May 2022 06:47:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0489F8D0001 for ; Fri, 27 May 2022 06:47:53 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C2F4F624 for ; Fri, 27 May 2022 10:47:52 +0000 (UTC) X-FDA: 79511197584.16.5392FC6 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf01.hostedemail.com (Postfix) with ESMTP id C5F2E40045 for ; Fri, 27 May 2022 10:47:47 +0000 (UTC) Received: by mail-ej1-f48.google.com with SMTP id jx22so7895690ejb.12 for ; Fri, 27 May 2022 03:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ru3bkI46cGQXTo5hSsyPFl2jKgUjs/czASVw2IToYVQ=; b=RMTaAYIppb0iNIn8vopS7Cr2YesNwAFvN/zYLCGwwZnvleh4l/JCv/+kYqZdwKcee7 Lg8Y0lEiEHp3O09jIAvbsiGMCAk2PZLifDV+rHZwg6OjCTYBay55PolLGo88EDQKUQzp jHq+2oUZSI1bUZ9KYQWpjHoUG20T4W5PoDpyuhrIENGOFMEnWt2f1XGgYwRPVuXE9HRw VwED6xqVVAjjq118lr/sHCgSJY/yf7BWOQ6oUFwAsWilhCkmpdXWa5oj5o/UHkBlMxJx eWECxyCBfNP6rjELVFh8yNPx8Qzp04sdzNjrCKbIKzs4icqcY93H62xhW7qn7pXWOHQ/ lZMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ru3bkI46cGQXTo5hSsyPFl2jKgUjs/czASVw2IToYVQ=; b=r6b5ciidoJJUhG23Xnd0NKyb+9O+4aF97hfBdfuo5f9sQkHgNRHe/z9PhOcH7RDFVa tUmPkEX1B27YEUxc5VtVcpHFM9R423R27m/Z4720uQeGXLyWCBUcKM6P/FHbx5El07rP 5lxNpvcePBneQfalk7A66n8+8ZhozCDeye9CnQWlLt3/IWaI/LOeIVbG4Uunb9ZQUDng b/1aJlbS4TPgwGCphuiSsQuUX8Ijj9IBYcpwioezh+A/PhqbzqbDAbstIsoiKHG715pZ CWr7jdwgKdQwJW6jyHLR1+ilKfKfcyQIhpxbrT8wdLWZ481oIq3ukoWsqMszh/HyEfxN jQKQ== X-Gm-Message-State: AOAM530QJXUxqMZ5igaz55Jz1dA9VVMpG4cDJ7rdG7RfPMwssAbQIjHy f8utFINKkkd73vRV3ijosOs= X-Google-Smtp-Source: ABdhPJwAjCaIQcHY9NNC4R29tI2fkFnzMUb9DOAXPD6YcxngDNqPknClEYm3JJz3kuq3hLflhiqVCg== X-Received: by 2002:a17:907:868e:b0:6ff:39e0:2653 with SMTP id qa14-20020a170907868e00b006ff39e02653mr1898679ejc.734.1653648471075; Fri, 27 May 2022 03:47:51 -0700 (PDT) Received: from orion.localdomain ([93.99.228.15]) by smtp.gmail.com with ESMTPSA id pg5-20020a170907204500b006f3ef214e6asm1321234ejb.208.2022.05.27.03.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 May 2022 03:47:49 -0700 (PDT) Received: by orion.localdomain (Postfix, from userid 1003) id 4DC43A028E; Fri, 27 May 2022 12:48:18 +0200 (CEST) From: =?utf-8?q?Jakub_Mat=C4=9Bna?= To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, vbabka@suse.cz, mhocko@kernel.org, mgorman@techsingularity.net, willy@infradead.org, liam.howlett@oracle.com, hughd@google.com, kirill@shutemov.name, riel@surriel.com, rostedt@goodmis.org, peterz@infradead.org, =?utf-8?q?Jakub_Mat=C4=9Bna?= Subject: [PATCH 1/2] [PATCH 1/2] mm: refactor of vma_merge() Date: Fri, 27 May 2022 12:48:09 +0200 Message-Id: <20220527104810.24736-2-matenajakub@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220527104810.24736-1-matenajakub@gmail.com> References: <20220527104810.24736-1-matenajakub@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C5F2E40045 X-Stat-Signature: myxqtf6mz6cn1b4t6ee3p4ysgn4r3fxp X-Rspam-User: Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=RMTaAYIp; spf=pass (imf01.hostedemail.com: domain of matenajakub@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=matenajakub@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1653648467-217354 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Refactor vma_merge() to make it shorter and more understandable. Main change is the elimination of code duplicity in the case of merge next check. This is done by first doing checks and caching the results before executing the merge itself. The variable 'area' is divided into 'mid' and 'res' as previously it was used for two purposes, as the middle VMA between prev and next and also as the result of the merge itself. Exit paths are also unified. Signed-off-by: Jakub Matěna --- mm/mmap.c | 87 +++++++++++++++++++++++-------------------------------- 1 file changed, 37 insertions(+), 50 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 3aa839f81e63..bd0dde2ad3e2 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1170,8 +1170,10 @@ struct vm_area_struct *vma_merge(struct mm_struct *mm, struct anon_vma_name *anon_name) { pgoff_t pglen = (end - addr) >> PAGE_SHIFT; - struct vm_area_struct *area, *next; - int err; + struct vm_area_struct *mid, *next, *res; + int err = -1; + bool merge_prev = false; + bool merge_next = false; /* * We later require that vma->vm_flags == vm_flags, @@ -1181,75 +1183,60 @@ struct vm_area_struct *vma_merge(struct mm_struct *mm, return NULL; next = vma_next(mm, prev); - area = next; - if (area && area->vm_end == end) /* cases 6, 7, 8 */ + mid = next; + if (next && next->vm_end == end) /* cases 6, 7, 8 */ next = next->vm_next; /* verify some invariant that must be enforced by the caller */ VM_WARN_ON(prev && addr <= prev->vm_start); - VM_WARN_ON(area && end > area->vm_end); + VM_WARN_ON(mid && end > mid->vm_end); VM_WARN_ON(addr >= end); - /* - * Can it merge with the predecessor? - */ + /* Can we merge the predecessor? */ if (prev && prev->vm_end == addr && mpol_equal(vma_policy(prev), policy) && can_vma_merge_after(prev, vm_flags, anon_vma, file, pgoff, vm_userfaultfd_ctx, anon_name)) { - /* - * OK, it can. Can we now merge in the successor as well? - */ - if (next && end == next->vm_start && - mpol_equal(policy, vma_policy(next)) && - can_vma_merge_before(next, vm_flags, - anon_vma, file, - pgoff+pglen, - vm_userfaultfd_ctx, anon_name) && - is_mergeable_anon_vma(prev->anon_vma, - next->anon_vma, NULL)) { - /* cases 1, 6 */ - err = __vma_adjust(prev, prev->vm_start, - next->vm_end, prev->vm_pgoff, NULL, - prev); - } else /* cases 2, 5, 7 */ - err = __vma_adjust(prev, prev->vm_start, - end, prev->vm_pgoff, NULL, prev); - if (err) - return NULL; - khugepaged_enter_vma_merge(prev, vm_flags); - return prev; + merge_prev = true; } - - /* - * Can this new request be merged in front of next? - */ + /* Can we merge the successor? */ if (next && end == next->vm_start && mpol_equal(policy, vma_policy(next)) && can_vma_merge_before(next, vm_flags, anon_vma, file, pgoff+pglen, vm_userfaultfd_ctx, anon_name)) { + merge_next = true; + } + /* Can we merge both the predecessor and the successor? */ + if (merge_prev && merge_next && + is_mergeable_anon_vma(prev->anon_vma, + next->anon_vma, NULL)) { /* cases 1, 6 */ + err = __vma_adjust(prev, prev->vm_start, + next->vm_end, prev->vm_pgoff, NULL, + prev); + res = prev; + } else if (merge_prev) { /* cases 2, 5, 7 */ + err = __vma_adjust(prev, prev->vm_start, + end, prev->vm_pgoff, NULL, prev); + res = prev; + } else if (merge_next) { if (prev && addr < prev->vm_end) /* case 4 */ err = __vma_adjust(prev, prev->vm_start, - addr, prev->vm_pgoff, NULL, next); - else { /* cases 3, 8 */ - err = __vma_adjust(area, addr, next->vm_end, - next->vm_pgoff - pglen, NULL, next); - /* - * In case 3 area is already equal to next and - * this is a noop, but in case 8 "area" has - * been removed and next was expanded over it. - */ - area = next; - } - if (err) - return NULL; - khugepaged_enter_vma_merge(area, vm_flags); - return area; + addr, prev->vm_pgoff, NULL, next); + else /* cases 3, 8 */ + err = __vma_adjust(mid, addr, next->vm_end, + next->vm_pgoff - pglen, NULL, next); + res = next; } - return NULL; + /* + * Cannot merge with predecessor or successor or error in __vma_adjust? + */ + if (err) + return NULL; + khugepaged_enter_vma_merge(res, vm_flags); + return res; } /* From patchwork Fri May 27 10:48:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Jakub_Mat=C4=9Bna?= X-Patchwork-Id: 12863266 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE463C433FE for ; Fri, 27 May 2022 10:47:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75A138D0005; Fri, 27 May 2022 06:47:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6E32A8D0001; Fri, 27 May 2022 06:47:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46EC88D0007; Fri, 27 May 2022 06:47:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2FDFC8D0005 for ; Fri, 27 May 2022 06:47:53 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E948A20974 for ; Fri, 27 May 2022 10:47:52 +0000 (UTC) X-FDA: 79511197584.09.AFFEF03 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by imf31.hostedemail.com (Postfix) with ESMTP id C165D20045 for ; Fri, 27 May 2022 10:47:15 +0000 (UTC) Received: by mail-ed1-f54.google.com with SMTP id c93so3587826edf.7 for ; Fri, 27 May 2022 03:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E8hw1YxRdyFQiBunbdOQafA0vLjTqdy9/4gfOonLrFg=; b=cMw3nqupRwb9iJ0bAiEPeAgnOBYM1beNn1TObGXfHMIOMS1sTGGaToiA5nWxbTPG23 7Bz8Tf9v6QAYxGl16dxbYfTSm7cMnx29cgprRiQll1k8U5uaKV9vxmnxwM/ichP5gcNC 2MhV/Fgj+5yEwdzZBeNIph23msVCY1g5rCZhKgrErykccuFP7ubdDpN+0sMnTjuyAXgD WlQzdnKIlZD5ZXUjz2BiatWWlSOPzWc4K+AATgFhBrhSUWAhmb/cxMDw9moE2hFmKj7M tL+7T6Ejx28GCXKEAeHuZe351Sitn3HNDzC0SQ+cSYB/9oTan6ZNM36uDM5QH6P2nfo8 yO6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E8hw1YxRdyFQiBunbdOQafA0vLjTqdy9/4gfOonLrFg=; b=CBlK2wYTpUoIumZb+QS8Y6kSn420nrktwGvgAkT+YxGzbTj+qjvbxMMMDkqOveBQ2L ztXmiauWvoTd6WauTElbbo9N4nj9KYMUFsFHrGLGUMMa3YD3WIqyU0caHJxz+9czGE7X xxr0KF8LY1/+F3VQCqbK9OcdLxGoMekO2MDCYLUxKwK2Rm1HUFmmjryIdKnrB07M52xZ V/AQlAZy6wZ37uGuJ+EFQ/Wx1f9bc2FiBVTU0ORpH1boJlvX1sT72sKJIiH3f7PQCSQz oSB0p0PvyDV++pJl8x3OU82pWD3B1u2fYMYqZ/dceq1rvpVFKIIH58oapJLrGCIpekKy +q4A== X-Gm-Message-State: AOAM5313NAiqAgL1wVGawOQQd5ai9fqN9Wk3hiJlSrbyjYkcveLdlxit H7HjpkJnvhtz4ySJK4Gxk1M= X-Google-Smtp-Source: ABdhPJyTYr0OkS3+nT4wN/g4MnpnJWE858fclOQNBvUdUSBEzGJbKuxqnWjjUb04IleiIrgnKZ+Spw== X-Received: by 2002:a05:6402:1cc1:b0:413:2b12:fc49 with SMTP id ds1-20020a0564021cc100b004132b12fc49mr43833744edb.118.1653648471305; Fri, 27 May 2022 03:47:51 -0700 (PDT) Received: from orion.localdomain ([93.99.228.15]) by smtp.gmail.com with ESMTPSA id v4-20020a50a444000000b0042ab87ea713sm777389edb.22.2022.05.27.03.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 27 May 2022 03:47:49 -0700 (PDT) Received: by orion.localdomain (Postfix, from userid 1003) id 5148EA0293; Fri, 27 May 2022 12:48:18 +0200 (CEST) From: =?utf-8?q?Jakub_Mat=C4=9Bna?= To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, vbabka@suse.cz, mhocko@kernel.org, mgorman@techsingularity.net, willy@infradead.org, liam.howlett@oracle.com, hughd@google.com, kirill@shutemov.name, riel@surriel.com, rostedt@goodmis.org, peterz@infradead.org, =?utf-8?q?Jakub_Mat=C4=9Bna?= Subject: [PATCH 2/2] [PATCH 2/2] mm: add merging after mremap resize Date: Fri, 27 May 2022 12:48:10 +0200 Message-Id: <20220527104810.24736-3-matenajakub@gmail.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220527104810.24736-1-matenajakub@gmail.com> References: <20220527104810.24736-1-matenajakub@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C165D20045 X-Stat-Signature: rcanrzfb6qkuzfjzjn6xz46djrfyrcso Authentication-Results: imf31.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=cMw3nqup; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf31.hostedemail.com: domain of matenajakub@gmail.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=matenajakub@gmail.com X-Rspam-User: X-HE-Tag: 1653648435-67616 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When mremap call results in expansion, it might be possible to merge the VMA with the next VMA which might become adjacent. This patch adds vma_merge call after the expansion is done to try and merge. Signed-off-by: Jakub Matěna Reported-by: kernel test robot --- mm/mremap.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index 303d3290b938..c41237e62156 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -9,6 +9,7 @@ */ #include +#include #include #include #include @@ -1022,8 +1023,11 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, } } - if (vma_adjust(vma, vma->vm_start, addr + new_len, - vma->vm_pgoff, NULL)) { + vma = vma_merge(mm, vma, addr + old_len, addr + new_len, + vma->vm_flags, vma->anon_vma, vma->vm_file, + vma->vm_pgoff + (old_len >> PAGE_SHIFT), vma_policy(vma), + vma->vm_userfaultfd_ctx, anon_vma_name(vma)); + if (!vma) { vm_unacct_memory(pages); ret = -ENOMEM; goto out;