From patchwork Thu Mar 9 11:12:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13167261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69E66C61DA4 for ; Thu, 9 Mar 2023 11:13:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 416EF6B0080; Thu, 9 Mar 2023 06:13:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BC406B007D; Thu, 9 Mar 2023 06:13:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFD446B0075; Thu, 9 Mar 2023 06:13:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 878A5280001 for ; Thu, 9 Mar 2023 06:13:15 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5ADC240671 for ; Thu, 9 Mar 2023 11:13:15 +0000 (UTC) X-FDA: 80549098350.13.3BF3D3A Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf25.hostedemail.com (Postfix) with ESMTP id 5D44AA001B for ; Thu, 9 Mar 2023 11:13:13 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=smy++lI0; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=e4ZqfoND; dmarc=none; spf=pass (imf25.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678360393; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2Qu5xXTySwPTF8dLZgi/I8vkKzIjKqyV6OaDVaB++xA=; b=phxLpMECc4qmdxHNJFOHHgQlT9f9sJ9WjMVJvBdZBmbEdRdqtwNGAGG0ltAUGLC8ip04dm vI6k3bVGknbNg+mJh3Fz69FcWTn+kprwIOMS1OYLy+PqZPEbXXioZ3XNeMGu7JVgDBSsOP puFc1M++GjFIKXzzr9sAdIOgCU6NBPU= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=smy++lI0; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=e4ZqfoND; dmarc=none; spf=pass (imf25.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678360393; a=rsa-sha256; cv=none; b=13V5GfRgEde+lmNpHVavnVi6yHqHJVxxL/9jBFDx8W+RqXdscW/kBhXmC6eOAwrEIRnl5J piUO5ye2vEhr5iE6LxU5RZnlo+ehwa96RpqI9OcdwBdCIvD9E/KZxVZUM1FxsuA+vchM3C bf0lXRDls55tC7FSb2YcFR7S+mnGGLA= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 28AA521E8D; Thu, 9 Mar 2023 11:13:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1678360392; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Qu5xXTySwPTF8dLZgi/I8vkKzIjKqyV6OaDVaB++xA=; b=smy++lI06RTR8P0db6O40WjBYDL0CeoTsxIDL+R5wzE6TncyoEn05HoKhjDWqaGzli6JEI 3GPGSrV0LDlKChKJhDhqhxVaooJcuqBjs0RN+6O6N1NHB8W9t1Iyg0t2hLpFQzhKeULqs2 dgjXsXdgZiumNxXHEFV8o0dUFaFQGfI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1678360392; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Qu5xXTySwPTF8dLZgi/I8vkKzIjKqyV6OaDVaB++xA=; b=e4ZqfoNDC7qdrfkIKt0ip7rtx+nWJSRX54u1pQUZUHNLoo3srPoDVcEHGdEnh7PdXiVbw2 e9byDlL3I6x1iZAA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 09FA513A73; Thu, 9 Mar 2023 11:13:12 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YLbTAUi/CWRiRwAAMHmgww (envelope-from ); Thu, 09 Mar 2023 11:13:12 +0000 From: Vlastimil Babka To: Andrew Morton Cc: "Liam R. Howlett" , Matthew Wilcox , linux-mm@kvack.org, linux-kernel@vger.kernel.org, patches@lists.linux.dev, maple-tree@lists.infradead.org, Vlastimil Babka Subject: [PATCH 09/10] mm/mmap: start distinguishing if vma can be removed in mergeability test Date: Thu, 9 Mar 2023 12:12:57 +0100 Message-Id: <20230309111258.24079-10-vbabka@suse.cz> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230309111258.24079-1-vbabka@suse.cz> References: <20230309111258.24079-1-vbabka@suse.cz> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5D44AA001B X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 6se6eq4ifidq9947cxznt4uhpdwtserk X-HE-Tag: 1678360393-560519 X-HE-Meta: U2FsdGVkX18YGabsZXQDI+4AzPsz6HNbGOi3EyJ9l5WEAz54bLlIfKeqmJKj38R0iF1Xzkgxj1xgsYaYL51M38Xb5CiV2vc/oXLrDs1wkbc7mHn0JxC9q89ExtTqpsah9gzmpviq4IXaagK0k0sCaB53df1BkB1NNZXF4hr0VwM3mIfMTtc+6P8e/Rif9DBZb2cUVDtTl+zKwRpg7tLkN2jE6VMxHhTlXR3q1FDm1ihQJyb2wN2wYKle7nqVCxufThTwf2V5zkvdo1waegVfTtmNkWuYFRa+fLcBbilx9ExojyPPCtGP1wMzjmovTrS9vQc63lD+fqPoneqTmG2NoFUt9BwtSZ+zq5kLVe+IjIjR/E7z7Un7UTJcYvU8r6X6kN0BbN7Va1lRPfj2CCQjlgwnB61XphEjFnIs25HF1IEV59qz5X0NJaJyLiojtXaBEPmwv0lBnoGAkyOGWW+Z7rjlo/2LufGYO0HglOryRrWoU3jod7GaN5akN4KrD56PMvl18ThP3iw4HcxV45GQI9yS6A2LHgME0bPa1D+3zl7/AYHFjdYaLks7Ckh99NKiOI/oCRtdrDOHADvPPS5hLLq2EjZ2Iw11DXpR2UTxwUBfv+giAjSfvL5tqLQtSyzPOArsLCAOJOCJxEO99duh3ym572VsLfInWjG3TiSBNRQUlbMfuaffTUd+4qwIt4Hp5cMlB4MV1hcSxPrO3foAWwerN5iqG0/zilRYuba7qDWqB2Wu8oXXNiEa/ekeTOP6T7JgPTXVZRPwupVCGsoZWm2cVUZt3gN6Q+HrLpnXTO5/A3nRQMRvPsYQS5YvqQRac+DfUCJnSe/aWPrJspNbXy8rBiZI494iTJLS4X2bAdQjCfQTpKZnjzSOnC6HyeK33mKzwpgQieXtpkTBrT0zJGit6bwolRR1uB8f8R2AqS3wiacf9Cc3fP71kjz1xNhOXrLp/nuskpUo9XdK5pX xVfxIiOI eFUpPS+EklcH+M9w0Y7S761svSvMHHjASLndZF8LNFTRJELwHtD/j90AmY8We8lmoErEvYfSQnHdMGOWHT3iIhuP5nCtJQPQhc918tCdK/1e2sw5Sty6mCq5D8kQZS7Ov3o2fkj3CwGkRrHJ3qcYU2Gnxmyq0FjoAqsWL3yVhbY4P+WEaaS7n3txXqoJFDc6R9gwI+wN/H2ObhsE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since pre-git times, is_mergeable_vma() returns false for a vma with vm_ops->close, so that no owner assumptions are violated in case the vma is removed as part of the merge. This check is currently very conservative and can prevent merging even situations where vma can't be removed, such as simple expansion of previous vma, as evidenced by commit d014cd7c1c35 ("mm, mremap: fix mremap() expanding for vma's with vm_ops->close()") In order to allow more merging when appropriate and simplify the code that was made more complex by commit d014cd7c1c35, start distinguishing cases where the vma can be really removed, and allow merging with vm_ops->close otherwise. As a first step, add a may_remove_vma parameter to is_mergeable_vma(). can_vma_merge_before() sets it to true, because when called from vma_merge(), a removal of the vma is possible. In can_vma_merge_after(), pass the parameter as false, because no removal can occur in each of its callers: - vma_merge() calls it on the 'prev' vma, which is never removed - mmap_region() and do_brk_flags() call it to determine if it can expand a vma, which is not removed As a result, vma's with vm_ops->close may now merge with compatible ranges in more situations than previously. We can also revert commit d014cd7c1c35 as the next step to simplify mremap code again. Signed-off-by: Vlastimil Babka Reviewed-By: Lorenzo Stoakes Signed-off-by: Vlastimil Babka --- mm/mmap.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index d20bbe9ec613..65503ea07f32 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -742,12 +742,13 @@ int vma_shrink(struct vma_iterator *vmi, struct vm_area_struct *vma, /* * If the vma has a ->close operation then the driver probably needs to release - * per-vma resources, so we don't attempt to merge those. + * per-vma resources, so we don't attempt to merge those in case the caller + * indicates the current vma may be removed as part of the merge. */ static inline bool is_mergeable_vma(struct vm_area_struct *vma, struct file *file, unsigned long vm_flags, struct vm_userfaultfd_ctx vm_userfaultfd_ctx, - struct anon_vma_name *anon_name) + struct anon_vma_name *anon_name, bool may_remove_vma) { /* * VM_SOFTDIRTY should not prevent from VMA merging, if we @@ -761,7 +762,7 @@ static inline bool is_mergeable_vma(struct vm_area_struct *vma, return false; if (vma->vm_file != file) return false; - if (vma->vm_ops && vma->vm_ops->close) + if (may_remove_vma && vma->vm_ops && vma->vm_ops->close) return false; if (!is_mergeable_vm_userfaultfd_ctx(vma, vm_userfaultfd_ctx)) return false; @@ -793,6 +794,8 @@ static inline bool is_mergeable_anon_vma(struct anon_vma *anon_vma1, * We don't check here for the merged mmap wrapping around the end of pagecache * indices (16TB on ia32) because do_mmap() does not permit mmap's which * wrap, nor mmaps which cover the final page at index -1UL. + * + * We assume the vma may be removed as part of the merge. */ static bool can_vma_merge_before(struct vm_area_struct *vma, unsigned long vm_flags, @@ -800,7 +803,7 @@ can_vma_merge_before(struct vm_area_struct *vma, unsigned long vm_flags, pgoff_t vm_pgoff, struct vm_userfaultfd_ctx vm_userfaultfd_ctx, struct anon_vma_name *anon_name) { - if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name) && + if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name, true) && is_mergeable_anon_vma(anon_vma, vma->anon_vma, vma)) { if (vma->vm_pgoff == vm_pgoff) return true; @@ -814,6 +817,8 @@ can_vma_merge_before(struct vm_area_struct *vma, unsigned long vm_flags, * * We cannot merge two vmas if they have differently assigned (non-NULL) * anon_vmas, nor if same anon_vma is assigned but offsets incompatible. + * + * We assume that vma is not removed as part of the merge. */ static bool can_vma_merge_after(struct vm_area_struct *vma, unsigned long vm_flags, @@ -821,7 +826,7 @@ can_vma_merge_after(struct vm_area_struct *vma, unsigned long vm_flags, pgoff_t vm_pgoff, struct vm_userfaultfd_ctx vm_userfaultfd_ctx, struct anon_vma_name *anon_name) { - if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name) && + if (is_mergeable_vma(vma, file, vm_flags, vm_userfaultfd_ctx, anon_name, false) && is_mergeable_anon_vma(anon_vma, vma->anon_vma, vma)) { pgoff_t vm_pglen; vm_pglen = vma_pages(vma);