From patchwork Wed Mar 3 17:52:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Geffon X-Patchwork-Id: 12113989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.6 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64793C433DB for ; Wed, 3 Mar 2021 17:53:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D514B64EE8 for ; Wed, 3 Mar 2021 17:53:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D514B64EE8 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 43AC48D018E; Wed, 3 Mar 2021 12:53:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 411F38D0157; Wed, 3 Mar 2021 12:53:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D9EE8D018E; Wed, 3 Mar 2021 12:53:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 11D2E8D0157 for ; Wed, 3 Mar 2021 12:53:07 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C7AFA9418 for ; Wed, 3 Mar 2021 17:53:06 +0000 (UTC) X-FDA: 77879309172.08.0DB61F2 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf18.hostedemail.com (Postfix) with ESMTP id 83BF920053E9 for ; Wed, 3 Mar 2021 17:53:02 +0000 (UTC) Received: by mail-yb1-f202.google.com with SMTP id l10so27883050ybt.6 for ; Wed, 03 Mar 2021 09:53:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=eQ1N4syBqhEslbSb0mdNs2OlZNLITnu4bJ8FJAo60h4=; b=OLQjJT0J9gD2SzMgodm1GHn6v88aUuM7RgUuzmOJG3tKUlRF4p/HNeovouVQxPrtDQ W3cfRVvYPVJf0PUgUTTg9KacNDQGjtV0t41hSbwQbAvjBMnZuTFT0x9eS3UEQXMttKn5 ltADl2+jM89/kQclQtWNkvS4ijrLyucXx0nZ0G4250rFdxWhgpLFaNyjqkIjosUfi07f sxRA7jMw0Py89UrFDAZ05uf6A7fvoblcOv74/leSiKt2Rn7WthMRqXMWZxSbJNoxNPsb Yyw+KEashPMBUEdqc6pmvhLyZ6GOk4foNIMP/6DrXY9D1XhFPoR0Yi1Y+cwIFFGlZmbv 8CCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=eQ1N4syBqhEslbSb0mdNs2OlZNLITnu4bJ8FJAo60h4=; b=hBVYa4CiBKa4UqXzZF3V5yerK+pjRcQ6kYq582zcIjShYlfppd5ImPCxcK2jnskvAm zKXLscOE8+UjKm2N6awtvWTxu6AAGj3Y7vJUq3pwYW03rDOQm0Vhg+cMAHRT7B1rf8GT ZKrTU6NS85GvsyCTBX4ns1Sqv+L2ajdKmiUXYT/fG18zpmaj21SJOuliilIhFFhGGdUt qJgMUsmADM8WI4ZunoR8Lt4FQS13+fUDuINTFyXtfwHyi+DdRAndw2/e0f1W5J2B9mf5 r+ApjwJLw3t1LzL2OokoEqlOqMTr/5WtMaLtI7i/gzRA6vb3h7pCMQ1lYiGWlV+A8K26 BlFg== X-Gm-Message-State: AOAM531WQITJD26pEtty2wyOi9YxZz95OM9Tt3/LNw7m9Zfx/Gl+XhTt 7PGY48iDG2NDq7WdpVuWUxHCO2jVdkSE X-Google-Smtp-Source: ABdhPJwnyqDHV+MfT760BeCIxgKo4yidtHHzI6Ce9X1wfCm1ty1jRfE6ZB+qqj7wRE676p0RC6J7UICzudl5 X-Received: from bg.sfo.corp.google.com ([2620:15c:8:10:60ab:b78:20ae:fc05]) (user=bgeffon job=sendgmr) by 2002:a25:e795:: with SMTP id e143mr523308ybh.509.1614793981739; Wed, 03 Mar 2021 09:53:01 -0800 (PST) Date: Wed, 3 Mar 2021 09:52:35 -0800 Message-Id: <20210303175235.3308220-1-bgeffon@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.31.0.rc0.254.gbdcc3b1a9d-goog Subject: [PATCH] mm: Allow shmem mappings with MREMAP_DONTUNMAP From: Brian Geffon To: Andrew Morton Cc: Axel Rasmussen , Lokesh Gidra , Mike Rapoport , Peter Xu , Hugh Dickins , "Michael S . Tsirkin" , Brian Geffon , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Vlastimil Babka , Andrea Arcangeli , Sonny Rao , Minchan Kim , "Kirill A . Shutemov" X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 83BF920053E9 X-Stat-Signature: 9p1rrffxd168kqrsp9cbtae93bhapfkc Received-SPF: none (flex--bgeffon.bounces.google.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from="<3_cw_YAcKCJ49ECDDMLEMMEJC.AMKJGLSV-KKIT8AI.MPE@flex--bgeffon.bounces.google.com>"; helo=mail-yb1-f202.google.com; client-ip=209.85.219.202 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614793982-644073 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently MREMAP_DONTUNMAP only accepts private anonymous mappings. This change will widen the support to include shmem mappings. The primary use case is to support MREMAP_DONTUNMAP on mappings which may have been created from a memfd. Lokesh Gidra who works on the Android JVM, provided an explanation of how such a feature will improve Android JVM garbage collection: "Android is developing a new garbage collector (GC), based on userfaultfd. The garbage collector will use userfaultfd (uffd) on the java heap during compaction. On accessing any uncompacted page, the application threads will find it missing, at which point the thread will create the compacted page and then use UFFDIO_COPY ioctl to get it mapped and then resume execution. Before starting this compaction, in a stop-the-world pause the heap will be mremap(MREMAP_DONTUNMAP) so that the java heap is ready to receive UFFD_EVENT_PAGEFAULT events after resuming execution. To speedup mremap operations, pagetable movement was optimized by moving PUD entries instead of PTE entries [1]. It was necessary as mremap of even modest sized memory ranges also took several milliseconds, and stopping the application for that long isn't acceptable in response-time sensitive cases. With UFFDIO_CONTINUE feature [2], it will be even more efficient to implement this GC, particularly the 'non-moveable' portions of the heap. It will also help in reducing the need to copy (UFFDIO_COPY) the pages. However, for this to work, the java heap has to be on a 'shared' vma. Currently MREMAP_DONTUNMAP only supports private anonymous mappings, this patch will enable using UFFDIO_CONTINUE for the new userfaultfd-based heap compaction." [1] https://lore.kernel.org/linux-mm/20201215030730.NC3CU98e4%25akpm@linux-foundation.org/ [2] https://lore.kernel.org/linux-mm/20210302000133.272579-1-axelrasmussen@google.com/ Signed-off-by: Brian Geffon Tested-by: Lokesh Gidra Signed-off-by: Linus Torvalds --- mm/mremap.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index ec8f840399ed..6934d199da54 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -653,8 +653,7 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, return ERR_PTR(-EINVAL); } - if (flags & MREMAP_DONTUNMAP && (!vma_is_anonymous(vma) || - vma->vm_flags & VM_SHARED)) + if (flags & MREMAP_DONTUNMAP && !(vma_is_anonymous(vma) || vma_is_shmem(vma))) return ERR_PTR(-EINVAL); if (is_vm_hugetlb_page(vma)) From patchwork Wed Mar 17 19:13:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Geffon X-Patchwork-Id: 12146907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A26BFC433E0 for ; Wed, 17 Mar 2021 19:39:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 190D164EEC for ; Wed, 17 Mar 2021 19:39:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 190D164EEC Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C2BB6B006E; Wed, 17 Mar 2021 15:39:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69C9D6B0070; Wed, 17 Mar 2021 15:39:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 53B406B0071; Wed, 17 Mar 2021 15:39:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id ED7156B006E for ; Wed, 17 Mar 2021 15:39:14 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4ED30180AD817 for ; Wed, 17 Mar 2021 19:39:14 +0000 (UTC) X-FDA: 77930379828.03.D8389CD Received: from mail-qv1-f74.google.com (mail-qv1-f74.google.com [209.85.219.74]) by imf02.hostedemail.com (Postfix) with ESMTP id 4EBE5402E475 for ; Wed, 17 Mar 2021 19:13:42 +0000 (UTC) Received: by mail-qv1-f74.google.com with SMTP id n1so28315469qvi.4 for ; Wed, 17 Mar 2021 12:13:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=vXuIw2r0KUths+dHDGslenhj94hjKPvWBq4rQwLj4o0=; b=vTiA3fHbv6U+fwsYlMXPFTMtzhTMXvkDESZyUN5z3abXBl83IpebxQk0tn/4N0Eqfk J/1oiRtAPkgxAetDq3sgkiSOqaE8EDyGM7R1FmbNnKll3WeQVCVzxuWL6mR0yIAga4Ck dKlb/YTwVhVei5UjQ+TH2DyHwM+r9wa17lvMFLU1AmDoB3MkrNRvRNiHcA3JtA4Xd/Bm IcO46vwWOQGjLptxQTLsoEziY/WRHi6NuhFNiSk5DQxPFp3JZUyoo5G5k8kaYVXe0hox Gp1CHZnuzS6HZ8ZXOWCcaFWOm8+z1AlNcMQ1gIUOYLLjVbejw1oqdXEJ8USpu8/snnE7 po1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=vXuIw2r0KUths+dHDGslenhj94hjKPvWBq4rQwLj4o0=; b=Pg3R8XU9ZB03mKW3aFIM+pgjWO58TjZ0Q1ZumHQS8BTasNIJuzojM1u3vANLTe9U0X RyuG0ihTe4hPwA/Pk/rGtYfHL2GWymVey4z4qFwAiPtDkcgI+q+8+u7VBz0f3KK204BS rTgo82SgtRpEKyO+LkEZbX8WWUkqb/df8ceNQWQCaXzqBTRxHgVw6r1mjmr5aIv/PFls DuxjN7mdc91qN6iAZaVR1fZTZMqHdZX9ONixNJu8MUHpn7vrxOp9rVn6brpDgMH3dPC1 luQXyb/12xqXBxnZi2RKj6f3m8Fo+hpO65eh6AihIyH8u8DZBYAJG/vHTmk+nmvKwbjE hWHA== X-Gm-Message-State: AOAM5313BzDOS74JYM0BRcolcJjQL8JsRu3Od/zkZeSAPrrPhcPAQSXu 1rwCGb5u92b4qPAy15/HjqnqrmoPji4r X-Google-Smtp-Source: ABdhPJzN/koKc7ECpfZRve3vcimXug/j+r6R3c5wMYiU4dPi+khV8lqZ0DEpfirn8y64cPtkqDkTIZQ8c4NX X-Received: from bg.sfo.corp.google.com ([2620:15c:8:10:8c6a:2e6f:ed81:d930]) (user=bgeffon job=sendgmr) by 2002:ad4:4c4c:: with SMTP id cs12mr630569qvb.35.1616008421589; Wed, 17 Mar 2021 12:13:41 -0700 (PDT) Date: Wed, 17 Mar 2021 12:13:34 -0700 In-Reply-To: <20210317191334.564944-1-bgeffon@google.com> Message-Id: <20210317191334.564944-2-bgeffon@google.com> Mime-Version: 1.0 References: <20210303175235.3308220-1-bgeffon@google.com> <20210317191334.564944-1-bgeffon@google.com> X-Mailer: git-send-email 2.31.0.rc2.261.g7f71774620-goog Subject: [PATCH v2 2/2] Revert "mremap: don't allow MREMAP_DONTUNMAP on special_mappings and aio" From: Brian Geffon To: Andrew Morton Cc: Axel Rasmussen , Lokesh Gidra , Mike Rapoport , Peter Xu , Hugh Dickins , "Michael S . Tsirkin" , Brian Geffon , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Vlastimil Babka , Andrea Arcangeli , Sonny Rao , Minchan Kim , "Kirill A . Shutemov" , Dmitry Safonov X-Stat-Signature: skx46d7r6a5r3inwzm8u45sm7e3zn47q X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4EBE5402E475 Received-SPF: none (flex--bgeffon.bounces.google.com>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from="<35VRSYAcKCOADIGHHQPIQQING.EQONKPWZ-OOMXCEM.QTI@flex--bgeffon.bounces.google.com>"; helo=mail-qv1-f74.google.com; client-ip=209.85.219.74 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616008422-989120 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This reverts commit cd544fd1dc9293c6702fab6effa63dac1cc67e99. As discussed in [1] this commit was a no-op because the mapping type was checked in vma_to_resize before move_vma is ever called. This meant that vm_ops->mremap() would never be called on such mappings. Furthermore, we've since expanded support of MREMAP_DONTUNMAP to non-anonymous mappings, and these special mappings are still protected by the existing check of !VM_DONTEXPAND and !VM_PFNMAP which will result in a -EFAULT. 1. https://lkml.org/lkml/2020/12/28/2340 Signed-off-by: Brian Geffon --- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- fs/aio.c | 5 +---- include/linux/mm.h | 2 +- mm/mmap.c | 6 +----- mm/mremap.c | 2 +- 5 files changed, 5 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index e916646adc69..0daf2f1cf7a8 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -1458,7 +1458,7 @@ static int pseudo_lock_dev_release(struct inode *inode, struct file *filp) return 0; } -static int pseudo_lock_dev_mremap(struct vm_area_struct *area, unsigned long flags) +static int pseudo_lock_dev_mremap(struct vm_area_struct *area) { /* Not supported */ return -EINVAL; diff --git a/fs/aio.c b/fs/aio.c index 1f32da13d39e..76ce0cc3ee4e 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -323,16 +323,13 @@ static void aio_free_ring(struct kioctx *ctx) } } -static int aio_ring_mremap(struct vm_area_struct *vma, unsigned long flags) +static int aio_ring_mremap(struct vm_area_struct *vma) { struct file *file = vma->vm_file; struct mm_struct *mm = vma->vm_mm; struct kioctx_table *table; int i, res = -EINVAL; - if (flags & MREMAP_DONTUNMAP) - return -EINVAL; - spin_lock(&mm->ioctx_lock); rcu_read_lock(); table = rcu_dereference(mm->ioctx_table); diff --git a/include/linux/mm.h b/include/linux/mm.h index 77e64e3eac80..8c3729eb3e38 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -570,7 +570,7 @@ struct vm_operations_struct { void (*close)(struct vm_area_struct * area); /* Called any time before splitting to check if it's allowed */ int (*may_split)(struct vm_area_struct *area, unsigned long addr); - int (*mremap)(struct vm_area_struct *area, unsigned long flags); + int (*mremap)(struct vm_area_struct *area); /* * Called by mprotect() to make driver-specific permission * checks before mprotect() is finalised. The VMA must not diff --git a/mm/mmap.c b/mm/mmap.c index 3f287599a7a3..9d7651e4e1fe 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3403,14 +3403,10 @@ static const char *special_mapping_name(struct vm_area_struct *vma) return ((struct vm_special_mapping *)vma->vm_private_data)->name; } -static int special_mapping_mremap(struct vm_area_struct *new_vma, - unsigned long flags) +static int special_mapping_mremap(struct vm_area_struct *new_vma) { struct vm_special_mapping *sm = new_vma->vm_private_data; - if (flags & MREMAP_DONTUNMAP) - return -EINVAL; - if (WARN_ON_ONCE(current->mm != new_vma->vm_mm)) return -EFAULT; diff --git a/mm/mremap.c b/mm/mremap.c index 2c57dc4bc8b6..b1f7bc43ece9 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -545,7 +545,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, if (moved_len < old_len) { err = -ENOMEM; } else if (vma->vm_ops && vma->vm_ops->mremap) { - err = vma->vm_ops->mremap(new_vma, flags); + err = vma->vm_ops->mremap(new_vma); } if (unlikely(err)) {