From patchwork Wed Mar 17 21:41:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Geffon X-Patchwork-Id: 12147025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55C2DC433E0 for ; Wed, 17 Mar 2021 21:41:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C502464E64 for ; Wed, 17 Mar 2021 21:41:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C502464E64 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 094606B0070; Wed, 17 Mar 2021 17:41:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 06C6B6B0071; Wed, 17 Mar 2021 17:41:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E4EA16B0072; Wed, 17 Mar 2021 17:41:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id CBC686B0070 for ; Wed, 17 Mar 2021 17:41:51 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 901F563FA for ; Wed, 17 Mar 2021 21:41:51 +0000 (UTC) X-FDA: 77930688822.08.7A65561 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf27.hostedemail.com (Postfix) with ESMTP id 3A06C801914C for ; Wed, 17 Mar 2021 21:41:51 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id j4so46002755ybt.23 for ; Wed, 17 Mar 2021 14:41:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=6JKBBoz8CQGNb7LdFHQu8eZZo9mpl/jAnB6fSdRHbgM=; b=TVFJl/Bz+n9V0z7OP99iR4mlt7aH5gkCLB0f5sibPPJF64t0iU47HjkYgm9BRwwIS7 oW7WYz8lWfu/urEkfdhsFqyqXbAo6PCTzLxinfBAuJ6ciBbFgyPLFY0fQ5jvv8KbLng6 4QomHgwzG4CRgQenSNg85//32OdBi2aoO09fwIT2HLgtkhxzp/WWBKjz+mbfwfWVGwZB 7/i8O7zLqSKH04aDXTcyUq7HinalLh0vqCliDXbE5PDUaMMu+NOCh+csdLEWB6wqYnJW uj2i9n54E4/Hvqkk8pcuMNLXxsO6luRxA/+QL0bOysGkvb9zdSMIWXXBqzv7nbM5F3wW fSbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=6JKBBoz8CQGNb7LdFHQu8eZZo9mpl/jAnB6fSdRHbgM=; b=sSWangA/lI8QjgrcD0DfJcg8muhHR/qNKE8esNu5Cef8vyqfGdLKUe9esWg9ZhomVe 7T7AZUY21u3nZ3hmb9uMR144F1gfsfaG9Ijk2W/OvCCLdVuBg3wKVjMPk9RLHu5SOpCq 3S5gGHKtaxrTqiOQTdH7fTIQB375xDsU0j2X6IINvbQzlBBUW684rM0CJhMc+BBXXoO2 ilaidYgSBUxXwK1/P5T7X9uOciqwGdeK6a+K1Iiq0kodvKRjnrgt9fh+ZdCxPf/q38BB UKmz1ximX4EqzIf+wNbbuibSz+W20kUmSGG4Lr4K4yGfned3vOcFxcRBBtECpUh+gFB/ eF5g== X-Gm-Message-State: AOAM531kc3aa89BjuYbDhK4pVhGzaWB09z7jIciFJ8eZ+ED8y0D3/FlY XN4Oq8oM7j1YNzwEymTHG/gCikjJoNqO X-Google-Smtp-Source: ABdhPJyWVCh6V0a73bmY964YJglCxN05PngE4FP3YYnZ6KhzjGe1g6OQoBJkIVakX9cR7RD8uwys7JxEyw8C X-Received: from bg.sfo.corp.google.com ([2620:15c:8:10:8c6a:2e6f:ed81:d930]) (user=bgeffon job=sendgmr) by 2002:a25:a561:: with SMTP id h88mr7030922ybi.235.1616017310240; Wed, 17 Mar 2021 14:41:50 -0700 (PDT) Date: Wed, 17 Mar 2021 14:41:46 -0700 In-Reply-To: <20210303175235.3308220-1-bgeffon@google.com> Message-Id: <20210317214147.682687-1-bgeffon@google.com> Mime-Version: 1.0 References: <20210303175235.3308220-1-bgeffon@google.com> X-Mailer: git-send-email 2.31.0.rc2.261.g7f71774620-goog Subject: [PATCH v3 1/2] mm: Allow non-VM_DONTEXPAND and VM_PFNMAP mappings with MREMAP_DONTUNMAP From: Brian Geffon To: Andrew Morton Cc: Axel Rasmussen , Lokesh Gidra , Mike Rapoport , Peter Xu , Hugh Dickins , "Michael S . Tsirkin" , Brian Geffon , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Vlastimil Babka , Andrea Arcangeli , Sonny Rao , Minchan Kim , "Kirill A . Shutemov" , Dmitry Safonov X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3A06C801914C X-Stat-Signature: s5qu4rtzk9fx9q6f35arbwwut6cz5e9w Received-SPF: none (flex--bgeffon.bounces.google.com>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from="<3nndSYAcKCN8CHFGGPOHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--bgeffon.bounces.google.com>"; helo=mail-yb1-f201.google.com; client-ip=209.85.219.201 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616017311-150348 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently MREMAP_DONTUNMAP only accepts private anonymous mappings. This change will widen the support to include any mappings which are not VM_DONTEXPAND or VM_PFNMAP. The primary use case is to support MREMAP_DONTUNMAP on mappings which may have been created from a memfd. This change will result in mremap(MREMAP_DONTUNMAP) returning -EINVAL if VM_DONTEXPAND or VM_PFNMAP mappings are specified. Lokesh Gidra who works on the Android JVM, provided an explanation of how such a feature will improve Android JVM garbage collection: "Android is developing a new garbage collector (GC), based on userfaultfd. The garbage collector will use userfaultfd (uffd) on the java heap during compaction. On accessing any uncompacted page, the application threads will find it missing, at which point the thread will create the compacted page and then use UFFDIO_COPY ioctl to get it mapped and then resume execution. Before starting this compaction, in a stop-the-world pause the heap will be mremap(MREMAP_DONTUNMAP) so that the java heap is ready to receive UFFD_EVENT_PAGEFAULT events after resuming execution. To speedup mremap operations, pagetable movement was optimized by moving PUD entries instead of PTE entries [1]. It was necessary as mremap of even modest sized memory ranges also took several milliseconds, and stopping the application for that long isn't acceptable in response-time sensitive cases. With UFFDIO_CONTINUE feature [2], it will be even more efficient to implement this GC, particularly the 'non-moveable' portions of the heap. It will also help in reducing the need to copy (UFFDIO_COPY) the pages. However, for this to work, the java heap has to be on a 'shared' vma. Currently MREMAP_DONTUNMAP only supports private anonymous mappings, this patch will enable using UFFDIO_CONTINUE for the new userfaultfd-based heap compaction." [1] https://lore.kernel.org/linux-mm/20201215030730.NC3CU98e4%25akpm@linux-foundation.org/ [2] https://lore.kernel.org/linux-mm/20210302000133.272579-1-axelrasmussen@google.com/ Signed-off-by: Brian Geffon Acked-by: Hugh Dickins --- mm/mremap.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index ec8f840399ed..db5b8b28c2dd 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -653,8 +653,8 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, return ERR_PTR(-EINVAL); } - if (flags & MREMAP_DONTUNMAP && (!vma_is_anonymous(vma) || - vma->vm_flags & VM_SHARED)) + if ((flags & MREMAP_DONTUNMAP) && + (vma->vm_flags & (VM_DONTEXPAND | VM_PFNMAP))) return ERR_PTR(-EINVAL); if (is_vm_hugetlb_page(vma)) From patchwork Wed Mar 17 21:41:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brian Geffon X-Patchwork-Id: 12147027 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4A8CC433E6 for ; Wed, 17 Mar 2021 21:41:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4845E64F17 for ; Wed, 17 Mar 2021 21:41:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4845E64F17 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C30376B0071; Wed, 17 Mar 2021 17:41:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C07498D0002; Wed, 17 Mar 2021 17:41:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A34276B0073; Wed, 17 Mar 2021 17:41:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 87C516B0071 for ; Wed, 17 Mar 2021 17:41:53 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 45C75181AF5D0 for ; Wed, 17 Mar 2021 21:41:53 +0000 (UTC) X-FDA: 77930688906.21.1567E3D Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf13.hostedemail.com (Postfix) with ESMTP id BF0F1E000B97 for ; Wed, 17 Mar 2021 21:41:52 +0000 (UTC) Received: by mail-yb1-f201.google.com with SMTP id u1so46355571ybu.14 for ; Wed, 17 Mar 2021 14:41:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PE76OhS7gOhisfnYJ34BsUCZhRzyUT9agI71FqfBCUQ=; b=Kddr/UUoa+woTOsBjRGakKyox6pozzIhXdt3RnHLuXmSDuUfQSgFuVdknVN533ca87 sDtghIkKoA3a5oDDpEQSnXDFML8r8/+kvqi7gfwqRsrDClpBBJaLPWUm3uzcAF5Ygsqg Wr0Q7glWMar1BLgExphmB0ty7JZ86ny9Hda9QaX4hyVhE/cbC9uTQq1M3XrpkZ1LBqHq gNgSYUg5xCJeoqXYhXuA3VYeCL5BsAFIsC3+6FcwvK1198CDnMfx6cPxFqIIim992vtB jy2dr4q+kXSxZzGb5+Xd0lDum8mPtghFG0nO/7ACV0tl+fsb7QzU6wJAlI/aBdij1p0J 2Vfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PE76OhS7gOhisfnYJ34BsUCZhRzyUT9agI71FqfBCUQ=; b=YdBa8DZQNMEWANEoej/lBLXUJEbIWVEg1/VtfjhiCVdznjpIHcEgVNIzgBKKQlmsjp kES8mPPThRrEyzdEbyHgD+RYu6JtRecqEuQYwX8058w48/S6ciz1kNac/Io0LCASXQKQ 7kMNbXWyv4Tq5/OQl5lp5tW9gO/0jXQ9TuclvX9QaZWW/zbAvT6G4mk3ElPM8/Hdq2ps nAq5AhSeVOoqQzfcYM9SAuW+8WDPr4E6vEVfR9XAvRpBCo5raVWCOvvz1oiN/0Letse9 ZvCyECK2DPQk4zOd7MxDt6fG4rqQNzamIe+9eYTmA0vnGxPRf71Fj7cfsu174n9Xdfee yA9Q== X-Gm-Message-State: AOAM531sYVhqWJDazD+96qe+XwhzJB73KYzG/iPWuj4qxUQQ1rgnmQdk Oa5l8JXOa55rrR55Kz+p/GLG2lHR4l1U X-Google-Smtp-Source: ABdhPJyBuc4RzSE6YsLPIioC7HoYjlEa6rLlvNPBwi6c0E0CrnwV5mtzkXtzfpc5FobREa6yi1FZ9SZ6h/54 X-Received: from bg.sfo.corp.google.com ([2620:15c:8:10:8c6a:2e6f:ed81:d930]) (user=bgeffon job=sendgmr) by 2002:a25:ca13:: with SMTP id a19mr6217553ybg.446.1616017312057; Wed, 17 Mar 2021 14:41:52 -0700 (PDT) Date: Wed, 17 Mar 2021 14:41:47 -0700 In-Reply-To: <20210317214147.682687-1-bgeffon@google.com> Message-Id: <20210317214147.682687-2-bgeffon@google.com> Mime-Version: 1.0 References: <20210303175235.3308220-1-bgeffon@google.com> <20210317214147.682687-1-bgeffon@google.com> X-Mailer: git-send-email 2.31.0.rc2.261.g7f71774620-goog Subject: [PATCH v3 2/2] Revert "mremap: don't allow MREMAP_DONTUNMAP on special_mappings and aio" From: Brian Geffon To: Andrew Morton Cc: Axel Rasmussen , Lokesh Gidra , Mike Rapoport , Peter Xu , Hugh Dickins , "Michael S . Tsirkin" , Brian Geffon , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andy Lutomirski , Vlastimil Babka , Andrea Arcangeli , Sonny Rao , Minchan Kim , "Kirill A . Shutemov" , Dmitry Safonov X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BF0F1E000B97 X-Stat-Signature: mbyc9d75bttw7jgftrohc354s7jjh4jk Received-SPF: none (flex--bgeffon.bounces.google.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from="<3oHdSYAcKCOEEJHIIRQJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--bgeffon.bounces.google.com>"; helo=mail-yb1-f201.google.com; client-ip=209.85.219.201 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616017312-824750 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This reverts commit cd544fd1dc9293c6702fab6effa63dac1cc67e99. As discussed in [1] this commit was a no-op because the mapping type was checked in vma_to_resize before move_vma is ever called. This meant that vm_ops->mremap() would never be called on such mappings. Furthermore, we've since expanded support of MREMAP_DONTUNMAP to non-anonymous mappings, and these special mappings are still protected by the existing check of !VM_DONTEXPAND and !VM_PFNMAP which will result in a -EFAULT. 1. https://lkml.org/lkml/2020/12/28/2340 Signed-off-by: Brian Geffon Acked-by: Hugh Dickins --- arch/x86/kernel/cpu/resctrl/pseudo_lock.c | 2 +- fs/aio.c | 5 +---- include/linux/mm.h | 2 +- mm/mmap.c | 6 +----- mm/mremap.c | 2 +- 5 files changed, 5 insertions(+), 12 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c index e916646adc69..0daf2f1cf7a8 100644 --- a/arch/x86/kernel/cpu/resctrl/pseudo_lock.c +++ b/arch/x86/kernel/cpu/resctrl/pseudo_lock.c @@ -1458,7 +1458,7 @@ static int pseudo_lock_dev_release(struct inode *inode, struct file *filp) return 0; } -static int pseudo_lock_dev_mremap(struct vm_area_struct *area, unsigned long flags) +static int pseudo_lock_dev_mremap(struct vm_area_struct *area) { /* Not supported */ return -EINVAL; diff --git a/fs/aio.c b/fs/aio.c index 1f32da13d39e..76ce0cc3ee4e 100644 --- a/fs/aio.c +++ b/fs/aio.c @@ -323,16 +323,13 @@ static void aio_free_ring(struct kioctx *ctx) } } -static int aio_ring_mremap(struct vm_area_struct *vma, unsigned long flags) +static int aio_ring_mremap(struct vm_area_struct *vma) { struct file *file = vma->vm_file; struct mm_struct *mm = vma->vm_mm; struct kioctx_table *table; int i, res = -EINVAL; - if (flags & MREMAP_DONTUNMAP) - return -EINVAL; - spin_lock(&mm->ioctx_lock); rcu_read_lock(); table = rcu_dereference(mm->ioctx_table); diff --git a/include/linux/mm.h b/include/linux/mm.h index 77e64e3eac80..8c3729eb3e38 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -570,7 +570,7 @@ struct vm_operations_struct { void (*close)(struct vm_area_struct * area); /* Called any time before splitting to check if it's allowed */ int (*may_split)(struct vm_area_struct *area, unsigned long addr); - int (*mremap)(struct vm_area_struct *area, unsigned long flags); + int (*mremap)(struct vm_area_struct *area); /* * Called by mprotect() to make driver-specific permission * checks before mprotect() is finalised. The VMA must not diff --git a/mm/mmap.c b/mm/mmap.c index 3f287599a7a3..9d7651e4e1fe 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3403,14 +3403,10 @@ static const char *special_mapping_name(struct vm_area_struct *vma) return ((struct vm_special_mapping *)vma->vm_private_data)->name; } -static int special_mapping_mremap(struct vm_area_struct *new_vma, - unsigned long flags) +static int special_mapping_mremap(struct vm_area_struct *new_vma) { struct vm_special_mapping *sm = new_vma->vm_private_data; - if (flags & MREMAP_DONTUNMAP) - return -EINVAL; - if (WARN_ON_ONCE(current->mm != new_vma->vm_mm)) return -EFAULT; diff --git a/mm/mremap.c b/mm/mremap.c index db5b8b28c2dd..d22629ff8f3c 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -545,7 +545,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, if (moved_len < old_len) { err = -ENOMEM; } else if (vma->vm_ops && vma->vm_ops->mremap) { - err = vma->vm_ops->mremap(new_vma, flags); + err = vma->vm_ops->mremap(new_vma); } if (unlikely(err)) {