From patchwork Mon May 20 14:00:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951277 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C223933 for ; Mon, 20 May 2019 14:00:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0AD342872E for ; Mon, 20 May 2019 14:00:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F294328775; Mon, 20 May 2019 14:00:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ADE672875A for ; Mon, 20 May 2019 14:00:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 348196B0005; Mon, 20 May 2019 10:00:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2F8B06B0007; Mon, 20 May 2019 10:00:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E8746B0008; Mon, 20 May 2019 10:00:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by kanga.kvack.org (Postfix) with ESMTP id ADBC26B0005 for ; Mon, 20 May 2019 10:00:20 -0400 (EDT) Received: by mail-lf1-f71.google.com with SMTP id k27so2626600lfj.21 for ; Mon, 20 May 2019 07:00:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=P9jpEkBpQw4ZOTJz/Lsq/b2TN30uIH42X6duHPmvzHQ=; b=sYhOSgTsdFnRBGcmO0M9WgQ9Z4i2jgQDKvJ6q0W5Pyew/EGEX4GVxc5l8RYRmHi9+L khA2b2KUcF1B/Xm5XkB1LtchHh3mMjzZS1TLEeD+79iBHV14JN05FoVnbMvqnpDMlifn 978/zs4NjJVdlN8mxCD+Ab5e/aqtFFqTfVlIwHBkyeufHpgRnTLluqu0r8T4RZ8HtWFJ RnKz+rqJHj6XNqjZTjDMJtV+IOXqY1jLntCSSBhxSO0c6dPxl9sKsHuXP3Nm2ht1cvCV Nvi9Jrrq7Sso9N1CRzZz7Y2Cj1q3Jpqf4E/F2fgySSC3PTtz6rFmLtNkt9Ns5wFu60Ld c8FQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAV2GWivJ+w9rPdnX4dRbC/YZiKWY/gYJhBxwv8Uop3rk7w8tlca kM6Jj/W9gC7qI6pwdEALiCK2g+DP0hTU4xHfehQnrG6zUp+u09JX7SJLS/i8xOI/l18Q+NovUV7 C+wqwJOJtRrJU2Be5wLxQWMpcH0AE2Naq7WSaiUnzgUW7OHxme9Kjsm/VDLNh4I0vmg== X-Received: by 2002:a2e:6c01:: with SMTP id h1mr11453377ljc.103.1558360820084; Mon, 20 May 2019 07:00:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqwvIFY2gmflkBFAm0egeSOWhkwbXSrar61W7xgpDkbLNIg6pQ9PwBSiOOuHSDXl195uZsCD X-Received: by 2002:a2e:6c01:: with SMTP id h1mr11453292ljc.103.1558360818877; Mon, 20 May 2019 07:00:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360818; cv=none; d=google.com; s=arc-20160816; b=G6oblSEckf18kH+mrThEc8lgErKiViB37+QilLIJGQ33pk3GO3AJSbaNc0Xu76HH7K wKbmSR1MUwUUIXuukEQbkbdYZ58D50Qswdi18klAetAs8+IzHHmtZ+LBBtUOj6orrCkv Gx7ZsRV4dEsQ6458mCyXmuEXx0/BT07aeBad0wYxbjl4xDAW6mExD5yH6kXVam0Ur7mE VkAmvc8oMLk9L4TFfKsSCMcA4uAwUZ3YHRlwtndi2j5WPUTJ1s77evjWFIKuO6MIPIwW joFpHMVocPjukvbDxuGznwTdKEf1HHX2VBWyfxwk0NgHjTSACJmcEqxEn3IYW5793uAf MMxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=P9jpEkBpQw4ZOTJz/Lsq/b2TN30uIH42X6duHPmvzHQ=; b=St1IlvzuHRfF5/1q5aSqh/iu4u7UaXHn/hjFCOIybU3InMRE012X+cq4MwVz4VmoXF XIrMYZaYOTYf+n8emxGrKx3V5ImqtpcK7qHqqP4niFbi/PLq7p8Hl8JSkmSZy+saH6n9 GEXqQwM+EOYC0CMnnzaEKzGFSAb7DPbesLclpEYnmAGB/5niXkNS7kVv0KZ9d6l6MzSP hLBY1un8cKjLa/XqW3Kge1qAsxqNrXtKa2OAYBDY1lFqTFFzSvR7EjFMMnMd/5ZPvg5q XHkvXNCnGFtW/2bmGHQkjNqdjYJli5hI5rWkw+ve9Hq7gyLQYhgdMc9iqRg5nMOl2WtX kslQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id b24si11314377lji.187.2019.05.20.07.00.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:18 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSipj-00082Z-CF; Mon, 20 May 2019 17:00:07 +0300 Subject: [PATCH v2 1/7] mm: Add process_vm_mmap() syscall declaration From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:07 +0300 Message-ID: <155836080726.2441.11153759042802992469.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Similar to process_vm_readv() and process_vm_writev(), add declarations of a new syscall, which will allow to map memory from or to another process. Signed-off-by: Kirill Tkhai --- arch/x86/entry/syscalls/syscall_32.tbl | 1 + arch/x86/entry/syscalls/syscall_64.tbl | 2 ++ include/linux/syscalls.h | 5 +++++ include/uapi/asm-generic/unistd.h | 5 ++++- init/Kconfig | 9 +++++---- kernel/sys_ni.c | 2 ++ 6 files changed, 19 insertions(+), 5 deletions(-) diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl index ad968b7bac72..99d6e0085576 100644 --- a/arch/x86/entry/syscalls/syscall_32.tbl +++ b/arch/x86/entry/syscalls/syscall_32.tbl @@ -438,3 +438,4 @@ 431 i386 fsconfig sys_fsconfig __ia32_sys_fsconfig 432 i386 fsmount sys_fsmount __ia32_sys_fsmount 433 i386 fspick sys_fspick __ia32_sys_fspick +434 i386 process_vm_mmap sys_process_vm_mmap __ia32_compat_sys_process_vm_mmap diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl index b4e6f9e6204a..46d7d2898f7a 100644 --- a/arch/x86/entry/syscalls/syscall_64.tbl +++ b/arch/x86/entry/syscalls/syscall_64.tbl @@ -355,6 +355,7 @@ 431 common fsconfig __x64_sys_fsconfig 432 common fsmount __x64_sys_fsmount 433 common fspick __x64_sys_fspick +434 common process_vm_mmap __x64_sys_process_vm_mmap # # x32-specific system call numbers start at 512 to avoid cache impact @@ -398,3 +399,4 @@ 545 x32 execveat __x32_compat_sys_execveat/ptregs 546 x32 preadv2 __x32_compat_sys_preadv64v2 547 x32 pwritev2 __x32_compat_sys_pwritev64v2 +548 x32 process_vm_mmap __x32_compat_sys_process_vm_mmap diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h index e2870fe1be5b..7d8ae36589cf 100644 --- a/include/linux/syscalls.h +++ b/include/linux/syscalls.h @@ -997,6 +997,11 @@ asmlinkage long sys_fspick(int dfd, const char __user *path, unsigned int flags) asmlinkage long sys_pidfd_send_signal(int pidfd, int sig, siginfo_t __user *info, unsigned int flags); +asmlinkage long sys_process_vm_mmap(pid_t pid, + unsigned long src_addr, + unsigned long len, + unsigned long dst_addr, + unsigned long flags); /* * Architecture-specific system calls diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h index a87904daf103..b7aaa5ae02da 100644 --- a/include/uapi/asm-generic/unistd.h +++ b/include/uapi/asm-generic/unistd.h @@ -844,9 +844,12 @@ __SYSCALL(__NR_fsconfig, sys_fsconfig) __SYSCALL(__NR_fsmount, sys_fsmount) #define __NR_fspick 433 __SYSCALL(__NR_fspick, sys_fspick) +#define __NR_process_vm_mmap 424 +__SC_COMP(__NR_process_vm_mmap, sys_process_vm_mmap, \ + compat_sys_process_vm_mmap) #undef __NR_syscalls -#define __NR_syscalls 434 +#define __NR_syscalls 435 /* * 32 bit systems traditionally used different diff --git a/init/Kconfig b/init/Kconfig index 8b9ffe236e4f..604db5f14718 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -320,13 +320,14 @@ config POSIX_MQUEUE_SYSCTL default y config CROSS_MEMORY_ATTACH - bool "Enable process_vm_readv/writev syscalls" + bool "Enable process_vm_readv/writev/mmap syscalls" depends on MMU default y help - Enabling this option adds the system calls process_vm_readv and - process_vm_writev which allow a process with the correct privileges - to directly read from or write to another process' address space. + Enabling this option adds the system calls process_vm_readv, + process_vm_writev and process_vm_mmap, which allow a process + with the correct privileges to directly read from or write to + or mmap another process' address space. See the man page for more details. config USELIB diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c index 4d9ae5ea6caf..6f51634f4f7e 100644 --- a/kernel/sys_ni.c +++ b/kernel/sys_ni.c @@ -316,6 +316,8 @@ COND_SYSCALL(process_vm_readv); COND_SYSCALL_COMPAT(process_vm_readv); COND_SYSCALL(process_vm_writev); COND_SYSCALL_COMPAT(process_vm_writev); +COND_SYSCALL(process_vm_mmap); +COND_SYSCALL_COMPAT(process_vm_mmap); /* compare kernel pointers */ COND_SYSCALL(kcmp); From patchwork Mon May 20 14:00:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951281 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3DD1176 for ; Mon, 20 May 2019 14:00:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A7E728821 for ; Mon, 20 May 2019 14:00:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E12028823; Mon, 20 May 2019 14:00:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1AFEB287E9 for ; Mon, 20 May 2019 14:00:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBF876B0007; Mon, 20 May 2019 10:00:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C95596B0008; Mon, 20 May 2019 10:00:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B83796B000A; Mon, 20 May 2019 10:00:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f69.google.com (mail-lf1-f69.google.com [209.85.167.69]) by kanga.kvack.org (Postfix) with ESMTP id 50C586B0007 for ; Mon, 20 May 2019 10:00:21 -0400 (EDT) Received: by mail-lf1-f69.google.com with SMTP id q3so2621817lfp.7 for ; Mon, 20 May 2019 07:00:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=WNDs88Ea4tbx39imqpysHHjvu2lJrySurfWdBdvnglE=; b=Clt67qRHcwZtaVmtuSpBAC/THKBthd1XkrLLIREERH4MKgzbCku0QGEzJBiyPInFZI HgYhSTgaK/grxcCXh0NOBxY4KTG5dWusMV47+rmisLtRipOU7LfytIEci/kaz7dzQtZW pEdr9thQLdinJ1LB4pcKJpIBGy8wIzM4KlctZL01cas/hDrR3alm9Fa9AKpyqbYtudhL WETYSULabPd1nJdYvGC/rT22fLiuiqixcrmUTf+xFOkB8YP8NaVwQt6tDbMLFN6YNaI2 mfCUYb0N5HOYu4ivpRCskK0m/6ihrBmQz8SW8b2pj2sAnq5g+tHfev0oTU2oJi/9KUs6 wIbg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAXbMHD0Yq4jSqoKyZCBLH2tRVC6aWryMcNfw3k4PFKKzfJCEJxH A5RZb8LSXU1X7Ujhdwt3jcI22lET0ndqOsZw3bUEmBhW+uPmAZLgWzJ7U5ygxS2Gh9pQf+kgKwU CuVVGCsRyxwrnl1rFNSazdZL7uFhMnkzPUpqKB3NbJNUr886pIkiqSBm4roy/T34OXA== X-Received: by 2002:ac2:4c98:: with SMTP id d24mr10790801lfl.146.1558360820731; Mon, 20 May 2019 07:00:20 -0700 (PDT) X-Google-Smtp-Source: APXvYqzHe47P9rAlKD+iFq6o9s0oRKGucI1DHwLEVkKKqgfnc5OkFg7VnoaSSozLpsdyxPXnH1xG X-Received: by 2002:ac2:4c98:: with SMTP id d24mr10790701lfl.146.1558360819036; Mon, 20 May 2019 07:00:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360819; cv=none; d=google.com; s=arc-20160816; b=HMsHNSFFDPVnZ1J1tBKBepc66AKI70mVis4XtrHzo6g9u5vH9PMqwubYUnJf/RCYe3 JXutyNw3KjFWbSSgauuapuYaRwPIPhoqEff3pc19rgFPdiKf1kcJayGO58sXTUQqE5aY 4lbLxczJpuBrsTl4ua3pDelxATnzgSAUwu5hyHWwxZUkw1HdAmo/46fitynTHJxNVFCz 9kPjmdwoWdP+NQxT85zbT8V0+1ps/gl+oPMNDfcS0k8QBkw3vtnAz21FE1b/BoagfgPg vmGNZcGbfrrxVCEvdIA1wv2S/Ytk6tENA0JlCvMO0NurCex67exdnH9YIwV9ADirX5iO TvJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=WNDs88Ea4tbx39imqpysHHjvu2lJrySurfWdBdvnglE=; b=hJB/62iuGt9CuxcF2lryirjWI9cH1B2PjluJBXYAk01+k0wLuZYyp2GeZUaZM7IOkq Gi9Ul0v9H5mMLvshb2zirYTJrxYGSEpoS6wuTLk5Ms+tDTvWLnZVPVb/f0kVg+Dyjvjg 0SS31/xLf5SIre8b3pQVCzzBYvb+iWJLOucRfw03XApj9UH8356ZFwUVLf0Qe/bAfpk7 Fvx+03W4TtJ0VFvkjOVf18lNW8cIjXYzCl1MekMe4I7MTnBxqCt6oGIN7r5t5TkPWSPV 03tlTY/zuvSFv5vXPP14h46SAe2IoFFYCnfGama4DnB3zwCGVuhBGk8cJr00koQgh+Io iMQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id n19si14940194lji.197.2019.05.20.07.00.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:19 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSipo-00082c-KS; Mon, 20 May 2019 17:00:12 +0300 Subject: [PATCH v2 2/7] mm: Extend copy_vma() From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:12 +0300 Message-ID: <155836081252.2441.9024100415314519956.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This prepares the function to copy a vma between two processes. Two new arguments are introduced. Signed-off-by: Kirill Tkhai --- include/linux/mm.h | 4 ++-- mm/mmap.c | 33 ++++++++++++++++++++++++--------- mm/mremap.c | 4 ++-- 3 files changed, 28 insertions(+), 13 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 0e8834ac32b7..afe07e4a76f8 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2329,8 +2329,8 @@ extern void __vma_link_rb(struct mm_struct *, struct vm_area_struct *, struct rb_node **, struct rb_node *); extern void unlink_file_vma(struct vm_area_struct *); extern struct vm_area_struct *copy_vma(struct vm_area_struct **, - unsigned long addr, unsigned long len, pgoff_t pgoff, - bool *need_rmap_locks); + struct mm_struct *, unsigned long addr, unsigned long len, + pgoff_t pgoff, bool *need_rmap_locks, bool clear_flags_ctx); extern void exit_mmap(struct mm_struct *); static inline int check_data_rlimit(unsigned long rlim, diff --git a/mm/mmap.c b/mm/mmap.c index 57803a0a3a5c..99778e724ad1 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3195,19 +3195,21 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma) } /* - * Copy the vma structure to a new location in the same mm, - * prior to moving page table entries, to effect an mremap move. + * Copy the vma structure to new location in the same vma + * prior to moving page table entries, to effect an mremap move; */ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, - unsigned long addr, unsigned long len, pgoff_t pgoff, - bool *need_rmap_locks) + struct mm_struct *mm, unsigned long addr, + unsigned long len, pgoff_t pgoff, + bool *need_rmap_locks, bool clear_flags_ctx) { struct vm_area_struct *vma = *vmap; unsigned long vma_start = vma->vm_start; - struct mm_struct *mm = vma->vm_mm; + struct vm_userfaultfd_ctx uctx; struct vm_area_struct *new_vma, *prev; struct rb_node **rb_link, *rb_parent; bool faulted_in_anon_vma = true; + unsigned long flags; /* * If anonymous vma has not yet been faulted, update new pgoff @@ -3220,15 +3222,25 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, if (find_vma_links(mm, addr, addr + len, &prev, &rb_link, &rb_parent)) return NULL; /* should never get here */ - new_vma = vma_merge(mm, prev, addr, addr + len, vma->vm_flags, - vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma), - vma->vm_userfaultfd_ctx); + + uctx = vma->vm_userfaultfd_ctx; + flags = vma->vm_flags; + if (clear_flags_ctx) { + uctx = NULL_VM_UFFD_CTX; + flags &= ~(VM_UFFD_MISSING | VM_UFFD_WP | VM_MERGEABLE | + VM_LOCKED | VM_LOCKONFAULT | VM_WIPEONFORK | + VM_DONTCOPY); + } + + new_vma = vma_merge(mm, prev, addr, addr + len, flags, vma->anon_vma, + vma->vm_file, pgoff, vma_policy(vma), uctx); if (new_vma) { /* * Source vma may have been merged into new_vma */ if (unlikely(vma_start >= new_vma->vm_start && - vma_start < new_vma->vm_end)) { + vma_start < new_vma->vm_end) && + vma->vm_mm == mm) { /* * The only way we can get a vma_merge with * self during an mremap is if the vma hasn't @@ -3249,6 +3261,9 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, new_vma = vm_area_dup(vma); if (!new_vma) goto out; + new_vma->vm_mm = mm; + new_vma->vm_flags = flags; + new_vma->vm_userfaultfd_ctx = uctx; new_vma->vm_start = addr; new_vma->vm_end = addr + len; new_vma->vm_pgoff = pgoff; diff --git a/mm/mremap.c b/mm/mremap.c index 37b5b2ad91be..9a96cfc28675 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -352,8 +352,8 @@ static unsigned long move_vma(struct vm_area_struct *vma, return err; new_pgoff = vma->vm_pgoff + ((old_addr - vma->vm_start) >> PAGE_SHIFT); - new_vma = copy_vma(&vma, new_addr, new_len, new_pgoff, - &need_rmap_locks); + new_vma = copy_vma(&vma, mm, new_addr, new_len, new_pgoff, + &need_rmap_locks, false); if (!new_vma) return -ENOMEM; From patchwork Mon May 20 14:00:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951283 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9CB6976 for ; Mon, 20 May 2019 14:00:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B0CC28841 for ; Mon, 20 May 2019 14:00:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7F0D028785; Mon, 20 May 2019 14:00:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3D73428834 for ; Mon, 20 May 2019 14:00:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE0486B000C; Mon, 20 May 2019 10:00:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EBAF86B000D; Mon, 20 May 2019 10:00:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA7A56B000E; Mon, 20 May 2019 10:00:26 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) by kanga.kvack.org (Postfix) with ESMTP id 637CC6B000C for ; Mon, 20 May 2019 10:00:26 -0400 (EDT) Received: by mail-lj1-f199.google.com with SMTP id 7so2407243ljr.23 for ; Mon, 20 May 2019 07:00:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=hONox51jQo6A/MbOkGQBVTkQwV5hGFmjaLe7PmVmgL0=; b=O5AzKiKscIyLoecBgsv5elUrzpIubtejmgpsJ+Mm5d2UYp2p8P+jiGDUDF9Cc1OYA0 ib3RC5gF8Nc9kehdfO6Krjoi/mbba6NYk3vEG08GwqIKX3P3Nmji0e7VcucCB6Xxq+tP r8ar7v9ZH4L80cZtSYopu2fpCZOk2YDGgjfM8Ie5kcEf9jnNNfJWhavAi5/jm9JJEaPu ptwZOksdXiniV2LMXYe0HfHfLUiAPRU/nn5QxlXdJNXoYgRQT4HxKd27xejH2ajx5Txy tEfif7+3li8zS2EAII1Ib7ILn0Tg/oADaSyhDzi31ziXyhL3tLFGdHNduaXouZ4TGwJA g93g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAWsdzhTDP0x+oWuEufvk3SKJO8WcbI4sOv48fasflttlSffEHaa 1BOy9ZrGTEyPbRY0UUeXm3w2/OpV7AlETGd3jVHt4YvdtGo3cpav5Rd8TDEuT87RQSsBjmpMcZe 8s0X517Pd+P1pIne2savj6+XrAAfNsGqZrkMYgEllot/YlkxrAIDKsTugb3cpZ/vWKQ== X-Received: by 2002:a2e:5b18:: with SMTP id p24mr37392273ljb.50.1558360825599; Mon, 20 May 2019 07:00:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqxTRtD1NsOEHn/lmweV9mXuM07ZR+i9dgj7Dm/EnR0T3rIY8EuIsv2BfXCLmpovg3ZvPfB+ X-Received: by 2002:a2e:5b18:: with SMTP id p24mr37392146ljb.50.1558360823064; Mon, 20 May 2019 07:00:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360823; cv=none; d=google.com; s=arc-20160816; b=Nx6XdmOfLodg93iIUymlKwCftLmU3Oy03/lak+K3qk6jML5yZlsz0SeDakT5PmWyYW VsZRi2Z2cW5Kjsyg4rPBDpWL9U0Hldc+RgZ8coWc8nMKPhvw1L0Cbx5LnWjTZmyTm1gV uJaOXOExmr+mQZaYJ1pKMWi6/02VMWFoqVPHhjdfVFixoXCkcwjHOnlMmZwYFFkeFU4l H8OO+bMN+S5c3f0t74F6QYUHrfq4l3p06025gw31JvrIAZJG94FeJGxmeuyXcNnShs8m iamQyu2qhoTIJmoNz18I18SqL/Z3bnmf1mQeSO/kxJ6taBDSm7itRDetvXnw5ey0e7i+ Do9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=hONox51jQo6A/MbOkGQBVTkQwV5hGFmjaLe7PmVmgL0=; b=iDkuipGG/605dOBH6V9/ZVXHzcC2mp7ZZvBdtA1xTWkz+UpOgrKDYsFw+vaOcCnUit bXFiQMHvcXqgaRlIRzfXWiImkNSW7rrxgCoavN1EmnWhTnEEH+5cmrGOci+xhdFSfIxi VFkJmqeZu+NuJx9pfKkLBdiIw8Ywo4IKYHg8tpXp8qytydY5QmfP1Imu0FAmz5O7G4OY 8EiHq0HfF00mu4v1oKVwTZf/JKikjU0CPWy+D0AxELB2I694TuiuMWGihEO72UbHNHYV WfKG1cOzp3xkIMKUusbz5Ctc/Ta3H9g+a23DHtB+Ew0W1oN309Oier+x+X8tirYUgqCf 13bw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id r5si13493343lfm.123.2019.05.20.07.00.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:23 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSipu-00082u-48; Mon, 20 May 2019 17:00:18 +0300 Subject: [PATCH v2 3/7] mm: Extend copy_page_range() From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:17 +0300 Message-ID: <155836081774.2441.16449538600904205092.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This allows to copy pages not only to the same addreses in another process, but also to a specified address. Huge pages and unaligned address cases are handled by splitting. Signed-off-by: Kirill Tkhai --- include/linux/huge_mm.h | 6 +- include/linux/mm.h | 3 + kernel/fork.c | 5 + mm/huge_memory.c | 30 ++++++--- mm/memory.c | 165 +++++++++++++++++++++++++++++++---------------- 5 files changed, 141 insertions(+), 68 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 7cd5c150c21d..1e6002ee7c44 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -9,11 +9,13 @@ extern vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf); extern int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, + pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long dst_addr, + unsigned long src_addr, unsigned long len, struct vm_area_struct *vma); extern void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd); extern int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pud_t *dst_pud, pud_t *src_pud, unsigned long addr, + pud_t *dst_pud, pud_t *src_pud, unsigned long dst_addr, + unsigned long src_addr, unsigned long len, struct vm_area_struct *vma); #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD diff --git a/include/linux/mm.h b/include/linux/mm.h index afe07e4a76f8..54328d08dbdd 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1485,7 +1485,8 @@ int walk_page_vma(struct vm_area_struct *vma, struct mm_walk *walk); void free_pgd_range(struct mmu_gather *tlb, unsigned long addr, unsigned long end, unsigned long floor, unsigned long ceiling); int copy_page_range(struct mm_struct *dst, struct mm_struct *src, - struct vm_area_struct *vma); + struct vm_area_struct *vma, unsigned long dst_addr, + unsigned long src_addr, unsigned long src_end); int follow_pte_pmd(struct mm_struct *mm, unsigned long address, struct mmu_notifier_range *range, pte_t **ptepp, pmd_t **pmdpp, spinlock_t **ptlp); diff --git a/kernel/fork.c b/kernel/fork.c index 45fde571c5dd..35f7240ed5c9 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -584,7 +584,10 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm, mm->map_count++; if (!(tmp->vm_flags & VM_WIPEONFORK)) - retval = copy_page_range(mm, oldmm, mpnt); + retval = copy_page_range(mm, oldmm, mpnt, + mpnt->vm_start, + mpnt->vm_start, + mpnt->vm_end); if (tmp->vm_ops && tmp->vm_ops->open) tmp->vm_ops->open(tmp); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 9f8bce9a6b32..f338b06f42c6 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -956,7 +956,8 @@ struct page *follow_devmap_pmd(struct vm_area_struct *vma, unsigned long addr, } int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, + pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long dst_addr, + unsigned long src_addr, unsigned long len, struct vm_area_struct *vma) { spinlock_t *dst_ptl, *src_ptl; @@ -969,6 +970,11 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (!vma_is_anonymous(vma)) return 0; + if (len != HPAGE_PMD_SIZE) { + split_huge_pmd(vma, src_pmd, src_addr); + return -EAGAIN; + } + pgtable = pte_alloc_one(dst_mm); if (unlikely(!pgtable)) goto out; @@ -990,12 +996,12 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd = swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*src_pmd)) pmd = pmd_swp_mksoft_dirty(pmd); - set_pmd_at(src_mm, addr, src_pmd, pmd); + set_pmd_at(src_mm, src_addr, src_pmd, pmd); } add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); mm_inc_nr_ptes(dst_mm); pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); - set_pmd_at(dst_mm, addr, dst_pmd, pmd); + set_pmd_at(dst_mm, dst_addr, dst_pmd, pmd); ret = 0; goto out_unlock; } @@ -1018,7 +1024,7 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, * reference. */ zero_page = mm_get_huge_zero_page(dst_mm); - set_huge_zero_page(pgtable, dst_mm, vma, addr, dst_pmd, + set_huge_zero_page(pgtable, dst_mm, vma, dst_addr, dst_pmd, zero_page); ret = 0; goto out_unlock; @@ -1032,9 +1038,9 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, mm_inc_nr_ptes(dst_mm); pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); - pmdp_set_wrprotect(src_mm, addr, src_pmd); + pmdp_set_wrprotect(src_mm, src_addr, src_pmd); pmd = pmd_mkold(pmd_wrprotect(pmd)); - set_pmd_at(dst_mm, addr, dst_pmd, pmd); + set_pmd_at(dst_mm, dst_addr, dst_pmd, pmd); ret = 0; out_unlock: @@ -1096,13 +1102,19 @@ struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr, } int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pud_t *dst_pud, pud_t *src_pud, unsigned long addr, + pud_t *dst_pud, pud_t *src_pud, unsigned long dst_addr, + unsigned long src_addr, unsigned long len, struct vm_area_struct *vma) { spinlock_t *dst_ptl, *src_ptl; pud_t pud; int ret; + if (len != HPAGE_PUD_SIZE) { + split_huge_pud(vma, src_pud, src_addr); + return -EAGAIN; + } + dst_ptl = pud_lock(dst_mm, dst_pud); src_ptl = pud_lockptr(src_mm, src_pud); spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); @@ -1121,9 +1133,9 @@ int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, /* No huge zero pud yet */ } - pudp_set_wrprotect(src_mm, addr, src_pud); + pudp_set_wrprotect(src_mm, src_addr, src_pud); pud = pud_mkold(pud_wrprotect(pud)); - set_pud_at(dst_mm, addr, dst_pud, pud); + set_pud_at(dst_mm, dst_addr, dst_pud, pud); ret = 0; out_unlock: diff --git a/mm/memory.c b/mm/memory.c index 0d0711a912de..9d0fe2aee5f2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -699,7 +699,7 @@ struct page *vm_normal_page_pmd(struct vm_area_struct *vma, unsigned long addr, static inline unsigned long copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *vma, - unsigned long addr, int *rss) + unsigned long src_addr, int *rss, unsigned long dst_addr) { unsigned long vm_flags = vma->vm_flags; pte_t pte = *src_pte; @@ -737,7 +737,7 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, pte = swp_entry_to_pte(entry); if (pte_swp_soft_dirty(*src_pte)) pte = pte_swp_mksoft_dirty(pte); - set_pte_at(src_mm, addr, src_pte, pte); + set_pte_at(src_mm, src_addr, src_pte, pte); } } else if (is_device_private_entry(entry)) { page = device_private_entry_to_page(entry); @@ -766,7 +766,7 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, is_cow_mapping(vm_flags)) { make_device_private_entry_read(&entry); pte = swp_entry_to_pte(entry); - set_pte_at(src_mm, addr, src_pte, pte); + set_pte_at(src_mm, src_addr, src_pte, pte); } } goto out_set_pte; @@ -777,7 +777,7 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, * in the parent and the child */ if (is_cow_mapping(vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); + ptep_set_wrprotect(src_mm, src_addr, src_pte); pte = pte_wrprotect(pte); } @@ -789,7 +789,7 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, pte = pte_mkclean(pte); pte = pte_mkold(pte); - page = vm_normal_page(vma, addr, pte); + page = vm_normal_page(vma, src_addr, pte); if (page) { get_page(page); page_dup_rmap(page, false); @@ -810,13 +810,14 @@ copy_one_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, } out_set_pte: - set_pte_at(dst_mm, addr, dst_pte, pte); + set_pte_at(dst_mm, dst_addr, dst_pte, pte); return 0; } static int copy_pte_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, struct vm_area_struct *vma, - unsigned long addr, unsigned long end) + unsigned long src_addr, unsigned long src_end, + unsigned long dst_addr) { pte_t *orig_src_pte, *orig_dst_pte; pte_t *src_pte, *dst_pte; @@ -828,10 +829,10 @@ static int copy_pte_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, again: init_rss_vec(rss); - dst_pte = pte_alloc_map_lock(dst_mm, dst_pmd, addr, &dst_ptl); + dst_pte = pte_alloc_map_lock(dst_mm, dst_pmd, dst_addr, &dst_ptl); if (!dst_pte) return -ENOMEM; - src_pte = pte_offset_map(src_pmd, addr); + src_pte = pte_offset_map(src_pmd, src_addr); src_ptl = pte_lockptr(src_mm, src_pmd); spin_lock_nested(src_ptl, SINGLE_DEPTH_NESTING); orig_src_pte = src_pte; @@ -854,11 +855,12 @@ static int copy_pte_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, continue; } entry.val = copy_one_pte(dst_mm, src_mm, dst_pte, src_pte, - vma, addr, rss); + vma, src_addr, rss, dst_addr); if (entry.val) break; progress += 8; - } while (dst_pte++, src_pte++, addr += PAGE_SIZE, addr != end); + } while (dst_pte++, src_pte++, dst_addr += PAGE_SIZE, + src_addr += PAGE_SIZE, src_addr != src_end); arch_leave_lazy_mmu_mode(); spin_unlock(src_ptl); @@ -872,108 +874,147 @@ static int copy_pte_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, return -ENOMEM; progress = 0; } - if (addr != end) + if (src_addr != src_end) goto again; return 0; } static inline int copy_pmd_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, pud_t *dst_pud, pud_t *src_pud, struct vm_area_struct *vma, - unsigned long addr, unsigned long end) + unsigned long src_addr, unsigned long src_end, unsigned long dst_addr) { + unsigned long src_next, dst_next, src_len, dst_len, dst_end, len; pmd_t *src_pmd, *dst_pmd; - unsigned long next; - dst_pmd = pmd_alloc(dst_mm, dst_pud, addr); + dst_pmd = pmd_alloc(dst_mm, dst_pud, dst_addr); if (!dst_pmd) return -ENOMEM; - src_pmd = pmd_offset(src_pud, addr); + src_pmd = pmd_offset(src_pud, src_addr); + dst_end = dst_addr + (src_end - src_addr); do { - next = pmd_addr_end(addr, end); + src_next = pmd_addr_end(src_addr, src_end); + dst_next = pmd_addr_end(dst_addr, dst_end); + src_len = src_next - src_addr; + dst_len = dst_next - dst_addr; + + len = min(src_len, dst_len); + src_next = src_addr + len; + dst_next = dst_addr + len; if (is_swap_pmd(*src_pmd) || pmd_trans_huge(*src_pmd) || pmd_devmap(*src_pmd)) { int err; - VM_BUG_ON_VMA(next-addr != HPAGE_PMD_SIZE, vma); - err = copy_huge_pmd(dst_mm, src_mm, - dst_pmd, src_pmd, addr, vma); + err = copy_huge_pmd(dst_mm, src_mm, dst_pmd, src_pmd, + dst_addr, src_addr, len, vma); if (err == -ENOMEM) return -ENOMEM; if (!err) - continue; + goto next; /* fall through */ } if (pmd_none_or_clear_bad(src_pmd)) - continue; + goto next; if (copy_pte_range(dst_mm, src_mm, dst_pmd, src_pmd, - vma, addr, next)) + vma, src_addr, src_next, dst_addr)) return -ENOMEM; - } while (dst_pmd++, src_pmd++, addr = next, addr != end); +next: + if (src_len == len) + src_pmd++; + if (dst_len == len) + dst_pmd++; + } while (src_addr = src_next, dst_addr = dst_next, src_addr != src_end); return 0; } static inline int copy_pud_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, p4d_t *dst_p4d, p4d_t *src_p4d, struct vm_area_struct *vma, - unsigned long addr, unsigned long end) + unsigned long src_addr, unsigned long src_end, unsigned long dst_addr) { + unsigned long src_next, dst_next, src_len, dst_len, dst_end, len; pud_t *src_pud, *dst_pud; - unsigned long next; - dst_pud = pud_alloc(dst_mm, dst_p4d, addr); + dst_pud = pud_alloc(dst_mm, dst_p4d, dst_addr); if (!dst_pud) return -ENOMEM; - src_pud = pud_offset(src_p4d, addr); + src_pud = pud_offset(src_p4d, src_addr); + dst_end = dst_addr + (src_end - src_addr); do { - next = pud_addr_end(addr, end); + src_next = pud_addr_end(src_addr, src_end); + dst_next = pud_addr_end(dst_addr, dst_end); + src_len = src_next - src_addr; + dst_len = dst_next - dst_addr; + + len = min(src_len, dst_len); + src_next = src_addr + len; + dst_next = dst_addr + len; + if (pud_trans_huge(*src_pud) || pud_devmap(*src_pud)) { int err; - VM_BUG_ON_VMA(next-addr != HPAGE_PUD_SIZE, vma); - err = copy_huge_pud(dst_mm, src_mm, - dst_pud, src_pud, addr, vma); + err = copy_huge_pud(dst_mm, src_mm, dst_pud, src_pud, + dst_addr, src_addr, len, vma); if (err == -ENOMEM) return -ENOMEM; if (!err) - continue; + goto next; /* fall through */ } if (pud_none_or_clear_bad(src_pud)) - continue; + goto next; if (copy_pmd_range(dst_mm, src_mm, dst_pud, src_pud, - vma, addr, next)) + vma, src_addr, src_next, dst_addr)) return -ENOMEM; - } while (dst_pud++, src_pud++, addr = next, addr != end); +next: + if (src_len == len) + src_pud++; + if (dst_len == len) + dst_pud++; + } while (src_addr = src_next, dst_addr = dst_next, src_addr != src_end); return 0; } static inline int copy_p4d_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, pgd_t *dst_pgd, pgd_t *src_pgd, struct vm_area_struct *vma, - unsigned long addr, unsigned long end) + unsigned long src_addr, unsigned long src_end, unsigned long dst_addr) { + unsigned long src_next, dst_next, src_len, dst_len, dst_end, len; p4d_t *src_p4d, *dst_p4d; - unsigned long next; - dst_p4d = p4d_alloc(dst_mm, dst_pgd, addr); + dst_p4d = p4d_alloc(dst_mm, dst_pgd, dst_addr); if (!dst_p4d) return -ENOMEM; - src_p4d = p4d_offset(src_pgd, addr); + + src_p4d = p4d_offset(src_pgd, src_addr); + dst_end = dst_addr + (src_end - src_addr); do { - next = p4d_addr_end(addr, end); + src_next = p4d_addr_end(src_addr, src_end); + dst_next = p4d_addr_end(dst_addr, dst_end); + src_len = src_next - src_addr; + dst_len = dst_next - dst_addr; + + len = min(src_len, dst_len); + src_next = src_addr + len; + dst_next = dst_addr + len; + if (p4d_none_or_clear_bad(src_p4d)) - continue; + goto next; if (copy_pud_range(dst_mm, src_mm, dst_p4d, src_p4d, - vma, addr, next)) + vma, src_addr, src_next, dst_addr)) return -ENOMEM; - } while (dst_p4d++, src_p4d++, addr = next, addr != end); +next: + if (src_len == len) + src_p4d++; + if (dst_len == len) + dst_p4d++; + } while (src_addr = src_next, dst_addr = dst_next, src_addr != src_end); return 0; } int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, - struct vm_area_struct *vma) + struct vm_area_struct *vma, unsigned long dst_addr, + unsigned long src_addr, unsigned long src_end) { pgd_t *src_pgd, *dst_pgd; - unsigned long next; - unsigned long addr = vma->vm_start; - unsigned long end = vma->vm_end; + unsigned long src_next, dst_next, src_len, dst_len, dst_end, len; struct mmu_notifier_range range; bool is_cow; int ret; @@ -1011,23 +1052,37 @@ int copy_page_range(struct mm_struct *dst_mm, struct mm_struct *src_mm, if (is_cow) { mmu_notifier_range_init(&range, MMU_NOTIFY_PROTECTION_PAGE, - 0, vma, src_mm, addr, end); + 0, vma, src_mm, src_addr, src_end); mmu_notifier_invalidate_range_start(&range); } ret = 0; - dst_pgd = pgd_offset(dst_mm, addr); - src_pgd = pgd_offset(src_mm, addr); + dst_pgd = pgd_offset(dst_mm, dst_addr); + src_pgd = pgd_offset(src_mm, src_addr); + dst_end = dst_addr + (src_end - src_addr); do { - next = pgd_addr_end(addr, end); + src_next = pgd_addr_end(src_addr, src_end); + dst_next = pgd_addr_end(dst_addr, dst_end); + src_len = src_next - src_addr; + dst_len = dst_next - dst_addr; + + len = min(src_len, dst_len); + src_next = src_addr + len; + dst_next = dst_addr + len; + if (pgd_none_or_clear_bad(src_pgd)) - continue; + goto next; if (unlikely(copy_p4d_range(dst_mm, src_mm, dst_pgd, src_pgd, - vma, addr, next))) { + vma, src_addr, src_next, dst_addr))) { ret = -ENOMEM; break; } - } while (dst_pgd++, src_pgd++, addr = next, addr != end); +next: + if (src_len == len) + src_pgd++; + if (dst_len == len) + dst_pgd++; + } while (src_addr = src_next, dst_addr = dst_next, src_addr != src_end); if (is_cow) mmu_notifier_invalidate_range_end(&range); From patchwork Mon May 20 14:00:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951285 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E87D76 for ; Mon, 20 May 2019 14:00:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0D8928846 for ; Mon, 20 May 2019 14:00:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EEA822887B; Mon, 20 May 2019 14:00:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E5D428898 for ; Mon, 20 May 2019 14:00:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D5CF36B000E; Mon, 20 May 2019 10:00:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CBDAF6B0266; Mon, 20 May 2019 10:00:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B860E6B0010; Mon, 20 May 2019 10:00:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by kanga.kvack.org (Postfix) with ESMTP id 5639A6B000D for ; Mon, 20 May 2019 10:00:28 -0400 (EDT) Received: by mail-lj1-f198.google.com with SMTP id m2so2424659ljj.13 for ; Mon, 20 May 2019 07:00:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=UT1yo5i/SDXJHxF74x02x37C8rHVZsb1Uhc8HALyM8A=; b=B261YOBk90cGiJS6/GoHotqE4hOpkNOFeTC74v47+NkTudtfL0DJ8JpT3++TejnCyR 63L2R9764bfRJVUudJ6+Wmj2PYoVYM+EHJfq9K3LbW74zyI7VHtCJFs/w8NDMEyhV5mZ nQyOGGeqRp9Ye2iOg9nxfZSLssqe7+vdxKT57KvVHZ3ZZS0AfBYN02/9LT5cvDBA+Fwm tptnMOSjiJf+Rw4YywVF/hdgnxNGH5ltNUM/IipHIzZ+iteWjkRdcIVLF4PzYMZmK5Fe wtoCE4DAEIOdbBRzuy4ePITDNVas61RsS4KrOSpTAAWj2nocU+95LWP5alQiIixHAgfg EcpQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAX29hr2l98vfZinFHd86Ey9WDIIBOrZdZl1X6IWvVbGMXYZXkGr dl4K1ndr0kvlKHGPWgLwYwKCGtYV3j2BTndcqODjR/iEXlnckq0u4iPWt01DNrqx5/5gVkccKtA Eo9AA1SRsBcDdKtAsyR0fVR1IfFGju4Hat4dyRdaHDJjpKw73KjKOnqXUBCl/xBE4bg== X-Received: by 2002:ac2:48a8:: with SMTP id u8mr13965813lfg.141.1558360827731; Mon, 20 May 2019 07:00:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqyrSu5iln8tIfEzGyn4baM8KwLrgDXLIjp8slx8BYhrOtiH9SiL/ibRQeQOaQMBwkCeGHUi X-Received: by 2002:ac2:48a8:: with SMTP id u8mr13965770lfg.141.1558360826792; Mon, 20 May 2019 07:00:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360826; cv=none; d=google.com; s=arc-20160816; b=r+QWwc1r49JMjKwgxRGZ0WYwSyeqByqcjDlwzD4tsw1JpoSOORhb11EUk+1jxeF42L DW5Xn8IY6p4SDheBD8aW2BSIyejN0OnOLDhPSyJ1vOAngcQwenBuO6mNQ7og/dVJZcGk fBxXiddFhySZmgy37jcEbPIRcMo/M1jeDJdD9HpTLVnF6vhJUFVyYVtgYiprCVEkllgQ FbqiW5IBSGYU9p+YU71HPNIQN4P4IPgOX9AIjRbDbS4rTfckoMuBK/RrKXCD+9LA+HqQ BC/VF3fMh2eVIUr7K6lDrj+0R3y2URbPVJEzbxRQfflpFF366iTp7mFi/he14YACnyxl Husw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=UT1yo5i/SDXJHxF74x02x37C8rHVZsb1Uhc8HALyM8A=; b=MhuHg6JjOrLjAS880eiRUcZdtD+ks3wPMfApXB/u3JtkJAEZsBId/HszdkBMBBs0P8 3w32nXXJztH1NfdCyGelR5Tu5UwEFuETUggidbq69M/pzBVLf/7Uyih45dZ+3hGqrOeK kMBwy7m0+MigKbbxNTaWuOpGvJtmXWv63krUzpfMaFGaJzuiqYTmzvtzR2AqC2VmU2wX s3ynCP/P0nVH/TuAodrDZO5HUNR/AU4A3JsFayCnfyPVQtVOVyd7Mm6XtO4HgO7x/xXz nZYKVQ2/bvSbMCkbA+8Ud4ZIROGct7qa4Or2Ld+cyZMpAH466i/RUNe1ygKC3Sx4m0SO 5t0Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id u27si13873560lfq.11.2019.05.20.07.00.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:26 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSipz-00083D-HH; Mon, 20 May 2019 17:00:23 +0300 Subject: [PATCH v2 4/7] mm: Export round_hint_to_min() From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:23 +0300 Message-ID: <155836082337.2441.15115541609966690918.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Signed-off-by: Kirill Tkhai --- include/linux/mman.h | 14 ++++++++++++++ mm/mmap.c | 13 ------------- 2 files changed, 14 insertions(+), 13 deletions(-) diff --git a/include/linux/mman.h b/include/linux/mman.h index 4b08e9c9c538..69feb3144c12 100644 --- a/include/linux/mman.h +++ b/include/linux/mman.h @@ -4,6 +4,7 @@ #include #include +#include #include #include @@ -73,6 +74,19 @@ static inline void vm_unacct_memory(long pages) vm_acct_memory(-pages); } +/* + * If a hint addr is less than mmap_min_addr change hint to be as + * low as possible but still greater than mmap_min_addr + */ +static inline unsigned long round_hint_to_min(unsigned long hint) +{ + hint &= PAGE_MASK; + if (((void *)hint != NULL) && + (hint < mmap_min_addr)) + return PAGE_ALIGN(mmap_min_addr); + return hint; +} + /* * Allow architectures to handle additional protection bits */ diff --git a/mm/mmap.c b/mm/mmap.c index 99778e724ad1..e4ced5366643 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1318,19 +1318,6 @@ struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *vma) return NULL; } -/* - * If a hint addr is less than mmap_min_addr change hint to be as - * low as possible but still greater than mmap_min_addr - */ -static inline unsigned long round_hint_to_min(unsigned long hint) -{ - hint &= PAGE_MASK; - if (((void *)hint != NULL) && - (hint < mmap_min_addr)) - return PAGE_ALIGN(mmap_min_addr); - return hint; -} - static inline int mlock_future_check(struct mm_struct *mm, unsigned long flags, unsigned long len) From patchwork Mon May 20 14:00:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951287 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8113276 for ; Mon, 20 May 2019 14:00:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F1BF28843 for ; Mon, 20 May 2019 14:00:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 634302886F; Mon, 20 May 2019 14:00:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EDE5A2887B for ; Mon, 20 May 2019 14:00:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9C7C6B0266; Mon, 20 May 2019 10:00:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A74C66B0269; Mon, 20 May 2019 10:00:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8EEFC6B026A; Mon, 20 May 2019 10:00:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f72.google.com (mail-lf1-f72.google.com [209.85.167.72]) by kanga.kvack.org (Postfix) with ESMTP id 28A986B0266 for ; Mon, 20 May 2019 10:00:34 -0400 (EDT) Received: by mail-lf1-f72.google.com with SMTP id k27so2626708lfj.21 for ; Mon, 20 May 2019 07:00:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=Lzk/TJG16V8mXaPrVAS/SPcGg77NW4YCV5EqRztW4Nw=; b=Ws9xIL9pHUbJ+K9/eFSwTAKGrr4Vvc6XVV9gJQuI+/XUHaHSoy6jMvP7I9EJYMvBCc caW8rAZEtSX7WJA74yFP7v1QhuAh/MB7kisg24pu5yZ+iLfCiPA0Ya2Ijdrux37gDBcc 64uP2mUaDg2GtDP9sr2M+qq9iDrH2Fvb0S4J1OQJkhFessE0ZDc94Wn7MInvHAABlCZS R1wjJTp7M8Mj9aQIZmZLzkrrsNobBe0tw6CGY01IwPtbm2o7ORX7C+lOlLX3eY1kmS1T W5WeQwpZzuQGPr2CMIQMlR9QpjnOkMMkTxuqwP/XjHxj11jxdF2MzJNlC8UeaM+mK5zA qBvw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAURlYOqJlUup6vUNKlxL0xbGs8QPDeDA3BVvAuBCl8S4LjOMtMp xUM+M8XICc2OTDbCxgHYdDYrp3Pz3CAy/A/wy+Dah+fKGtQT26bjli1AclPT29hOetLHgpp3PId ogocH6XIzXjKHU7gxyEqaMYd0eaQ3OcPnqSu9Ny+c607W+PTih4QQfM5N56HlfGY3ww== X-Received: by 2002:ac2:494f:: with SMTP id o15mr22584342lfi.22.1558360833555; Mon, 20 May 2019 07:00:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqzEUJWkFQoQZUHjCkKN2Fn025WPJpGHGGfCjunBwh2t+3OFN2S17anJMnRXBL2GPB7e4bNF X-Received: by 2002:ac2:494f:: with SMTP id o15mr22584266lfi.22.1558360832042; Mon, 20 May 2019 07:00:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360832; cv=none; d=google.com; s=arc-20160816; b=e+/ZtapWkkDVgTk7UDWxJ5FFTfhvaXI7TE2dOrahW12XCYMGrnT385zCryKlwg4lRf tLia1w293OKCpcIQQWFYk1XRt0Vl1UyC8TEIgb5OGbNWQELOQzK+9VVvcr/fDe37YBI0 eekpM66LqSphm0n/8q4aj5scEkFI4iikFdJJUtw0SP9JHJD2AUCaaZv2+A5ooHf988ri nrn08z71HrpjNAzGDYzE6rhchvyKjS5XLMpJ5zf5M8BYPlFFW23qT9kGziyIa66o7F9W r9+wGbpGrhybsYPKvBQI+nXdy5doDcwrexg2NifWAkUjz9ZFhJ6Jtd7xJGto+lGAFJ9a wNDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=Lzk/TJG16V8mXaPrVAS/SPcGg77NW4YCV5EqRztW4Nw=; b=bdz7l9YTiafDSlUB0o3wANZK8N6JuFrRZNOgX29qn+/eMk75w0ZoiUN6jokb3jGOIn 4PKHITnrpJxmGRdP4UoUrtQIFaH2wj3orSSBjmKimT8FG205/ACFCD0J4xCMhrFmliA0 HSVngTWUlqvQ0+9ZZjW0tuRZFRXdQUMjAqS2u1stOQ0tbuyKF4MM425wzXx/4zqEIxcD F2ogLM7pTOpTm5dHw1kJOue3KlW9sAN+VJ/u346VUDIWcr/furIjb5yeCD/ZvVAeLNjA X9m5Q1pk7RDy66KuVyy+ExQ4qdTQ2DpQ7EEG15kdurlEoEmdGVNHv67dhToU/drF6tRh turQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id m13si5851967ljh.20.2019.05.20.07.00.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:32 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSiq4-00083Y-T0; Mon, 20 May 2019 17:00:29 +0300 Subject: [PATCH v2 5/7] mm: Introduce may_mmap_overlapped_region() helper From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:28 +0300 Message-ID: <155836082877.2441.3415778176783960096.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Extract address space limit check for overlapped regions in a separate helper. v2: New Signed-off-by: Kirill Tkhai --- mm/mmap.c | 33 ++++++++++++++++++++------------- 1 file changed, 20 insertions(+), 13 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index e4ced5366643..260e47e917e6 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -583,6 +583,24 @@ static unsigned long count_vma_pages_range(struct mm_struct *mm, return nr_pages; } +/* + * Check against address space limit, whether we may expand mm + * with a new mapping. Currently mapped in the given range pages + * are not accounted in the limit. + */ +static bool may_mmap_overlapped_region(struct mm_struct *mm, + unsigned long vm_flags, unsigned long addr, unsigned long len) +{ + unsigned long nr_pages = len >> PAGE_SHIFT; + + if (!may_expand_vm(mm, vm_flags, nr_pages)) { + nr_pages -= count_vma_pages_range(mm, addr, addr + len); + if (!may_expand_vm(mm, vm_flags, nr_pages)) + return false; + } + return true; +} + void __vma_link_rb(struct mm_struct *mm, struct vm_area_struct *vma, struct rb_node **rb_link, struct rb_node *rb_parent) { @@ -1697,19 +1715,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long charged = 0; /* Check against address space limit. */ - if (!may_expand_vm(mm, vm_flags, len >> PAGE_SHIFT)) { - unsigned long nr_pages; - - /* - * MAP_FIXED may remove pages of mappings that intersects with - * requested mapping. Account for the pages it would unmap. - */ - nr_pages = count_vma_pages_range(mm, addr, addr + len); - - if (!may_expand_vm(mm, vm_flags, - (len >> PAGE_SHIFT) - nr_pages)) - return -ENOMEM; - } + if (!may_mmap_overlapped_region(mm, vm_flags, addr, len)) + return -ENOMEM; /* Clear old maps */ while (find_vma_links(mm, addr, addr + len, &prev, &rb_link, From patchwork Mon May 20 14:00:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F8CD76 for ; Mon, 20 May 2019 14:00:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4A93D28426 for ; Mon, 20 May 2019 14:00:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3D15828846; Mon, 20 May 2019 14:00:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 90D1A28426 for ; Mon, 20 May 2019 14:00:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B87956B026B; Mon, 20 May 2019 10:00:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B5D806B026C; Mon, 20 May 2019 10:00:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4C926B026D; Mon, 20 May 2019 10:00:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lf1-f71.google.com (mail-lf1-f71.google.com [209.85.167.71]) by kanga.kvack.org (Postfix) with ESMTP id 3E4C96B026B for ; Mon, 20 May 2019 10:00:39 -0400 (EDT) Received: by mail-lf1-f71.google.com with SMTP id e11so2632732lfn.19 for ; Mon, 20 May 2019 07:00:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=qzFh73EFiUphac3v9UKerHFDloFYF/p8i538X3csTX4=; b=BgPYxkYsdIAqzgbImGuFaiYjH93M0NqogfJquOmGPpz/bopj0lfJd8prrLPmaef1Ym 0x1UM+kZwtrHwuswpUFGx95eREYmHRbHHOdqYdpe6HhHoEOj2ys0tZoy2TcNou8FDqhM xbqRGayJL24hgmIOy7X7YvF7rzZ/dkE+BOIkYOn421y8hqFJUKxhYKNbsgMl6/eyEOTr IonGClv/m6VZLv8MqHmOuc1M0p6aCIgPoczZi98Ar0PGP5yUcLuUXPVf6fnqQ64rBSA8 rZGtKNzMpxx1nLss4jMZYCFaggIcocL2y1iTkM8m+7scWH//3efV0HWga0Nd5VcmC1+t 8goA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAVUT/41zj7+Ye3Ka48WCxzRvvMyjOfcJbBY3zlHHt9r1tmHTOgz 6I8ZO8z6zPxLwc1MbOL7A4UkwrRkrwrSCA4S+JUDZAPurbqGWpN+S16WcgT1SpnyGfd0XAI531z PqUtUB3exS0aFwhncU73qN3gHv3MH0X/jUIyN3qyVsQupjCppiX1Hw+nDkjw7SbzYPA== X-Received: by 2002:a2e:9092:: with SMTP id l18mr3209641ljg.8.1558360838693; Mon, 20 May 2019 07:00:38 -0700 (PDT) X-Google-Smtp-Source: APXvYqyP0tNK9INNt8xyEdYq/CWEjP/DyUN2dr7taZocM+mvxWIj4M5GpsC8wLHakXHoYuCTwL6S X-Received: by 2002:a2e:9092:: with SMTP id l18mr3209582ljg.8.1558360837656; Mon, 20 May 2019 07:00:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360837; cv=none; d=google.com; s=arc-20160816; b=d83w/zTvAmkTgKC+RhBtXFcsD2KzD11s2F2aRT9aDDhsVcQ58ny36HB/qcTViW/NfC EEcfQhj43Re15l5jmR5X9oYTp1Bu6vQA/7ABsrFFYd+ys+MyEfdoFF3HS6MTZiJ08FR4 E+MwufZXDz1G4dmvWK8G3HVkCYu/+t+pg4vi9J+bfEoMFQJKfw8RcEHfCsPteL/wp12m EqCrNiSmGDMAFCW/J+SXS1HGU5C9XiDC70hqR9YZR7nPA+VOx9W9fs/eUdXSDwLlQNjV oVgHI/UizSx5IGZeblwp+u3LnFIFJy0xLI6lx2qjtNVqBQE1ni+eYtUxaYSBn6hkppqb 8vrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=qzFh73EFiUphac3v9UKerHFDloFYF/p8i538X3csTX4=; b=YxgYONYwBXbRUgViJiYofvg1nRhI8IIP3oszBxoQEJC7IaQH/fq9EAka5K1aJxLKgO JbA5qH2Dt0LB7Bjn3EUBEDfRFcb/bJaTMWfOMBCfHGlaKw39DQZ1me0E3hZXWDE5E6Cy uUat6+wYV+XFxpXq1Ej4l6eo8ZhMIHc9XpFLvWCkmGbaRidl+jKhm4QOdgWRfuRkYRnN izHgNSTH0H9M7ah9bXz9vPi/flQEyqMeahY/CnMq/tdmVfyoh0sNLhxyiIz0DJ2kKCwi ha3uroF117cai11S2z9IH5JqpqSJnSsEOidDZouuWHKV1TtGN+0AqlGoNuXer5OHa4FO fkfQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id v2si14275154ljg.12.2019.05.20.07.00.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:37 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSiqA-00083v-6q; Mon, 20 May 2019 17:00:34 +0300 Subject: [PATCH v2 6/7] mm: Introduce find_vma_filter_flags() helper From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:34 +0300 Message-ID: <155836083406.2441.7999607190635457587.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This patch introduce a new helper, which returns vma of enough length at given address, but only if it does not contain passed flags. v2: New Signed-off-by: Kirill Tkhai --- include/linux/mm.h | 3 +++ mm/mremap.c | 39 ++++++++++++++++++++++++++------------- 2 files changed, 29 insertions(+), 13 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 54328d08dbdd..65ceb56acd44 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1515,6 +1515,9 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start, pgoff_t nr, bool even_cows); void unmap_mapping_range(struct address_space *mapping, loff_t const holebegin, loff_t const holelen, int even_cows); +struct vm_area_struct *find_vma_without_flags(struct mm_struct *mm, + unsigned long addr, unsigned long len, + unsigned long prohibited_flags); #else static inline vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags) diff --git a/mm/mremap.c b/mm/mremap.c index 9a96cfc28675..dabae6a70287 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -430,14 +430,37 @@ static unsigned long move_vma(struct vm_area_struct *vma, return new_addr; } +struct vm_area_struct *find_vma_without_flags(struct mm_struct *mm, + unsigned long addr, unsigned long len, + unsigned long prohibited_flags) +{ + struct vm_area_struct *vma = find_vma(mm, addr); + + if (!vma || vma->vm_start > addr) + return ERR_PTR(-EFAULT); + + /* vm area boundaries crossing */ + if (len > vma->vm_end - addr) + return ERR_PTR(-EFAULT); + + if (vma->vm_flags & prohibited_flags) + return ERR_PTR(-EFAULT); + + return vma; +} + static struct vm_area_struct *vma_to_resize(unsigned long addr, unsigned long old_len, unsigned long new_len, unsigned long *p) { struct mm_struct *mm = current->mm; - struct vm_area_struct *vma = find_vma(mm, addr); - unsigned long pgoff; + struct vm_area_struct *vma; + unsigned long pgoff, prohibited_flags = VM_HUGETLB; - if (!vma || vma->vm_start > addr) + if (old_len != new_len) + prohibited_flags |= VM_DONTEXPAND | VM_PFNMAP; + + vma = find_vma_without_flags(mm, addr, old_len, prohibited_flags); + if (IS_ERR(vma)) return ERR_PTR(-EFAULT); /* @@ -453,13 +476,6 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, return ERR_PTR(-EINVAL); } - if (is_vm_hugetlb_page(vma)) - return ERR_PTR(-EINVAL); - - /* We can't remap across vm area boundaries */ - if (old_len > vma->vm_end - addr) - return ERR_PTR(-EFAULT); - if (new_len == old_len) return vma; @@ -469,9 +485,6 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, if (pgoff + (new_len >> PAGE_SHIFT) < pgoff) return ERR_PTR(-EINVAL); - if (vma->vm_flags & (VM_DONTEXPAND | VM_PFNMAP)) - return ERR_PTR(-EFAULT); - if (vma->vm_flags & VM_LOCKED) { unsigned long locked, lock_limit; locked = atomic64_read(&mm->locked_vm) << PAGE_SHIFT; From patchwork Mon May 20 14:00:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kirill Tkhai X-Patchwork-Id: 10951291 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2B0E8933 for ; Mon, 20 May 2019 14:00:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 188412872E for ; Mon, 20 May 2019 14:00:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0CE5D2887D; Mon, 20 May 2019 14:00:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B0102887B for ; Mon, 20 May 2019 14:00:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE5136B026D; Mon, 20 May 2019 10:00:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CBC2E6B026E; Mon, 20 May 2019 10:00:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD3796B026F; Mon, 20 May 2019 10:00:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by kanga.kvack.org (Postfix) with ESMTP id 5287D6B026D for ; Mon, 20 May 2019 10:00:44 -0400 (EDT) Received: by mail-lj1-f198.google.com with SMTP id v12so2160292ljv.7 for ; Mon, 20 May 2019 07:00:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:date:message-id:in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=ioYOyKuzdQzKpmA7+EHjpRFnK6vAt19ymD2CLyn7WXo=; b=ZH5K4bai+ay6RUlzxIThQnCqp4zDSdwXAhrRmPQDTp/30fBvxGP2+5OKx0zi+405tY xZllwYB/kro7x+AAn5tqF6TGhxi/es4k8WbxRB01mQKIGf2sqX+NCIW9tcpT0OQfSl7o RR6xCEuDEGsLV73GELANsSMV7CdaoNzaFJKeqGDcZSeMeGJZNxireAjWhm2LuKipsnYc EiFUvc858sqS+VOAWJgVew5H0NMu+rSHj64Kk1BZTwpIrdyR5i07N32ocMIT+nF4lEzS UZGX0xrcG3XLfdsjlVDECMWi2SiyGLG+eD5+OrbrNJtXv/ffDe9PVHEBOczwZ/xO+7cm yIsw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com X-Gm-Message-State: APjAAAVhT/aU0r6T7/jzzUe52KPMxrKHoaySPdoc17fAwc3hJiM7MsXs ak0+GAzJRO6gP3NYYJW2HDnfiF8R7gxL7ft1sIs4T/xzW9ysF72T01TguJFNPYrG+L7Bb31xhv2 rEMtqgke+UlKPRywLjLDe2WjQ72HLXSv4wZICY7lhAPhl2VxfA3dH+KTwHx+YZ1w1LQ== X-Received: by 2002:a2e:1412:: with SMTP id u18mr2974471ljd.197.1558360843740; Mon, 20 May 2019 07:00:43 -0700 (PDT) X-Google-Smtp-Source: APXvYqz1ylpmSwx+Q0ZDXVZNBSFXbZ4aKUBsOtxoPQf9EgfvXoJZObJLcWphIl2g24bQJn22RlMJ X-Received: by 2002:a2e:1412:: with SMTP id u18mr2974408ljd.197.1558360842530; Mon, 20 May 2019 07:00:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558360842; cv=none; d=google.com; s=arc-20160816; b=DjByWxe8B335tJ7ZvtiPTTQOJeQVpAecqmr3orjBxQE6WJ/EvdYgYz9Wo0lzZCoAV6 YA1ajQh+xv+S/MC8FodIx5hHLbwmqzoPCW6Ci08pmKsyPlcB3tv9x/aF7bYNFr2cq6pn AMqyCj40IY+cqvtm6MrzR2oeUqsjITHPgj0nks08YEQg+YmivksKZirvDvSXc7MRdlKh LOGV47Xp6zWNfgYlPJh5aN9Kn+bAhF0b/4m9buP06IHoaPmyzzT53yzphHBqfowGHUZj cCoBQwaaobyNC1Uc4EyftGpMQqsAYN5Kx6wh0CQqM8e5ticBeRz21mKOpxKgKH8JSqPi t+FQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:to:from:subject; bh=ioYOyKuzdQzKpmA7+EHjpRFnK6vAt19ymD2CLyn7WXo=; b=w84xNhPMW9xAmmHNVMCufPwuJ9LmEluxlZNXp/IaUWs3ybyNQwF4q7omYc12y7Le7g i4T5ykUpBpBBUbvUrBURA7vgCQro8a+hBvSey/Es96PpGohzN7diuvSMojJQe5WmGaHX j5941NQ/jvxaWIyhp91NA5+CsgEJXWVjeop0Rdva+Gijrs7B50WkxBMjoY/l24b5MuNY 2QaHnVoGBfOIi2D3pSy3FhtVVH5/I4qYWgDHXNoTjDUH7XVCRdBGFY2EGbvEGyPcXast SUDSRBVYgUUZamnVcVeUxbA7Z/rXoJpNQFBiAP4Z6hX91TSfn5VDudHOul5I1ChMzqtg dHFQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from relay.sw.ru (relay.sw.ru. [185.231.240.75]) by mx.google.com with ESMTPS id m70si16442248lje.194.2019.05.20.07.00.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 May 2019 07:00:42 -0700 (PDT) Received-SPF: pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) client-ip=185.231.240.75; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ktkhai@virtuozzo.com designates 185.231.240.75 as permitted sender) smtp.mailfrom=ktkhai@virtuozzo.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=virtuozzo.com Received: from [172.16.25.169] (helo=localhost.localdomain) by relay.sw.ru with esmtp (Exim 4.91) (envelope-from ) id 1hSiqF-00084l-Gn; Mon, 20 May 2019 17:00:39 +0300 Subject: [PATCH v2 7/7] mm: Add process_vm_mmap() From: Kirill Tkhai To: akpm@linux-foundation.org, dan.j.williams@intel.com, ktkhai@virtuozzo.com, mhocko@suse.com, keith.busch@intel.com, kirill.shutemov@linux.intel.com, alexander.h.duyck@linux.intel.com, ira.weiny@intel.com, andreyknvl@google.com, arunks@codeaurora.org, vbabka@suse.cz, cl@linux.com, riel@surriel.com, keescook@chromium.org, hannes@cmpxchg.org, npiggin@gmail.com, mathieu.desnoyers@efficios.com, shakeelb@google.com, guro@fb.com, aarcange@redhat.com, hughd@google.com, jglisse@redhat.com, mgorman@techsingularity.net, daniel.m.jordan@oracle.com, jannh@google.com, kilobyte@angband.pl, linux-api@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Date: Mon, 20 May 2019 17:00:39 +0300 Message-ID: <155836083941.2441.1939229815005692644.stgit@localhost.localdomain> In-Reply-To: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> References: <155836064844.2441.10911127801797083064.stgit@localhost.localdomain> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This adds a new syscall to duplicate a VMA of other process to current. Flag PVMMAP_FIXED may be specified, its meaning is similar to mmap()'s MAP_FIXED. VMA are merged on destination, i.e. if source task has VMA with address [start; end], and we map it sequentially twice: process_vm_mmap(@pid, start, start + (end - start)/2, ...); process_vm_mmap(@pid, start + (end - start)/2, end, ...); the destination task will have single vma [start, end]. v2: Add PVMMAP_FIXED_NOREPLACE flag. Use find_vma_without_flags() and may_mmap_overlapped_region() helpers. Fix whitespaces. Previous version has a possibility to duplicate VMA from current to remote process, but there was a error, so I removed that. It's needed to advance get_unmapped_area to make it working with remote VMA (which I missed initially). This requires a lot of refactoring, which may hide the main logic away from a reader, so let's I do that later in a separate series. Signed-off-by: Kirill Tkhai --- include/linux/mm.h | 4 + include/linux/mm_types.h | 2 + include/uapi/asm-generic/mman-common.h | 6 ++ mm/mmap.c | 107 ++++++++++++++++++++++++++++++++ mm/process_vm_access.c | 69 +++++++++++++++++++++ 5 files changed, 188 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 65ceb56acd44..9d1c79a44128 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2385,6 +2385,10 @@ extern int __do_munmap(struct mm_struct *, unsigned long, size_t, struct list_head *uf, bool downgrade); extern int do_munmap(struct mm_struct *, unsigned long, size_t, struct list_head *uf); +extern unsigned long mmap_process_vm(struct mm_struct *, unsigned long, + struct mm_struct *, unsigned long, + unsigned long, unsigned long, + struct list_head *); static inline unsigned long do_mmap_pgoff(struct file *file, unsigned long addr, diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 1815fbc40926..885f256f2fb7 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -261,11 +261,13 @@ struct vm_region { #ifdef CONFIG_USERFAULTFD #define NULL_VM_UFFD_CTX ((struct vm_userfaultfd_ctx) { NULL, }) +#define IS_NULL_VM_UFFD_CTX(uctx) ((uctx)->ctx == NULL) struct vm_userfaultfd_ctx { struct userfaultfd_ctx *ctx; }; #else /* CONFIG_USERFAULTFD */ #define NULL_VM_UFFD_CTX ((struct vm_userfaultfd_ctx) {}) +#define IS_NULL_VM_UFFD_CTX(uctx) (true) struct vm_userfaultfd_ctx {}; #endif /* CONFIG_USERFAULTFD */ diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h index abd238d0f7a4..91803e6ada7c 100644 --- a/include/uapi/asm-generic/mman-common.h +++ b/include/uapi/asm-generic/mman-common.h @@ -28,6 +28,12 @@ /* 0x0100 - 0x80000 flags are defined in asm-generic/mman.h */ #define MAP_FIXED_NOREPLACE 0x100000 /* MAP_FIXED which doesn't unmap underlying mapping */ +/* + * Flags for process_vm_mmap + */ +#define PVMMAP_FIXED 0x01 +#define PVMMAP_FIXED_NOREPLACE 0x02 /* PVMAP_FIXED which doesn't unmap underlying mapping */ + /* * Flags for mlock */ diff --git a/mm/mmap.c b/mm/mmap.c index 260e47e917e6..3123ecee5fb8 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3282,6 +3282,113 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap, return NULL; } +static int do_mmap_process_vm(struct vm_area_struct *src_vma, + unsigned long src_addr, + struct mm_struct *dst_mm, + unsigned long dst_addr, + unsigned long len, + struct list_head *uf) +{ + struct vm_area_struct *dst_vma; + unsigned long pgoff, ret; + bool unused; + + if (do_munmap(dst_mm, dst_addr, len, uf)) + return -ENOMEM; + + if (src_vma->vm_flags & VM_ACCOUNT) { + if (security_vm_enough_memory_mm(dst_mm, len >> PAGE_SHIFT)) + return -ENOMEM; + } + + pgoff = src_vma->vm_pgoff + + ((src_addr - src_vma->vm_start) >> PAGE_SHIFT); + dst_vma = copy_vma(&src_vma, dst_mm, dst_addr, + len, pgoff, &unused, false); + if (!dst_vma) { + ret = -ENOMEM; + goto unacct; + } + + ret = copy_page_range(dst_mm, src_vma->vm_mm, src_vma, + dst_addr, src_addr, src_addr + len); + if (ret) { + do_munmap(dst_mm, dst_addr, len, uf); + return -ENOMEM; + } + + if (dst_vma->vm_file) + uprobe_mmap(dst_vma); + perf_event_mmap(dst_vma); + + dst_vma->vm_flags |= VM_SOFTDIRTY; + vma_set_page_prot(dst_vma); + + vm_stat_account(dst_mm, dst_vma->vm_flags, len >> PAGE_SHIFT); + return 0; + +unacct: + vm_unacct_memory(len >> PAGE_SHIFT); + return ret; +} + +unsigned long mmap_process_vm(struct mm_struct *src_mm, + unsigned long src_addr, + struct mm_struct *dst_mm, + unsigned long dst_addr, + unsigned long len, + unsigned long flags, + struct list_head *uf) +{ + struct vm_area_struct *src_vma, *dst_vma; + unsigned long gua_flags = 0; + unsigned long ret; + + src_vma = find_vma_without_flags(src_mm, src_addr, len, + VM_HUGETLB|VM_DONTEXPAND|VM_PFNMAP|VM_IO); + if (IS_ERR(src_vma)) + return -EFAULT; + if (dst_mm->map_count > sysctl_max_map_count - 2) + return -ENOMEM; + if (!IS_NULL_VM_UFFD_CTX(&src_vma->vm_userfaultfd_ctx)) + return -ENOTSUPP; + + if (src_vma->vm_flags & VM_SHARED) + gua_flags |= MAP_SHARED; + else + gua_flags |= MAP_PRIVATE; + if (vma_is_anonymous(src_vma) || vma_is_shmem(src_vma)) + gua_flags |= MAP_ANONYMOUS; + if (flags & PVMMAP_FIXED) + gua_flags |= MAP_FIXED; + if (flags & PVMMAP_FIXED_NOREPLACE) + gua_flags |= MAP_FIXED | MAP_FIXED_NOREPLACE; + + ret = get_unmapped_area(src_vma->vm_file, dst_addr, len, + src_vma->vm_pgoff + + ((src_addr - src_vma->vm_start) >> PAGE_SHIFT), + gua_flags); + if (offset_in_page(ret)) + return ret; + if (flags & PVMMAP_FIXED_NOREPLACE) { + dst_vma = find_vma(dst_mm, dst_addr); + if (dst_vma && dst_vma->vm_start < dst_addr + len) + return -EEXIST; + } + + dst_addr = ret; + + /* Check against address space limit. */ + if (!may_mmap_overlapped_region(dst_mm, src_vma->vm_flags, dst_addr, len)) + return -ENOMEM; + + ret = do_mmap_process_vm(src_vma, src_addr, dst_mm, dst_addr, len, uf); + if (ret) + return ret; + + return dst_addr; +} + /* * Return true if the calling process may expand its vm space by the passed * number of pages diff --git a/mm/process_vm_access.c b/mm/process_vm_access.c index a447092d4635..e2073f52415b 100644 --- a/mm/process_vm_access.c +++ b/mm/process_vm_access.c @@ -17,6 +17,8 @@ #include #include #include +#include +#include #ifdef CONFIG_COMPAT #include @@ -295,6 +297,66 @@ static ssize_t process_vm_rw(pid_t pid, return rc; } +static unsigned long process_vm_mmap(pid_t pid, unsigned long src_addr, + unsigned long len, unsigned long dst_addr, + unsigned long flags) +{ + struct mm_struct *src_mm, *dst_mm; + struct task_struct *task; + unsigned long ret; + int depth = 0; + LIST_HEAD(uf); + + len = PAGE_ALIGN(len); + src_addr = round_down(src_addr, PAGE_SIZE); + if (flags & PVMMAP_FIXED) + dst_addr = round_down(dst_addr, PAGE_SIZE); + else + dst_addr = round_hint_to_min(dst_addr); + + if ((flags & ~(PVMMAP_FIXED|PVMMAP_FIXED_NOREPLACE)) || + len == 0 || len > TASK_SIZE || src_addr == 0 || + dst_addr > TASK_SIZE - len || pid <= 0) + return -EINVAL; + task = find_get_task_by_vpid(pid); + if (!task) + return -ESRCH; + if (unlikely(task->flags & PF_KTHREAD)) { + ret = -EINVAL; + goto out_put_task; + } + + src_mm = mm_access(task, PTRACE_MODE_ATTACH_REALCREDS); + if (!src_mm || IS_ERR(src_mm)) { + ret = IS_ERR(src_mm) ? PTR_ERR(src_mm) : -ESRCH; + goto out_put_task; + } + dst_mm = current->mm; + mmget(dst_mm); + + /* Double lock mm in address order: smallest is the first */ + if (src_mm < dst_mm) { + down_write(&src_mm->mmap_sem); + depth = SINGLE_DEPTH_NESTING; + } + down_write_nested(&dst_mm->mmap_sem, depth); + if (src_mm > dst_mm) + down_write_nested(&src_mm->mmap_sem, SINGLE_DEPTH_NESTING); + + ret = mmap_process_vm(src_mm, src_addr, dst_mm, dst_addr, len, flags, &uf); + + up_write(&dst_mm->mmap_sem); + if (dst_mm != src_mm) + up_write(&src_mm->mmap_sem); + + userfaultfd_unmap_complete(dst_mm, &uf); + mmput(src_mm); + mmput(dst_mm); +out_put_task: + put_task_struct(task); + return ret; +} + SYSCALL_DEFINE6(process_vm_readv, pid_t, pid, const struct iovec __user *, lvec, unsigned long, liovcnt, const struct iovec __user *, rvec, unsigned long, riovcnt, unsigned long, flags) @@ -310,6 +372,13 @@ SYSCALL_DEFINE6(process_vm_writev, pid_t, pid, return process_vm_rw(pid, lvec, liovcnt, rvec, riovcnt, flags, 1); } +SYSCALL_DEFINE5(process_vm_mmap, pid_t, pid, + unsigned long, src_addr, unsigned long, len, + unsigned long, dst_addr, unsigned long, flags) +{ + return process_vm_mmap(pid, src_addr, len, dst_addr, flags); +} + #ifdef CONFIG_COMPAT static ssize_t