From patchwork Sat Oct 17 23:13:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843081 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C479B15E6 for ; Sat, 17 Oct 2020 23:13:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 61669208E4 for ; Sat, 17 Oct 2020 23:13:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="vaRz5bnS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61669208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99C376B0068; Sat, 17 Oct 2020 19:13:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 94B926B006E; Sat, 17 Oct 2020 19:13:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 814666B0070; Sat, 17 Oct 2020 19:13:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0200.hostedemail.com [216.40.44.200]) by kanga.kvack.org (Postfix) with ESMTP id 50DE36B0068 for ; Sat, 17 Oct 2020 19:13:40 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DFC898249980 for ; Sat, 17 Oct 2020 23:13:39 +0000 (UTC) X-FDA: 77382971358.03.sheep98_010503927229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id C41E328A4E8 for ; Sat, 17 Oct 2020 23:13:39 +0000 (UTC) X-Spam-Summary: 1,0,0,43ae7c70467669c6,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2689:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3867:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4043:4321:5007:6261:6653:7576:7903:8784:9008:9025:9545:10004:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12986:13069:13221:13229:13311:13357:13846:14181:14384:14721:21080:21433:21451:21627:21795:21939:30045:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yr58fu3pza1q968ute5skfgww66ocik6t46u8hr1dse8xiubup6jtr6pc11ho.qd8qdqurthn6a5egsfbyjeg6dn6eg1qr6najtchzbb9mjoskf9fhusws9j1hghs.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutra l,Custom X-HE-Tag: sheep98_010503927229 X-Filterd-Recvd-Size: 2819 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:39 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 293D320878; Sat, 17 Oct 2020 23:13:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976418; bh=HS242CeKxVh1MmljEzFukdz2v+/3iS3zJnh37idj+xQ=; h=Date:From:To:Subject:In-Reply-To:From; b=vaRz5bnSZs48U0meeCB9SzxTN8zGoOGOvAHbfLCExWl4sd+H44kJaI2owjxe3jJsR bzgp26I/6MRqs8v0+eI0emTs0ZJaTL/oLgm/djmsF+Lb/WoN3t3z4JqFC61rkbe5I2 +SE7uqAR3mXNHeTQbIk/zgjKCa0txgJRK3id4EFc= Date: Sat, 17 Oct 2020 16:13:37 -0700 From: Andrew Morton To: akpm@linux-foundation.org, fenghua.yu@intel.com, krzk@kernel.org, linux-mm@kvack.org, lkp@intel.com, mm-commits@vger.kernel.org, stable@vger.kernel.org, tony.luck@intel.com, torvalds@linux-foundation.org Subject: [patch 01/40] ia64: fix build error with !COREDUMP Message-ID: <20201017231337.Vvr0NUSA4%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Krzysztof Kozlowski Subject: ia64: fix build error with !COREDUMP Fix linkage error when CONFIG_BINFMT_ELF is selected but CONFIG_COREDUMP is not: ia64-linux-ld: arch/ia64/kernel/elfcore.o: in function `elf_core_write_extra_phdrs': elfcore.c:(.text+0x172): undefined reference to `dump_emit' ia64-linux-ld: arch/ia64/kernel/elfcore.o: in function `elf_core_write_extra_data': elfcore.c:(.text+0x2b2): undefined reference to `dump_emit' Link: https://lkml.kernel.org/r/20200819064146.12529-1-krzk@kernel.org Fixes: 1fcccbac89f5 ("elf coredump: replace ELF_CORE_EXTRA_* macros by functions") Signed-off-by: Krzysztof Kozlowski Reported-by: kernel test robot Cc: Tony Luck Cc: Fenghua Yu Cc: Signed-off-by: Andrew Morton --- arch/ia64/kernel/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/arch/ia64/kernel/Makefile~ia64-fix-build-error-with-coredump +++ a/arch/ia64/kernel/Makefile @@ -40,7 +40,7 @@ obj-y += esi_stub.o # must be in kern endif obj-$(CONFIG_INTEL_IOMMU) += pci-dma.o -obj-$(CONFIG_BINFMT_ELF) += elfcore.o +obj-$(CONFIG_ELF_CORE) += elfcore.o # fp_emulate() expects f2-f5,f16-f31 to contain the user-level state. CFLAGS_traps.o += -mfixed-range=f2-f5,f16-f31 From patchwork Sat Oct 17 23:13:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843083 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A22914B5 for ; Sat, 17 Oct 2020 23:13:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1E392208B6 for ; Sat, 17 Oct 2020 23:13:45 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="jSzXO4js" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E392208B6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5089F6B006E; Sat, 17 Oct 2020 19:13:44 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4B7516B0070; Sat, 17 Oct 2020 19:13:44 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CE836B0071; Sat, 17 Oct 2020 19:13:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id 08B646B006E for ; Sat, 17 Oct 2020 19:13:43 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A5A4E180AD806 for ; Sat, 17 Oct 2020 23:13:43 +0000 (UTC) X-FDA: 77382971526.01.alley79_5f03d1827229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 89C4410046472 for ; Sat, 17 Oct 2020 23:13:43 +0000 (UTC) X-Spam-Summary: 50,0,0,a28c2885ca0032f1,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:2:41:69:355:379:800:960:967:973:981:988:989:1260:1345:1359:1381:1431:1437:1535:1605:1730:1747:1777:1792:1801:2194:2199:2393:2525:2553:2560:2564:2682:2685:2693:2859:2894:2902:2904:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4119:4250:4321:4605:5007:6119:6261:6653:7576:7875:7903:8660:9025:9545:9592:9707:10004:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12895:12986:13148:13161:13229:13230:21080:21433:21450:21451:21627:21740:21788:21939:21990:30054:30064:30070:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04yfnwkm68sontfqxhx6ukc55mqjqopmxg3aftmsp85wndstgu1padmrhnefd8a.dty6ocuheo7nankuwo6nnz9at81xt6447tnbjs4rtm1rrb3jmkxckypfxrbnea4.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesia n:0.5,0. X-HE-Tag: alley79_5f03d1827229 X-Filterd-Recvd-Size: 8233 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:43 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5F41F208B6; Sat, 17 Oct 2020 23:13:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976422; bh=lCCj9/RlmlQae3iuweNEKS14LX3hhbe04ab+YPkQsMo=; h=Date:From:To:Subject:In-Reply-To:From; b=jSzXO4jsFiTOWhEqvj+2ICZcZokTjFd/X4bXh9xY5PwekclvcnZLVc7htgtDVscns OLdTYHPSHWJFfOrsl57M59l8rocAIot0piRdpuYlmITypcE43V6fwtvEYUq9PmLekB dW9LqLQPu4HZqZdnt4xon4YaN7zanrUoGTcvoxuY= Date: Sat, 17 Oct 2020 16:13:40 -0700 From: Andrew Morton To: akpm@linux-foundation.org, dschatzberg@fb.com, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 02/40] mm, memcg: rework remote charging API to support nesting Message-ID: <20201017231340.JBcWsleuj%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm, memcg: rework remote charging API to support nesting Currently the remote memcg charging API consists of two functions: memalloc_use_memcg() and memalloc_unuse_memcg(), which set and clear the memcg value, which overwrites the memcg of the current task. memalloc_use_memcg(target_memcg); <...> memalloc_unuse_memcg(); It works perfectly for allocations performed from a normal context, however an attempt to call it from an interrupt context or just nest two remote charging blocks will lead to an incorrect accounting. On exit from the inner block the active memcg will be cleared instead of being restored. memalloc_use_memcg(target_memcg); memalloc_use_memcg(target_memcg_2); <...> memalloc_unuse_memcg(); Error: allocation here are charged to the memcg of the current process instead of target_memcg. memalloc_unuse_memcg(); This patch extends the remote charging API by switching to a single function: struct mem_cgroup *set_active_memcg(struct mem_cgroup *memcg), which sets the new value and returns the old one. So a remote charging block will look like: old_memcg = set_active_memcg(target_memcg); <...> set_active_memcg(old_memcg); This patch is heavily based on the patch by Johannes Weiner, which can be found here: https://lkml.org/lkml/2020/5/28/806 . Link: https://lkml.kernel.org/r/20200821212056.3769116-1-guro@fb.com Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Johannes Weiner Cc: Dan Schatzberg Signed-off-by: Andrew Morton --- fs/buffer.c | 6 ++--- fs/notify/fanotify/fanotify.c | 5 ++-- fs/notify/inotify/inotify_fsnotify.c | 5 ++-- include/linux/sched/mm.h | 30 ++++++++----------------- mm/memcontrol.c | 6 ++--- 5 files changed, 22 insertions(+), 30 deletions(-) --- a/fs/buffer.c~mm-rework-remote-memcg-charging-api-to-support-nesting +++ a/fs/buffer.c @@ -842,13 +842,13 @@ struct buffer_head *alloc_page_buffers(s struct buffer_head *bh, *head; gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT; long offset; - struct mem_cgroup *memcg; + struct mem_cgroup *memcg, *old_memcg; if (retry) gfp |= __GFP_NOFAIL; memcg = get_mem_cgroup_from_page(page); - memalloc_use_memcg(memcg); + old_memcg = set_active_memcg(memcg); head = NULL; offset = PAGE_SIZE; @@ -867,7 +867,7 @@ struct buffer_head *alloc_page_buffers(s set_bh_page(bh, page, offset); } out: - memalloc_unuse_memcg(); + set_active_memcg(old_memcg); mem_cgroup_put(memcg); return head; /* --- a/fs/notify/fanotify/fanotify.c~mm-rework-remote-memcg-charging-api-to-support-nesting +++ a/fs/notify/fanotify/fanotify.c @@ -531,6 +531,7 @@ static struct fanotify_event *fanotify_a struct inode *dirid = fanotify_dfid_inode(mask, data, data_type, dir); const struct path *path = fsnotify_data_path(data, data_type); unsigned int fid_mode = FAN_GROUP_FLAG(group, FANOTIFY_FID_BITS); + struct mem_cgroup *old_memcg; struct inode *child = NULL; bool name_event = false; @@ -580,7 +581,7 @@ static struct fanotify_event *fanotify_a gfp |= __GFP_RETRY_MAYFAIL; /* Whoever is interested in the event, pays for the allocation. */ - memalloc_use_memcg(group->memcg); + old_memcg = set_active_memcg(group->memcg); if (fanotify_is_perm_event(mask)) { event = fanotify_alloc_perm_event(path, gfp); @@ -608,7 +609,7 @@ static struct fanotify_event *fanotify_a event->pid = get_pid(task_tgid(current)); out: - memalloc_unuse_memcg(); + set_active_memcg(old_memcg); return event; } --- a/fs/notify/inotify/inotify_fsnotify.c~mm-rework-remote-memcg-charging-api-to-support-nesting +++ a/fs/notify/inotify/inotify_fsnotify.c @@ -66,6 +66,7 @@ static int inotify_one_event(struct fsno int ret; int len = 0; int alloc_len = sizeof(struct inotify_event_info); + struct mem_cgroup *old_memcg; if ((inode_mark->mask & FS_EXCL_UNLINK) && path && d_unlinked(path->dentry)) @@ -87,9 +88,9 @@ static int inotify_one_event(struct fsno * trigger OOM killer in the target monitoring memcg as it may have * security repercussion. */ - memalloc_use_memcg(group->memcg); + old_memcg = set_active_memcg(group->memcg); event = kmalloc(alloc_len, GFP_KERNEL_ACCOUNT | __GFP_RETRY_MAYFAIL); - memalloc_unuse_memcg(); + set_active_memcg(old_memcg); if (unlikely(!event)) { /* --- a/include/linux/sched/mm.h~mm-rework-remote-memcg-charging-api-to-support-nesting +++ a/include/linux/sched/mm.h @@ -280,38 +280,28 @@ static inline void memalloc_nocma_restor #ifdef CONFIG_MEMCG /** - * memalloc_use_memcg - Starts the remote memcg charging scope. + * set_active_memcg - Starts the remote memcg charging scope. * @memcg: memcg to charge. * * This function marks the beginning of the remote memcg charging scope. All the * __GFP_ACCOUNT allocations till the end of the scope will be charged to the * given memcg. * - * NOTE: This function is not nesting safe. + * NOTE: This function can nest. Users must save the return value and + * reset the previous value after their own charging scope is over. */ -static inline void memalloc_use_memcg(struct mem_cgroup *memcg) +static inline struct mem_cgroup * +set_active_memcg(struct mem_cgroup *memcg) { - WARN_ON_ONCE(current->active_memcg); + struct mem_cgroup *old = current->active_memcg; current->active_memcg = memcg; -} - -/** - * memalloc_unuse_memcg - Ends the remote memcg charging scope. - * - * This function marks the end of the remote memcg charging scope started by - * memalloc_use_memcg(). - */ -static inline void memalloc_unuse_memcg(void) -{ - current->active_memcg = NULL; + return old; } #else -static inline void memalloc_use_memcg(struct mem_cgroup *memcg) -{ -} - -static inline void memalloc_unuse_memcg(void) +static inline struct mem_cgroup * +set_active_memcg(struct mem_cgroup *memcg) { + return NULL; } #endif --- a/mm/memcontrol.c~mm-rework-remote-memcg-charging-api-to-support-nesting +++ a/mm/memcontrol.c @@ -5290,12 +5290,12 @@ static struct cgroup_subsys_state * __re mem_cgroup_css_alloc(struct cgroup_subsys_state *parent_css) { struct mem_cgroup *parent = mem_cgroup_from_css(parent_css); - struct mem_cgroup *memcg; + struct mem_cgroup *memcg, *old_memcg; long error = -ENOMEM; - memalloc_use_memcg(parent); + old_memcg = set_active_memcg(parent); memcg = mem_cgroup_alloc(); - memalloc_unuse_memcg(); + set_active_memcg(old_memcg); if (IS_ERR(memcg)) return ERR_CAST(memcg); From patchwork Sat Oct 17 23:13:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843085 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8EBE815E6 for ; Sat, 17 Oct 2020 23:13:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 42B5620878 for ; Sat, 17 Oct 2020 23:13:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="SyrELc9z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 42B5620878 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 619046B0070; Sat, 17 Oct 2020 19:13:47 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5CA266B0071; Sat, 17 Oct 2020 19:13:47 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DEF56B0072; Sat, 17 Oct 2020 19:13:47 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id 1826F6B0070 for ; Sat, 17 Oct 2020 19:13:47 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BD509362B for ; Sat, 17 Oct 2020 23:13:46 +0000 (UTC) X-FDA: 77382971652.01.dad36_4400e6c27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 9DD3E10046472 for ; Sat, 17 Oct 2020 23:13:46 +0000 (UTC) X-Spam-Summary: 1,0,0,56cf811ad5cd04a8,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1359:1381:1431:1437:1535:1544:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2564:2682:2685:2693:2731:2859:2892:2902:2904:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4605:5007:6119:6261:6653:7576:7903:7974:8599:8660:8957:9025:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13148:13161:13227:13229:13230:13255:14181:14721:14849:21080:21212:21433:21450:21451:21627:21740:21795:21939:21990:30012:30034:30051:30054:30064:30075,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrs394dss3ndbnwrazp9sw47yfjypbyumitc76ztxptj8xf7zfb8buz6i79da.pg6wikp59j77pn3h7yr43ic489nh51on97aiczogxzwr3gfmbzh1k53yfuqgj1i .6-lbl8. X-HE-Tag: dad36_4400e6c27229 X-Filterd-Recvd-Size: 5482 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:46 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0095520E65; Sat, 17 Oct 2020 23:13:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976425; bh=AqPbhk1/9EDvvsGkmL2/V/8ERjOJSl9illXilsGwdq0=; h=Date:From:To:Subject:In-Reply-To:From; b=SyrELc9zsGzqET708GEV4125mPEg2WUvdKHQYg2rEt8bRNv4lVDYpMqJgitt66r/7 jyUqxnSpiwt06odojxIyXeUHhRACnu9miIOwikwxaoiaU/VRlnyTn1Vp+XUi+ppwFB xO+Kk5u+VJj5PGcCMZCRSWl6yWr4A5tDo/tvXbcY= Date: Sat, 17 Oct 2020 16:13:44 -0700 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 03/40] mm: kmem: move memcg_kmem_bypass() calls to get_mem/obj_cgroup_from_current() Message-ID: <20201017231344.gdtuTtYpQ%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm: kmem: move memcg_kmem_bypass() calls to get_mem/obj_cgroup_from_current() Patch series "mm: kmem: kernel memory accounting in an interrupt context". This patchset implements memcg-based memory accounting of allocations made from an interrupt context. Historically, such allocations were passed unaccounted mostly because charging the memory cgroup of the current process wasn't an option. Also performance reasons were likely a reason too. The remote charging API allows to temporarily overwrite the currently active memory cgroup, so that all memory allocations are accounted towards some specified memory cgroup instead of the memory cgroup of the current process. This patchset extends the remote charging API so that it can be used from an interrupt context. Then it removes the fence that prevented the accounting of allocations made from an interrupt context. It also contains a couple of optimizations/code refactorings. This patchset doesn't directly enable accounting for any specific allocations, but prepares the code base for it. The bpf memory accounting will likely be the first user of it: a typical example is a bpf program parsing an incoming network packet, which allocates an entry in hashmap map to store some information. This patch (of 4): Currently memcg_kmem_bypass() is called before obtaining the current memory/obj cgroup using get_mem/obj_cgroup_from_current(). Moving memcg_kmem_bypass() into get_mem/obj_cgroup_from_current() reduces the number of call sites and allows further code simplifications. Link: http://lkml.kernel.org/r/20200827225843.1270629-1-guro@fb.com Link: http://lkml.kernel.org/r/20200827225843.1270629-2-guro@fb.com Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Johannes Weiner Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memcontrol.c | 13 ++++++++----- mm/percpu.c | 3 +-- mm/slab.h | 3 --- 3 files changed, 9 insertions(+), 10 deletions(-) --- a/mm/memcontrol.c~mm-kmem-move-memcg_kmem_bypass-calls-to-get_mem-obj_cgroup_from_current +++ a/mm/memcontrol.c @@ -1066,6 +1066,9 @@ EXPORT_SYMBOL(get_mem_cgroup_from_page); */ static __always_inline struct mem_cgroup *get_mem_cgroup_from_current(void) { + if (memcg_kmem_bypass()) + return NULL; + if (unlikely(current->active_memcg)) { struct mem_cgroup *memcg; @@ -2933,6 +2936,9 @@ __always_inline struct obj_cgroup *get_o struct obj_cgroup *objcg = NULL; struct mem_cgroup *memcg; + if (memcg_kmem_bypass()) + return NULL; + if (unlikely(!current->mm && !current->active_memcg)) return NULL; @@ -3059,19 +3065,16 @@ int __memcg_kmem_charge_page(struct page struct mem_cgroup *memcg; int ret = 0; - if (memcg_kmem_bypass()) - return 0; - memcg = get_mem_cgroup_from_current(); - if (!mem_cgroup_is_root(memcg)) { + if (memcg && !mem_cgroup_is_root(memcg)) { ret = __memcg_kmem_charge(memcg, gfp, 1 << order); if (!ret) { page->mem_cgroup = memcg; __SetPageKmemcg(page); return 0; } + css_put(&memcg->css); } - css_put(&memcg->css); return ret; } --- a/mm/percpu.c~mm-kmem-move-memcg_kmem_bypass-calls-to-get_mem-obj_cgroup_from_current +++ a/mm/percpu.c @@ -1584,8 +1584,7 @@ static enum pcpu_chunk_type pcpu_memcg_p { struct obj_cgroup *objcg; - if (!memcg_kmem_enabled() || !(gfp & __GFP_ACCOUNT) || - memcg_kmem_bypass()) + if (!memcg_kmem_enabled() || !(gfp & __GFP_ACCOUNT)) return PCPU_CHUNK_ROOT; objcg = get_obj_cgroup_from_current(); --- a/mm/slab.h~mm-kmem-move-memcg_kmem_bypass-calls-to-get_mem-obj_cgroup_from_current +++ a/mm/slab.h @@ -280,9 +280,6 @@ static inline struct obj_cgroup *memcg_s { struct obj_cgroup *objcg; - if (memcg_kmem_bypass()) - return NULL; - objcg = get_obj_cgroup_from_current(); if (!objcg) return NULL; From patchwork Sat Oct 17 23:13:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8CF7A14B5 for ; Sat, 17 Oct 2020 23:13:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4F5B2208B6 for ; Sat, 17 Oct 2020 23:13:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="RK1FgBx6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4F5B2208B6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 46BF26B0071; Sat, 17 Oct 2020 19:13:50 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 41A776B0072; Sat, 17 Oct 2020 19:13:50 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 330DB6B0073; Sat, 17 Oct 2020 19:13:50 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0094.hostedemail.com [216.40.44.94]) by kanga.kvack.org (Postfix) with ESMTP id F24F96B0071 for ; Sat, 17 Oct 2020 19:13:49 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 89BC5180AD806 for ; Sat, 17 Oct 2020 23:13:49 +0000 (UTC) X-FDA: 77382971778.06.ghost62_4a168cd27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 66B36101E2D12 for ; Sat, 17 Oct 2020 23:13:49 +0000 (UTC) X-Spam-Summary: 1,0,0,18e9c8337fe998fa,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1263:1345:1359:1381:1431:1437:1534:1541:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3351:3868:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4303:4321:5007:6261:6653:7576:8599:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12114:12296:12297:12438:12517:12519:12555:12679:12783:12986:13069:13161:13229:13255:13311:13357:14181:14384:14721:14849:21080:21451:21627:21939:21990:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04yg6i6tzr7ian6ux9g1aq9qdw4mbypynu5qy4yssjy7979151upwryf3potnn3.iqhk6mcu6kcyhi4yzdktkarg7gngarboy7t5cswft4jt56qcrtgwxcuqhkgnbfw.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutra l,Custom X-HE-Tag: ghost62_4a168cd27229 X-Filterd-Recvd-Size: 2493 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:48 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0D937208B6; Sat, 17 Oct 2020 23:13:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976428; bh=Oas1oCgjpRfVaJzPK1h8+/05dKzRzYPON4YtCKvjbDk=; h=Date:From:To:Subject:In-Reply-To:From; b=RK1FgBx65clXcg3ng30bXJ+s6+EzApHPRobGHILF0d+xkd7RHMCebGgqybonmiIAw kluzfl7xQ0WVHsEu6wci77MnyVhQr/D1wlVVYmesKOCfovEjkdxllLmSt6Tji7S8BV bZP9XIYsRGgx076pWZ6F/5b4CVtIN+Y3GNVZc198= Date: Sat, 17 Oct 2020 16:13:47 -0700 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 04/40] mm: kmem: remove redundant checks from get_obj_cgroup_from_current() Message-ID: <20201017231347.o9xpcQkwN%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000236, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm: kmem: remove redundant checks from get_obj_cgroup_from_current() There are checks for current->mm and current->active_memcg in get_obj_cgroup_from_current(), but these checks are redundant: memcg_kmem_bypass() called just above performs same checks. Link: http://lkml.kernel.org/r/20200827225843.1270629-3-guro@fb.com Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Johannes Weiner Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memcontrol.c | 3 --- 1 file changed, 3 deletions(-) --- a/mm/memcontrol.c~mm-kmem-remove-redundant-checks-from-get_obj_cgroup_from_current +++ a/mm/memcontrol.c @@ -2939,9 +2939,6 @@ __always_inline struct obj_cgroup *get_o if (memcg_kmem_bypass()) return NULL; - if (unlikely(!current->mm && !current->active_memcg)) - return NULL; - rcu_read_lock(); if (unlikely(current->active_memcg)) memcg = rcu_dereference(current->active_memcg); From patchwork Sat Oct 17 23:13:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843089 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B2B0D15E6 for ; Sat, 17 Oct 2020 23:13:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 71F00214DB for ; Sat, 17 Oct 2020 23:13:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="wUrC4ZcJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71F00214DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 816336B0072; Sat, 17 Oct 2020 19:13:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7C4D06B0073; Sat, 17 Oct 2020 19:13:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DA3A6B0074; Sat, 17 Oct 2020 19:13:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 3CC246B0072 for ; Sat, 17 Oct 2020 19:13:53 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DA5F5181AEF07 for ; Sat, 17 Oct 2020 23:13:52 +0000 (UTC) X-FDA: 77382971904.11.bee21_4f0bfaf27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id B76D9180F8B81 for ; Sat, 17 Oct 2020 23:13:52 +0000 (UTC) X-Spam-Summary: 1,0,0,8a5d4642917357df,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1359:1381:1431:1437:1535:1544:1605:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2693:2859:2902:2904:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4250:4321:4605:5007:6261:6653:7576:7875:8599:9025:9203:9545:10004:10913:11026:11473:11658:11914:12043:12048:12219:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13161:13178:13227:13229:13255:14096:14181:14721:14849:21080:21433:21450:21451:21627:21740:21939:21966:21990:30054:30064:30074,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yfoge1ummb9g4bpyaq7dmcx7s1xypu1b5cdrgrkayugcuo35hatrkx6zshbyg.rfjzq7xefoo9g1tpsh8fhc4o44957khak6hs4rixymcscsmn5htnrfka3cacdbg.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Ba yesian:0 X-HE-Tag: bee21_4f0bfaf27229 X-Filterd-Recvd-Size: 6305 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:52 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2DD7420878; Sat, 17 Oct 2020 23:13:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976431; bh=4O3cY30FaNn8aYMShgTdSRDAm7hy+nEhjTVz8ELFYEg=; h=Date:From:To:Subject:In-Reply-To:From; b=wUrC4ZcJnUEbubFJn6ffrJCQO8UwAzWMYTaMk0AHe9wt8hFvwcMXU+G7BDhFbvSq3 FWFjOFwztbLG6Lit2R6uUi3wPX1NHrvjLwA0W9xNnL5dkjfHPcWVDxx1uT97ogO2ax oQwixJ9Sp5UiH4ibJd55ajB0ZTFHZjaooRnCZn/E= Date: Sat, 17 Oct 2020 16:13:50 -0700 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 05/40] mm: kmem: prepare remote memcg charging infra for interrupt contexts Message-ID: <20201017231350.KU3k56hf5%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm: kmem: prepare remote memcg charging infra for interrupt contexts Remote memcg charging API uses current->active_memcg to store the currently active memory cgroup, which overwrites the memory cgroup of the current process. It works well for normal contexts, but doesn't work for interrupt contexts: indeed, if an interrupt occurs during the execution of a section with an active memcg set, all allocations inside the interrupt will be charged to the active memcg set (given that we'll enable accounting for allocations from an interrupt context). But because the interrupt might have no relation to the active memcg set outside, it's obviously wrong from the accounting prospective. To resolve this problem, let's add a global percpu int_active_memcg variable, which will be used to store an active memory cgroup which will be used from interrupt contexts. set_active_memcg() will transparently use current->active_memcg or int_active_memcg depending on the context. To make the read part simple and transparent for the caller, let's introduce two new functions: - struct mem_cgroup *active_memcg(void), - struct mem_cgroup *get_active_memcg(void). They are returning the active memcg if it's set, hiding all implementation details: where to get it depending on the current context. Link: http://lkml.kernel.org/r/20200827225843.1270629-4-guro@fb.com Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Johannes Weiner Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/sched/mm.h | 13 ++++++++-- mm/memcontrol.c | 48 ++++++++++++++++++++++++++----------- 2 files changed, 45 insertions(+), 16 deletions(-) --- a/include/linux/sched/mm.h~mm-kmem-prepare-remote-memcg-charging-infra-for-interrupt-contexts +++ a/include/linux/sched/mm.h @@ -279,6 +279,7 @@ static inline void memalloc_nocma_restor #endif #ifdef CONFIG_MEMCG +DECLARE_PER_CPU(struct mem_cgroup *, int_active_memcg); /** * set_active_memcg - Starts the remote memcg charging scope. * @memcg: memcg to charge. @@ -293,8 +294,16 @@ static inline void memalloc_nocma_restor static inline struct mem_cgroup * set_active_memcg(struct mem_cgroup *memcg) { - struct mem_cgroup *old = current->active_memcg; - current->active_memcg = memcg; + struct mem_cgroup *old; + + if (in_interrupt()) { + old = this_cpu_read(int_active_memcg); + this_cpu_write(int_active_memcg, memcg); + } else { + old = current->active_memcg; + current->active_memcg = memcg; + } + return old; } #else --- a/mm/memcontrol.c~mm-kmem-prepare-remote-memcg-charging-infra-for-interrupt-contexts +++ a/mm/memcontrol.c @@ -73,6 +73,9 @@ EXPORT_SYMBOL(memory_cgrp_subsys); struct mem_cgroup *root_mem_cgroup __read_mostly; +/* Active memory cgroup to use from an interrupt context */ +DEFINE_PER_CPU(struct mem_cgroup *, int_active_memcg); + /* Socket memory accounting disabled? */ static bool cgroup_memory_nosocket; @@ -1061,26 +1064,43 @@ struct mem_cgroup *get_mem_cgroup_from_p } EXPORT_SYMBOL(get_mem_cgroup_from_page); -/** - * If current->active_memcg is non-NULL, do not fallback to current->mm->memcg. - */ -static __always_inline struct mem_cgroup *get_mem_cgroup_from_current(void) +static __always_inline struct mem_cgroup *active_memcg(void) { - if (memcg_kmem_bypass()) - return NULL; + if (in_interrupt()) + return this_cpu_read(int_active_memcg); + else + return current->active_memcg; +} - if (unlikely(current->active_memcg)) { - struct mem_cgroup *memcg; +static __always_inline struct mem_cgroup *get_active_memcg(void) +{ + struct mem_cgroup *memcg; - rcu_read_lock(); + rcu_read_lock(); + memcg = active_memcg(); + if (memcg) { /* current->active_memcg must hold a ref. */ - if (WARN_ON_ONCE(!css_tryget(¤t->active_memcg->css))) + if (WARN_ON_ONCE(!css_tryget(&memcg->css))) memcg = root_mem_cgroup; else memcg = current->active_memcg; - rcu_read_unlock(); - return memcg; } + rcu_read_unlock(); + + return memcg; +} + +/** + * If active memcg is set, do not fallback to current->mm->memcg. + */ +static __always_inline struct mem_cgroup *get_mem_cgroup_from_current(void) +{ + if (memcg_kmem_bypass()) + return NULL; + + if (unlikely(active_memcg())) + return get_active_memcg(); + return get_mem_cgroup_from_mm(current->mm); } @@ -2940,8 +2960,8 @@ __always_inline struct obj_cgroup *get_o return NULL; rcu_read_lock(); - if (unlikely(current->active_memcg)) - memcg = rcu_dereference(current->active_memcg); + if (unlikely(active_memcg())) + memcg = active_memcg(); else memcg = mem_cgroup_from_task(current); From patchwork Sat Oct 17 23:13:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843091 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CEEDC15E6 for ; Sat, 17 Oct 2020 23:13:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 925A420878 for ; Sat, 17 Oct 2020 23:13:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="VX2OI9jT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 925A420878 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A11C96B0073; Sat, 17 Oct 2020 19:13:56 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9985E6B0074; Sat, 17 Oct 2020 19:13:56 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AEFC6B0075; Sat, 17 Oct 2020 19:13:56 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 5DC7B6B0073 for ; Sat, 17 Oct 2020 19:13:56 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id EE8EA1EE6 for ; Sat, 17 Oct 2020 23:13:55 +0000 (UTC) X-FDA: 77382972030.26.week89_0604aa527229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id CAA371804B656 for ; Sat, 17 Oct 2020 23:13:55 +0000 (UTC) X-Spam-Summary: 1,0,0,26aea3266ab8f8c7,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2693:2731:2859:2902:2904:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7576:8599:8603:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:14181:14721:14849:14877:21080:21433:21450:21451:21627:21740:21796:21939:21990:30036:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04yfitw4p689h8987ujiu3r64uoxxypzjxxa9oj1ynzhuyey833qbakxw6ospmj.fgi8g6xfcpepcytiehbpboi986tgn5e8ngqgjtfw7fhdgak1zcb5qzt8ymtnxnk.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,Doma inCache: X-HE-Tag: week89_0604aa527229 X-Filterd-Recvd-Size: 3709 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:55 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 54E14208B6; Sat, 17 Oct 2020 23:13:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976434; bh=VlZELGYYfGYDdh/QokchyFR5JVtzlL8vI17tRfmCMQQ=; h=Date:From:To:Subject:In-Reply-To:From; b=VX2OI9jTJtrXJaKppvYkJztGHQk9s0N/61deAAfSvnVc/tAL8GnT8H4wxJwHDpJje 8r/hDbgluxwQJk+XocHJ0+LbbNSgb1fZ24/EayyqZXRIp6oJUnyMCRaYumHo3HWhzM VpIQlLAWG0UqjwXzyHH9eN47XDNRyb0YCacqKnrY= Date: Sat, 17 Oct 2020 16:13:53 -0700 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 06/40] mm: kmem: enable kernel memcg accounting from interrupt contexts Message-ID: <20201017231353.tyvwMYEGc%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm: kmem: enable kernel memcg accounting from interrupt contexts If a memcg to charge can be determined (using remote charging API), there are no reasons to exclude allocations made from an interrupt context from the accounting. Such allocations will pass even if the resulting memcg size will exceed the hard limit, but it will affect the application of the memory pressure and an inability to put the workload under the limit will eventually trigger the OOM. To use active_memcg() helper, memcg_kmem_bypass() is moved back to memcontrol.c. Link: http://lkml.kernel.org/r/20200827225843.1270629-5-guro@fb.com Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Johannes Weiner Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 12 ------------ mm/memcontrol.c | 13 +++++++++++++ 2 files changed, 13 insertions(+), 12 deletions(-) --- a/include/linux/memcontrol.h~mm-kmem-enable-kernel-memcg-accounting-from-interrupt-contexts +++ a/include/linux/memcontrol.h @@ -1531,18 +1531,6 @@ static inline bool memcg_kmem_enabled(vo return static_branch_likely(&memcg_kmem_enabled_key); } -static inline bool memcg_kmem_bypass(void) -{ - if (in_interrupt()) - return true; - - /* Allow remote memcg charging in kthread contexts. */ - if ((!current->mm || (current->flags & PF_KTHREAD)) && - !current->active_memcg) - return true; - return false; -} - static inline int memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order) { --- a/mm/memcontrol.c~mm-kmem-enable-kernel-memcg-accounting-from-interrupt-contexts +++ a/mm/memcontrol.c @@ -1090,6 +1090,19 @@ static __always_inline struct mem_cgroup return memcg; } +static __always_inline bool memcg_kmem_bypass(void) +{ + /* Allow remote memcg charging from any context. */ + if (unlikely(active_memcg())) + return false; + + /* Memcg to charge can't be determined. */ + if (in_interrupt() || !current->mm || (current->flags & PF_KTHREAD)) + return true; + + return false; +} + /** * If active memcg is set, do not fallback to current->mm->memcg. */ From patchwork Sat Oct 17 23:13:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843093 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C244714B5 for ; Sat, 17 Oct 2020 23:14:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 887EF214DB for ; Sat, 17 Oct 2020 23:14:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="aW6j9K8B" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 887EF214DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99B056B0074; Sat, 17 Oct 2020 19:13:59 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 94A3D6B0075; Sat, 17 Oct 2020 19:13:59 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 812EF6B0078; Sat, 17 Oct 2020 19:13:59 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id 510A46B0074 for ; Sat, 17 Oct 2020 19:13:59 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id EE037180AD806 for ; Sat, 17 Oct 2020 23:13:58 +0000 (UTC) X-FDA: 77382972156.20.tray46_510984527229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin20.hostedemail.com (Postfix) with ESMTP id CE978180C07A3 for ; Sat, 17 Oct 2020 23:13:58 +0000 (UTC) X-Spam-Summary: 1,0,0,1d334d8a6f355427,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:966:967:968:973:988:989:1260:1263:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2559:2563:2682:2685:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3866:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6119:6121:6261:6653:6737:7576:7903:8957:9025:9391:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12297:12438:12517:12519:12555:12679:12783:12986:13161:13229:13846:14181:14721:14849:21060:21080:21220:21433:21451:21627:21819:21939:21987:21990:30054:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y8rfag95tkckwwkjtbg7mp7byq8ycosf45r7nhcaj17xccdwg6ot5d1cqnupx.hpttw15syx9czssdhb4z76w867st8nbejnrcsxwjgzw5eedijqjnttoewxpqrzx.6-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian: 0.5,0.5, X-HE-Tag: tray46_510984527229 X-Filterd-Recvd-Size: 3634 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:13:58 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 69F50208E4; Sat, 17 Oct 2020 23:13:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976437; bh=jcP/Kw8FmZ0zKkpo4EW4E6OO3YfOWpZaxtg6HSyqCmA=; h=Date:From:To:Subject:In-Reply-To:From; b=aW6j9K8BEWTCK6TVlG3TGnHhBb+EOigOVv5KbPvLOCRCDtRY9DgxMdXO/mbiCVktf RGKX4l7XfjuJwU0lVzHwQ2H1JC6qj0ZTApjufbDr755nqfO8F2ATZj/+GykbSjsYds brWEaHeiYiePUKOzXnB7HK7byzvG3bzrfcIXZ8Xo= Date: Sat, 17 Oct 2020 16:13:57 -0700 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hch@infradead.org, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 07/40] mm/memory-failure: remove a wrapper for alloc_migration_target() Message-ID: <20201017231357.LjnfaMDj_%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joonsoo Kim Subject: mm/memory-failure: remove a wrapper for alloc_migration_target() There is a well-defined standard migration target callback. Use it directly. Link: http://lkml.kernel.org/r/1594622517-20681-9-git-send-email-iamjoonsoo.kim@lge.com Signed-off-by: Joonsoo Kim Acked-by: Vlastimil Babka Cc: Christoph Hellwig Cc: Michal Hocko Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Roman Gushchin Signed-off-by: Andrew Morton --- mm/memory-failure.c | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) --- a/mm/memory-failure.c~mm-memory-failure-remove-a-wrapper-for-alloc_migration_target +++ a/mm/memory-failure.c @@ -1673,16 +1673,6 @@ int unpoison_memory(unsigned long pfn) } EXPORT_SYMBOL(unpoison_memory); -static struct page *new_page(struct page *p, unsigned long private) -{ - struct migration_target_control mtc = { - .nid = page_to_nid(p), - .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, - }; - - return alloc_migration_target(p, (unsigned long)&mtc); -} - /* * Safely get reference count of an arbitrary page. * Returns 0 for a free page, -EIO for a zero refcount page @@ -1797,6 +1787,10 @@ static int __soft_offline_page(struct pa char const *msg_page[] = {"page", "hugepage"}; bool huge = PageHuge(page); LIST_HEAD(pagelist); + struct migration_target_control mtc = { + .nid = NUMA_NO_NODE, + .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + }; /* * Check PageHWPoison again inside page lock because PageHWPoison @@ -1833,8 +1827,8 @@ static int __soft_offline_page(struct pa } if (isolate_page(hpage, &pagelist)) { - ret = migrate_pages(&pagelist, new_page, NULL, MPOL_MF_MOVE_ALL, - MIGRATE_SYNC, MR_MEMORY_FAILURE); + ret = migrate_pages(&pagelist, alloc_migration_target, NULL, + (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_FAILURE); if (!ret) { bool release = !huge; From patchwork Sat Oct 17 23:14:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843095 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2AD1F15E6 for ; Sat, 17 Oct 2020 23:14:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA7A820897 for ; Sat, 17 Oct 2020 23:14:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="URw6+IHx" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA7A820897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F1A286B0075; Sat, 17 Oct 2020 19:14:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EC9DF6B0078; Sat, 17 Oct 2020 19:14:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB78B6B007B; Sat, 17 Oct 2020 19:14:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0133.hostedemail.com [216.40.44.133]) by kanga.kvack.org (Postfix) with ESMTP id AD9F36B0075 for ; Sat, 17 Oct 2020 19:14:02 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5A0B3362B for ; Sat, 17 Oct 2020 23:14:02 +0000 (UTC) X-FDA: 77382972324.21.star77_4313a9627229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 4227E180442C0 for ; Sat, 17 Oct 2020 23:14:02 +0000 (UTC) X-Spam-Summary: 1,0,0,d4d0deee1f359db9,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:967:968:973:988:989:1260:1263:1345:1359:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2194:2198:2199:2200:2393:2525:2559:2563:2682:2685:2741:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:6737:7576:7903:9025:9391:9545:9592:10004:10913:11026:11473:11658:11914:12043:12048:12114:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13846:14096:14181:14721:14849:21060:21080:21220:21433:21451:21627:21819:21939:21990:30054:30064:30070:30080,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrhwwhwjzkgubnuzcro3jmc7cq8yc8nkeugy53y5ddjkuif1cgm16td9orctr.ca73aag1t6dftem5buyjhyq44hionxmym4o6rawf1qrd67tpp3q9fz1jwfeumn1.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,B ayesian: X-HE-Tag: star77_4313a9627229 X-Filterd-Recvd-Size: 4763 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:01 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9BF9121527; Sat, 17 Oct 2020 23:14:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976441; bh=D9/srntcgQT61qLxAZplDyTN/RW+cx+Iz/b9R0DrjCM=; h=Date:From:To:Subject:In-Reply-To:From; b=URw6+IHxbH1kRcQl+HLujGD/zMe+OJWcu4IE2c2WfOlBC5ATtoBEqoaja0JjoOzsz yeY31hrHrzGclUxF1ZoOJeSC6BnJUZ2oumJerxkZZ0UIDWUNjdk8D+SiTdxJpzJLbC KTSjf471WwQ6fHHkYb7jEhqiYOqjOt9G8VZQP+z0= Date: Sat, 17 Oct 2020 16:14:00 -0700 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hch@infradead.org, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 08/40] mm/memory_hotplug: remove a wrapper for alloc_migration_target() Message-ID: <20201017231400.LHDabo5KG%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joonsoo Kim Subject: mm/memory_hotplug: remove a wrapper for alloc_migration_target() To calculate the correct node to migrate the page for hotplug, we need to check node id of the page. Wrapper for alloc_migration_target() exists for this purpose. However, Vlastimil informs that all migration source pages come from a single node. In this case, we don't need to check the node id for each page and we don't need to re-set the target nodemask for each page by using the wrapper. Set up the migration_target_control once and use it for all pages. Link: http://lkml.kernel.org/r/1594622517-20681-10-git-send-email-iamjoonsoo.kim@lge.com Signed-off-by: Joonsoo Kim Acked-by: Vlastimil Babka Acked-by: Michal Hocko Cc: Christoph Hellwig Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Roman Gushchin Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 46 ++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 24 deletions(-) --- a/mm/memory_hotplug.c~mm-memory_hotplug-remove-a-wrapper-for-alloc_migration_target +++ a/mm/memory_hotplug.c @@ -1290,27 +1290,6 @@ found: return 0; } -static struct page *new_node_page(struct page *page, unsigned long private) -{ - nodemask_t nmask = node_states[N_MEMORY]; - struct migration_target_control mtc = { - .nid = page_to_nid(page), - .nmask = &nmask, - .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, - }; - - /* - * try to allocate from a different node but reuse this node if there - * are no other online nodes to be used (e.g. we are offlining a part - * of the only existing node) - */ - node_clear(mtc.nid, nmask); - if (nodes_empty(nmask)) - node_set(mtc.nid, nmask); - - return alloc_migration_target(page, (unsigned long)&mtc); -} - static int do_migrate_range(unsigned long start_pfn, unsigned long end_pfn) { @@ -1370,9 +1349,28 @@ do_migrate_range(unsigned long start_pfn put_page(page); } if (!list_empty(&source)) { - /* Allocate a new page from the nearest neighbor node */ - ret = migrate_pages(&source, new_node_page, NULL, 0, - MIGRATE_SYNC, MR_MEMORY_HOTPLUG); + nodemask_t nmask = node_states[N_MEMORY]; + struct migration_target_control mtc = { + .nmask = &nmask, + .gfp_mask = GFP_USER | __GFP_MOVABLE | __GFP_RETRY_MAYFAIL, + }; + + /* + * We have checked that migration range is on a single zone so + * we can use the nid of the first page to all the others. + */ + mtc.nid = page_to_nid(list_first_entry(&source, struct page, lru)); + + /* + * try to allocate from a different node but reuse this node + * if there are no other online nodes to be used (e.g. we are + * offlining a part of the only existing node) + */ + node_clear(mtc.nid, nmask); + if (nodes_empty(nmask)) + node_set(mtc.nid, nmask); + ret = migrate_pages(&source, alloc_migration_target, NULL, + (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_HOTPLUG); if (ret) { list_for_each_entry(page, &source, lru) { pr_warn("migrating pfn %lx failed ret:%d ", From patchwork Sat Oct 17 23:14:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843097 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3A37514B5 for ; Sat, 17 Oct 2020 23:14:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 004DC20897 for ; Sat, 17 Oct 2020 23:14:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="IjRvNDF3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 004DC20897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E7DF56B0078; Sat, 17 Oct 2020 19:14:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E11F96B007B; Sat, 17 Oct 2020 19:14:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1B176B007D; Sat, 17 Oct 2020 19:14:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id A64716B0078 for ; Sat, 17 Oct 2020 19:14:05 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 43AB5180AD806 for ; Sat, 17 Oct 2020 23:14:05 +0000 (UTC) X-FDA: 77382972450.19.boat43_630793327229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 2ADA21AD1B2 for ; Sat, 17 Oct 2020 23:14:05 +0000 (UTC) X-Spam-Summary: 1,0,0,ded772330afcbee1,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2894:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3871:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8660:9010:9025:9545:9592:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12986:13148:13161:13229:13230:13255:13846:14181:14721:21080:21324:21451:21627:21939:21990:30046:30051:30054:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04ygxrqhfs7juqubfjo8c7yfaejspopge7owgur1ucrbtndrs43gbuxszxzb1n6.7sz9dayyyrcht368un4zgd9yg43wyp1e8yw9cbc59dtynznzwktw9zew3hds8rt.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp ,MSBL:0, X-HE-Tag: boat43_630793327229 X-Filterd-Recvd-Size: 5171 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:04 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id B3E6D208B6; Sat, 17 Oct 2020 23:14:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976444; bh=kFSoIWWs4XOxOsX9f43QcJB4KhMzRi51ggBuKQyD7mk=; h=Date:From:To:Subject:In-Reply-To:From; b=IjRvNDF3tpyxDdGpYlb62tyEelqo362Mbr0ug/dw0rvceJazuUCjK2gP6JpXttIr5 fiMPQKTtnwcFsshAp1kEKpb24THVKPfns0cF7vKWHclUZ2G5u7/8M6GKa9OccRrObz ui5OejwMCn0u0PI1kKub5314tsiFUleS57H7uMNU= Date: Sat, 17 Oct 2020 16:14:03 -0700 From: Andrew Morton To: akpm@linux-foundation.org, cl@linux.com, linmiaohe@huawei.com, linux-mm@kvack.org, louhongxiang@huawei.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 09/40] mm/migrate: avoid possible unnecessary process right check in kernel_move_pages() Message-ID: <20201017231403.WBGUY4nLK%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/migrate: avoid possible unnecessary process right check in kernel_move_pages() There is no need to check if this process has the right to modify the specified process when they are same. And we could also skip the security hook call if a process is modifying its own pages. Add helper function to handle these. Link: https://lkml.kernel.org/r/20200819083331.19012-1-linmiaohe@huawei.com Signed-off-by: Hongxiang Lou Signed-off-by: Miaohe Lin Suggested-by: Matthew Wilcox Cc: Christopher Lameter Signed-off-by: Andrew Morton --- mm/migrate.c | 71 +++++++++++++++++++++++++++++-------------------- 1 file changed, 43 insertions(+), 28 deletions(-) --- a/mm/migrate.c~mm-migrate-avoid-possible-unnecessary-process-right-check-in-kernel_move_pages +++ a/mm/migrate.c @@ -1864,33 +1864,27 @@ static int do_pages_stat(struct mm_struc return nr_pages ? -EFAULT : 0; } -/* - * Move a list of pages in the address space of the currently executing - * process. - */ -static int kernel_move_pages(pid_t pid, unsigned long nr_pages, - const void __user * __user *pages, - const int __user *nodes, - int __user *status, int flags) +static struct mm_struct *find_mm_struct(pid_t pid, nodemask_t *mem_nodes) { struct task_struct *task; struct mm_struct *mm; - int err; - nodemask_t task_nodes; - - /* Check flags */ - if (flags & ~(MPOL_MF_MOVE|MPOL_MF_MOVE_ALL)) - return -EINVAL; - if ((flags & MPOL_MF_MOVE_ALL) && !capable(CAP_SYS_NICE)) - return -EPERM; + /* + * There is no need to check if current process has the right to modify + * the specified process when they are same. + */ + if (!pid) { + mmget(current->mm); + *mem_nodes = cpuset_mems_allowed(current); + return current->mm; + } /* Find the mm_struct */ rcu_read_lock(); - task = pid ? find_task_by_vpid(pid) : current; + task = find_task_by_vpid(pid); if (!task) { rcu_read_unlock(); - return -ESRCH; + return ERR_PTR(-ESRCH); } get_task_struct(task); @@ -1900,22 +1894,47 @@ static int kernel_move_pages(pid_t pid, */ if (!ptrace_may_access(task, PTRACE_MODE_READ_REALCREDS)) { rcu_read_unlock(); - err = -EPERM; + mm = ERR_PTR(-EPERM); goto out; } rcu_read_unlock(); - err = security_task_movememory(task); - if (err) + mm = ERR_PTR(security_task_movememory(task)); + if (IS_ERR(mm)) goto out; - - task_nodes = cpuset_mems_allowed(task); + *mem_nodes = cpuset_mems_allowed(task); mm = get_task_mm(task); +out: put_task_struct(task); - if (!mm) + mm = ERR_PTR(-EINVAL); + return mm; +} + +/* + * Move a list of pages in the address space of the currently executing + * process. + */ +static int kernel_move_pages(pid_t pid, unsigned long nr_pages, + const void __user * __user *pages, + const int __user *nodes, + int __user *status, int flags) +{ + struct mm_struct *mm; + int err; + nodemask_t task_nodes; + + /* Check flags */ + if (flags & ~(MPOL_MF_MOVE|MPOL_MF_MOVE_ALL)) return -EINVAL; + if ((flags & MPOL_MF_MOVE_ALL) && !capable(CAP_SYS_NICE)) + return -EPERM; + + mm = find_mm_struct(pid, &task_nodes); + if (IS_ERR(mm)) + return PTR_ERR(mm); + if (nodes) err = do_pages_move(mm, task_nodes, nr_pages, pages, nodes, status, flags); @@ -1924,10 +1943,6 @@ static int kernel_move_pages(pid_t pid, mmput(mm); return err; - -out: - put_task_struct(task); - return err; } SYSCALL_DEFINE6(move_pages, pid_t, pid, unsigned long, nr_pages, From patchwork Sat Oct 17 23:14:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843099 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 66E0014B5 for ; Sat, 17 Oct 2020 23:14:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2914B208E4 for ; Sat, 17 Oct 2020 23:14:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="qxWpwbQ7" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2914B208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1D2946B007B; Sat, 17 Oct 2020 19:14:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1813A6B007D; Sat, 17 Oct 2020 19:14:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C0406B007E; Sat, 17 Oct 2020 19:14:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0115.hostedemail.com [216.40.44.115]) by kanga.kvack.org (Postfix) with ESMTP id D0C146B007B for ; Sat, 17 Oct 2020 19:14:08 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 77E191EE6 for ; Sat, 17 Oct 2020 23:14:08 +0000 (UTC) X-FDA: 77382972576.22.glue45_601108927229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 3849D18038E60 for ; Sat, 17 Oct 2020 23:14:08 +0000 (UTC) X-Spam-Summary: 1,0,0,d0412edb6cea0cd2,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1263:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:7576:8599:9025:9545:10004:10913:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12783:12986:13141:13161:13229:13230:13846:14096:14181:14721:14849:21080:21324:21451:21627:21939:21990:30012:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y8h8jxm1pywk9kkn674roey1npnycnxenh3pbiw6qwo8qx8wrquep73brys3x.td8qdo8ptcmp45mpui14dt3rbrpqceufb5rkzbzrx5hobkptrdq89momn7adsi6.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0, DNSBL:ne X-HE-Tag: glue45_601108927229 X-Filterd-Recvd-Size: 3552 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf38.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:07 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BE74E208E4; Sat, 17 Oct 2020 23:14:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976447; bh=0XHlP719nJRI+EVhTk6Cs4CWAHZk5C5nuGJnmRbAUzc=; h=Date:From:To:Subject:In-Reply-To:From; b=qxWpwbQ7s3Rx+Oj/Muw0KEc6BkAGTcTYZ9KHfbYn2OXtPiat+HMTdtNtqYd2xrchi m9vv+e3TspkL4JyynwycGUPXd0oNiTclqj6qGPXGKO+YTySUbyebM4QLW8jPArIU4L yljpKWaCnjpW+DLQYAS2n/jSjEoIr1+ZJsVNCIeQ= Date: Sat, 17 Oct 2020 16:14:06 -0700 From: Andrew Morton To: akpm@linux-foundation.org, Liam.Howlett@Oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 10/40] mm/mmap: add inline vma_next() for readability of mmap code Message-ID: <20201017231406.Fm-sT8ouL%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Liam R. Howlett" Subject: mm/mmap: add inline vma_next() for readability of mmap code There are three places that the next vma is required which uses the same block of code. Replace the block with a function and add comments on what happens in the case where NULL is encountered. Link: http://lkml.kernel.org/r/20200818154707.2515169-1-Liam.Howlett@Oracle.com Signed-off-by: Liam R. Howlett Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/mmap.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) --- a/mm/mmap.c~mm-mmap-add-inline-vma_next-for-readability-of-mmap-code +++ a/mm/mmap.c @@ -558,6 +558,23 @@ static int find_vma_links(struct mm_stru return 0; } +/* + * vma_next() - Get the next VMA. + * @mm: The mm_struct. + * @vma: The current vma. + * + * If @vma is NULL, return the first vma in the mm. + * + * Returns: The next VMA after @vma. + */ +static inline struct vm_area_struct *vma_next(struct mm_struct *mm, + struct vm_area_struct *vma) +{ + if (!vma) + return mm->mmap; + + return vma->vm_next; +} static unsigned long count_vma_pages_range(struct mm_struct *mm, unsigned long addr, unsigned long end) { @@ -1128,10 +1145,7 @@ struct vm_area_struct *vma_merge(struct if (vm_flags & VM_SPECIAL) return NULL; - if (prev) - next = prev->vm_next; - else - next = mm->mmap; + next = vma_next(mm, prev); area = next; if (area && area->vm_end == end) /* cases 6, 7, 8 */ next = next->vm_next; @@ -2632,7 +2646,7 @@ static void unmap_region(struct mm_struc struct vm_area_struct *vma, struct vm_area_struct *prev, unsigned long start, unsigned long end) { - struct vm_area_struct *next = prev ? prev->vm_next : mm->mmap; + struct vm_area_struct *next = vma_next(mm, prev); struct mmu_gather tlb; lru_add_drain(); @@ -2831,7 +2845,7 @@ int __do_munmap(struct mm_struct *mm, un if (error) return error; } - vma = prev ? prev->vm_next : mm->mmap; + vma = vma_next(mm, prev); if (unlikely(uf)) { /* From patchwork Sat Oct 17 23:14:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843101 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C32A14B5 for ; Sat, 17 Oct 2020 23:14:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D8723208E4 for ; Sat, 17 Oct 2020 23:14:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="sBegiK50" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8723208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DE3436B007D; Sat, 17 Oct 2020 19:14:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D6E036B007E; Sat, 17 Oct 2020 19:14:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C80776B0080; Sat, 17 Oct 2020 19:14:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 93D556B007D for ; Sat, 17 Oct 2020 19:14:11 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3BDC7362B for ; Sat, 17 Oct 2020 23:14:11 +0000 (UTC) X-FDA: 77382972702.06.egg16_0300e7c27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 1C0FD101E2D12 for ; Sat, 17 Oct 2020 23:14:11 +0000 (UTC) X-Spam-Summary: 13,1.2,0,44b683fd47a0b88a,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:967:973:988:989:1260:1263:1345:1359:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2693:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3355:3865:3866:3867:3868:3870:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:7576:7875:7903:8599:9025:9036:9545:9592:10008:10913:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13161:13229:13846:14096:14181:14721:14799:14849:21080:21324:21451:21627:21939:21990:30003:30012:30054:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrdkoy6icbb54wimsqia9a1wbdmocyhqary3w6rnpd7wqt6bhn5gi9psqy5js.gt4cqejo8rknd1hm5wfnx1xy7gaugknj147875i534xjcdsacnyoeaox5qtkcfb.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netchec k:none,D X-HE-Tag: egg16_0300e7c27229 X-Filterd-Recvd-Size: 4391 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:10 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id ACA1420897; Sat, 17 Oct 2020 23:14:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976449; bh=f5CCkANMVT19gApMAe7FnKgAv0Hukgr+Bo/TXSkzKMA=; h=Date:From:To:Subject:In-Reply-To:From; b=sBegiK50yBBpN0N/fQlPnwRmDjA2MknrG751J2jEsuKDEYkkMmGA3TtDGkgnyc5ZM Ynbs7pwYiS3jYtC60BgZB0g7aejI4YRd8UYjJEcuXgmTWGCoYJfQkSFmz1VJMRm+1L JzDlXdzgIVscM2UQtIO/SJyU6qtELGV4mjbr0S5Q= Date: Sat, 17 Oct 2020 16:14:09 -0700 From: Andrew Morton To: akpm@linux-foundation.org, Liam.Howlett@Oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 11/40] mm/mmap: add inline munmap_vma_range() for code readability Message-ID: <20201017231409.Qiho8zkqA%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Liam R. Howlett" Subject: mm/mmap: add inline munmap_vma_range() for code readability There are two locations that have a block of code for munmapping a vma range. Change those two locations to use a function and add meaningful comments about what happens to the arguments, which was unclear in the previous code. Link: http://lkml.kernel.org/r/20200818154707.2515169-2-Liam.Howlett@Oracle.com Signed-off-by: Liam R. Howlett Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/mmap.c | 48 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 33 insertions(+), 15 deletions(-) --- a/mm/mmap.c~mm-mmap-add-inline-munmap_vma_range-for-code-readability +++ a/mm/mmap.c @@ -575,6 +575,33 @@ static inline struct vm_area_struct *vma return vma->vm_next; } + +/* + * munmap_vma_range() - munmap VMAs that overlap a range. + * @mm: The mm struct + * @start: The start of the range. + * @len: The length of the range. + * @pprev: pointer to the pointer that will be set to previous vm_area_struct + * @rb_link: the rb_node + * @rb_parent: the parent rb_node + * + * Find all the vm_area_struct that overlap from @start to + * @end and munmap them. Set @pprev to the previous vm_area_struct. + * + * Returns: -ENOMEM on munmap failure or 0 on success. + */ +static inline int +munmap_vma_range(struct mm_struct *mm, unsigned long start, unsigned long len, + struct vm_area_struct **pprev, struct rb_node ***link, + struct rb_node **parent, struct list_head *uf) +{ + + while (find_vma_links(mm, start, start + len, pprev, link, parent)) + if (do_munmap(mm, start, len, uf)) + return -ENOMEM; + + return 0; +} static unsigned long count_vma_pages_range(struct mm_struct *mm, unsigned long addr, unsigned long end) { @@ -1721,13 +1748,9 @@ unsigned long mmap_region(struct file *f return -ENOMEM; } - /* Clear old maps */ - while (find_vma_links(mm, addr, addr + len, &prev, &rb_link, - &rb_parent)) { - if (do_munmap(mm, addr, len, uf)) - return -ENOMEM; - } - + /* Clear old maps, set up prev, rb_link, rb_parent, and uf */ + if (munmap_vma_range(mm, addr, len, &prev, &rb_link, &rb_parent, uf)) + return -ENOMEM; /* * Private writable mapping: check memory availability */ @@ -3063,14 +3086,9 @@ static int do_brk_flags(unsigned long ad if (error) return error; - /* - * Clear old maps. this also does some error checking for us - */ - while (find_vma_links(mm, addr, addr + len, &prev, &rb_link, - &rb_parent)) { - if (do_munmap(mm, addr, len, uf)) - return -ENOMEM; - } + /* Clear old maps, set up prev, rb_link, rb_parent, and uf */ + if (munmap_vma_range(mm, addr, len, &prev, &rb_link, &rb_parent, uf)) + return -ENOMEM; /* Check against address space limits *after* clearing old maps... */ if (!may_expand_vm(mm, flags, len >> PAGE_SHIFT)) From patchwork Sat Oct 17 23:14:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843103 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 47DA314B5 for ; Sat, 17 Oct 2020 23:14:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F2F0220897 for ; Sat, 17 Oct 2020 23:14:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="f/1qtrAS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2F0220897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 073016B007E; Sat, 17 Oct 2020 19:14:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 021346B0080; Sat, 17 Oct 2020 19:14:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA3996B0081; Sat, 17 Oct 2020 19:14:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id BED1C6B007E for ; Sat, 17 Oct 2020 19:14:14 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5934A181AEF07 for ; Sat, 17 Oct 2020 23:14:14 +0000 (UTC) X-FDA: 77382972828.19.wall97_5d1655d27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin19.hostedemail.com (Postfix) with ESMTP id 372AB1AD1B2 for ; Sat, 17 Oct 2020 23:14:14 +0000 (UTC) X-Spam-Summary: 1,0,0,9c56863b678170de,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2194:2196:2199:2200:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3872:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4385:5007:6119:6261:6653:6737:7576:7903:7904:8603:8784:8957:9025:9121:9545:10004:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12679:14096:14181:14721:21080:21451:21627:21939:21990:30012:30054:30056:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04y8c16qp9euo6c4wd5nfo7oh54gsyctaj8wyxicy5ptc8zrxari8pohe153yw4.adqc7cjxkxe75owh51mzrkpi59tu5odro3sbbafpj3kxzt1uwi5cxa8rsyfhtt3.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,D NSBL:neu X-HE-Tag: wall97_5d1655d27229 X-Filterd-Recvd-Size: 3724 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:13 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id ACF6C20878; Sat, 17 Oct 2020 23:14:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976453; bh=hOaqVUk6Ts2NldEbt7e5w6xi/BvKskyBfySCZv/4e50=; h=Date:From:To:Subject:In-Reply-To:From; b=f/1qtrASMSGmqI/A1treObMrUIMEjxPeFnS7b6/Zi5ujWA2up6uu6KaLTlBKfURq4 /ieI7SKNKrouyjRgVnNhTWwVRxbJJTxQU+lKFPwlCQ1nzmU/IkVvC8tfSfNCT4Avw4 CEku5ubewvt2A9NZTFBU6Hcn5hsIC3mjYTH7/JuI= Date: Sat, 17 Oct 2020 16:14:12 -0700 From: Andrew Morton To: akpm@linux-foundation.org, ebiederm@xmission.com, jannh@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mchehab@kernel.org, mm-commits@vger.kernel.org, sakari.ailus@linux.intel.com, torvalds@linux-foundation.org, walken@google.com Subject: [patch 12/40] mm/gup_benchmark: take the mmap lock around GUP Message-ID: <20201017231412.94CmEnsAR%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jann Horn Subject: mm/gup_benchmark: take the mmap lock around GUP To be safe against concurrent changes to the VMA tree, we must take the mmap lock around GUP operations (excluding the GUP-fast family of operations, which will take the mmap lock by themselves if necessary). This code is only for testing, and it's only reachable by root through debugfs, so this doesn't really have any impact; however, if we want to add lockdep asserts into the GUP path, we need to have clean locking here. Link: https://lkml.kernel.org/r/CAG48ez3SG6ngZLtasxJ6LABpOnqCz5-QHqb0B4k44TQ8F9n6+w@mail.gmail.com Signed-off-by: Jann Horn Reviewed-by: Jason Gunthorpe Reviewed-by: John Hubbard Acked-by: Michel Lespinasse Cc: "Eric W . Biederman" Cc: Mauro Carvalho Chehab Cc: Sakari Ailus Signed-off-by: Andrew Morton --- mm/gup_benchmark.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) --- a/mm/gup_benchmark.c~mm-gup_benchmark-take-the-mmap-lock-around-gup +++ a/mm/gup_benchmark.c @@ -72,6 +72,8 @@ static int __gup_benchmark_ioctl(unsigne int nr; struct page **pages; int ret = 0; + bool needs_mmap_lock = + cmd != GUP_FAST_BENCHMARK && cmd != PIN_FAST_BENCHMARK; if (gup->size > ULONG_MAX) return -EINVAL; @@ -81,6 +83,11 @@ static int __gup_benchmark_ioctl(unsigne if (!pages) return -ENOMEM; + if (needs_mmap_lock && mmap_read_lock_killable(current->mm)) { + ret = -EINTR; + goto free_pages; + } + i = 0; nr = gup->nr_pages_per_call; start_time = ktime_get(); @@ -120,9 +127,8 @@ static int __gup_benchmark_ioctl(unsigne pages + i, NULL); break; default: - kvfree(pages); ret = -EINVAL; - goto out; + goto unlock; } if (nr <= 0) @@ -150,8 +156,11 @@ static int __gup_benchmark_ioctl(unsigne end_time = ktime_get(); gup->put_delta_usec = ktime_us_delta(end_time, start_time); +unlock: + if (needs_mmap_lock) + mmap_read_unlock(current->mm); +free_pages: kvfree(pages); -out: return ret; } From patchwork Sat Oct 17 23:14:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5998915E6 for ; Sat, 17 Oct 2020 23:14:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C746208E4 for ; Sat, 17 Oct 2020 23:14:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="2J556YqS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C746208E4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2BE6F6B0080; Sat, 17 Oct 2020 19:14:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2444F6B0081; Sat, 17 Oct 2020 19:14:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 181C06B0082; Sat, 17 Oct 2020 19:14:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id DFD9A6B0080 for ; Sat, 17 Oct 2020 19:14:17 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8C237181AEF07 for ; Sat, 17 Oct 2020 23:14:17 +0000 (UTC) X-FDA: 77382972954.10.legs72_1c0aca227229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id 67A8516A0AB for ; Sat, 17 Oct 2020 23:14:17 +0000 (UTC) X-Spam-Summary: 1,0,0,b734a99dd49062c4,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2198:2199:2393:2525:2553:2559:2563:2682:2685:2691:2731:2859:2901:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3353:3865:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:5007:6261:6653:6737:7576:7903:7904:7974:8957:9025:9545:10004:11026:11658:11914:12043:12048:12296:12297:12438:12517:12519:12663:12679:13161:13229:14181:14721:21080:21324:21451:21627:21795:21939:21990:30054:30064:30070:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yfunu87bskbj9xgp1juao5io3quyctdxt56q5r1qxwacrgzus79crmjypgkd6.kfbway8s1ks8jp8h1mrpct34u4j8chk6siiketrmbrcif8icz393ipgnnfm9gm9.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL :0,DNSBL X-HE-Tag: legs72_1c0aca227229 X-Filterd-Recvd-Size: 3487 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:17 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E683D208B6; Sat, 17 Oct 2020 23:14:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976456; bh=3kOwiTpnCK1Bv4+1XPwqTtAubtIlwLSRR5F/+KJVms8=; h=Date:From:To:Subject:In-Reply-To:From; b=2J556YqShNJAJGEGS4R90SU++sGEP0FUvOaP09UqQ+RsiSz/LUKoo+v8AI8asZ3m1 2T5b8U2quZENka6NICO0hmXKOYWfqptB2eZSLbAZjpEcHjp5qG7khiPDJmRpN180N8 uIgCUqt8RNNstkmB4kkrpWR+xwPevDrCzacYxDcU= Date: Sat, 17 Oct 2020 16:14:15 -0700 From: Andrew Morton To: akpm@linux-foundation.org, ebiederm@xmission.com, jannh@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mchehab@kernel.org, mm-commits@vger.kernel.org, sakari.ailus@linux.intel.com, torvalds@linux-foundation.org, walken@google.com Subject: [patch 13/40] binfmt_elf: take the mmap lock around find_extend_vma() Message-ID: <20201017231415.XtZWgOT7Q%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jann Horn Subject: binfmt_elf: take the mmap lock around find_extend_vma() create_elf_tables() runs after setup_new_exec(), so other tasks can already access our new mm and do things like process_madvise() on it. (At the time I'm writing this commit, process_madvise() is not in mainline yet, but has been in akpm's tree for some time.) While I believe that there are currently no APIs that would actually allow another process to mess up our VMA tree (process_madvise() is limited to MADV_COLD and MADV_PAGEOUT, and uring and userfaultfd cannot reach an mm under which no syscalls have been executed yet), this seems like an accident waiting to happen. Let's make sure that we always take the mmap lock around GUP paths as long as another process might be able to see the mm. (Yes, this diff looks suspicious because we drop the lock before doing anything with `vma`, but that's because we actually don't do anything with it apart from the NULL check.) Link: https://lkml.kernel.org/r/CAG48ez1-PBCdv3y8pn-Ty-b+FmBSLwDuVKFSt8h7wARLy0dF-Q@mail.gmail.com Signed-off-by: Jann Horn Acked-by: Michel Lespinasse Cc: "Eric W . Biederman" Cc: Jason Gunthorpe Cc: John Hubbard Cc: Mauro Carvalho Chehab Cc: Sakari Ailus Signed-off-by: Andrew Morton --- fs/binfmt_elf.c | 3 +++ 1 file changed, 3 insertions(+) --- a/fs/binfmt_elf.c~binfmt_elf-take-the-mmap-lock-around-find_extend_vma +++ a/fs/binfmt_elf.c @@ -310,7 +310,10 @@ create_elf_tables(struct linux_binprm *b * Grow the stack manually; some architectures have a limit on how * far ahead a user-space access may be in order to grow the stack. */ + if (mmap_read_lock_killable(mm)) + return -EINTR; vma = find_extend_vma(mm, bprm->p); + mmap_read_unlock(mm); if (!vma) return -EFAULT; From patchwork Sat Oct 17 23:14:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843107 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9639E15E6 for ; Sat, 17 Oct 2020 23:14:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5E0F821582 for ; Sat, 17 Oct 2020 23:14:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="A1F2qudu" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E0F821582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 776596B0081; Sat, 17 Oct 2020 19:14:21 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 727746B0082; Sat, 17 Oct 2020 19:14:21 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 616EE6B0083; Sat, 17 Oct 2020 19:14:21 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id 32D086B0081 for ; Sat, 17 Oct 2020 19:14:21 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id CF21F180AD806 for ; Sat, 17 Oct 2020 23:14:20 +0000 (UTC) X-FDA: 77382973080.28.chain21_340927e27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id AECB86C05 for ; Sat, 17 Oct 2020 23:14:20 +0000 (UTC) X-Spam-Summary: 1,0,0,cc1c88834fa4b718,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2564:2682:2685:2859:2902:2903:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3165:3352:3865:3867:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:5007:6119:6261:6653:6737:7576:7903:7904:8957:9025:9545:10004:11026:11658:11914:12043:12048:12297:12438:12517:12519:12679:13069:13311:13357:14181:14384:14721:14777:21080:21433:21451:21627:21939:21990:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrr84gtmqecc8q35g6qpf4dzgdooppympc1dak8uqgmdtmqtsk66yg6x11h6w.fq8gni8ewadhtua4tkpb3ox9y99ebau58sscrd4yrb67rgg1jac9bse7fwznykz.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,L Ftime:22 X-HE-Tag: chain21_340927e27229 X-Filterd-Recvd-Size: 2869 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf39.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:20 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2C49621527; Sat, 17 Oct 2020 23:14:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976459; bh=GFKyW9Z0KnVTy7xMzO734UShpCWOyDBiv+3IMmcJJug=; h=Date:From:To:Subject:In-Reply-To:From; b=A1F2quduP7mKEKwcB4XnHZG1yB6RXN6FSJYAem9l+DGKudmxhbXSfCUZ879XIok1p apRdfaV6hZBlos1YW/FHwcRkQ8IVLav6QHZdd+66LdpgeDnV/Sx9xNfxa5Jc0wWpyd XPe6bRBw55/pkZC0uax/QBpwQHYP1PUPH+7WQOe8= Date: Sat, 17 Oct 2020 16:14:18 -0700 From: Andrew Morton To: akpm@linux-foundation.org, ebiederm@xmission.com, jannh@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mchehab@kernel.org, mm-commits@vger.kernel.org, sakari.ailus@linux.intel.com, torvalds@linux-foundation.org, walken@google.com Subject: [patch 14/40] mm/gup: assert that the mmap lock is held in __get_user_pages() Message-ID: <20201017231418.twQriq6_i%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jann Horn Subject: mm/gup: assert that the mmap lock is held in __get_user_pages() After having cleaned up all GUP callers (except for the atomisp staging driver, which currently gets mmap locking completely wrong [1]) to always ensure that they hold the mmap lock when calling into GUP (unless the mm is not yet globally visible), add an assertion to make sure it stays that way going forward. [1] https://lore.kernel.org/lkml/CAG48ez3tZAb9JVhw4T5e-i=h2_DUZxfNRTDsagSRCVazNXx5qA@mail.gmail.com/ Link: https://lkml.kernel.org/r/CAG48ez1GM==OnHpS=ghqZNJPn02FCDUEHc7GQmGRMXUD_aKudg@mail.gmail.com Signed-off-by: Jann Horn Reviewed-by: Jason Gunthorpe Acked-by: Michel Lespinasse Cc: "Eric W . Biederman" Cc: John Hubbard Cc: Mauro Carvalho Chehab Cc: Sakari Ailus Signed-off-by: Andrew Morton --- mm/gup.c | 2 ++ 1 file changed, 2 insertions(+) --- a/mm/gup.c~mm-gup-assert-that-the-mmap-lock-is-held-in-__get_user_pages +++ a/mm/gup.c @@ -1027,6 +1027,8 @@ static long __get_user_pages(struct mm_s struct vm_area_struct *vma = NULL; struct follow_page_context ctx = { NULL }; + mmap_assert_locked(mm); + if (!nr_pages) return 0; From patchwork Sat Oct 17 23:14:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7BDBC14B5 for ; Sat, 17 Oct 2020 23:14:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 281BE20897 for ; Sat, 17 Oct 2020 23:14:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="csnhjtuZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 281BE20897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 26DD26B0082; Sat, 17 Oct 2020 19:14:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1F6D06B0083; Sat, 17 Oct 2020 19:14:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 099986B0085; Sat, 17 Oct 2020 19:14:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0166.hostedemail.com [216.40.44.166]) by kanga.kvack.org (Postfix) with ESMTP id C73286B0082 for ; Sat, 17 Oct 2020 19:14:24 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 6B887181AEF07 for ; Sat, 17 Oct 2020 23:14:24 +0000 (UTC) X-FDA: 77382973248.12.news37_5309b0627229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin12.hostedemail.com (Postfix) with ESMTP id 492301800972A for ; Sat, 17 Oct 2020 23:14:24 +0000 (UTC) X-Spam-Summary: 1,0,0,61a06fb7dd89c7e3,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:152:327:355:379:800:960:966:967:968:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1593:1594:1605:1730:1747:1777:1792:1801:2194:2196:2198:2199:2200:2201:2393:2525:2559:2564:2682:2685:2692:2859:2892:2895:2899:2902:2915:2924:2925:2926:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6119:6261:6653:7576:7619:7875:7903:7904:8603:9025:9545:9592:10004:11026:11232:11473:11658:11914:12043:12048:12219:12296:12297:12438:12517:12519:12555:12679:12683:12986:13149:13161:13229:13230:13846:21080:21433:21451:21627:21740:21795:21939:21987:21990:30012:30029:30034:30045:30051:30054:30056:30062:30064:30070:30074:30075,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04yrbhccpeq3 uzrg5ojm X-HE-Tag: news37_5309b0627229 X-Filterd-Recvd-Size: 26911 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:23 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6A9AF2158C; Sat, 17 Oct 2020 23:14:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976462; bh=0so5ITxpFlMhSHyJEA9IDQxRvB9srj/i6GoXvqOIlm0=; h=Date:From:To:Subject:In-Reply-To:From; b=csnhjtuZsdmSGY/ge6FOlYFuBTMAJMZIyGUocBXheUtnYW2XPylfgWw6zqtk2EVxJ fYE3C04nJvP4TTAJtL57H97gCgEiKPBqiPtrh6tiIYc4dYpiMl5NHMiFLt5UrhjQJF +MWldxxaSat5Z8XqKGyUJ4MEDaNOUEJR+Jlc7Tzk= Date: Sat, 17 Oct 2020 16:14:22 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 15/40] mm/gup_benchmark: rename to mm/gup_test Message-ID: <20201017231422.Tmr-EXVMd%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: mm/gup_benchmark: rename to mm/gup_test Patch series "selftests/vm: gup_test, hmm-tests, assorted improvements", v2. This series provides two main things, and a number of smaller supporting goodies. The two main points are: 1) Add a new sub-test to gup_test, which in turn is a renamed version of gup_benchmark. This sub-test allows nicer testing of dump_pages(), at least on user-space pages. For quite a while, I was doing a quick hack to gup_test.c whenever I wanted to try out changes to dump_page(). Then Matthew Wilcox asked me what I meant when I said "I used my dump_page() unit test", and I realized that it might be nice to check in a polished up version of that. Details about how it works and how to use it are in the commit description for patch #6. 2) Fixes a limitation of hmm-tests: these tests are incredibly useful, but only if people actually build and run them. And it turns out that libhugetlbfs is a little too effective at throwing a wrench in the works, there. So I've added a little configuration check that removes just two of the 21 hmm-tests, if libhugetlbfs is not available. Further details in the commit description of patch #8. Other smaller things that this series does: a) Remove code duplication by creating gup_test.h. b) Clear up the sub-test organization, and their invocation within run_vmtests.sh. c) Other minor assorted improvements. This patch (of 8): Rename nearly every "gup_benchmark" reference and file name to "gup_test". The one exception is for the actual gup benchmark test itself. The current code already does a *little* bit more than benchmarking, and definitely covers more than get_user_pages_fast(). More importantly, however, subsequent patches are about to add some functionality that is non-benchmark related. Closely related changes: * Kconfig: in addition to renaming the options from GUP_BENCHMARK to GUP_TEST, update the help text to reflect that it's no longer a benchmark-only test. Link: https://lkml.kernel.org/r/20200929212747.251804-1-jhubbard@nvidia.com Link: https://lkml.kernel.org/r/20200929212747.251804-2-jhubbard@nvidia.com Signed-off-by: John Hubbard CC: Jonathan Corbet Cc: Jérôme Glisse Cc: Ralph Campbell Cc: Shuah Khan Cc: John Hubbard Signed-off-by: Andrew Morton --- Documentation/core-api/pin_user_pages.rst | 6 arch/s390/configs/debug_defconfig | 2 arch/s390/configs/defconfig | 2 mm/Kconfig | 15 - mm/Makefile | 2 mm/gup_benchmark.c | 210 ------------------- mm/gup_test.c | 210 +++++++++++++++++++ tools/testing/selftests/vm/.gitignore | 2 tools/testing/selftests/vm/Makefile | 2 tools/testing/selftests/vm/config | 2 tools/testing/selftests/vm/gup_benchmark.c | 143 ------------ tools/testing/selftests/vm/gup_test.c | 143 ++++++++++++ tools/testing/selftests/vm/run_vmtests | 8 13 files changed, 376 insertions(+), 371 deletions(-) --- a/arch/s390/configs/debug_defconfig~mm-gup_benchmark-rename-to-mm-gup_test +++ a/arch/s390/configs/debug_defconfig @@ -100,7 +100,7 @@ CONFIG_ZSMALLOC_STAT=y CONFIG_DEFERRED_STRUCT_PAGE_INIT=y CONFIG_IDLE_PAGE_TRACKING=y CONFIG_PERCPU_STATS=y -CONFIG_GUP_BENCHMARK=y +CONFIG_GUP_TEST=y CONFIG_NET=y CONFIG_PACKET=y CONFIG_PACKET_DIAG=m --- a/arch/s390/configs/defconfig~mm-gup_benchmark-rename-to-mm-gup_test +++ a/arch/s390/configs/defconfig @@ -94,7 +94,7 @@ CONFIG_ZSMALLOC_STAT=y CONFIG_DEFERRED_STRUCT_PAGE_INIT=y CONFIG_IDLE_PAGE_TRACKING=y CONFIG_PERCPU_STATS=y -CONFIG_GUP_BENCHMARK=y +CONFIG_GUP_TEST=y CONFIG_NET=y CONFIG_PACKET=y CONFIG_PACKET_DIAG=m --- a/Documentation/core-api/pin_user_pages.rst~mm-gup_benchmark-rename-to-mm-gup_test +++ a/Documentation/core-api/pin_user_pages.rst @@ -221,12 +221,12 @@ Unit testing ============ This file:: - tools/testing/selftests/vm/gup_benchmark.c + tools/testing/selftests/vm/gup_test.c has the following new calls to exercise the new pin*() wrapper functions: -* PIN_FAST_BENCHMARK (./gup_benchmark -a) -* PIN_BENCHMARK (./gup_benchmark -b) +* PIN_FAST_BENCHMARK (./gup_test -a) +* PIN_BENCHMARK (./gup_test -b) You can monitor how many total dma-pinned pages have been acquired and released since the system was booted, via two new /proc/vmstat entries: :: --- a/mm/gup_benchmark.c +++ /dev/null @@ -1,210 +0,0 @@ -#include -#include -#include -#include -#include -#include - -#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) -#define GUP_BENCHMARK _IOWR('g', 2, struct gup_benchmark) -#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_benchmark) -#define PIN_BENCHMARK _IOWR('g', 4, struct gup_benchmark) -#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_benchmark) - -struct gup_benchmark { - __u64 get_delta_usec; - __u64 put_delta_usec; - __u64 addr; - __u64 size; - __u32 nr_pages_per_call; - __u32 flags; - __u64 expansion[10]; /* For future use */ -}; - -static void put_back_pages(unsigned int cmd, struct page **pages, - unsigned long nr_pages) -{ - unsigned long i; - - switch (cmd) { - case GUP_FAST_BENCHMARK: - case GUP_BENCHMARK: - for (i = 0; i < nr_pages; i++) - put_page(pages[i]); - break; - - case PIN_FAST_BENCHMARK: - case PIN_BENCHMARK: - case PIN_LONGTERM_BENCHMARK: - unpin_user_pages(pages, nr_pages); - break; - } -} - -static void verify_dma_pinned(unsigned int cmd, struct page **pages, - unsigned long nr_pages) -{ - unsigned long i; - struct page *page; - - switch (cmd) { - case PIN_FAST_BENCHMARK: - case PIN_BENCHMARK: - case PIN_LONGTERM_BENCHMARK: - for (i = 0; i < nr_pages; i++) { - page = pages[i]; - if (WARN(!page_maybe_dma_pinned(page), - "pages[%lu] is NOT dma-pinned\n", i)) { - - dump_page(page, "gup_benchmark failure"); - break; - } - } - break; - } -} - -static int __gup_benchmark_ioctl(unsigned int cmd, - struct gup_benchmark *gup) -{ - ktime_t start_time, end_time; - unsigned long i, nr_pages, addr, next; - int nr; - struct page **pages; - int ret = 0; - bool needs_mmap_lock = - cmd != GUP_FAST_BENCHMARK && cmd != PIN_FAST_BENCHMARK; - - if (gup->size > ULONG_MAX) - return -EINVAL; - - nr_pages = gup->size / PAGE_SIZE; - pages = kvcalloc(nr_pages, sizeof(void *), GFP_KERNEL); - if (!pages) - return -ENOMEM; - - if (needs_mmap_lock && mmap_read_lock_killable(current->mm)) { - ret = -EINTR; - goto free_pages; - } - - i = 0; - nr = gup->nr_pages_per_call; - start_time = ktime_get(); - for (addr = gup->addr; addr < gup->addr + gup->size; addr = next) { - if (nr != gup->nr_pages_per_call) - break; - - next = addr + nr * PAGE_SIZE; - if (next > gup->addr + gup->size) { - next = gup->addr + gup->size; - nr = (next - addr) / PAGE_SIZE; - } - - /* Filter out most gup flags: only allow a tiny subset here: */ - gup->flags &= FOLL_WRITE; - - switch (cmd) { - case GUP_FAST_BENCHMARK: - nr = get_user_pages_fast(addr, nr, gup->flags, - pages + i); - break; - case GUP_BENCHMARK: - nr = get_user_pages(addr, nr, gup->flags, pages + i, - NULL); - break; - case PIN_FAST_BENCHMARK: - nr = pin_user_pages_fast(addr, nr, gup->flags, - pages + i); - break; - case PIN_BENCHMARK: - nr = pin_user_pages(addr, nr, gup->flags, pages + i, - NULL); - break; - case PIN_LONGTERM_BENCHMARK: - nr = pin_user_pages(addr, nr, - gup->flags | FOLL_LONGTERM, - pages + i, NULL); - break; - default: - ret = -EINVAL; - goto unlock; - } - - if (nr <= 0) - break; - i += nr; - } - end_time = ktime_get(); - - /* Shifting the meaning of nr_pages: now it is actual number pinned: */ - nr_pages = i; - - gup->get_delta_usec = ktime_us_delta(end_time, start_time); - gup->size = addr - gup->addr; - - /* - * Take an un-benchmark-timed moment to verify DMA pinned - * state: print a warning if any non-dma-pinned pages are found: - */ - verify_dma_pinned(cmd, pages, nr_pages); - - start_time = ktime_get(); - - put_back_pages(cmd, pages, nr_pages); - - end_time = ktime_get(); - gup->put_delta_usec = ktime_us_delta(end_time, start_time); - -unlock: - if (needs_mmap_lock) - mmap_read_unlock(current->mm); -free_pages: - kvfree(pages); - return ret; -} - -static long gup_benchmark_ioctl(struct file *filep, unsigned int cmd, - unsigned long arg) -{ - struct gup_benchmark gup; - int ret; - - switch (cmd) { - case GUP_FAST_BENCHMARK: - case GUP_BENCHMARK: - case PIN_FAST_BENCHMARK: - case PIN_BENCHMARK: - case PIN_LONGTERM_BENCHMARK: - break; - default: - return -EINVAL; - } - - if (copy_from_user(&gup, (void __user *)arg, sizeof(gup))) - return -EFAULT; - - ret = __gup_benchmark_ioctl(cmd, &gup); - if (ret) - return ret; - - if (copy_to_user((void __user *)arg, &gup, sizeof(gup))) - return -EFAULT; - - return 0; -} - -static const struct file_operations gup_benchmark_fops = { - .open = nonseekable_open, - .unlocked_ioctl = gup_benchmark_ioctl, -}; - -static int gup_benchmark_init(void) -{ - debugfs_create_file_unsafe("gup_benchmark", 0600, NULL, NULL, - &gup_benchmark_fops); - - return 0; -} - -late_initcall(gup_benchmark_init); --- /dev/null +++ a/mm/gup_test.c @@ -0,0 +1,210 @@ +#include +#include +#include +#include +#include +#include + +#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_test) +#define GUP_BENCHMARK _IOWR('g', 2, struct gup_test) +#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_test) +#define PIN_BENCHMARK _IOWR('g', 4, struct gup_test) +#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_test) + +struct gup_test { + __u64 get_delta_usec; + __u64 put_delta_usec; + __u64 addr; + __u64 size; + __u32 nr_pages_per_call; + __u32 flags; + __u64 expansion[10]; /* For future use */ +}; + +static void put_back_pages(unsigned int cmd, struct page **pages, + unsigned long nr_pages) +{ + unsigned long i; + + switch (cmd) { + case GUP_FAST_BENCHMARK: + case GUP_BENCHMARK: + for (i = 0; i < nr_pages; i++) + put_page(pages[i]); + break; + + case PIN_FAST_BENCHMARK: + case PIN_BENCHMARK: + case PIN_LONGTERM_BENCHMARK: + unpin_user_pages(pages, nr_pages); + break; + } +} + +static void verify_dma_pinned(unsigned int cmd, struct page **pages, + unsigned long nr_pages) +{ + unsigned long i; + struct page *page; + + switch (cmd) { + case PIN_FAST_BENCHMARK: + case PIN_BENCHMARK: + case PIN_LONGTERM_BENCHMARK: + for (i = 0; i < nr_pages; i++) { + page = pages[i]; + if (WARN(!page_maybe_dma_pinned(page), + "pages[%lu] is NOT dma-pinned\n", i)) { + + dump_page(page, "gup_test failure"); + break; + } + } + break; + } +} + +static int __gup_test_ioctl(unsigned int cmd, + struct gup_test *gup) +{ + ktime_t start_time, end_time; + unsigned long i, nr_pages, addr, next; + int nr; + struct page **pages; + int ret = 0; + bool needs_mmap_lock = + cmd != GUP_FAST_BENCHMARK && cmd != PIN_FAST_BENCHMARK; + + if (gup->size > ULONG_MAX) + return -EINVAL; + + nr_pages = gup->size / PAGE_SIZE; + pages = kvcalloc(nr_pages, sizeof(void *), GFP_KERNEL); + if (!pages) + return -ENOMEM; + + if (needs_mmap_lock && mmap_read_lock_killable(current->mm)) { + ret = -EINTR; + goto free_pages; + } + + i = 0; + nr = gup->nr_pages_per_call; + start_time = ktime_get(); + for (addr = gup->addr; addr < gup->addr + gup->size; addr = next) { + if (nr != gup->nr_pages_per_call) + break; + + next = addr + nr * PAGE_SIZE; + if (next > gup->addr + gup->size) { + next = gup->addr + gup->size; + nr = (next - addr) / PAGE_SIZE; + } + + /* Filter out most gup flags: only allow a tiny subset here: */ + gup->flags &= FOLL_WRITE; + + switch (cmd) { + case GUP_FAST_BENCHMARK: + nr = get_user_pages_fast(addr, nr, gup->flags, + pages + i); + break; + case GUP_BENCHMARK: + nr = get_user_pages(addr, nr, gup->flags, pages + i, + NULL); + break; + case PIN_FAST_BENCHMARK: + nr = pin_user_pages_fast(addr, nr, gup->flags, + pages + i); + break; + case PIN_BENCHMARK: + nr = pin_user_pages(addr, nr, gup->flags, pages + i, + NULL); + break; + case PIN_LONGTERM_BENCHMARK: + nr = pin_user_pages(addr, nr, + gup->flags | FOLL_LONGTERM, + pages + i, NULL); + break; + default: + ret = -EINVAL; + goto unlock; + } + + if (nr <= 0) + break; + i += nr; + } + end_time = ktime_get(); + + /* Shifting the meaning of nr_pages: now it is actual number pinned: */ + nr_pages = i; + + gup->get_delta_usec = ktime_us_delta(end_time, start_time); + gup->size = addr - gup->addr; + + /* + * Take an un-benchmark-timed moment to verify DMA pinned + * state: print a warning if any non-dma-pinned pages are found: + */ + verify_dma_pinned(cmd, pages, nr_pages); + + start_time = ktime_get(); + + put_back_pages(cmd, pages, nr_pages); + + end_time = ktime_get(); + gup->put_delta_usec = ktime_us_delta(end_time, start_time); + +unlock: + if (needs_mmap_lock) + mmap_read_unlock(current->mm); +free_pages: + kvfree(pages); + return ret; +} + +static long gup_test_ioctl(struct file *filep, unsigned int cmd, + unsigned long arg) +{ + struct gup_test gup; + int ret; + + switch (cmd) { + case GUP_FAST_BENCHMARK: + case GUP_BENCHMARK: + case PIN_FAST_BENCHMARK: + case PIN_BENCHMARK: + case PIN_LONGTERM_BENCHMARK: + break; + default: + return -EINVAL; + } + + if (copy_from_user(&gup, (void __user *)arg, sizeof(gup))) + return -EFAULT; + + ret = __gup_test_ioctl(cmd, &gup); + if (ret) + return ret; + + if (copy_to_user((void __user *)arg, &gup, sizeof(gup))) + return -EFAULT; + + return 0; +} + +static const struct file_operations gup_test_fops = { + .open = nonseekable_open, + .unlocked_ioctl = gup_test_ioctl, +}; + +static int gup_test_init(void) +{ + debugfs_create_file_unsafe("gup_test", 0600, NULL, NULL, + &gup_test_fops); + + return 0; +} + +late_initcall(gup_test_init); --- a/mm/Kconfig~mm-gup_benchmark-rename-to-mm-gup_test +++ a/mm/Kconfig @@ -831,13 +831,18 @@ config PERCPU_STATS information includes global and per chunk statistics, which can be used to help understand percpu memory usage. -config GUP_BENCHMARK - bool "Enable infrastructure for get_user_pages() and related calls benchmarking" +config GUP_TEST + bool "Enable infrastructure for get_user_pages()-related unit tests" help - Provides /sys/kernel/debug/gup_benchmark that helps with testing - performance of get_user_pages() and related calls. + Provides /sys/kernel/debug/gup_test, which in turn provides a way + to make ioctl calls that can launch kernel-based unit tests for + the get_user_pages*() and pin_user_pages*() family of API calls. - See tools/testing/selftests/vm/gup_benchmark.c + These tests include benchmark testing of the _fast variants of + get_user_pages*() and pin_user_pages*(), as well as smoke tests of + the non-_fast variants. + + See tools/testing/selftests/vm/gup_test.c config GUP_GET_PTE_LOW_HIGH bool --- a/mm/Makefile~mm-gup_benchmark-rename-to-mm-gup_test +++ a/mm/Makefile @@ -90,7 +90,7 @@ obj-$(CONFIG_PAGE_COUNTER) += page_count obj-$(CONFIG_MEMCG) += memcontrol.o vmpressure.o obj-$(CONFIG_MEMCG_SWAP) += swap_cgroup.o obj-$(CONFIG_CGROUP_HUGETLB) += hugetlb_cgroup.o -obj-$(CONFIG_GUP_BENCHMARK) += gup_benchmark.o +obj-$(CONFIG_GUP_TEST) += gup_test.o obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o obj-$(CONFIG_HWPOISON_INJECT) += hwpoison-inject.o obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o --- a/tools/testing/selftests/vm/config~mm-gup_benchmark-rename-to-mm-gup_test +++ a/tools/testing/selftests/vm/config @@ -3,4 +3,4 @@ CONFIG_USERFAULTFD=y CONFIG_TEST_VMALLOC=m CONFIG_DEVICE_PRIVATE=y CONFIG_TEST_HMM=m -CONFIG_GUP_BENCHMARK=y +CONFIG_GUP_TEST=y --- a/tools/testing/selftests/vm/.gitignore~mm-gup_benchmark-rename-to-mm-gup_test +++ a/tools/testing/selftests/vm/.gitignore @@ -15,7 +15,7 @@ userfaultfd mlock-intersect-test mlock-random-test virtual_address_range -gup_benchmark +gup_test va_128TBswitch map_fixed_noreplace write_to_hugetlbfs --- a/tools/testing/selftests/vm/gup_benchmark.c +++ /dev/null @@ -1,143 +0,0 @@ -#include -#include -#include -#include - -#include -#include -#include -#include -#include - -#include - -#define MB (1UL << 20) -#define PAGE_SIZE sysconf(_SC_PAGESIZE) - -#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) -#define GUP_BENCHMARK _IOWR('g', 2, struct gup_benchmark) - -/* Similar to above, but use FOLL_PIN instead of FOLL_GET. */ -#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_benchmark) -#define PIN_BENCHMARK _IOWR('g', 4, struct gup_benchmark) -#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_benchmark) - -/* Just the flags we need, copied from mm.h: */ -#define FOLL_WRITE 0x01 /* check pte is writable */ - -struct gup_benchmark { - __u64 get_delta_usec; - __u64 put_delta_usec; - __u64 addr; - __u64 size; - __u32 nr_pages_per_call; - __u32 flags; - __u64 expansion[10]; /* For future use */ -}; - -int main(int argc, char **argv) -{ - struct gup_benchmark gup; - unsigned long size = 128 * MB; - int i, fd, filed, opt, nr_pages = 1, thp = -1, repeats = 1, write = 0; - int cmd = GUP_FAST_BENCHMARK, flags = MAP_PRIVATE; - char *file = "/dev/zero"; - char *p; - - while ((opt = getopt(argc, argv, "m:r:n:f:abtTLUuwSH")) != -1) { - switch (opt) { - case 'a': - cmd = PIN_FAST_BENCHMARK; - break; - case 'b': - cmd = PIN_BENCHMARK; - break; - case 'L': - cmd = PIN_LONGTERM_BENCHMARK; - break; - case 'm': - size = atoi(optarg) * MB; - break; - case 'r': - repeats = atoi(optarg); - break; - case 'n': - nr_pages = atoi(optarg); - break; - case 't': - thp = 1; - break; - case 'T': - thp = 0; - break; - case 'U': - cmd = GUP_BENCHMARK; - break; - case 'u': - cmd = GUP_FAST_BENCHMARK; - break; - case 'w': - write = 1; - break; - case 'f': - file = optarg; - break; - case 'S': - flags &= ~MAP_PRIVATE; - flags |= MAP_SHARED; - break; - case 'H': - flags |= (MAP_HUGETLB | MAP_ANONYMOUS); - break; - default: - return -1; - } - } - - filed = open(file, O_RDWR|O_CREAT); - if (filed < 0) { - perror("open"); - exit(filed); - } - - gup.nr_pages_per_call = nr_pages; - if (write) - gup.flags |= FOLL_WRITE; - - fd = open("/sys/kernel/debug/gup_benchmark", O_RDWR); - if (fd == -1) { - perror("open"); - exit(1); - } - - p = mmap(NULL, size, PROT_READ | PROT_WRITE, flags, filed, 0); - if (p == MAP_FAILED) { - perror("mmap"); - exit(1); - } - gup.addr = (unsigned long)p; - - if (thp == 1) - madvise(p, size, MADV_HUGEPAGE); - else if (thp == 0) - madvise(p, size, MADV_NOHUGEPAGE); - - for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE) - p[0] = 0; - - for (i = 0; i < repeats; i++) { - gup.size = size; - if (ioctl(fd, cmd, &gup)) { - perror("ioctl"); - exit(1); - } - - printf("Time: get:%lld put:%lld us", gup.get_delta_usec, - gup.put_delta_usec); - if (gup.size != size) - printf(", truncated (size: %lld)", gup.size); - printf("\n"); - } - - return 0; -} --- /dev/null +++ a/tools/testing/selftests/vm/gup_test.c @@ -0,0 +1,143 @@ +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#include + +#define MB (1UL << 20) +#define PAGE_SIZE sysconf(_SC_PAGESIZE) + +#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) +#define GUP_BENCHMARK _IOWR('g', 2, struct gup_benchmark) + +/* Similar to above, but use FOLL_PIN instead of FOLL_GET. */ +#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_benchmark) +#define PIN_BENCHMARK _IOWR('g', 4, struct gup_benchmark) +#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_benchmark) + +/* Just the flags we need, copied from mm.h: */ +#define FOLL_WRITE 0x01 /* check pte is writable */ + +struct gup_benchmark { + __u64 get_delta_usec; + __u64 put_delta_usec; + __u64 addr; + __u64 size; + __u32 nr_pages_per_call; + __u32 flags; + __u64 expansion[10]; /* For future use */ +}; + +int main(int argc, char **argv) +{ + struct gup_benchmark gup; + unsigned long size = 128 * MB; + int i, fd, filed, opt, nr_pages = 1, thp = -1, repeats = 1, write = 0; + int cmd = GUP_FAST_BENCHMARK, flags = MAP_PRIVATE; + char *file = "/dev/zero"; + char *p; + + while ((opt = getopt(argc, argv, "m:r:n:f:abtTLUuwSH")) != -1) { + switch (opt) { + case 'a': + cmd = PIN_FAST_BENCHMARK; + break; + case 'b': + cmd = PIN_BENCHMARK; + break; + case 'L': + cmd = PIN_LONGTERM_BENCHMARK; + break; + case 'm': + size = atoi(optarg) * MB; + break; + case 'r': + repeats = atoi(optarg); + break; + case 'n': + nr_pages = atoi(optarg); + break; + case 't': + thp = 1; + break; + case 'T': + thp = 0; + break; + case 'U': + cmd = GUP_BENCHMARK; + break; + case 'u': + cmd = GUP_FAST_BENCHMARK; + break; + case 'w': + write = 1; + break; + case 'f': + file = optarg; + break; + case 'S': + flags &= ~MAP_PRIVATE; + flags |= MAP_SHARED; + break; + case 'H': + flags |= (MAP_HUGETLB | MAP_ANONYMOUS); + break; + default: + return -1; + } + } + + filed = open(file, O_RDWR|O_CREAT); + if (filed < 0) { + perror("open"); + exit(filed); + } + + gup.nr_pages_per_call = nr_pages; + if (write) + gup.flags |= FOLL_WRITE; + + fd = open("/sys/kernel/debug/gup_benchmark", O_RDWR); + if (fd == -1) { + perror("open"); + exit(1); + } + + p = mmap(NULL, size, PROT_READ | PROT_WRITE, flags, filed, 0); + if (p == MAP_FAILED) { + perror("mmap"); + exit(1); + } + gup.addr = (unsigned long)p; + + if (thp == 1) + madvise(p, size, MADV_HUGEPAGE); + else if (thp == 0) + madvise(p, size, MADV_NOHUGEPAGE); + + for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE) + p[0] = 0; + + for (i = 0; i < repeats; i++) { + gup.size = size; + if (ioctl(fd, cmd, &gup)) { + perror("ioctl"); + exit(1); + } + + printf("Time: get:%lld put:%lld us", gup.get_delta_usec, + gup.put_delta_usec); + if (gup.size != size) + printf(", truncated (size: %lld)", gup.size); + printf("\n"); + } + + return 0; +} --- a/tools/testing/selftests/vm/Makefile~mm-gup_benchmark-rename-to-mm-gup_test +++ a/tools/testing/selftests/vm/Makefile @@ -23,7 +23,7 @@ MAKEFLAGS += --no-builtin-rules CFLAGS = -Wall -I ../../../../usr/include $(EXTRA_CFLAGS) LDLIBS = -lrt TEST_GEN_FILES = compaction_test -TEST_GEN_FILES += gup_benchmark +TEST_GEN_FILES += gup_test TEST_GEN_FILES += hmm-tests TEST_GEN_FILES += hugepage-mmap TEST_GEN_FILES += hugepage-shm --- a/tools/testing/selftests/vm/run_vmtests~mm-gup_benchmark-rename-to-mm-gup_test +++ a/tools/testing/selftests/vm/run_vmtests @@ -124,9 +124,9 @@ else fi echo "--------------------------------------------" -echo "running 'gup_benchmark -U' (normal/slow gup)" +echo "running 'gup_test -U' (normal/slow gup)" echo "--------------------------------------------" -./gup_benchmark -U +./gup_test -U if [ $? -ne 0 ]; then echo "[FAIL]" exitcode=1 @@ -135,9 +135,9 @@ else fi echo "------------------------------------------" -echo "running gup_benchmark -b (pin_user_pages)" +echo "running gup_test -b (pin_user_pages)" echo "------------------------------------------" -./gup_benchmark -b +./gup_test -b if [ $? -ne 0 ]; then echo "[FAIL]" exitcode=1 From patchwork Sat Oct 17 23:14:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843111 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 58FEA14B5 for ; Sat, 17 Oct 2020 23:14:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F21F216C4 for ; Sat, 17 Oct 2020 23:14:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="fFmOkW/3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F21F216C4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1AB0C6B0083; Sat, 17 Oct 2020 19:14:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 15C9D6B0085; Sat, 17 Oct 2020 19:14:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 04A776B0087; Sat, 17 Oct 2020 19:14:27 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0052.hostedemail.com [216.40.44.52]) by kanga.kvack.org (Postfix) with ESMTP id C8B176B0083 for ; Sat, 17 Oct 2020 19:14:27 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6BFB88249980 for ; Sat, 17 Oct 2020 23:14:27 +0000 (UTC) X-FDA: 77382973374.18.bait57_190dc7827229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id 499FC101E4FA1 for ; Sat, 17 Oct 2020 23:14:27 +0000 (UTC) X-Spam-Summary: 1,0,0,032bc066b1a2e1ed,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:152:355:379:800:960:967:973:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1535:1544:1593:1594:1605:1711:1730:1747:1777:1792:1801:2194:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4117:4250:4321:4605:5007:6119:6261:6653:7576:7903:8603:9025:9545:9592:10004:11026:11473:11657:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12986:13138:13149:13178:13229:13230:13231:13846:14181:14721:21080:21451:21627:21795:21939:30051:30054:30056:30064:30070:30074,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04yfi4temap4n9u8rr3kekoooo183ycxnzdbyjoninmm6fhp43hhgfire8by8ru.stez7tprw634gugwbdjmbjffp1orizw6b4g9o6izbby68nzwwc9buc4jgze7mhe.o- lbl8.mai X-HE-Tag: bait57_190dc7827229 X-Filterd-Recvd-Size: 6389 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:26 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 98C6520EDD; Sat, 17 Oct 2020 23:14:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976465; bh=9xVVYY6w/84m+mqBteiQNvreVZLMaTle53ly7efDV4M=; h=Date:From:To:Subject:In-Reply-To:From; b=fFmOkW/3uJG+LTuTnQsSvEEwL59sDctB1WzGpNpKHyjxAwYgqvZGcd1bwS2SWkQZW cz9VQ3Y0SnsIjnSWOvtWuRtj2RcRsm3hZfj7609mvUkNwXJ0yM2zrjka9VAPlPRbkM bPqJrK5hJQ3CEKxicFuOTu4B50WRRAJiDsoZFdDw= Date: Sat, 17 Oct 2020 16:14:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 16/40] selftests/vm: use a common gup_test.h Message-ID: <20201017231425.0OZYgQdDr%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: use a common gup_test.h Avoid the need to copy-paste the gup_test ioctl commands and the struct gup_test definition, between the kernel and the user space application, by providing a new header file for these. This allows easier and safer adding of new ioctl calls, as well as reducing the overall line count. Details: The header file has to be able to compile independently, because of the arguably unfortunate way that the Makefile is written: the Makefile tries to build all of its prerequisites, when really it should be only building the .c files, and leaving the other prerequisites (LOCAL_HDRS) as pure dependencies. That Makefile limitation is probably not worth fixing, but it explains why one of the includes had to be moved into the new header file. Also: simplify the ioctl struct (struct gup_test), by deleting the unused __expansion[10] field. This sort of thing is what you might see in a stable ABI, but this low-level, kernel-developer-oriented selftests/vm system is very much not subject to ABI stability. So "expansion" and "reserved" fields are unnecessary here. Link: https://lkml.kernel.org/r/20200929212747.251804-3-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Ralph Campbell Cc: Shuah Khan Signed-off-by: Andrew Morton --- mm/gup_test.c | 17 +---------------- mm/gup_test.h | 22 ++++++++++++++++++++++ tools/testing/selftests/vm/Makefile | 2 ++ tools/testing/selftests/vm/gup_test.c | 22 +--------------------- 4 files changed, 26 insertions(+), 37 deletions(-) --- a/mm/gup_test.c~selftests-vm-use-a-common-gup_testh +++ a/mm/gup_test.c @@ -4,22 +4,7 @@ #include #include #include - -#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_test) -#define GUP_BENCHMARK _IOWR('g', 2, struct gup_test) -#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_test) -#define PIN_BENCHMARK _IOWR('g', 4, struct gup_test) -#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_test) - -struct gup_test { - __u64 get_delta_usec; - __u64 put_delta_usec; - __u64 addr; - __u64 size; - __u32 nr_pages_per_call; - __u32 flags; - __u64 expansion[10]; /* For future use */ -}; +#include "../../../../mm/gup_test.h" static void put_back_pages(unsigned int cmd, struct page **pages, unsigned long nr_pages) --- /dev/null +++ a/mm/gup_test.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +#ifndef __GUP_TEST_H +#define __GUP_TEST_H + +#include + +#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_test) +#define GUP_BENCHMARK _IOWR('g', 2, struct gup_test) +#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_test) +#define PIN_BENCHMARK _IOWR('g', 4, struct gup_test) +#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_test) + +struct gup_test { + __u64 get_delta_usec; + __u64 put_delta_usec; + __u64 addr; + __u64 size; + __u32 nr_pages_per_call; + __u32 flags; +}; + +#endif /* __GUP_TEST_H */ --- a/tools/testing/selftests/vm/gup_test.c~selftests-vm-use-a-common-gup_testh +++ a/tools/testing/selftests/vm/gup_test.c @@ -2,39 +2,19 @@ #include #include #include - #include #include #include #include #include - -#include +#include "../../../../mm/gup_test.h" #define MB (1UL << 20) #define PAGE_SIZE sysconf(_SC_PAGESIZE) -#define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_benchmark) -#define GUP_BENCHMARK _IOWR('g', 2, struct gup_benchmark) - -/* Similar to above, but use FOLL_PIN instead of FOLL_GET. */ -#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_benchmark) -#define PIN_BENCHMARK _IOWR('g', 4, struct gup_benchmark) -#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_benchmark) - /* Just the flags we need, copied from mm.h: */ #define FOLL_WRITE 0x01 /* check pte is writable */ -struct gup_benchmark { - __u64 get_delta_usec; - __u64 put_delta_usec; - __u64 addr; - __u64 size; - __u32 nr_pages_per_call; - __u32 flags; - __u64 expansion[10]; /* For future use */ -}; - int main(int argc, char **argv) { struct gup_benchmark gup; --- a/tools/testing/selftests/vm/Makefile~selftests-vm-use-a-common-gup_testh +++ a/tools/testing/selftests/vm/Makefile @@ -130,3 +130,5 @@ endif $(OUTPUT)/userfaultfd: LDLIBS += -lpthread $(OUTPUT)/mlock-random-test: LDLIBS += -lcap + +$(OUTPUT)/gup_test: ../../../../mm/gup_test.h From patchwork Sat Oct 17 23:14:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843113 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7238014B5 for ; Sat, 17 Oct 2020 23:14:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F8E621556 for ; Sat, 17 Oct 2020 23:14:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="pwysd1ue" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F8E621556 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 15CFE6B0085; Sat, 17 Oct 2020 19:14:31 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 132EB6B0087; Sat, 17 Oct 2020 19:14:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 049CE6B0088; Sat, 17 Oct 2020 19:14:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0062.hostedemail.com [216.40.44.62]) by kanga.kvack.org (Postfix) with ESMTP id C21106B0085 for ; Sat, 17 Oct 2020 19:14:30 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5B8A38249980 for ; Sat, 17 Oct 2020 23:14:30 +0000 (UTC) X-FDA: 77382973500.06.clam51_590b6ca27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id 3C68C101E2D39 for ; Sat, 17 Oct 2020 23:14:30 +0000 (UTC) X-Spam-Summary: 1,0,0,53eab8572917b8ee,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:4:41:69:152:355:379:800:960:966:967:973:982:988:989:1260:1277:1311:1313:1314:1345:1431:1437:1513:1515:1516:1518:1521:1593:1594:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2565:2570:2682:2685:2703:2731:2859:2890:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3653:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4042:4250:4321:4385:4605:5007:6119:6238:6261:7576:7875:8603:8957:9025:9545:10004:11658:12048:13146:13161:13229:13230:30056,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yf8ct166soq7ao7yznpsiy96n34ocwamoacdsyey6pa7uc3h7e3s781jwhrh4.h8zyujmjz5zqewkamnrp9q7gcdh98kh5peucxyc17tk6t8bthndw8fd9d1wjm1j.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:24,LUA_SUMMARY:none X-HE-Tag: clam51_590b6ca27229 X-Filterd-Recvd-Size: 18171 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:29 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id ADA9920E65; Sat, 17 Oct 2020 23:14:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976469; bh=n1vh/svw7FuDDIoEWaeCDbN5Sq2hhNt0iDhA/JEvXrM=; h=Date:From:To:Subject:In-Reply-To:From; b=pwysd1ueI//i8sSZJQEdt3v4aMIkgsznnzVobaeNOhsarhVRKXljhGlHu9zFkjuiC rAbMBH3TENLpEeia/ksPc+ac1hYux/PExX6k5VudJww9tpIBG6c3es+Oahk8px7xDp k7Kyw/jpjLutg1DP8ur6Y4ud3H+gcA7If2D9UgSQ= Date: Sat, 17 Oct 2020 16:14:28 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 17/40] selftests/vm: rename run_vmtests --> run_vmtests.sh Message-ID: <20201017231428.OuSvQ2jzW%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: rename run_vmtests --> run_vmtests.sh Rename to *.sh, in order to match the conventions of all of the other items in selftest/vm. The only reason not to use a .sh suffix a shell script like this, might be to make it look more like a normal program, but that's not an issue here. Link: https://lkml.kernel.org/r/20200929212747.251804-4-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Ralph Campbell Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/Makefile | 2 tools/testing/selftests/vm/run_vmtests | 326 -------------------- tools/testing/selftests/vm/run_vmtests.sh | 326 ++++++++++++++++++++ 3 files changed, 327 insertions(+), 327 deletions(-) --- a/tools/testing/selftests/vm/Makefile~selftests-vm-rename-run_vmtests-run_vmtestssh +++ a/tools/testing/selftests/vm/Makefile @@ -69,7 +69,7 @@ TEST_GEN_FILES += virtual_address_range TEST_GEN_FILES += write_to_hugetlbfs endif -TEST_PROGS := run_vmtests +TEST_PROGS := run_vmtests.sh TEST_FILES := test_vmalloc.sh --- a/tools/testing/selftests/vm/run_vmtests +++ /dev/null @@ -1,326 +0,0 @@ -#!/bin/bash -# SPDX-License-Identifier: GPL-2.0 -#please run as root - -# Kselftest framework requirement - SKIP code is 4. -ksft_skip=4 - -mnt=./huge -exitcode=0 - -#get huge pagesize and freepages from /proc/meminfo -while read name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs=$size - fi - if [ "$name" = "Hugepagesize:" ]; then - hpgsize_KB=$size - fi -done < /proc/meminfo - -# Simple hugetlbfs tests have a hardcoded minimum requirement of -# huge pages totaling 256MB (262144KB) in size. The userfaultfd -# hugetlb test requires a minimum of 2 * nr_cpus huge pages. Take -# both of these requirements into account and attempt to increase -# number of huge pages available. -nr_cpus=$(nproc) -hpgsize_MB=$((hpgsize_KB / 1024)) -half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128)) -needmem_KB=$((half_ufd_size_MB * 2 * 1024)) - -#set proper nr_hugepages -if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then - nr_hugepgs=`cat /proc/sys/vm/nr_hugepages` - needpgs=$((needmem_KB / hpgsize_KB)) - tries=2 - while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do - lackpgs=$(( $needpgs - $freepgs )) - echo 3 > /proc/sys/vm/drop_caches - echo $(( $lackpgs + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages - if [ $? -ne 0 ]; then - echo "Please run this test as root" - exit $ksft_skip - fi - while read name size unit; do - if [ "$name" = "HugePages_Free:" ]; then - freepgs=$size - fi - done < /proc/meminfo - tries=$((tries - 1)) - done - if [ $freepgs -lt $needpgs ]; then - printf "Not enough huge pages available (%d < %d)\n" \ - $freepgs $needpgs - exit 1 - fi -else - echo "no hugetlbfs support in kernel?" - exit 1 -fi - -#filter 64bit architectures -ARCH64STR="arm64 ia64 mips64 parisc64 ppc64 ppc64le riscv64 s390x sh64 sparc64 x86_64" -if [ -z $ARCH ]; then - ARCH=`uname -m 2>/dev/null | sed -e 's/aarch64.*/arm64/'` -fi -VADDR64=0 -echo "$ARCH64STR" | grep $ARCH && VADDR64=1 - -mkdir $mnt -mount -t hugetlbfs none $mnt - -echo "---------------------" -echo "running hugepage-mmap" -echo "---------------------" -./hugepage-mmap -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -shmmax=`cat /proc/sys/kernel/shmmax` -shmall=`cat /proc/sys/kernel/shmall` -echo 268435456 > /proc/sys/kernel/shmmax -echo 4194304 > /proc/sys/kernel/shmall -echo "--------------------" -echo "running hugepage-shm" -echo "--------------------" -./hugepage-shm -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi -echo $shmmax > /proc/sys/kernel/shmmax -echo $shmall > /proc/sys/kernel/shmall - -echo "-------------------" -echo "running map_hugetlb" -echo "-------------------" -./map_hugetlb -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "NOTE: The above hugetlb tests provide minimal coverage. Use" -echo " https://github.com/libhugetlbfs/libhugetlbfs.git for" -echo " hugetlb regression testing." - -echo "---------------------------" -echo "running map_fixed_noreplace" -echo "---------------------------" -./map_fixed_noreplace -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "--------------------------------------------" -echo "running 'gup_test -U' (normal/slow gup)" -echo "--------------------------------------------" -./gup_test -U -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "------------------------------------------" -echo "running gup_test -b (pin_user_pages)" -echo "------------------------------------------" -./gup_test -b -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "-------------------" -echo "running userfaultfd" -echo "-------------------" -./userfaultfd anon 128 32 -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "---------------------------" -echo "running userfaultfd_hugetlb" -echo "---------------------------" -# Test requires source and destination huge pages. Size of source -# (half_ufd_size_MB) is passed as argument to test. -./userfaultfd hugetlb $half_ufd_size_MB 32 $mnt/ufd_test_file -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi -rm -f $mnt/ufd_test_file - -echo "-------------------------" -echo "running userfaultfd_shmem" -echo "-------------------------" -./userfaultfd shmem 128 32 -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -#cleanup -umount $mnt -rm -rf $mnt -echo $nr_hugepgs > /proc/sys/vm/nr_hugepages - -echo "-----------------------" -echo "running compaction_test" -echo "-----------------------" -./compaction_test -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "----------------------" -echo "running on-fault-limit" -echo "----------------------" -sudo -u nobody ./on-fault-limit -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "--------------------" -echo "running map_populate" -echo "--------------------" -./map_populate -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "-------------------------" -echo "running mlock-random-test" -echo "-------------------------" -./mlock-random-test -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "--------------------" -echo "running mlock2-tests" -echo "--------------------" -./mlock2-tests -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "-----------------" -echo "running thuge-gen" -echo "-----------------" -./thuge-gen -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -if [ $VADDR64 -ne 0 ]; then -echo "-----------------------------" -echo "running virtual_address_range" -echo "-----------------------------" -./virtual_address_range -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi - -echo "-----------------------------" -echo "running virtual address 128TB switch test" -echo "-----------------------------" -./va_128TBswitch -if [ $? -ne 0 ]; then - echo "[FAIL]" - exitcode=1 -else - echo "[PASS]" -fi -fi # VADDR64 - -echo "------------------------------------" -echo "running vmalloc stability smoke test" -echo "------------------------------------" -./test_vmalloc.sh smoke -ret_val=$? - -if [ $ret_val -eq 0 ]; then - echo "[PASS]" -elif [ $ret_val -eq $ksft_skip ]; then - echo "[SKIP]" - exitcode=$ksft_skip -else - echo "[FAIL]" - exitcode=1 -fi - -echo "------------------------------------" -echo "running MREMAP_DONTUNMAP smoke test" -echo "------------------------------------" -./mremap_dontunmap -ret_val=$? - -if [ $ret_val -eq 0 ]; then - echo "[PASS]" -elif [ $ret_val -eq $ksft_skip ]; then - echo "[SKIP]" - exitcode=$ksft_skip -else - echo "[FAIL]" - exitcode=1 -fi - -echo "running HMM smoke test" -echo "------------------------------------" -./test_hmm.sh smoke -ret_val=$? - -if [ $ret_val -eq 0 ]; then - echo "[PASS]" -elif [ $ret_val -eq $ksft_skip ]; then - echo "[SKIP]" - exitcode=$ksft_skip -else - echo "[FAIL]" - exitcode=1 -fi - -exit $exitcode --- /dev/null +++ a/tools/testing/selftests/vm/run_vmtests.sh @@ -0,0 +1,326 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +#please run as root + +# Kselftest framework requirement - SKIP code is 4. +ksft_skip=4 + +mnt=./huge +exitcode=0 + +#get huge pagesize and freepages from /proc/meminfo +while read name size unit; do + if [ "$name" = "HugePages_Free:" ]; then + freepgs=$size + fi + if [ "$name" = "Hugepagesize:" ]; then + hpgsize_KB=$size + fi +done < /proc/meminfo + +# Simple hugetlbfs tests have a hardcoded minimum requirement of +# huge pages totaling 256MB (262144KB) in size. The userfaultfd +# hugetlb test requires a minimum of 2 * nr_cpus huge pages. Take +# both of these requirements into account and attempt to increase +# number of huge pages available. +nr_cpus=$(nproc) +hpgsize_MB=$((hpgsize_KB / 1024)) +half_ufd_size_MB=$((((nr_cpus * hpgsize_MB + 127) / 128) * 128)) +needmem_KB=$((half_ufd_size_MB * 2 * 1024)) + +#set proper nr_hugepages +if [ -n "$freepgs" ] && [ -n "$hpgsize_KB" ]; then + nr_hugepgs=`cat /proc/sys/vm/nr_hugepages` + needpgs=$((needmem_KB / hpgsize_KB)) + tries=2 + while [ $tries -gt 0 ] && [ $freepgs -lt $needpgs ]; do + lackpgs=$(( $needpgs - $freepgs )) + echo 3 > /proc/sys/vm/drop_caches + echo $(( $lackpgs + $nr_hugepgs )) > /proc/sys/vm/nr_hugepages + if [ $? -ne 0 ]; then + echo "Please run this test as root" + exit $ksft_skip + fi + while read name size unit; do + if [ "$name" = "HugePages_Free:" ]; then + freepgs=$size + fi + done < /proc/meminfo + tries=$((tries - 1)) + done + if [ $freepgs -lt $needpgs ]; then + printf "Not enough huge pages available (%d < %d)\n" \ + $freepgs $needpgs + exit 1 + fi +else + echo "no hugetlbfs support in kernel?" + exit 1 +fi + +#filter 64bit architectures +ARCH64STR="arm64 ia64 mips64 parisc64 ppc64 ppc64le riscv64 s390x sh64 sparc64 x86_64" +if [ -z $ARCH ]; then + ARCH=`uname -m 2>/dev/null | sed -e 's/aarch64.*/arm64/'` +fi +VADDR64=0 +echo "$ARCH64STR" | grep $ARCH && VADDR64=1 + +mkdir $mnt +mount -t hugetlbfs none $mnt + +echo "---------------------" +echo "running hugepage-mmap" +echo "---------------------" +./hugepage-mmap +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +shmmax=`cat /proc/sys/kernel/shmmax` +shmall=`cat /proc/sys/kernel/shmall` +echo 268435456 > /proc/sys/kernel/shmmax +echo 4194304 > /proc/sys/kernel/shmall +echo "--------------------" +echo "running hugepage-shm" +echo "--------------------" +./hugepage-shm +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi +echo $shmmax > /proc/sys/kernel/shmmax +echo $shmall > /proc/sys/kernel/shmall + +echo "-------------------" +echo "running map_hugetlb" +echo "-------------------" +./map_hugetlb +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "NOTE: The above hugetlb tests provide minimal coverage. Use" +echo " https://github.com/libhugetlbfs/libhugetlbfs.git for" +echo " hugetlb regression testing." + +echo "---------------------------" +echo "running map_fixed_noreplace" +echo "---------------------------" +./map_fixed_noreplace +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "--------------------------------------------" +echo "running 'gup_test -U' (normal/slow gup)" +echo "--------------------------------------------" +./gup_test -U +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "------------------------------------------" +echo "running gup_test -b (pin_user_pages)" +echo "------------------------------------------" +./gup_test -b +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "-------------------" +echo "running userfaultfd" +echo "-------------------" +./userfaultfd anon 128 32 +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "---------------------------" +echo "running userfaultfd_hugetlb" +echo "---------------------------" +# Test requires source and destination huge pages. Size of source +# (half_ufd_size_MB) is passed as argument to test. +./userfaultfd hugetlb $half_ufd_size_MB 32 $mnt/ufd_test_file +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi +rm -f $mnt/ufd_test_file + +echo "-------------------------" +echo "running userfaultfd_shmem" +echo "-------------------------" +./userfaultfd shmem 128 32 +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +#cleanup +umount $mnt +rm -rf $mnt +echo $nr_hugepgs > /proc/sys/vm/nr_hugepages + +echo "-----------------------" +echo "running compaction_test" +echo "-----------------------" +./compaction_test +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "----------------------" +echo "running on-fault-limit" +echo "----------------------" +sudo -u nobody ./on-fault-limit +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "--------------------" +echo "running map_populate" +echo "--------------------" +./map_populate +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "-------------------------" +echo "running mlock-random-test" +echo "-------------------------" +./mlock-random-test +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "--------------------" +echo "running mlock2-tests" +echo "--------------------" +./mlock2-tests +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "-----------------" +echo "running thuge-gen" +echo "-----------------" +./thuge-gen +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +if [ $VADDR64 -ne 0 ]; then +echo "-----------------------------" +echo "running virtual_address_range" +echo "-----------------------------" +./virtual_address_range +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "-----------------------------" +echo "running virtual address 128TB switch test" +echo "-----------------------------" +./va_128TBswitch +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi +fi # VADDR64 + +echo "------------------------------------" +echo "running vmalloc stability smoke test" +echo "------------------------------------" +./test_vmalloc.sh smoke +ret_val=$? + +if [ $ret_val -eq 0 ]; then + echo "[PASS]" +elif [ $ret_val -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip +else + echo "[FAIL]" + exitcode=1 +fi + +echo "------------------------------------" +echo "running MREMAP_DONTUNMAP smoke test" +echo "------------------------------------" +./mremap_dontunmap +ret_val=$? + +if [ $ret_val -eq 0 ]; then + echo "[PASS]" +elif [ $ret_val -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip +else + echo "[FAIL]" + exitcode=1 +fi + +echo "running HMM smoke test" +echo "------------------------------------" +./test_hmm.sh smoke +ret_val=$? + +if [ $ret_val -eq 0 ]; then + echo "[PASS]" +elif [ $ret_val -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip +else + echo "[FAIL]" + exitcode=1 +fi + +exit $exitcode From patchwork Sat Oct 17 23:14:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843115 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5A20315E6 for ; Sat, 17 Oct 2020 23:14:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 17F1F20897 for ; Sat, 17 Oct 2020 23:14:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Kd4z1YrE" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 17F1F20897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 222B66B0087; Sat, 17 Oct 2020 19:14:34 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1CEB26B0088; Sat, 17 Oct 2020 19:14:34 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E5D26B0089; Sat, 17 Oct 2020 19:14:34 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D665F6B0087 for ; Sat, 17 Oct 2020 19:14:33 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 81D2B8249980 for ; Sat, 17 Oct 2020 23:14:33 +0000 (UTC) X-FDA: 77382973626.01.hand61_4c0316627229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin01.hostedemail.com (Postfix) with ESMTP id 618CC10046472 for ; Sat, 17 Oct 2020 23:14:33 +0000 (UTC) X-Spam-Summary: 1,0,0,0d2cbe61a03c0a3c,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:152:355:379:800:960:967:968:973:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1534:1542:1593:1594:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2563:2682:2685:2731:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:2987:3022:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6119:6261:6653:7576:9025:9545:9592:10004:10400:11026:11473:11657:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12986:13053:13846:14096:14097:14181:14721:21080:21451:21627:21939:30054:30056:30064:30069,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrzdfncuyh6fh1sxeef14ce7ac7ypffuoknnixxgqyntfdyews3dubqnmmxwx.m4deh33rfizp19ck6zmjcd5eorcymnqxq5r1dpw4ziocby93br3fihg3kzedmr7.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5, 0.5,Netc X-HE-Tag: hand61_4c0316627229 X-Filterd-Recvd-Size: 4439 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:32 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id D20A220EDD; Sat, 17 Oct 2020 23:14:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976472; bh=iNU4Ovy5Elf8THMdQ4gFcXzeppLE19hDE4DfXbdWJjo=; h=Date:From:To:Subject:In-Reply-To:From; b=Kd4z1YrEnoPaKr8HeUUnRVS6LS4HhEuCoT+VVLgtUqUrRWwSFrxRq03ja+fW3mAvK hZv3UsJlkyE89Wcjf6NV5dEzYvfIQYYfXPqVhyUd6NJ/ERUfqmcNgqg5DhJ/czGc4L svv77y0Gt4CC0zjU9SFSZiomj2WI7/oTFLUz8Ef4= Date: Sat, 17 Oct 2020 16:14:31 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 18/40] selftests/vm: minor cleanup: Makefile and gup_test.c Message-ID: <20201017231431.a9u8D_pCE%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: minor cleanup: Makefile and gup_test.c A few cleanups that don't deserve separate patches, but that also should not clutter up other functional changes: 1. Remove an unnecessary #include 2. Restore the sorted order of TEST_GEN_FILES. 3. Add -lpthread to the common LDLIBS, as it is harmless and several tests use it. This gets rid of one special rule already. Link: https://lkml.kernel.org/r/20200929212747.251804-5-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Ralph Campbell Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/Makefile | 10 ++++------ tools/testing/selftests/vm/gup_test.c | 1 - 2 files changed, 4 insertions(+), 7 deletions(-) --- a/tools/testing/selftests/vm/gup_test.c~selftests-vm-minor-cleanup-makefile-and-gup_testc +++ a/tools/testing/selftests/vm/gup_test.c @@ -4,7 +4,6 @@ #include #include #include -#include #include #include #include "../../../../mm/gup_test.h" --- a/tools/testing/selftests/vm/Makefile~selftests-vm-minor-cleanup-makefile-and-gup_testc +++ a/tools/testing/selftests/vm/Makefile @@ -21,14 +21,15 @@ MACHINE ?= $(shell echo $(uname_M) | sed MAKEFLAGS += --no-builtin-rules CFLAGS = -Wall -I ../../../../usr/include $(EXTRA_CFLAGS) -LDLIBS = -lrt +LDLIBS = -lrt -lpthread TEST_GEN_FILES = compaction_test TEST_GEN_FILES += gup_test TEST_GEN_FILES += hmm-tests TEST_GEN_FILES += hugepage-mmap TEST_GEN_FILES += hugepage-shm -TEST_GEN_FILES += map_hugetlb +TEST_GEN_FILES += khugepaged TEST_GEN_FILES += map_fixed_noreplace +TEST_GEN_FILES += map_hugetlb TEST_GEN_FILES += map_populate TEST_GEN_FILES += mlock-random-test TEST_GEN_FILES += mlock2-tests @@ -37,7 +38,6 @@ TEST_GEN_FILES += on-fault-limit TEST_GEN_FILES += thuge-gen TEST_GEN_FILES += transhuge-stress TEST_GEN_FILES += userfaultfd -TEST_GEN_FILES += khugepaged ifeq ($(ARCH),x86_64) CAN_BUILD_I386 := $(shell ./../x86/check_cc.sh $(CC) ../x86/trivial_32bit_program.c -m32) @@ -76,7 +76,7 @@ TEST_FILES := test_vmalloc.sh KSFT_KHDR_INSTALL := 1 include ../lib.mk -$(OUTPUT)/hmm-tests: LDLIBS += -lhugetlbfs -lpthread +$(OUTPUT)/hmm-tests: LDLIBS += -lhugetlbfs ifeq ($(ARCH),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32)) @@ -127,8 +127,6 @@ warn_32bit_failure: endif endif -$(OUTPUT)/userfaultfd: LDLIBS += -lpthread - $(OUTPUT)/mlock-random-test: LDLIBS += -lcap $(OUTPUT)/gup_test: ../../../../mm/gup_test.h From patchwork Sat Oct 17 23:14:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843117 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8155015E6 for ; Sat, 17 Oct 2020 23:14:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3D4F2208B6 for ; Sat, 17 Oct 2020 23:14:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="iPaG6Kt+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D4F2208B6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 52CF36B0088; Sat, 17 Oct 2020 19:14:37 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 507DA6B0089; Sat, 17 Oct 2020 19:14:37 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F4E16B008A; Sat, 17 Oct 2020 19:14:37 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0224.hostedemail.com [216.40.44.224]) by kanga.kvack.org (Postfix) with ESMTP id 139726B0088 for ; Sat, 17 Oct 2020 19:14:37 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B0ABF1EE6 for ; Sat, 17 Oct 2020 23:14:36 +0000 (UTC) X-FDA: 77382973752.13.paint08_0c08db127229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 957E018140B60 for ; Sat, 17 Oct 2020 23:14:36 +0000 (UTC) X-Spam-Summary: 1,0,0,6ab1a574441402cc,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:2:41:152:355:379:560:800:960:967:973:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1535:1593:1594:1605:1730:1747:1777:1792:1801:2198:2199:2393:2525:2559:2563:2682:2685:2731:2859:2892:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4119:4321:4605:5007:6119:6261:6653:7576:7903:7904:8603:9025:9545:10004:11026:11473:11658:11914:12043:12048:12294:12296:12297:12438:12517:12519:12555:12679:12986:13053:13161:13229:13548:13846:21080:21451:21611:21627:21939:21987:21990:30054:30056:30062:30064:30070:30075,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y8qme9qo51ztu4o79k9xbzuc44typ8zc77hxhbo3wqjmzzhd6a41en3faou8k.doh8zbjeut8xnnrycofgqrq5fiipm4exf15wbuuh39cc4td7q8xeo5zdhm1iahm.s-lbl8.mailshell. net-223. X-HE-Tag: paint08_0c08db127229 X-Filterd-Recvd-Size: 8425 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf40.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:36 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 04DA820878; Sat, 17 Oct 2020 23:14:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976475; bh=lmEEKv2mIKNfdNCObR/g2iM92wjOFNdktBDk6OdfgUE=; h=Date:From:To:Subject:In-Reply-To:From; b=iPaG6Kt+lndUFR0+gpDVRP1+vt6RMJ6cU+eoz/cyTuLZDd7aeaskBGYNMy+uZBOXw VCDLQLEoOsPZpEieLD+YbIMUVOidCaTameI4+F5hDLI6hdRsyjGOrpiHaun3ud7BeT rTQ80YumBE+x8FRrfBnVCywUWx9Ub2HozgCcA+KM= Date: Sat, 17 Oct 2020 16:14:34 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 19/40] selftests/vm: only some gup_test items are really benchmarks Message-ID: <20201017231434.DkYlvReNz%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: only some gup_test items are really benchmarks Therefore, some minor cleanup and improvements are in order: 1. Rename the other items appropriately. 2. Stop reporting timing information on the non-benchmark items. It's still being recorded and is available, but there's no point in cluttering up the report with data that no one reasonably needs to check. 3. Don't do iterations, for non-benchmark items. 4. Print out a shorter, more appropriate report for the non-benchmark tests. 5. Add the command that was run, to the report. This really helps, as there are quite a lot of options now. Link: https://lkml.kernel.org/r/20200929212747.251804-6-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Ralph Campbell Cc: Shuah Khan Signed-off-by: Andrew Morton --- Documentation/core-api/pin_user_pages.rst | 2 mm/gup_test.c | 14 ++--- mm/gup_test.h | 8 +-- tools/testing/selftests/vm/gup_test.c | 47 ++++++++++++++++---- 4 files changed, 51 insertions(+), 20 deletions(-) --- a/Documentation/core-api/pin_user_pages.rst~selftests-vm-only-some-gup_test-items-are-really-benchmarks +++ a/Documentation/core-api/pin_user_pages.rst @@ -226,7 +226,7 @@ This file:: has the following new calls to exercise the new pin*() wrapper functions: * PIN_FAST_BENCHMARK (./gup_test -a) -* PIN_BENCHMARK (./gup_test -b) +* PIN_BASIC_TEST (./gup_test -b) You can monitor how many total dma-pinned pages have been acquired and released since the system was booted, via two new /proc/vmstat entries: :: --- a/mm/gup_test.c~selftests-vm-only-some-gup_test-items-are-really-benchmarks +++ a/mm/gup_test.c @@ -13,13 +13,13 @@ static void put_back_pages(unsigned int switch (cmd) { case GUP_FAST_BENCHMARK: - case GUP_BENCHMARK: + case GUP_BASIC_TEST: for (i = 0; i < nr_pages; i++) put_page(pages[i]); break; case PIN_FAST_BENCHMARK: - case PIN_BENCHMARK: + case PIN_BASIC_TEST: case PIN_LONGTERM_BENCHMARK: unpin_user_pages(pages, nr_pages); break; @@ -34,7 +34,7 @@ static void verify_dma_pinned(unsigned i switch (cmd) { case PIN_FAST_BENCHMARK: - case PIN_BENCHMARK: + case PIN_BASIC_TEST: case PIN_LONGTERM_BENCHMARK: for (i = 0; i < nr_pages; i++) { page = pages[i]; @@ -94,7 +94,7 @@ static int __gup_test_ioctl(unsigned int nr = get_user_pages_fast(addr, nr, gup->flags, pages + i); break; - case GUP_BENCHMARK: + case GUP_BASIC_TEST: nr = get_user_pages(addr, nr, gup->flags, pages + i, NULL); break; @@ -102,7 +102,7 @@ static int __gup_test_ioctl(unsigned int nr = pin_user_pages_fast(addr, nr, gup->flags, pages + i); break; - case PIN_BENCHMARK: + case PIN_BASIC_TEST: nr = pin_user_pages(addr, nr, gup->flags, pages + i, NULL); break; @@ -157,10 +157,10 @@ static long gup_test_ioctl(struct file * switch (cmd) { case GUP_FAST_BENCHMARK: - case GUP_BENCHMARK: case PIN_FAST_BENCHMARK: - case PIN_BENCHMARK: case PIN_LONGTERM_BENCHMARK: + case GUP_BASIC_TEST: + case PIN_BASIC_TEST: break; default: return -EINVAL; --- a/mm/gup_test.h~selftests-vm-only-some-gup_test-items-are-really-benchmarks +++ a/mm/gup_test.h @@ -5,10 +5,10 @@ #include #define GUP_FAST_BENCHMARK _IOWR('g', 1, struct gup_test) -#define GUP_BENCHMARK _IOWR('g', 2, struct gup_test) -#define PIN_FAST_BENCHMARK _IOWR('g', 3, struct gup_test) -#define PIN_BENCHMARK _IOWR('g', 4, struct gup_test) -#define PIN_LONGTERM_BENCHMARK _IOWR('g', 5, struct gup_test) +#define PIN_FAST_BENCHMARK _IOWR('g', 2, struct gup_test) +#define PIN_LONGTERM_BENCHMARK _IOWR('g', 3, struct gup_test) +#define GUP_BASIC_TEST _IOWR('g', 4, struct gup_test) +#define PIN_BASIC_TEST _IOWR('g', 5, struct gup_test) struct gup_test { __u64 get_delta_usec; --- a/tools/testing/selftests/vm/gup_test.c~selftests-vm-only-some-gup_test-items-are-really-benchmarks +++ a/tools/testing/selftests/vm/gup_test.c @@ -14,12 +14,30 @@ /* Just the flags we need, copied from mm.h: */ #define FOLL_WRITE 0x01 /* check pte is writable */ +static char *cmd_to_str(unsigned long cmd) +{ + switch (cmd) { + case GUP_FAST_BENCHMARK: + return "GUP_FAST_BENCHMARK"; + case PIN_FAST_BENCHMARK: + return "PIN_FAST_BENCHMARK"; + case PIN_LONGTERM_BENCHMARK: + return "PIN_LONGTERM_BENCHMARK"; + case GUP_BASIC_TEST: + return "GUP_BASIC_TEST"; + case PIN_BASIC_TEST: + return "PIN_BASIC_TEST"; + } + return "Unknown command"; +} + int main(int argc, char **argv) { struct gup_benchmark gup; unsigned long size = 128 * MB; int i, fd, filed, opt, nr_pages = 1, thp = -1, repeats = 1, write = 0; - int cmd = GUP_FAST_BENCHMARK, flags = MAP_PRIVATE; + int cmd = GUP_FAST_BENCHMARK; + int flags = MAP_PRIVATE; char *file = "/dev/zero"; char *p; @@ -29,7 +47,7 @@ int main(int argc, char **argv) cmd = PIN_FAST_BENCHMARK; break; case 'b': - cmd = PIN_BENCHMARK; + cmd = PIN_BASIC_TEST; break; case 'L': cmd = PIN_LONGTERM_BENCHMARK; @@ -50,7 +68,7 @@ int main(int argc, char **argv) thp = 0; break; case 'U': - cmd = GUP_BENCHMARK; + cmd = GUP_BASIC_TEST; break; case 'u': cmd = GUP_FAST_BENCHMARK; @@ -104,18 +122,31 @@ int main(int argc, char **argv) for (; (unsigned long)p < gup.addr + size; p += PAGE_SIZE) p[0] = 0; - for (i = 0; i < repeats; i++) { + /* Only report timing information on the *_BENCHMARK commands: */ + if ((cmd == PIN_FAST_BENCHMARK) || (cmd == GUP_FAST_BENCHMARK) || + (cmd == PIN_LONGTERM_BENCHMARK)) { + for (i = 0; i < repeats; i++) { + gup.size = size; + if (ioctl(fd, cmd, &gup)) + perror("ioctl"), exit(1); + + printf("%s: Time: get:%lld put:%lld us", + cmd_to_str(cmd), gup.get_delta_usec, + gup.put_delta_usec); + if (gup.size != size) + printf(", truncated (size: %lld)", gup.size); + printf("\n"); + } + } else { gup.size = size; if (ioctl(fd, cmd, &gup)) { perror("ioctl"); exit(1); } - printf("Time: get:%lld put:%lld us", gup.get_delta_usec, - gup.put_delta_usec); + printf("%s: done\n", cmd_to_str(cmd)); if (gup.size != size) - printf(", truncated (size: %lld)", gup.size); - printf("\n"); + printf("Truncated (size: %lld)\n", gup.size); } return 0; From patchwork Sat Oct 17 23:14:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843119 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BBE6F14B5 for ; Sat, 17 Oct 2020 23:14:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6F83220878 for ; Sat, 17 Oct 2020 23:14:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="MTZ5rkCt" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6F83220878 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7F78F6B0089; Sat, 17 Oct 2020 19:14:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7C2896B008A; Sat, 17 Oct 2020 19:14:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D94F6B008C; Sat, 17 Oct 2020 19:14:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0052.hostedemail.com [216.40.44.52]) by kanga.kvack.org (Postfix) with ESMTP id 3FD0B6B0089 for ; Sat, 17 Oct 2020 19:14:40 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id DE061362D for ; Sat, 17 Oct 2020 23:14:39 +0000 (UTC) X-FDA: 77382973878.15.train80_410219827229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id BDF381814B0C1 for ; Sat, 17 Oct 2020 23:14:39 +0000 (UTC) X-Spam-Summary: 1,0,0,10bb911550187c22,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:1:2:41:146:152:355:379:421:800:960:967:973:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1593:1594:1605:1730:1747:1777:1792:1801:2198:2199:2393:2525:2553:2559:2563:2682:2685:2731:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4052:4250:4321:4605:5007:6119:6261:6653:7576:7619:7875:7903:8603:8660:9025:9121:9545:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12895:12986:13141:13148:13161:13229:13230:13846:14093:14097:21080:21433:21451:21627:21795:21796:21939:21972:21990:30034:30036:30045:30051:30054:30056:30062:30064:30070:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04y8wpi1t61s1cnn1txutrepwp8poycmsagyxzrdfzmq1figqhdeac9otsoj7nt.dbrot5bkw8j 1nyzfcz9 X-HE-Tag: train80_410219827229 X-Filterd-Recvd-Size: 10697 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:39 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 31D5B20897; Sat, 17 Oct 2020 23:14:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976478; bh=AW72XPM2fSWkbvxbCsWzLWupH6l+30Cx5v3t9yyTjhU=; h=Date:From:To:Subject:In-Reply-To:From; b=MTZ5rkCtC13jPMqPsQ/Tsxg6E6FBv4d5M8of+THexKWOkwrPb64C1ddtgL5HOz5zc 7RLTRw6XQ+N6epFrOGFSyoBViuYS/zc21KIpcBtEEn9sfD3xd2TdJDZo0JQL9zDn8g LA8Jbeu6pBjIZ5xnOnEAB2RAkK3JSULssZbNifKA= Date: Sat, 17 Oct 2020 16:14:37 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 20/40] selftests/vm: gup_test: introduce the dump_pages() sub-test Message-ID: <20201017231437.0AtoOa8Jf%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: gup_test: introduce the dump_pages() sub-test For quite a while, I was doing a quick hack to gup_test.c (previously, gup_benchmark.c) whenever I wanted to try out my changes to dump_page(). This makes that hack unnecessary, and instead allows anyone to easily get the same coverage from a user space program. That saves a lot of time because you don't have to change the kernel, in order to test different pages and options. The new sub-test takes advantage of the existing gup_test infrastructure, which already provides a simple user space program, some allocated user space pages, an ioctl call, pinning of those pages (via either get_user_pages or pin_user_pages) and a corresponding kernel-side test invocation. There's not much more required, mainly just a couple of inputs from the user. In fact, the new test re-uses the existing command line options in order to get various helpful combinations (THP or normal, _fast or slow gup, gup vs. pup, and more). New command line options are: which pages to dump, and what type of "get/pin" to use. In order to figure out which pages to dump, the logic is: * If the user doesn't specify anything, the page 0 (the first page in the address range that the program sets up for testing) is dumped. * Or, the user can type up to 8 page indices anywhere on the command line. If you type more than 8, then it uses the first 8 and ignores the remaining items. For example: ./gup_test -ct -F 1 0 19 0x1000 Meaning: -c: dump pages sub-test -t: use THP pages -F 1: use pin_user_pages() instead of get_user_pages() 0 19 0x1000: dump pages 0, 19, and 4096 Link: https://lkml.kernel.org/r/20200929212747.251804-7-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Ralph Campbell Cc: Shuah Khan Signed-off-by: Andrew Morton --- mm/Kconfig | 6 ++ mm/gup_test.c | 54 +++++++++++++++++++++++- mm/gup_test.h | 10 ++++ tools/testing/selftests/vm/gup_test.c | 47 +++++++++++++++++++- 4 files changed, 112 insertions(+), 5 deletions(-) --- a/mm/gup_test.c~selftests-vm-gup_test-introduce-the-dump_pages-sub-test +++ a/mm/gup_test.c @@ -7,7 +7,7 @@ #include "../../../../mm/gup_test.h" static void put_back_pages(unsigned int cmd, struct page **pages, - unsigned long nr_pages) + unsigned long nr_pages, unsigned int gup_test_flags) { unsigned long i; @@ -23,6 +23,15 @@ static void put_back_pages(unsigned int case PIN_LONGTERM_BENCHMARK: unpin_user_pages(pages, nr_pages); break; + case DUMP_USER_PAGES_TEST: + if (gup_test_flags & GUP_TEST_FLAG_DUMP_PAGES_USE_PIN) { + unpin_user_pages(pages, nr_pages); + } else { + for (i = 0; i < nr_pages; i++) + put_page(pages[i]); + + } + break; } } @@ -49,6 +58,37 @@ static void verify_dma_pinned(unsigned i } } +static void dump_pages_test(struct gup_test *gup, struct page **pages, + unsigned long nr_pages) +{ + unsigned int index_to_dump; + unsigned int i; + + /* + * Zero out any user-supplied page index that is out of range. Remember: + * .which_pages[] contains a 1-based set of page indices. + */ + for (i = 0; i < GUP_TEST_MAX_PAGES_TO_DUMP; i++) { + if (gup->which_pages[i] > nr_pages) { + pr_warn("ZEROING due to out of range: .which_pages[%u]: %u\n", + i, gup->which_pages[i]); + gup->which_pages[i] = 0; + } + } + + for (i = 0; i < GUP_TEST_MAX_PAGES_TO_DUMP; i++) { + index_to_dump = gup->which_pages[i]; + + if (index_to_dump) { + index_to_dump--; // Decode from 1-based, to 0-based + pr_info("---- page #%u, starting from user virt addr: 0x%llx\n", + index_to_dump, gup->addr); + dump_page(pages[index_to_dump], + "gup_test: dump_pages() test"); + } + } +} + static int __gup_test_ioctl(unsigned int cmd, struct gup_test *gup) { @@ -111,6 +151,14 @@ static int __gup_test_ioctl(unsigned int gup->flags | FOLL_LONGTERM, pages + i, NULL); break; + case DUMP_USER_PAGES_TEST: + if (gup->flags & GUP_TEST_FLAG_DUMP_PAGES_USE_PIN) + nr = pin_user_pages(addr, nr, gup->flags, + pages + i, NULL); + else + nr = get_user_pages(addr, nr, gup->flags, + pages + i, NULL); + break; default: ret = -EINVAL; goto unlock; @@ -133,10 +181,11 @@ static int __gup_test_ioctl(unsigned int * state: print a warning if any non-dma-pinned pages are found: */ verify_dma_pinned(cmd, pages, nr_pages); + dump_pages_test(gup, pages, nr_pages); start_time = ktime_get(); - put_back_pages(cmd, pages, nr_pages); + put_back_pages(cmd, pages, nr_pages, gup->flags); end_time = ktime_get(); gup->put_delta_usec = ktime_us_delta(end_time, start_time); @@ -161,6 +210,7 @@ static long gup_test_ioctl(struct file * case PIN_LONGTERM_BENCHMARK: case GUP_BASIC_TEST: case PIN_BASIC_TEST: + case DUMP_USER_PAGES_TEST: break; default: return -EINVAL; --- a/mm/gup_test.h~selftests-vm-gup_test-introduce-the-dump_pages-sub-test +++ a/mm/gup_test.h @@ -9,6 +9,11 @@ #define PIN_LONGTERM_BENCHMARK _IOWR('g', 3, struct gup_test) #define GUP_BASIC_TEST _IOWR('g', 4, struct gup_test) #define PIN_BASIC_TEST _IOWR('g', 5, struct gup_test) +#define DUMP_USER_PAGES_TEST _IOWR('g', 6, struct gup_test) + +#define GUP_TEST_MAX_PAGES_TO_DUMP 8 + +#define GUP_TEST_FLAG_DUMP_PAGES_USE_PIN 0x1 struct gup_test { __u64 get_delta_usec; @@ -17,6 +22,11 @@ struct gup_test { __u64 size; __u32 nr_pages_per_call; __u32 flags; + /* + * Each non-zero entry is the number of the page (1-based: first page is + * page 1, so that zero entries mean "do nothing") from the .addr base. + */ + __u32 which_pages[GUP_TEST_MAX_PAGES_TO_DUMP]; }; #endif /* __GUP_TEST_H */ --- a/mm/Kconfig~selftests-vm-gup_test-introduce-the-dump_pages-sub-test +++ a/mm/Kconfig @@ -842,6 +842,12 @@ config GUP_TEST get_user_pages*() and pin_user_pages*(), as well as smoke tests of the non-_fast variants. + There is also a sub-test that allows running dump_page() on any + of up to eight pages (selected by command line args) within the + range of user-space addresses. These pages are either pinned via + pin_user_pages*(), or pinned via get_user_pages*(), as specified + by other command line arguments. + See tools/testing/selftests/vm/gup_test.c config GUP_GET_PTE_LOW_HIGH --- a/tools/testing/selftests/vm/gup_test.c~selftests-vm-gup_test-introduce-the-dump_pages-sub-test +++ a/tools/testing/selftests/vm/gup_test.c @@ -27,21 +27,23 @@ static char *cmd_to_str(unsigned long cm return "GUP_BASIC_TEST"; case PIN_BASIC_TEST: return "PIN_BASIC_TEST"; + case DUMP_USER_PAGES_TEST: + return "DUMP_USER_PAGES_TEST"; } return "Unknown command"; } int main(int argc, char **argv) { - struct gup_benchmark gup; + struct gup_test gup = { 0 }; unsigned long size = 128 * MB; int i, fd, filed, opt, nr_pages = 1, thp = -1, repeats = 1, write = 0; - int cmd = GUP_FAST_BENCHMARK; + unsigned long cmd = GUP_FAST_BENCHMARK; int flags = MAP_PRIVATE; char *file = "/dev/zero"; char *p; - while ((opt = getopt(argc, argv, "m:r:n:f:abtTLUuwSH")) != -1) { + while ((opt = getopt(argc, argv, "m:r:n:F:f:abctTLUuwSH")) != -1) { switch (opt) { case 'a': cmd = PIN_FAST_BENCHMARK; @@ -52,6 +54,21 @@ int main(int argc, char **argv) case 'L': cmd = PIN_LONGTERM_BENCHMARK; break; + case 'c': + cmd = DUMP_USER_PAGES_TEST; + /* + * Dump page 0 (index 1). May be overridden later, by + * user's non-option arguments. + * + * .which_pages is zero-based, so that zero can mean "do + * nothing". + */ + gup.which_pages[0] = 1; + break; + case 'F': + /* strtol, so you can pass flags in hex form */ + gup.flags = strtol(optarg, 0, 0); + break; case 'm': size = atoi(optarg) * MB; break; @@ -91,6 +108,30 @@ int main(int argc, char **argv) } } + if (optind < argc) { + int extra_arg_count = 0; + /* + * For example: + * + * ./gup_test -c 0 1 0x1001 + * + * ...to dump pages 0, 1, and 4097 + */ + + while ((optind < argc) && + (extra_arg_count < GUP_TEST_MAX_PAGES_TO_DUMP)) { + /* + * Do the 1-based indexing here, so that the user can + * use normal 0-based indexing on the command line. + */ + long page_index = strtol(argv[optind], 0, 0) + 1; + + gup.which_pages[extra_arg_count] = page_index; + extra_arg_count++; + optind++; + } + } + filed = open(file, O_RDWR|O_CREAT); if (filed < 0) { perror("open"); From patchwork Sat Oct 17 23:14:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843121 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 171C514B5 for ; Sat, 17 Oct 2020 23:14:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C978720878 for ; Sat, 17 Oct 2020 23:14:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="yIQhyZ80" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C978720878 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E7A176B0068; Sat, 17 Oct 2020 19:14:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E2A706B008A; Sat, 17 Oct 2020 19:14:43 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D68466B008C; Sat, 17 Oct 2020 19:14:43 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id AA1C46B0068 for ; Sat, 17 Oct 2020 19:14:43 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 50F2C8249980 for ; Sat, 17 Oct 2020 23:14:43 +0000 (UTC) X-FDA: 77382974046.27.net89_2509d7627229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id 267C63D663 for ; Sat, 17 Oct 2020 23:14:43 +0000 (UTC) X-Spam-Summary: 1,0,0,80caf6ac2fd6f8e0,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:152:355:379:800:960:967:973:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1534:1542:1593:1594:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2895:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3865:3866:3867:3868:3870:3872:3873:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:8660:9025:9545:10004:10400:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12683:12895:12986:13148:13230:13255:13846:14093:14097:14181:14721:21080:21451:21627:21796:21939:30036:30054:30056:30062:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yfnuu6chsa3ffx5r8ors7yqiukcycfca8r5xnk6ax17844krfyemr5eneo6du.nbyn7ffaf73mphoqksg38w9pjcnbzkkw9gj4th7fuj7s68qgjehd1dgrnj7g5d9.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0 .5,0.5,N X-HE-Tag: net89_2509d7627229 X-Filterd-Recvd-Size: 3784 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:42 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4AE08208B6; Sat, 17 Oct 2020 23:14:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976481; bh=eUbd2W90+RDXkzNB16/AG7mgOK8GmWj7wUP2cWH9jFE=; h=Date:From:To:Subject:In-Reply-To:From; b=yIQhyZ803u9buW9NwiwW76XSboLapA5nGZ1foW8UU9OFt9+YyWnwVG2Q8c18YJYgo foeAVglJ5fW/lbOgcQP2R7cVb36PICC6i/cng0m2IPhmW6kMGCLlj8TQXO8mZgZSsM z6SLf8m9jtHc+0VWfJsV4M4I7t2GNl/fpMDlTryM= Date: Sat, 17 Oct 2020 16:14:40 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 21/40] selftests/vm: run_vmtests.sh: update and clean up gup_test invocation Message-ID: <20201017231440.9OlmYk2Ol%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: run_vmtests.sh: update and clean up gup_test invocation Run benchmarks on the _fast variants of gup and pup, as originally intended. Run the new gup_test sub-test: dump pages. In addition to exercising the dump_page() call, it also demonstrates the various options you can use to specify which pages to dump, and how. Link: https://lkml.kernel.org/r/20200929212747.251804-8-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Ralph Campbell Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/run_vmtests.sh | 24 +++++++++++++++----- 1 file changed, 18 insertions(+), 6 deletions(-) --- a/tools/testing/selftests/vm/run_vmtests.sh~selftests-vm-run_vmtestsh-update-and-clean-up-gup_test-invocation +++ a/tools/testing/selftests/vm/run_vmtests.sh @@ -124,9 +124,9 @@ else fi echo "--------------------------------------------" -echo "running 'gup_test -U' (normal/slow gup)" +echo "running 'gup_test -u' (fast gup benchmark)" echo "--------------------------------------------" -./gup_test -U +./gup_test -u if [ $? -ne 0 ]; then echo "[FAIL]" exitcode=1 @@ -134,10 +134,22 @@ else echo "[PASS]" fi -echo "------------------------------------------" -echo "running gup_test -b (pin_user_pages)" -echo "------------------------------------------" -./gup_test -b +echo "---------------------------------------------------" +echo "running gup_test -a (pin_user_pages_fast benchmark)" +echo "---------------------------------------------------" +./gup_test -a +if [ $? -ne 0 ]; then + echo "[FAIL]" + exitcode=1 +else + echo "[PASS]" +fi + +echo "--------------------------------------------------------------" +echo "running gup_test -ct -F 0x1 0 19 0x1000" +echo " Dumps pages 0, 19, and 4096, using pin_user_pages (-F 0x1)" +echo "--------------------------------------------------------------" +./gup_test -ct -F 0x1 0 19 0x1000 if [ $? -ne 0 ]; then echo "[FAIL]" exitcode=1 From patchwork Sat Oct 17 23:14:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843123 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DFEB514B5 for ; Sat, 17 Oct 2020 23:14:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 998D020897 for ; Sat, 17 Oct 2020 23:14:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="lKoCTug5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 998D020897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A340A6B006E; Sat, 17 Oct 2020 19:14:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9BE6B6B008A; Sat, 17 Oct 2020 19:14:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FAC66B0092; Sat, 17 Oct 2020 19:14:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220]) by kanga.kvack.org (Postfix) with ESMTP id 659B36B008A for ; Sat, 17 Oct 2020 19:14:46 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0EB1A1EE6 for ; Sat, 17 Oct 2020 23:14:46 +0000 (UTC) X-FDA: 77382974172.03.book30_5c1659727229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id E778E28A4E8 for ; Sat, 17 Oct 2020 23:14:45 +0000 (UTC) X-Spam-Summary: 1,0,0,0c0d112f5403bf53,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:2:41:152:355:379:800:960:966:967:968:973:982:988:989:1260:1277:1311:1313:1314:1345:1359:1381:1431:1437:1513:1515:1516:1518:1521:1535:1593:1594:1605:1730:1747:1777:1792:1801:2196:2197:2198:2199:2200:2201:2393:2525:2553:2559:2564:2682:2685:2693:2731:2736:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3653:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4119:4250:4321:4385:4605:5007:6119:6261:6653:7576:7903:7974:9008:9025:9036:9545:9592:10004:11026:11473:11658:11914:12043:12048:12050:12296:12297:12438:12517:12519:12555:12679:12986:13161:13229:13846:21080:21221:21324:21325:21451:21627:21740:21939:21990:30054:30056:30064:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yr37pzrtj9wex6i888g19dwbfofocbyd6f4h4u6pn9gm3j33qf1fcfew11iz1.d7nxhwdm6izicsed8n8cpj48jkiyeh4o4kp ej3idm9m X-HE-Tag: book30_5c1659727229 X-Filterd-Recvd-Size: 8254 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:45 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 797CB208B6; Sat, 17 Oct 2020 23:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976484; bh=WSTkUoZ5EXpZv5AqZ6/MWVOMH286gRrio6fQGTQD4wM=; h=Date:From:To:Subject:In-Reply-To:From; b=lKoCTug5YV1YhXBoQnXCeqjrUEHYSHPyeJrV8aPQAps98fDSB65KTzH821QwdKYS+ o8NwLPGkPI1trXdjujvvW5GAyeDrpaa17hCmlh70MM+iCwalR7VnqFXUMz1UgGrbm3 VpzbU5uY23Le2OdwG6j8vKY9gVYK/N4iDEZAkgdU= Date: Sat, 17 Oct 2020 16:14:44 -0700 From: Andrew Morton To: akpm@linux-foundation.org, corbet@lwn.net, jglisse@redhat.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, torvalds@linux-foundation.org Subject: [patch 22/40] selftests/vm: hmm-tests: remove the libhugetlbfs dependency Message-ID: <20201017231444.avmsZaNbp%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: hmm-tests: remove the libhugetlbfs dependency HMM selftests are incredibly useful, but they are only effective if people actually build and run them. All the other tests in selftests/vm can be built with very standard, always-available libraries: libpthread, librt. The hmm-tests.c program, on the other hand, requires something that is (much) less readily available: libhugetlbfs. And so the build will typically fail for many developers. A simple attempt to install libhugetlbfs will also run into complications on some common distros these days: Fedora and Arch Linux (yes, Arch AUR has it, but that's fragile, as always with AUR). The library is not maintained actively enough at the moment, for distros to deal with it. I had to build it from source, for Fedora, and that didn't go too smoothly either. It turns out that, out of 21 tests in hmm-tests.c, only 2 actually require functionality from libhugetlbfs. Therefore, if libhugetlbfs is missing, simply ifdef those two tests out and allow the developer to at least have the other 19 tests, if they don't want to pause to work through the above issues. Also issue a warning, so that it's clear that there is an imperfection in the build. In order to do that, a tiny shell script (check_config.sh) runs a quick compile (not link, that's too prone to false failures with library paths), and basically, if the compiler doesn't find hugetlbfs.h in its standard locations, then the script concludes that libhugetlbfs is not available. The output is in two files, one for inclusion in hmm-test.c (local_config.h), and one for inclusion in the Makefile (local_config.mk). [jhubbard@nvidia.com: fix an improper dependency upon executable script permissions] Link: https://lkml.kernel.org/r/20201003002142.32671-2-jhubbard@nvidia.com Link: https://lkml.kernel.org/r/20200929212747.251804-9-jhubbard@nvidia.com Signed-off-by: John Hubbard Cc: Ralph Campbell Cc: Jérôme Glisse Cc: Jonathan Corbet Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/.gitignore | 1 tools/testing/selftests/vm/Makefile | 24 +++++++++++++- tools/testing/selftests/vm/check_config.sh | 31 +++++++++++++++++++ tools/testing/selftests/vm/hmm-tests.c | 10 +++++- 4 files changed, 63 insertions(+), 3 deletions(-) --- /dev/null +++ a/tools/testing/selftests/vm/check_config.sh @@ -0,0 +1,31 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# Probe for libraries and create header files to record the results. Both C +# header files and Makefile include fragments are created. + +OUTPUT_H_FILE=local_config.h +OUTPUT_MKFILE=local_config.mk + +# libhugetlbfs +tmpname=$(mktemp) +tmpfile_c=${tmpname}.c +tmpfile_o=${tmpname}.o + +echo "#include " > $tmpfile_c +echo "#include " >> $tmpfile_c +echo "int func(void) { return 0; }" >> $tmpfile_c + +CC=${1:?"Usage: $0 # example compiler: gcc"} +$CC -c $tmpfile_c -o $tmpfile_o >/dev/null 2>&1 + +if [ -f $tmpfile_o ]; then + echo "#define LOCAL_CONFIG_HAVE_LIBHUGETLBFS 1" > $OUTPUT_H_FILE + echo "HMM_EXTRA_LIBS = -lhugetlbfs" > $OUTPUT_MKFILE +else + echo "// No libhugetlbfs support found" > $OUTPUT_H_FILE + echo "# No libhugetlbfs support found, so:" > $OUTPUT_MKFILE + echo "HMM_EXTRA_LIBS = " >> $OUTPUT_MKFILE +fi + +rm ${tmpname}.* --- a/tools/testing/selftests/vm/.gitignore~selftests-vm-hmm-tests-remove-the-libhugetlbfs-dependency +++ a/tools/testing/selftests/vm/.gitignore @@ -20,3 +20,4 @@ va_128TBswitch map_fixed_noreplace write_to_hugetlbfs hmm-tests +local_config.* --- a/tools/testing/selftests/vm/hmm-tests.c~selftests-vm-hmm-tests-remove-the-libhugetlbfs-dependency +++ a/tools/testing/selftests/vm/hmm-tests.c @@ -21,12 +21,16 @@ #include #include #include -#include #include #include #include #include +#include "./local_config.h" +#ifdef LOCAL_CONFIG_HAVE_LIBHUGETLBFS +#include +#endif + /* * This is a private UAPI to the kernel test module so it isn't exported * in the usual include/uapi/... directory. @@ -662,6 +666,7 @@ TEST_F(hmm, anon_write_huge) hmm_buffer_free(buffer); } +#ifdef LOCAL_CONFIG_HAVE_LIBHUGETLBFS /* * Write huge TLBFS page. */ @@ -720,6 +725,7 @@ TEST_F(hmm, anon_write_hugetlbfs) buffer->ptr = NULL; hmm_buffer_free(buffer); } +#endif /* LOCAL_CONFIG_HAVE_LIBHUGETLBFS */ /* * Read mmap'ed file memory. @@ -1336,6 +1342,7 @@ TEST_F(hmm2, snapshot) hmm_buffer_free(buffer); } +#ifdef LOCAL_CONFIG_HAVE_LIBHUGETLBFS /* * Test the hmm_range_fault() HMM_PFN_PMD flag for large pages that * should be mapped by a large page table entry. @@ -1411,6 +1418,7 @@ TEST_F(hmm, compound) buffer->ptr = NULL; hmm_buffer_free(buffer); } +#endif /* LOCAL_CONFIG_HAVE_LIBHUGETLBFS */ /* * Test two devices reading the same memory (double mapped). --- a/tools/testing/selftests/vm/Makefile~selftests-vm-hmm-tests-remove-the-libhugetlbfs-dependency +++ a/tools/testing/selftests/vm/Makefile @@ -1,5 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 # Makefile for vm selftests + +include local_config.mk + uname_M := $(shell uname -m 2>/dev/null || echo not) MACHINE ?= $(shell echo $(uname_M) | sed -e 's/aarch64.*/arm64/') @@ -76,8 +79,6 @@ TEST_FILES := test_vmalloc.sh KSFT_KHDR_INSTALL := 1 include ../lib.mk -$(OUTPUT)/hmm-tests: LDLIBS += -lhugetlbfs - ifeq ($(ARCH),x86_64) BINARIES_32 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_32)) BINARIES_64 := $(patsubst %,$(OUTPUT)/%,$(BINARIES_64)) @@ -130,3 +131,22 @@ endif $(OUTPUT)/mlock-random-test: LDLIBS += -lcap $(OUTPUT)/gup_test: ../../../../mm/gup_test.h + +$(OUTPUT)/hmm-tests: local_config.h + +# HMM_EXTRA_LIBS may get set in local_config.mk, or it may be left empty. +$(OUTPUT)/hmm-tests: LDLIBS += $(HMM_EXTRA_LIBS) + +local_config.mk local_config.h: check_config.sh + /bin/sh ./check_config.sh $(CC) + +EXTRA_CLEAN += local_config.mk local_config.h + +ifeq ($(HMM_EXTRA_LIBS),) +all: warn_missing_hugelibs + +warn_missing_hugelibs: + @echo ; \ + echo "Warning: missing libhugetlbfs support. Some HMM tests will be skipped." ; \ + echo +endif From patchwork Sat Oct 17 23:14:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843125 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D0F8814B5 for ; Sat, 17 Oct 2020 23:14:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 97C7C20897 for ; Sat, 17 Oct 2020 23:14:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EmRDwEgZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 97C7C20897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A83986B008A; Sat, 17 Oct 2020 19:14:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A337E6B008C; Sat, 17 Oct 2020 19:14:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D54B6B0092; Sat, 17 Oct 2020 19:14:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 6068D6B008A for ; Sat, 17 Oct 2020 19:14:49 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0B22B181AEF07 for ; Sat, 17 Oct 2020 23:14:49 +0000 (UTC) X-FDA: 77382974298.02.knot52_07048ea27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id DBCEB101E2D3A for ; Sat, 17 Oct 2020 23:14:48 +0000 (UTC) X-Spam-Summary: 1,0,0,973b3978f4e8c113,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3352:3834:3865:3866:3867:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4560:5007:6261:6653:7514:7576:8957:9025:9545:10004:11026:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12986:13069:13311:13357:13846:14181:14384:14721:21080:21451:21627:21939:30034:30054:30056:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrgtge5wff4dye8ocr4y854n67doc78qiz79ba1e5s8xx9r6edtr1tzkwy87w.yy7n1q1ajef1hpcwanq8g97hf8skn83u5xy3p7ghsfnf313jdmnz3h6hmfagz9r.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,C ustom_ru X-HE-Tag: knot52_07048ea27229 X-Filterd-Recvd-Size: 2712 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:48 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8CBA3212CC; Sat, 17 Oct 2020 23:14:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976487; bh=25RfhTqr2Jfw6P3r6zDykxyPWbiJknJ7qwh27f3yQic=; h=Date:From:To:Subject:In-Reply-To:From; b=EmRDwEgZvAUwkCRkYXY55983hD7TvnkRXlNWpLbwzSl4v+Le3P6he8EmejHQiJAs2 nyGFbRwmRaLQwIGd+Zo8njT+KV5i1Befpyzw6oxT5coCTexTJ/pXddxDOvxBjlSWvx Y+C6HhiEgjQGyFDgjAN8cjYK997CjwJjtJrLu7ok= Date: Sat, 17 Oct 2020 16:14:47 -0700 From: Andrew Morton To: akpm@linux-foundation.org, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, shuah@kernel.org, sj38.park@gmail.com, torvalds@linux-foundation.org Subject: [patch 23/40] selftests/vm: 10x speedup for hmm-tests Message-ID: <20201017231447.uw23bzJ9_%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: John Hubbard Subject: selftests/vm: 10x speedup for hmm-tests This patch reduces the running time for hmm-tests from about 10+ seconds, to just under 1.0 second, for an approximately 10x speedup. That brings it in line with most of the other tests in selftests/vm, which mostly run in < 1 sec. This is done with a one-line change that simply reduces the number of iterations of several tests, from 256, to 10. Thanks to Ralph Campbell for suggesting changing NTIMES as a way to get the speedup. Link: https://lkml.kernel.org/r/20201003011721.44238-1-jhubbard@nvidia.com Signed-off-by: John Hubbard Suggested-by: Ralph Campbell Cc: SeongJae Park Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/hmm-tests.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/tools/testing/selftests/vm/hmm-tests.c~selftests-vm-10x-speedup-for-hmm-tests +++ a/tools/testing/selftests/vm/hmm-tests.c @@ -49,7 +49,7 @@ struct hmm_buffer { #define TWOMEG (1 << 21) #define HMM_BUFFER_SIZE (1024 << 12) #define HMM_PATH_MAX 64 -#define NTIMES 256 +#define NTIMES 10 #define ALIGN(x, a) (((x) + (a - 1)) & (~((a) - 1))) From patchwork Sat Oct 17 23:14:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843127 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 335EA15E6 for ; Sat, 17 Oct 2020 23:14:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E004D20897 for ; Sat, 17 Oct 2020 23:14:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="yk7jXghK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E004D20897 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0BF596B0071; Sat, 17 Oct 2020 19:14:54 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 07DA86B008C; Sat, 17 Oct 2020 19:14:54 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E79EA6B0092; Sat, 17 Oct 2020 19:14:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id BBC816B0071 for ; Sat, 17 Oct 2020 19:14:53 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 64B588249980 for ; Sat, 17 Oct 2020 23:14:53 +0000 (UTC) X-FDA: 77382974466.11.trees41_4912a4d27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin11.hostedemail.com (Postfix) with ESMTP id 41D0C180F8B81 for ; Sat, 17 Oct 2020 23:14:53 +0000 (UTC) X-Spam-Summary: 1,0,0,05290795ad7ca175,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:1:2:41:69:355:379:800:960:966:967:968:973:982:988:989:1260:1263:1345:1359:1381:1431:1437:1605:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2376:2393:2525:2559:2568:2627:2682:2685:2693:2859:2892:2901:2902:2911:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4051:4250:4321:4385:4425:4605:5007:6261:6653:6737:6738:7514:7576:7875:7901:7903:8599:8603:8660:8957:9025:9121:9545:10004:10913:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12783:12986:13053:13141:13148:13191:13192:13229:13230:13846:14096:21080:21094:21323:21324:21433:21451:21627:21796:21939:21987:21990:30003:30036:30054:30064:30070:30075,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yryn5yg8ab9dgeu8qqm441s4ibnypo3idcpqjnda417iwtm583ff87 jz1kd85. X-HE-Tag: trees41_4912a4d27229 X-Filterd-Recvd-Size: 10343 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:52 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0998D20878; Sat, 17 Oct 2020 23:14:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976492; bh=VAwWi4WCSi7loeB6+ApzSTtAS3FJGJG9OS1CfO0EI3g=; h=Date:From:To:Subject:In-Reply-To:From; b=yk7jXghKZNsFsTBXEnHxrQAuzdK3bvJhBXmcyPvThyi4TO0M3t2ckoPQ260OX3SN4 HyJDZRUfnV4a1hHpuee+SQzEa5wGJxGFW8nZ7+eHWQE+xoyP3e8h7johI0b/e2P75Q YDiAN7ffeLVC+92ITFo4lmcSJeTC55umZnq7E8zo= Date: Sat, 17 Oct 2020 16:14:50 -0700 From: Andrew Morton To: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com, axboe@kernel.dk, bgeffon@google.com, christian.brauner@ubuntu.com, christian@brauner.io, dancol@google.com, fw@deneb.enyo.de, hannes@cmpxchg.org, jannh@google.com, joaodias@google.com, joel@joelfernandes.org, ktkhai@virtuozzo.com, linux-man@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, oleksandr@redhat.com, rientjes@google.com, shakeelb@google.com, sj38.park@gmail.com, sjpark@amazon.de, sonnyrao@google.com, sspatil@google.com, surenb@google.com, timmurray@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 24/40] mm/madvise: pass mm to do_madvise Message-ID: <20201017231450.ZUTJgOnD5%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Minchan Kim Subject: mm/madvise: pass mm to do_madvise Patch series "introduce memory hinting API for external process", v9. Now, we have MADV_PAGEOUT and MADV_COLD as madvise hinting API. With that, application could give hints to kernel what memory range are preferred to be reclaimed. However, in some platform(e.g., Android), the information required to make the hinting decision is not known to the app. Instead, it is known to a centralized userspace daemon(e.g., ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement. To solve the concern, this patch introduces new syscall - process_madvise(2). Bascially, it's same with madvise(2) syscall but it has some differences. 1. It needs pidfd of target process to provide the hint 2. It supports only MADV_{COLD|PAGEOUT|MERGEABLE|UNMEREABLE} at this moment. Other hints in madvise will be opened when there are explicit requests from community to prevent unexpected bugs we couldn't support. 3. Only privileged processes can do something for other process's address space. For more detail of the new API, please see "mm: introduce external memory hinting API" description in this patchset. This patch (of 3): In upcoming patches, do_madvise will be called from external process context so we shouldn't asssume "current" is always hinted process's task_struct. Furthermore, we must not access mm_struct via task->mm, but obtain it via access_mm() once (in the following patch) and only use that pointer [1], so pass it to do_madvise() as well. Note the vma->vm_mm pointers are safe, so we can use them further down the call stack. And let's pass current->mm as arguments of do_madvise so it shouldn't change existing behavior but prepare next patch to make review easy. [vbabka@suse.cz: changelog tweak] [minchan@kernel.org: use current->mm for io_uring] Link: http://lkml.kernel.org/r/20200423145215.72666-1-minchan@kernel.org [akpm@linux-foundation.org: fix it for upstream changes] [akpm@linux-foundation.org: whoops] [rdunlap@infradead.org: add missing includes] Link: https://lkml.kernel.org/r/20200901000633.1920247-1-minchan@kernel.org Link: http://lkml.kernel.org/r/20200622192900.22757-1-minchan@kernel.org Link: http://lkml.kernel.org/r/20200302193630.68771-2-minchan@kernel.org Link: http://lkml.kernel.org/r/20200622192900.22757-2-minchan@kernel.org Link: https://lkml.kernel.org/r/20200901000633.1920247-2-minchan@kernel.org Signed-off-by: Minchan Kim Reviewed-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka Acked-by: David Rientjes Cc: Jens Axboe Cc: Jann Horn Cc: Tim Murray Cc: Daniel Colascione Cc: Sandeep Patil Cc: Sonny Rao Cc: Brian Geffon Cc: Michal Hocko Cc: Johannes Weiner Cc: Shakeel Butt Cc: John Dias Cc: Joel Fernandes Cc: Alexander Duyck Cc: SeongJae Park Cc: Christian Brauner Cc: Kirill Tkhai Cc: Oleksandr Natalenko Cc: SeongJae Park Cc: Christian Brauner Cc: Florian Weimer Cc: Signed-off-by: Andrew Morton --- fs/io_uring.c | 2 +- include/linux/mm.h | 2 +- mm/madvise.c | 32 ++++++++++++++++++-------------- 3 files changed, 20 insertions(+), 16 deletions(-) --- a/fs/io_uring.c~mm-madvise-pass-mm-to-do_madvise +++ a/fs/io_uring.c @@ -3989,7 +3989,7 @@ static int io_madvise(struct io_kiocb *r if (force_nonblock) return -EAGAIN; - ret = do_madvise(ma->addr, ma->len, ma->advice); + ret = do_madvise(current->mm, ma->addr, ma->len, ma->advice); if (ret < 0) req_set_fail_links(req); io_req_complete(req, ret); --- a/include/linux/mm.h~mm-madvise-pass-mm-to-do_madvise +++ a/include/linux/mm.h @@ -2579,7 +2579,7 @@ extern int __do_munmap(struct mm_struct struct list_head *uf, bool downgrade); extern int do_munmap(struct mm_struct *, unsigned long, size_t, struct list_head *uf); -extern int do_madvise(unsigned long start, size_t len_in, int behavior); +extern int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int behavior); #ifdef CONFIG_MMU extern int __mm_populate(unsigned long addr, unsigned long len, --- a/mm/madvise.c~mm-madvise-pass-mm-to-do_madvise +++ a/mm/madvise.c @@ -258,6 +258,7 @@ static long madvise_willneed(struct vm_a struct vm_area_struct **prev, unsigned long start, unsigned long end) { + struct mm_struct *mm = vma->vm_mm; struct file *file = vma->vm_file; loff_t offset; @@ -294,10 +295,10 @@ static long madvise_willneed(struct vm_a get_file(file); offset = (loff_t)(start - vma->vm_start) + ((loff_t)vma->vm_pgoff << PAGE_SHIFT); - mmap_read_unlock(current->mm); + mmap_read_unlock(mm); vfs_fadvise(file, offset, end - start, POSIX_FADV_WILLNEED); fput(file); - mmap_read_lock(current->mm); + mmap_read_lock(mm); return 0; } @@ -766,6 +767,8 @@ static long madvise_dontneed_free(struct unsigned long start, unsigned long end, int behavior) { + struct mm_struct *mm = vma->vm_mm; + *prev = vma; if (!can_madv_lru_vma(vma)) return -EINVAL; @@ -773,8 +776,8 @@ static long madvise_dontneed_free(struct if (!userfaultfd_remove(vma, start, end)) { *prev = NULL; /* mmap_lock has been dropped, prev is stale */ - mmap_read_lock(current->mm); - vma = find_vma(current->mm, start); + mmap_read_lock(mm); + vma = find_vma(mm, start); if (!vma) return -ENOMEM; if (start < vma->vm_start) { @@ -828,6 +831,7 @@ static long madvise_remove(struct vm_are loff_t offset; int error; struct file *f; + struct mm_struct *mm = vma->vm_mm; *prev = NULL; /* tell sys_madvise we drop mmap_lock */ @@ -855,13 +859,13 @@ static long madvise_remove(struct vm_are get_file(f); if (userfaultfd_remove(vma, start, end)) { /* mmap_lock was not released by userfaultfd_remove() */ - mmap_read_unlock(current->mm); + mmap_read_unlock(mm); } error = vfs_fallocate(f, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, offset, end - start); fput(f); - mmap_read_lock(current->mm); + mmap_read_lock(mm); return error; } @@ -1045,7 +1049,7 @@ madvise_behavior_valid(int behavior) * -EBADF - map exists, but area maps something that isn't a file. * -EAGAIN - a kernel resource was temporarily unavailable. */ -int do_madvise(unsigned long start, size_t len_in, int behavior) +int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int behavior) { unsigned long end, tmp; struct vm_area_struct *vma, *prev; @@ -1083,10 +1087,10 @@ int do_madvise(unsigned long start, size write = madvise_need_mmap_write(behavior); if (write) { - if (mmap_write_lock_killable(current->mm)) + if (mmap_write_lock_killable(mm)) return -EINTR; } else { - mmap_read_lock(current->mm); + mmap_read_lock(mm); } /* @@ -1094,7 +1098,7 @@ int do_madvise(unsigned long start, size * ranges, just ignore them, but return -ENOMEM at the end. * - different from the way of handling in mlock etc. */ - vma = find_vma_prev(current->mm, start, &prev); + vma = find_vma_prev(mm, start, &prev); if (vma && start > vma->vm_start) prev = vma; @@ -1131,19 +1135,19 @@ int do_madvise(unsigned long start, size if (prev) vma = prev->vm_next; else /* madvise_remove dropped mmap_lock */ - vma = find_vma(current->mm, start); + vma = find_vma(mm, start); } out: blk_finish_plug(&plug); if (write) - mmap_write_unlock(current->mm); + mmap_write_unlock(mm); else - mmap_read_unlock(current->mm); + mmap_read_unlock(mm); return error; } SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior) { - return do_madvise(start, len_in, behavior); + return do_madvise(current->mm, start, len_in, behavior); } From patchwork Sat Oct 17 23:14:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843129 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 232F514B5 for ; Sat, 17 Oct 2020 23:14:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA1F020E65 for ; Sat, 17 Oct 2020 23:14:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="hVhDvE41" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA1F020E65 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F3D716B008C; Sat, 17 Oct 2020 19:14:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EC5B86B0092; Sat, 17 Oct 2020 19:14:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB5AE6B0093; Sat, 17 Oct 2020 19:14:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0133.hostedemail.com [216.40.44.133]) by kanga.kvack.org (Postfix) with ESMTP id AF8DE6B008C for ; Sat, 17 Oct 2020 19:14:57 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5978E362C for ; Sat, 17 Oct 2020 23:14:57 +0000 (UTC) X-FDA: 77382974634.02.wave29_2407ca127229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 3D650101E4FA1 for ; Sat, 17 Oct 2020 23:14:57 +0000 (UTC) X-Spam-Summary: 1,0,0,e2f61f53ee1b2e60,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:967:973:982:988:989:1260:1263:1345:1359:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2198:2199:2393:2525:2559:2565:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3867:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4605:5007:6261:6653:6737:6738:7514:7576:7875:7903:8599:8957:9025:9545:9592:10004:10913:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12783:12986:13846:14181:14721:14849:21080:21094:21323:21324:21451:21627:21939:21990:30054:30062:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04ygz49a5pdt1buau1s148hzbqzbwope7ktfoo7xmpbyyft3rsa8f86hn4t78hf.3ik7t4hoqxysnheohnwxw5454s8uunpgqifsc1ze9kq9uf4if8suefqqm3em9rb.r-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL: 0,DNSBL: X-HE-Tag: wave29_2407ca127229 X-Filterd-Recvd-Size: 5140 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:14:56 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 22A33208B6; Sat, 17 Oct 2020 23:14:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976496; bh=3OiFuzdtCsGrUK5Zn8MzCz+Mf/ZTWtSwZm402RHEt3Y=; h=Date:From:To:Subject:In-Reply-To:From; b=hVhDvE41FSBKQv6Zc0NRG3xIjOR5eYSTYT19a5O/j6O0sHZqrs2bVXo7dHoLnAzG8 SXJuUWrjmJvsYKEch/z+5um5FvVDmt/HBNGiRgxoaXKcOn/+tdFN0oIkoUFrDIGBW1 0k+QPOjscsjA0nonGZ8Pu3tHx7LJwYUZHhjz8k1g= Date: Sat, 17 Oct 2020 16:14:54 -0700 From: Andrew Morton To: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com, axboe@kernel.dk, bgeffon@google.com, christian.brauner@ubuntu.com, dancol@google.com, fw@deneb.enyo.de, hannes@cmpxchg.org, jannh@google.com, joaodias@google.com, joel@joelfernandes.org, ktkhai@virtuozzo.com, linux-man@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, oleksandr@redhat.com, rientjes@google.com, shakeelb@google.com, sj38.park@gmail.com, sjpark@amazon.de, sonnyrao@google.com, sspatil@google.com, surenb@google.com, timmurray@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 25/40] pid: move pidfd_get_pid() to pid.c Message-ID: <20201017231454.v5NcoxBDP%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Minchan Kim Subject: pid: move pidfd_get_pid() to pid.c process_madvise syscall needs pidfd_get_pid function to translate pidfd to pid so this patch move the function to kernel/pid.c. Link: http://lkml.kernel.org/r/20200302193630.68771-5-minchan@kernel.org Link: http://lkml.kernel.org/r/20200622192900.22757-3-minchan@kernel.org Link: https://lkml.kernel.org/r/20200901000633.1920247-3-minchan@kernel.org Reviewed-by: Suren Baghdasaryan Suggested-by: Alexander Duyck Reviewed-by: Alexander Duyck Acked-by: Christian Brauner Reviewed-by: Vlastimil Babka Acked-by: David Rientjes Signed-off-by: Minchan Kim Cc: Jens Axboe Cc: Jann Horn Cc: Brian Geffon Cc: Daniel Colascione Cc: Joel Fernandes Cc: Johannes Weiner Cc: John Dias Cc: Kirill Tkhai Cc: Michal Hocko Cc: Oleksandr Natalenko Cc: Sandeep Patil Cc: SeongJae Park Cc: SeongJae Park Cc: Shakeel Butt Cc: Sonny Rao Cc: Tim Murray Cc: Christian Brauner Cc: Florian Weimer Cc: Signed-off-by: Andrew Morton --- include/linux/pid.h | 1 + kernel/exit.c | 19 ------------------- kernel/pid.c | 19 +++++++++++++++++++ 3 files changed, 20 insertions(+), 19 deletions(-) --- a/include/linux/pid.h~pid-move-pidfd_get_pid-to-pidc +++ a/include/linux/pid.h @@ -77,6 +77,7 @@ extern const struct file_operations pidf struct file; extern struct pid *pidfd_pid(const struct file *file); +struct pid *pidfd_get_pid(unsigned int fd, unsigned int *flags); static inline struct pid *get_pid(struct pid *pid) { --- a/kernel/exit.c~pid-move-pidfd_get_pid-to-pidc +++ a/kernel/exit.c @@ -1474,25 +1474,6 @@ end: return retval; } -static struct pid *pidfd_get_pid(unsigned int fd, unsigned int *flags) -{ - struct fd f; - struct pid *pid; - - f = fdget(fd); - if (!f.file) - return ERR_PTR(-EBADF); - - pid = pidfd_pid(f.file); - if (!IS_ERR(pid)) { - get_pid(pid); - *flags = f.file->f_flags; - } - - fdput(f); - return pid; -} - static long kernel_waitid(int which, pid_t upid, struct waitid_info *infop, int options, struct rusage *ru) { --- a/kernel/pid.c~pid-move-pidfd_get_pid-to-pidc +++ a/kernel/pid.c @@ -520,6 +520,25 @@ struct pid *find_ge_pid(int nr, struct p return idr_get_next(&ns->idr, &nr); } +struct pid *pidfd_get_pid(unsigned int fd, unsigned int *flags) +{ + struct fd f; + struct pid *pid; + + f = fdget(fd); + if (!f.file) + return ERR_PTR(-EBADF); + + pid = pidfd_pid(f.file); + if (!IS_ERR(pid)) { + get_pid(pid); + *flags = f.file->f_flags; + } + + fdput(f); + return pid; +} + /** * pidfd_create() - Create a new pid file descriptor. * From patchwork Sat Oct 17 23:14:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843131 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C0E81580 for ; Sat, 17 Oct 2020 23:15:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1A2A620EDD for ; Sat, 17 Oct 2020 23:15:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="mINCk+po" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1A2A620EDD Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 292106B0092; Sat, 17 Oct 2020 19:15:03 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 243136B0093; Sat, 17 Oct 2020 19:15:03 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 132336B0095; Sat, 17 Oct 2020 19:15:03 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id CCCFF6B0092 for ; Sat, 17 Oct 2020 19:15:02 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7CEAB180AD807 for ; Sat, 17 Oct 2020 23:15:02 +0000 (UTC) X-FDA: 77382974844.29.town59_3b16b4927229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 5CAD018086CD9 for ; Sat, 17 Oct 2020 23:15:02 +0000 (UTC) X-Spam-Summary: 10,1,0,1145f51df5fb8ced,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:327:341:355:379:800:960:966:967:968:973:982:988:989:1042:1260:1263:1345:1359:1381:1431:1437:1605:1730:1747:1777:1792:1801:1981:2194:2196:2198:2199:2200:2201:2393:2525:2553:2559:2568:2682:2685:2689:2693:2740:2859:2892:2895:2896:2901:2902:2911:2924:2925:2926:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4031:4250:4321:4385:4425:4605:5007:6119:6261:6653:6691:6737:6738:7514:7576:7875:7903:7974:8599:8603:8660:8784:8957:9025:9121:9545:10004:10226:10913:11026:11233:11473:11657:11658:11914:12043:12048:12294:12296:12297:12438:12517:12519:12555:12679:12696:12737:12783:12986:13141:13142:13148:13149:13153:13161:13228:13229:13230:13255:13846:14096:21080:21094:21212:21222:21323:21324:21347:21365:21433:21451:21627:21664:21740:21772:21789:21795:21811:21939:21987:21990:30003:30034:3004 5:30051: X-HE-Tag: town59_3b16b4927229 X-Filterd-Recvd-Size: 27074 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:01 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9EE0B20E65; Sat, 17 Oct 2020 23:14:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976500; bh=HUkpvdzjB+x4x/+1k9Huo5DeTDFFtaqoY9K1Kz6WgNA=; h=Date:From:To:Subject:In-Reply-To:From; b=mINCk+po3lvOicNQVI2g/y1TADHGdefGvE+THjFaET2dER4+fiLoy95PuhWLdN2Cp Fu9La4nU2tj4+vhEMBdCpNDvnnQKDwUPRZoHl2N3F5/gWLWhiz0KitO9xozekay9Hk ZGc4pilw8Qo3GEBZWz33q12GfAebpFdtDIXnbkVk= Date: Sat, 17 Oct 2020 16:14:59 -0700 From: Andrew Morton To: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com, axboe@kernel.dk, bgeffon@google.com, christian.brauner@ubuntu.com, christian@brauner.io, dancol@google.com, fw@deneb.enyo.de, hannes@cmpxchg.org, jannh@google.com, joaodias@google.com, joel@joelfernandes.org, ktkhai@virtuozzo.com, linux-man@vger.kernel.org, linux-mm@kvack.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, oleksandr@redhat.com, rientjes@google.com, sfr@canb.auug.org.au, shakeelb@google.com, sj38.park@gmail.com, sjpark@amazon.de, sonnyrao@google.com, sspatil@google.com, surenb@google.com, timmurray@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, yuehaibing@huawei.com Subject: [patch 26/40] mm/madvise: introduce process_madvise() syscall: an external memory hinting API Message-ID: <20201017231459.Lp8onBbxK%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Minchan Kim Subject: mm/madvise: introduce process_madvise() syscall: an external memory hinting API There is usecase that System Management Software(SMS) want to give a memory hint like MADV_[COLD|PAGEEOUT] to other processes and in the case of Android, it is the ActivityManagerService. The information required to make the reclaim decision is not known to the app. Instead, it is known to the centralized userspace daemon(ActivityManagerService), and that daemon must be able to initiate reclaim on its own without any app involvement. To solve the issue, this patch introduces a new syscall process_madvise(2). It uses pidfd of an external process to give the hint. It also supports vector address range because Android app has thousands of vmas due to zygote so it's totally waste of CPU and power if we should call the syscall one by one for each vma.(With testing 2000-vma syscall vs 1-vector syscall, it showed 15% performance improvement. I think it would be bigger in real practice because the testing ran very cache friendly environment). Another potential use case for the vector range is to amortize the cost ofTLB shootdowns for multiple ranges when using MADV_DONTNEED; this could benefit users like TCP receive zerocopy and malloc implementations. In future, we could find more usecases for other advises so let's make it happens as API since we introduce a new syscall at this moment. With that, existing madvise(2) user could replace it with process_madvise(2) with their own pid if they want to have batch address ranges support feature. ince it could affect other process's address range, only privileged process(PTRACE_MODE_ATTACH_FSCREDS) or something else(e.g., being the same UID) gives it the right to ptrace the process could use it successfully. The flag argument is reserved for future use if we need to extend the API. I think supporting all hints madvise has/will supported/support to process_madvise is rather risky. Because we are not sure all hints make sense from external process and implementation for the hint may rely on the caller being in the current context so it could be error-prone. Thus, I just limited hints as MADV_[COLD|PAGEOUT] in this patch. If someone want to add other hints, we could hear the usecase and review it for each hint. It's safer for maintenance rather than introducing a buggy syscall but hard to fix it later. So finally, the API is as follows, ssize_t process_madvise(int pidfd, const struct iovec *iovec, unsigned long vlen, int advice, unsigned int flags); DESCRIPTION The process_madvise() system call is used to give advice or directions to the kernel about the address ranges from external process as well as local process. It provides the advice to address ranges of process described by iovec and vlen. The goal of such advice is to improve system or application performance. The pidfd selects the process referred to by the PID file descriptor specified in pidfd. (See pidofd_open(2) for further information) The pointer iovec points to an array of iovec structures, defined in as: struct iovec { void *iov_base; /* starting address */ size_t iov_len; /* number of bytes to be advised */ }; The iovec describes address ranges beginning at address(iov_base) and with size length of bytes(iov_len). The vlen represents the number of elements in iovec. The advice is indicated in the advice argument, which is one of the following at this moment if the target process specified by pidfd is external. MADV_COLD MADV_PAGEOUT Permission to provide a hint to external process is governed by a ptrace access mode PTRACE_MODE_ATTACH_FSCREDS check; see ptrace(2). The process_madvise supports every advice madvise(2) has if target process is in same thread group with calling process so user could use process_madvise(2) to extend existing madvise(2) to support vector address ranges. RETURN VALUE On success, process_madvise() returns the number of bytes advised. This return value may be less than the total number of requested bytes, if an error occurred. The caller should check return value to determine whether a partial advice occurred. FAQ: Q.1 - Why does any external entity have better knowledge? Quote from Sandeep "For Android, every application (including the special SystemServer) are forked from Zygote. The reason of course is to share as many libraries and classes between the two as possible to benefit from the preloading during boot. After applications start, (almost) all of the APIs end up calling into this SystemServer process over IPC (binder) and back to the application. In a fully running system, the SystemServer monitors every single process periodically to calculate their PSS / RSS and also decides which process is "important" to the user for interactivity. So, because of how these processes start _and_ the fact that the SystemServer is looping to monitor each process, it does tend to *know* which address range of the application is not used / useful. Besides, we can never rely on applications to clean things up themselves. We've had the "hey app1, the system is low on memory, please trim your memory usage down" notifications for a long time[1]. They rely on applications honoring the broadcasts and very few do. So, if we want to avoid the inevitable killing of the application and restarting it, some way to be able to tell the OS about unimportant memory in these applications will be useful. - ssp Q.2 - How to guarantee the race(i.e., object validation) between when giving a hint from an external process and get the hint from the target process? process_madvise operates on the target process's address space as it exists at the instant that process_madvise is called. If the space target process can run between the time the process_madvise process inspects the target process address space and the time that process_madvise is actually called, process_madvise may operate on memory regions that the calling process does not expect. It's the responsibility of the process calling process_madvise to close this race condition. For example, the calling process can suspend the target process with ptrace, SIGSTOP, or the freezer cgroup so that it doesn't have an opportunity to change its own address space before process_madvise is called. Another option is to operate on memory regions that the caller knows a priori will be unchanged in the target process. Yet another option is to accept the race for certain process_madvise calls after reasoning that mistargeting will do no harm. The suggested API itself does not provide synchronization. It also apply other APIs like move_pages, process_vm_write. The race isn't really a problem though. Why is it so wrong to require that callers do their own synchronization in some manner? Nobody objects to write(2) merely because it's possible for two processes to open the same file and clobber each other's writes --- instead, we tell people to use flock or something. Think about mmap. It never guarantees newly allocated address space is still valid when the user tries to access it because other threads could unmap the memory right before. That's where we need synchronization by using other API or design from userside. It shouldn't be part of API itself. If someone needs more fine-grained synchronization rather than process level, there were two ideas suggested - cookie[2] and anon-fd[3]. Both are applicable via using last reserved argument of the API but I don't think it's necessary right now since we have already ways to prevent the race so don't want to add additional complexity with more fine-grained optimization model. To make the API extend, it reserved an unsigned long as last argument so we could support it in future if someone really needs it. Q.3 - Why doesn't ptrace work? Injecting an madvise in the target process using ptrace would not work for us because such injected madvise would have to be executed by the target process, which means that process would have to be runnable and that creates the risk of the abovementioned race and hinting a wrong VMA. Furthermore, we want to act the hint in caller's context, not the callee's, because the callee is usually limited in cpuset/cgroups or even freezed state so they can't act by themselves quick enough, which causes more thrashing/kill. It doesn't work if the target process are ptraced(e.g., strace, debugger, minidump) because a process can have at most one ptracer. [1] https://developer.android.com/topic/performance/memory" [2] process_getinfo for getting the cookie which is updated whenever vma of process address layout are changed - Daniel Colascione - https://lore.kernel.org/lkml/20190520035254.57579-1-minchan@kernel.org/T/#m7694416fd179b2066a2c62b5b139b14e3894e224 [3] anonymous fd which is used for the object(i.e., address range) validation - Michal Hocko - https://lore.kernel.org/lkml/20200120112722.GY18451@dhcp22.suse.cz/ [minchan@kernel.org: fix process_madvise build break for arm64] Link: http://lkml.kernel.org/r/20200303145756.GA219683@google.com [minchan@kernel.org: fix build error for mips of process_madvise] Link: http://lkml.kernel.org/r/20200508052517.GA197378@google.com [akpm@linux-foundation.org: fix patch ordering issue] [akpm@linux-foundation.org: fix arm64 whoops] [minchan@kernel.org: make process_madvise() vlen arg have type size_t, per Florian] [akpm@linux-foundation.org: fix i386 build] [sfr@canb.auug.org.au: fix syscall numbering] Link: https://lkml.kernel.org/r/20200905142639.49fc3f1a@canb.auug.org.au [sfr@canb.auug.org.au: madvise.c needs compat.h] Link: https://lkml.kernel.org/r/20200908204547.285646b4@canb.auug.org.au [minchan@kernel.org: fix mips build] Link: https://lkml.kernel.org/r/20200909173655.GC2435453@google.com [yuehaibing@huawei.com: remove duplicate header which is included twice] Link: https://lkml.kernel.org/r/20200915121550.30584-1-yuehaibing@huawei.com [minchan@kernel.org: do not use helper functions for process_madvise] Link: https://lkml.kernel.org/r/20200921175539.GB387368@google.com [akpm@linux-foundation.org: pidfd_get_pid() gained an argument] [sfr@canb.auug.org.au: fix up for "iov_iter: transparently handle compat iovecs in import_iovec"] Link: https://lkml.kernel.org/r/20200928212542.468e1fef@canb.auug.org.au Link: http://lkml.kernel.org/r/20200302193630.68771-3-minchan@kernel.org Link: http://lkml.kernel.org/r/20200508183320.GA125527@google.com Link: http://lkml.kernel.org/r/20200622192900.22757-4-minchan@kernel.org Link: https://lkml.kernel.org/r/20200901000633.1920247-4-minchan@kernel.org Reviewed-by: Suren Baghdasaryan Reviewed-by: Vlastimil Babka Acked-by: David Rientjes Signed-off-by: Minchan Kim Signed-off-by: YueHaibing Signed-off-by: Stephen Rothwell Cc: Alexander Duyck Cc: Brian Geffon Cc: Christian Brauner Cc: Daniel Colascione Cc: Jann Horn Cc: Jens Axboe Cc: Joel Fernandes Cc: Johannes Weiner Cc: John Dias Cc: Kirill Tkhai Cc: Michal Hocko Cc: Oleksandr Natalenko Cc: Sandeep Patil Cc: SeongJae Park Cc: SeongJae Park Cc: Shakeel Butt Cc: Sonny Rao Cc: Tim Murray Cc: Christian Brauner Cc: Florian Weimer Cc: Signed-off-by: Andrew Morton --- arch/alpha/kernel/syscalls/syscall.tbl | 1 arch/arm/tools/syscall.tbl | 1 arch/arm64/include/asm/unistd.h | 2 arch/arm64/include/asm/unistd32.h | 2 arch/ia64/kernel/syscalls/syscall.tbl | 1 arch/m68k/kernel/syscalls/syscall.tbl | 1 arch/microblaze/kernel/syscalls/syscall.tbl | 1 arch/mips/kernel/syscalls/syscall_n32.tbl | 1 arch/mips/kernel/syscalls/syscall_n64.tbl | 1 arch/mips/kernel/syscalls/syscall_o32.tbl | 1 arch/parisc/kernel/syscalls/syscall.tbl | 1 arch/powerpc/kernel/syscalls/syscall.tbl | 1 arch/s390/kernel/syscalls/syscall.tbl | 1 arch/sh/kernel/syscalls/syscall.tbl | 1 arch/sparc/kernel/syscalls/syscall.tbl | 1 arch/x86/entry/syscalls/syscall_32.tbl | 1 arch/x86/entry/syscalls/syscall_64.tbl | 1 arch/xtensa/kernel/syscalls/syscall.tbl | 1 include/linux/syscalls.h | 2 include/uapi/asm-generic/unistd.h | 4 kernel/sys_ni.c | 1 mm/madvise.c | 93 +++++++++++++++++- 22 files changed, 117 insertions(+), 3 deletions(-) --- a/arch/alpha/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/alpha/kernel/syscalls/syscall.tbl @@ -479,3 +479,4 @@ 547 common openat2 sys_openat2 548 common pidfd_getfd sys_pidfd_getfd 549 common faccessat2 sys_faccessat2 +550 common process_madvise sys_process_madvise --- a/arch/arm64/include/asm/unistd32.h~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/arm64/include/asm/unistd32.h @@ -887,6 +887,8 @@ __SYSCALL(__NR_openat2, sys_openat2) __SYSCALL(__NR_pidfd_getfd, sys_pidfd_getfd) #define __NR_faccessat2 439 __SYSCALL(__NR_faccessat2, sys_faccessat2) +#define __NR_process_madvise 440 +__SYSCALL(__NR_process_madvise, sys_process_madvise) /* * Please add new compat syscalls above this comment and update --- a/arch/arm64/include/asm/unistd.h~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/arm64/include/asm/unistd.h @@ -38,7 +38,7 @@ #define __ARM_NR_compat_set_tls (__ARM_NR_COMPAT_BASE + 5) #define __ARM_NR_COMPAT_END (__ARM_NR_COMPAT_BASE + 0x800) -#define __NR_compat_syscalls 440 +#define __NR_compat_syscalls 441 #endif #define __ARCH_WANT_SYS_CLONE --- a/arch/arm/tools/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/arm/tools/syscall.tbl @@ -453,3 +453,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/ia64/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/ia64/kernel/syscalls/syscall.tbl @@ -360,3 +360,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/m68k/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/m68k/kernel/syscalls/syscall.tbl @@ -439,3 +439,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/microblaze/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/microblaze/kernel/syscalls/syscall.tbl @@ -445,3 +445,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/mips/kernel/syscalls/syscall_n32.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/mips/kernel/syscalls/syscall_n32.tbl @@ -378,3 +378,4 @@ 437 n32 openat2 sys_openat2 438 n32 pidfd_getfd sys_pidfd_getfd 439 n32 faccessat2 sys_faccessat2 +440 n32 process_madvise sys_process_madvise --- a/arch/mips/kernel/syscalls/syscall_n64.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/mips/kernel/syscalls/syscall_n64.tbl @@ -354,3 +354,4 @@ 437 n64 openat2 sys_openat2 438 n64 pidfd_getfd sys_pidfd_getfd 439 n64 faccessat2 sys_faccessat2 +440 n64 process_madvise sys_process_madvise --- a/arch/mips/kernel/syscalls/syscall_o32.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/mips/kernel/syscalls/syscall_o32.tbl @@ -427,3 +427,4 @@ 437 o32 openat2 sys_openat2 438 o32 pidfd_getfd sys_pidfd_getfd 439 o32 faccessat2 sys_faccessat2 +440 o32 process_madvise sys_process_madvise --- a/arch/parisc/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/parisc/kernel/syscalls/syscall.tbl @@ -437,3 +437,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/powerpc/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/powerpc/kernel/syscalls/syscall.tbl @@ -529,3 +529,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/s390/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/s390/kernel/syscalls/syscall.tbl @@ -442,3 +442,4 @@ 437 common openat2 sys_openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise sys_process_madvise --- a/arch/sh/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/sh/kernel/syscalls/syscall.tbl @@ -442,3 +442,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/sparc/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/sparc/kernel/syscalls/syscall.tbl @@ -485,3 +485,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/arch/x86/entry/syscalls/syscall_32.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/x86/entry/syscalls/syscall_32.tbl @@ -444,3 +444,4 @@ 437 i386 openat2 sys_openat2 438 i386 pidfd_getfd sys_pidfd_getfd 439 i386 faccessat2 sys_faccessat2 +440 i386 process_madvise sys_process_madvise --- a/arch/x86/entry/syscalls/syscall_64.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/x86/entry/syscalls/syscall_64.tbl @@ -361,6 +361,7 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise # # x32-specific system call numbers start at 512 to avoid cache impact --- a/arch/xtensa/kernel/syscalls/syscall.tbl~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/arch/xtensa/kernel/syscalls/syscall.tbl @@ -410,3 +410,4 @@ 437 common openat2 sys_openat2 438 common pidfd_getfd sys_pidfd_getfd 439 common faccessat2 sys_faccessat2 +440 common process_madvise sys_process_madvise --- a/include/linux/syscalls.h~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/include/linux/syscalls.h @@ -879,6 +879,8 @@ asmlinkage long sys_munlockall(void); asmlinkage long sys_mincore(unsigned long start, size_t len, unsigned char __user * vec); asmlinkage long sys_madvise(unsigned long start, size_t len, int behavior); +asmlinkage long sys_process_madvise(int pidfd, const struct iovec __user *vec, + size_t vlen, int behavior, unsigned int flags); asmlinkage long sys_remap_file_pages(unsigned long start, unsigned long size, unsigned long prot, unsigned long pgoff, unsigned long flags); --- a/include/uapi/asm-generic/unistd.h~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/include/uapi/asm-generic/unistd.h @@ -857,9 +857,11 @@ __SYSCALL(__NR_openat2, sys_openat2) __SYSCALL(__NR_pidfd_getfd, sys_pidfd_getfd) #define __NR_faccessat2 439 __SYSCALL(__NR_faccessat2, sys_faccessat2) +#define __NR_process_madvise 440 +__SYSCALL(__NR_process_madvise, sys_process_madvise) #undef __NR_syscalls -#define __NR_syscalls 440 +#define __NR_syscalls 441 /* * 32 bit systems traditionally used different --- a/kernel/sys_ni.c~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/kernel/sys_ni.c @@ -280,6 +280,7 @@ COND_SYSCALL(mlockall); COND_SYSCALL(munlockall); COND_SYSCALL(mincore); COND_SYSCALL(madvise); +COND_SYSCALL(process_madvise); COND_SYSCALL(remap_file_pages); COND_SYSCALL(mbind); COND_SYSCALL_COMPAT(mbind); --- a/mm/madvise.c~mm-madvise-introduce-process_madvise-syscall-an-external-memory-hinting-api +++ a/mm/madvise.c @@ -17,6 +17,8 @@ #include #include #include +#include +#include #include #include #include @@ -27,7 +29,6 @@ #include #include #include -#include #include @@ -988,6 +989,18 @@ madvise_behavior_valid(int behavior) } } +static bool +process_madvise_behavior_valid(int behavior) +{ + switch (behavior) { + case MADV_COLD: + case MADV_PAGEOUT: + return true; + default: + return false; + } +} + /* * The madvise(2) system call. * @@ -1035,6 +1048,11 @@ madvise_behavior_valid(int behavior) * MADV_DONTDUMP - the application wants to prevent pages in the given range * from being included in its core dump. * MADV_DODUMP - cancel MADV_DONTDUMP: no longer exclude from core dump. + * MADV_COLD - the application is not expected to use this memory soon, + * deactivate pages in this range so that they can be reclaimed + * easily if memory pressure hanppens. + * MADV_PAGEOUT - the application is not expected to use this memory soon, + * page out the pages in this range immediately. * * return values: * zero - success @@ -1151,3 +1169,76 @@ SYSCALL_DEFINE3(madvise, unsigned long, { return do_madvise(current->mm, start, len_in, behavior); } + +SYSCALL_DEFINE5(process_madvise, int, pidfd, const struct iovec __user *, vec, + size_t, vlen, int, behavior, unsigned int, flags) +{ + ssize_t ret; + struct iovec iovstack[UIO_FASTIOV], iovec; + struct iovec *iov = iovstack; + struct iov_iter iter; + struct pid *pid; + struct task_struct *task; + struct mm_struct *mm; + size_t total_len; + unsigned int f_flags; + + if (flags != 0) { + ret = -EINVAL; + goto out; + } + + ret = import_iovec(READ, vec, vlen, ARRAY_SIZE(iovstack), &iov, &iter); + if (ret < 0) + goto out; + + pid = pidfd_get_pid(pidfd, &f_flags); + if (IS_ERR(pid)) { + ret = PTR_ERR(pid); + goto free_iov; + } + + task = get_pid_task(pid, PIDTYPE_PID); + if (!task) { + ret = -ESRCH; + goto put_pid; + } + + if (task->mm != current->mm && + !process_madvise_behavior_valid(behavior)) { + ret = -EINVAL; + goto release_task; + } + + mm = mm_access(task, PTRACE_MODE_ATTACH_FSCREDS); + if (IS_ERR_OR_NULL(mm)) { + ret = IS_ERR(mm) ? PTR_ERR(mm) : -ESRCH; + goto release_task; + } + + total_len = iov_iter_count(&iter); + + while (iov_iter_count(&iter)) { + iovec = iov_iter_iovec(&iter); + ret = do_madvise(mm, (unsigned long)iovec.iov_base, + iovec.iov_len, behavior); + if (ret < 0) + break; + iov_iter_advance(&iter, iovec.iov_len); + } + + if (ret == 0) + ret = total_len - iov_iter_count(&iter); + + mmput(mm); + return ret; + +release_task: + put_task_struct(task); +put_pid: + put_pid(pid); +free_iov: + kfree(iov); +out: + return ret; +} From patchwork Sat Oct 17 23:15:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843133 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A01AE14B5 for ; Sat, 17 Oct 2020 23:15:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 692F1214DB for ; Sat, 17 Oct 2020 23:15:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="SRbuA8Ya" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 692F1214DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 63B886B0093; Sat, 17 Oct 2020 19:15:06 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5ECC26B0095; Sat, 17 Oct 2020 19:15:06 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DAEE6B0096; Sat, 17 Oct 2020 19:15:06 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0235.hostedemail.com [216.40.44.235]) by kanga.kvack.org (Postfix) with ESMTP id 225DA6B0093 for ; Sat, 17 Oct 2020 19:15:06 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C51C11EE6 for ; Sat, 17 Oct 2020 23:15:05 +0000 (UTC) X-FDA: 77382974970.18.drink67_3a0e3e127229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin18.hostedemail.com (Postfix) with ESMTP id A702C101E4FA1 for ; Sat, 17 Oct 2020 23:15:05 +0000 (UTC) X-Spam-Summary: 30,2,0,41b60cab943cc63c,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:965:966:967:968:973:988:989:1260:1345:1359:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2194:2196:2198:2199:2200:2201:2393:2525:2553:2559:2564:2682:2685:2693:2859:2895:2902:2904:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3355:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4390:4395:5007:6261:6653:6737:7514:7576:7875:7903:8603:8660:9025:9545:10010:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12986:13148:13230:13846:14096:14181:14721:21080:21433:21451:21627:21740:21795:21939:21987:30003:30051:30054:30064:30070:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04ygyfre5gb3gz6aa3gfjr4iggpjaypkkpowq8a3iuk94xp7mnz665qbn4cwrz3.odkdaizncf8ciwu46cc5eiydxpkfuo4g83ircjysoaskqchf9e9usfuyqjubksc.n-lbl8.mail shell.ne X-HE-Tag: drink67_3a0e3e127229 X-Filterd-Recvd-Size: 5166 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:05 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AE37521527; Sat, 17 Oct 2020 23:15:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976504; bh=zscrx8kxlqwKF9gn/8Q/Wy4GW35yVFNRmCQaMYJkRzA=; h=Date:From:To:Subject:In-Reply-To:From; b=SRbuA8Ya83LRc+DJO8XSEwceiQp2inV9V4Kbz3KEZzbkbfrW3SySebDXhQsRQf/Cw GhgSkt8ufRV+jMoSYa9fbzozm+jhcsw/wHvUD+/wtsRcPaPYmJn1z1dUoXm+Hpism5 xWzLS3p3Zi/1wAhqEG3+ZK2YYowyjmYj750n3P4I= Date: Sat, 17 Oct 2020 16:15:03 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@linux.intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 27/40] mm: update the documentation for vfree Message-ID: <20201017231503.Mq52Vrou6%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm: update the documentation for vfree Patch series "remove alloc_vm_area", v4. This series removes alloc_vm_area, which was left over from the big vmalloc interface rework. It is a rather arkane interface, basicaly the equivalent of get_vm_area + actually faulting in all PTEs in the allocated area. It was originally addeds for Xen (which isn't modular to start with), and then grew users in zsmalloc and i915 which seems to mostly qualify as abuses of the interface, especially for i915 as a random driver should not set up PTE bits directly. This patch (of 11): * Document that you can call vfree() on an address returned from vmap() * Remove the note about the minimum size -- the minimum size of a vmalloc allocation is one page * Add a Context: section * Fix capitalisation * Reword the prohibition on calling from NMI context to avoid a double negative Link: https://lkml.kernel.org/r/20201002122204.1534411-1-hch@lst.de Link: https://lkml.kernel.org/r/20201002122204.1534411-2-hch@lst.de Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Christoph Hellwig Cc: Peter Zijlstra Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Tvrtko Ursulin Cc: Chris Wilson Cc: Matthew Auld Cc: Rodrigo Vivi Cc: Minchan Kim Cc: Matthew Wilcox Cc: Nitin Gupta Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- mm/vmalloc.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) --- a/mm/vmalloc.c~mm-update-the-documentation-for-vfree +++ a/mm/vmalloc.c @@ -2321,20 +2321,21 @@ static void __vfree(const void *addr) } /** - * vfree - release memory allocated by vmalloc() - * @addr: memory base address + * vfree - Release memory allocated by vmalloc() + * @addr: Memory base address * - * Free the virtually continuous memory area starting at @addr, as - * obtained from vmalloc(), vmalloc_32() or __vmalloc(). If @addr is - * NULL, no operation is performed. + * Free the virtually continuous memory area starting at @addr, as obtained + * from one of the vmalloc() family of APIs. This will usually also free the + * physical memory underlying the virtual allocation, but that memory is + * reference counted, so it will not be freed until the last user goes away. * - * Must not be called in NMI context (strictly speaking, only if we don't - * have CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG, but making the calling - * conventions for vfree() arch-depenedent would be a really bad idea) + * If @addr is NULL, no operation is performed. * + * Context: * May sleep if called *not* from interrupt context. - * - * NOTE: assumes that the object at @addr has a size >= sizeof(llist_node) + * Must not be called in NMI context (strictly speaking, it could be + * if we have CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG, but making the calling + * conventions for vfree() arch-depenedent would be a really bad idea). */ void vfree(const void *addr) { From patchwork Sat Oct 17 23:15:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843135 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 36B9514B5 for ; Sat, 17 Oct 2020 23:15:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E936C21527 for ; Sat, 17 Oct 2020 23:15:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="gwIk66Qv" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E936C21527 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F16586B0095; Sat, 17 Oct 2020 19:15:09 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EC4E76B0096; Sat, 17 Oct 2020 19:15:09 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB7486B0098; Sat, 17 Oct 2020 19:15:09 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0200.hostedemail.com [216.40.44.200]) by kanga.kvack.org (Postfix) with ESMTP id AD2876B0095 for ; Sat, 17 Oct 2020 19:15:09 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 476C8362B for ; Sat, 17 Oct 2020 23:15:09 +0000 (UTC) X-FDA: 77382975138.25.bite71_42153c227229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin25.hostedemail.com (Postfix) with ESMTP id 24E0E1804E3A0 for ; Sat, 17 Oct 2020 23:15:09 +0000 (UTC) X-Spam-Summary: 1,0,0,bcbbc43075ac43e3,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2693:2859:2897:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3354:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6261:6653:6737:7514:7576:9025:9545:10004:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12986:13846:14096:14181:14721:21080:21450:21451:21627:21939:21990:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y89fyj4rttz5src99b69mmqumyhocp9qfhpx773b9ye8e4oc4ubm3tppdhxna.tonumg9f59fpe8o33xbtag137165eof5axy71sejcn37sk6dd3eeer3r7hw9qd3.c-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0 ,DNSBL:n X-HE-Tag: bite71_42153c227229 X-Filterd-Recvd-Size: 4331 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:08 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 405BB21556; Sat, 17 Oct 2020 23:15:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976507; bh=yUfqaJ02dl6ucwXHLbBungiigrhtsAK717dpjBcJtz8=; h=Date:From:To:Subject:In-Reply-To:From; b=gwIk66QvBOO1cEnI1S05L8SgCv4tCPNAynvJ+m6o4hpBBAw6QA+wBqq8Y4Xdcfk0f GMgepihR/d++Fz6cRE70TS+YAlwSsHNLXjFeGKfCd2+RKX5R4egZ1EP4HxCvk9yZON klfBGDqjHSoXETOdTYiW0NTjCNQhOhNtUVdlEU00= Date: Sat, 17 Oct 2020 16:15:06 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 28/40] mm: add a VM_MAP_PUT_PAGES flag for vmap Message-ID: <20201017231506.vz7J_Eu6T%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm: add a VM_MAP_PUT_PAGES flag for vmap Add a flag so that vmap takes ownership of the passed in page array. When vfree is called on such an allocation it will put one reference on each page, and free the page array itself. Link: https://lkml.kernel.org/r/20201002122204.1534411-3-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- include/linux/vmalloc.h | 1 + mm/vmalloc.c | 9 +++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) --- a/include/linux/vmalloc.h~mm-add-a-vm_map_put_pages-flag-for-vmap +++ a/include/linux/vmalloc.h @@ -24,6 +24,7 @@ struct notifier_block; /* in notifier.h #define VM_UNINITIALIZED 0x00000020 /* vm_struct is not fully initialized */ #define VM_NO_GUARD 0x00000040 /* don't add guard page */ #define VM_KASAN 0x00000080 /* has allocated kasan shadow memory */ +#define VM_MAP_PUT_PAGES 0x00000100 /* put pages and free array in vfree */ /* * VM_KASAN is used slighly differently depending on CONFIG_KASAN_VMALLOC. --- a/mm/vmalloc.c~mm-add-a-vm_map_put_pages-flag-for-vmap +++ a/mm/vmalloc.c @@ -2377,8 +2377,11 @@ EXPORT_SYMBOL(vunmap); * @flags: vm_area->flags * @prot: page protection for the mapping * - * Maps @count pages from @pages into contiguous kernel virtual - * space. + * Maps @count pages from @pages into contiguous kernel virtual space. + * If @flags contains %VM_MAP_PUT_PAGES the ownership of the pages array itself + * (which must be kmalloc or vmalloc memory) and one reference per pages in it + * are transferred from the caller to vmap(), and will be freed / dropped when + * vfree() is called on the return value. * * Return: the address of the area or %NULL on failure */ @@ -2404,6 +2407,8 @@ void *vmap(struct page **pages, unsigned return NULL; } + if (flags & VM_MAP_PUT_PAGES) + area->pages = pages; return area->addr; } EXPORT_SYMBOL(vmap); From patchwork Sat Oct 17 23:15:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843137 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 88FBB14B5 for ; Sat, 17 Oct 2020 23:15:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5092521582 for ; Sat, 17 Oct 2020 23:15:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="fDsFUAhF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5092521582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5C7466B0096; Sat, 17 Oct 2020 19:15:13 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 59FB26B0098; Sat, 17 Oct 2020 19:15:13 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DC276B0099; Sat, 17 Oct 2020 19:15:13 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 1E3346B0096 for ; Sat, 17 Oct 2020 19:15:13 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BEC96180AD806 for ; Sat, 17 Oct 2020 23:15:12 +0000 (UTC) X-FDA: 77382975264.30.bit94_4f090d527229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 9D56B180B3AA7 for ; Sat, 17 Oct 2020 23:15:12 +0000 (UTC) X-Spam-Summary: 1,0,0,736f91ae34037be3,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2897:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3354:3865:3866:3867:3868:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6117:6261:6653:6737:7514:7576:9025:9545:10004:11026:11473:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12683:12986:13846:14096:14181:14721:21080:21451:21627:21939:21990:30003:30034:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yfcttgren4buuw1onofhqprakeiyp3871xmsm7i8ran3z9en41b75r1ezsdcu.cjqyqapu5a8wndhp9a957ib816hpwz1nu97prhtkgf5z8ab5hpmg4wx8qqb7yq6.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF X-HE-Tag: bit94_4f090d527229 X-Filterd-Recvd-Size: 5171 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf33.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:12 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C976E20EDD; Sat, 17 Oct 2020 23:15:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976511; bh=K3HEs8+uulq4zNfuTiInv60TjFABgs0nKl3bYhfpbnk=; h=Date:From:To:Subject:In-Reply-To:From; b=fDsFUAhF+RhMEG5bHGWb2ejWWvVUbSPWj3KmK13UvgUlvbqHhzXf7BrFdt/BGM345 NTbkVZq6ib3mJAXsfmud1/Ql86g5DEWUpofUoJraF+2i/UfhfuHJhLWo7UUjfbVpCE ZV19AYvmxX+Ucna3NLZelMPqzrlVqMy7H/qZWkXU= Date: Sat, 17 Oct 2020 16:15:10 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 29/40] mm: add a vmap_pfn function Message-ID: <20201017231510.EY4VsFGOW%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm: add a vmap_pfn function Add a proper helper to remap PFNs into kernel virtual space so that drivers don't have to abuse alloc_vm_area and open coded PTE manipulation for it. Link: https://lkml.kernel.org/r/20201002122204.1534411-4-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- include/linux/vmalloc.h | 1 mm/Kconfig | 3 ++ mm/vmalloc.c | 45 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 49 insertions(+) --- a/include/linux/vmalloc.h~mm-add-a-vmap_pfn-function +++ a/include/linux/vmalloc.h @@ -122,6 +122,7 @@ extern void vfree_atomic(const void *add extern void *vmap(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot); +void *vmap_pfn(unsigned long *pfns, unsigned int count, pgprot_t prot); extern void vunmap(const void *addr); extern int remap_vmalloc_range_partial(struct vm_area_struct *vma, --- a/mm/Kconfig~mm-add-a-vmap_pfn-function +++ a/mm/Kconfig @@ -816,6 +816,9 @@ config DEVICE_PRIVATE memory; i.e., memory that is only accessible from the device (or group of devices). You likely also want to select HMM_MIRROR. +config VMAP_PFN + bool + config FRAME_VECTOR bool --- a/mm/vmalloc.c~mm-add-a-vmap_pfn-function +++ a/mm/vmalloc.c @@ -2413,6 +2413,51 @@ void *vmap(struct page **pages, unsigned } EXPORT_SYMBOL(vmap); +#ifdef CONFIG_VMAP_PFN +struct vmap_pfn_data { + unsigned long *pfns; + pgprot_t prot; + unsigned int idx; +}; + +static int vmap_pfn_apply(pte_t *pte, unsigned long addr, void *private) +{ + struct vmap_pfn_data *data = private; + + if (WARN_ON_ONCE(pfn_valid(data->pfns[data->idx]))) + return -EINVAL; + *pte = pte_mkspecial(pfn_pte(data->pfns[data->idx++], data->prot)); + return 0; +} + +/** + * vmap_pfn - map an array of PFNs into virtually contiguous space + * @pfns: array of PFNs + * @count: number of pages to map + * @prot: page protection for the mapping + * + * Maps @count PFNs from @pfns into contiguous kernel virtual space and returns + * the start address of the mapping. + */ +void *vmap_pfn(unsigned long *pfns, unsigned int count, pgprot_t prot) +{ + struct vmap_pfn_data data = { .pfns = pfns, .prot = pgprot_nx(prot) }; + struct vm_struct *area; + + area = get_vm_area_caller(count * PAGE_SIZE, VM_IOREMAP, + __builtin_return_address(0)); + if (!area) + return NULL; + if (apply_to_page_range(&init_mm, (unsigned long)area->addr, + count * PAGE_SIZE, vmap_pfn_apply, &data)) { + free_vm_area(area); + return NULL; + } + return area->addr; +} +EXPORT_SYMBOL_GPL(vmap_pfn); +#endif /* CONFIG_VMAP_PFN */ + static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, pgprot_t prot, int node) { From patchwork Sat Oct 17 23:15:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843139 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BC321580 for ; Sat, 17 Oct 2020 23:15:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BABFA212CC for ; Sat, 17 Oct 2020 23:15:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="n9JGR0rG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BABFA212CC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CCCDC6B0098; Sat, 17 Oct 2020 19:15:16 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CA2166B0099; Sat, 17 Oct 2020 19:15:16 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDF7E6B009A; Sat, 17 Oct 2020 19:15:16 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0123.hostedemail.com [216.40.44.123]) by kanga.kvack.org (Postfix) with ESMTP id 922006B0098 for ; Sat, 17 Oct 2020 19:15:16 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 43C00181AEF07 for ; Sat, 17 Oct 2020 23:15:16 +0000 (UTC) X-FDA: 77382975432.16.shirt84_5515dd527229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 22C78100FC29F for ; Sat, 17 Oct 2020 23:15:16 +0000 (UTC) X-Spam-Summary: 1,0,0,6de32ce4e1be9a16,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3352:3865:3866:3867:3870:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:5007:6261:6653:6737:7514:7576:7901:7903:9025:9545:10004:11026:11473:11658:11914:12043:12048:12297:12438:12517:12519:12555:12679:12986:13069:13161:13229:13311:13357:13846:14181:14384:14721:21080:21212:21451:21627:21939:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-64.100.201.201 62.2.0.100;04yf3qep3q8uxyaxg6jqix74mgh1wypka6jezw6wugnt19334sui35hzc8f59q3.okmxhrogfea9j667tnhmp1us3o63btn48zi3om5g1j7w4r9bkau5wpi7keajyed.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom _rules:0 X-HE-Tag: shirt84_5515dd527229 X-Filterd-Recvd-Size: 3478 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:15 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6D0D221D7F; Sat, 17 Oct 2020 23:15:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976515; bh=LHwdFnkWtSLSOvSabjnALNiw0+W9/t6aQCjsd6fLkvM=; h=Date:From:To:Subject:In-Reply-To:From; b=n9JGR0rGY7aua+VWubvnVDD+xYEzL9/YmLj2/TSNl990PdI83q/KbiaECnC/bqjlv GQ/4BfxAPWheZkslO5LbvBKeYUCkahUpSoOvoS/ZyysqjuriTVohJTzYzOmtitK4pS RZSwQDgbR3UxWEwoFr2610poBV75+YrMTuqzp1BE= Date: Sat, 17 Oct 2020 16:15:14 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 30/40] mm: allow a NULL fn callback in apply_to_page_range Message-ID: <20201017231514.wi5ZbtIfe%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm: allow a NULL fn callback in apply_to_page_range Besides calling the callback on each page, apply_to_page_range also has the effect of pre-faulting all PTEs for the range. To support callers that only need the pre-faulting, make the callback optional. Based on a patch from Minchan Kim . Link: https://lkml.kernel.org/r/20201002122204.1534411-5-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- mm/memory.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) --- a/mm/memory.c~mm-allow-a-null-fn-callback-in-apply_to_page_range +++ a/mm/memory.c @@ -2391,13 +2391,15 @@ static int apply_to_pte_range(struct mm_ arch_enter_lazy_mmu_mode(); - do { - if (create || !pte_none(*pte)) { - err = fn(pte++, addr, data); - if (err) - break; - } - } while (addr += PAGE_SIZE, addr != end); + if (fn) { + do { + if (create || !pte_none(*pte)) { + err = fn(pte++, addr, data); + if (err) + break; + } + } while (addr += PAGE_SIZE, addr != end); + } *mask |= PGTBL_PTE_MODIFIED; arch_leave_lazy_mmu_mode(); From patchwork Sat Oct 17 23:15:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843141 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BAB421580 for ; Sat, 17 Oct 2020 23:15:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7F59C214DB for ; Sat, 17 Oct 2020 23:15:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="tJjLCV/G" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F59C214DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 96FC36B0099; Sat, 17 Oct 2020 19:15:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 91FF06B009A; Sat, 17 Oct 2020 19:15:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80F466B009B; Sat, 17 Oct 2020 19:15:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0141.hostedemail.com [216.40.44.141]) by kanga.kvack.org (Postfix) with ESMTP id 55DD36B0099 for ; Sat, 17 Oct 2020 19:15:20 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id ECFC5181AEF07 for ; Sat, 17 Oct 2020 23:15:19 +0000 (UTC) X-FDA: 77382975558.30.wind39_5112b4d27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id B546A180B3C8B for ; Sat, 17 Oct 2020 23:15:19 +0000 (UTC) X-Spam-Summary: 1,0,0,80bfe2575e3c9305,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3352:3865:3866:3867:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:6737:7514:7576:9025:9545:10004:11026:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12986:13069:13311:13357:13846:14181:14384:14721:21080:21433:21451:21627:21939:21990:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y8c7t759dhtajow5j6oz97g9ktgocziqgfyxh6s3g9e8q7xfknnftz6a6ozwj.66j33zb65hyfw1eeo5ys3dz3htg1tgot1bbrctyiuff4daho413qpam5ua85xff.a-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMA RY:none X-HE-Tag: wind39_5112b4d27229 X-Filterd-Recvd-Size: 3393 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:19 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E129B21D41; Sat, 17 Oct 2020 23:15:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976518; bh=wjTQ5WwxCcfoifS1ViMKtTZCF9uZGnBGRtTwjU53J2I=; h=Date:From:To:Subject:In-Reply-To:From; b=tJjLCV/GBsrhjinJfqBIus3cZEz9QDzRq2efXukO7I6my+SMH5UQJ3TuAIL8pHk91 sFDHMNAqXJtLrjOQIOGJztuypXbBVyVVpJ0pyRIDS10wjFdtPGgKSn0llYEXr8f07r JOLYzBaMRvDgTlmdx1jD8r8os3DcMT0SBYwK3pVA= Date: Sat, 17 Oct 2020 16:15:17 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 31/40] zsmalloc: switch from alloc_vm_area to get_vm_area Message-ID: <20201017231517.RsEnw_-y7%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: zsmalloc: switch from alloc_vm_area to get_vm_area Just manually pre-fault the PTEs using apply_to_page_range. Link: https://lkml.kernel.org/r/20201002122204.1534411-6-hch@lst.de Signed-off-by: Christoph Hellwig Co-developed-by: Minchan Kim Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- mm/zsmalloc.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) --- a/mm/zsmalloc.c~zsmalloc-switch-from-alloc_vm_area-to-get_vm_area +++ a/mm/zsmalloc.c @@ -1122,10 +1122,16 @@ static inline int __zs_cpu_up(struct map */ if (area->vm) return 0; - area->vm = alloc_vm_area(PAGE_SIZE * 2, NULL); + area->vm = get_vm_area(PAGE_SIZE * 2, 0); if (!area->vm) return -ENOMEM; - return 0; + + /* + * Populate ptes in advance to avoid pte allocation with GFP_KERNEL + * in non-preemtible context of zs_map_object. + */ + return apply_to_page_range(&init_mm, (unsigned long)area->vm->addr, + PAGE_SIZE * 2, NULL, NULL); } static inline void __zs_cpu_down(struct mapping_area *area) From patchwork Sat Oct 17 23:15:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843143 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADD8B14B5 for ; Sat, 17 Oct 2020 23:15:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 745E121556 for ; Sat, 17 Oct 2020 23:15:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="Q77/iBd5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 745E121556 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7747C6B0068; Sat, 17 Oct 2020 19:15:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 724E96B0071; Sat, 17 Oct 2020 19:15:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5EE8B6B0073; Sat, 17 Oct 2020 19:15:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id 302956B0068 for ; Sat, 17 Oct 2020 19:15:24 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C88811EE6 for ; Sat, 17 Oct 2020 23:15:23 +0000 (UTC) X-FDA: 77382975726.30.cave57_31101c827229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id A70D5180B3C83 for ; Sat, 17 Oct 2020 23:15:23 +0000 (UTC) X-Spam-Summary: 1,0,0,9709b421db261448,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3354:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6119:6261:6653:6737:7514:7576:7903:9025:9036:9545:9592:10004:11026:11233:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12986:13141:13230:13846:14181:14721:21080:21451:21627:21939:21990:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y8oyehxhznemtgn1wz7q9f4de43opfbaipec8yb7koyhupcipu1ftsq3fesnw.17p3e1577t9r5awpty8tfh56h81zo31r8dn76536uw1gkjjo3ptcnsnffrz17k4.g-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk ,SPF:fp, X-HE-Tag: cave57_31101c827229 X-Filterd-Recvd-Size: 5514 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:23 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 749FB212CC; Sat, 17 Oct 2020 23:15:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976522; bh=sAiTG+S/mBKS2my6tWraPLhSa9rfatLBXJs4lzVUxQI=; h=Date:From:To:Subject:In-Reply-To:From; b=Q77/iBd5qudymV3144AfXEh51ZrQygaksUt+k7ZA6fjdQeeFkGE6DufJGZzAbLYSe Qr1IzGupnmxEOJXt9CnS7GlU5w0tywWIa8wYJ+ktwkXBSujfv2AITeeiUweWlA7URK AQymucRCUf53i8bHyXbKskzRugNZXq6P6Y3Qo50Y= Date: Sat, 17 Oct 2020 16:15:21 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 32/40] drm/i915: use vmap in shmem_pin_map Message-ID: <20201017231521.9DF7wjUfG%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: drm/i915: use vmap in shmem_pin_map shmem_pin_map somewhat awkwardly reimplements vmap using alloc_vm_area and manual pte setup. The only practical difference is that alloc_vm_area prefeaults the vmalloc area PTEs, which doesn't seem to be required here (and could be added to vmap using a flag if actually required). Switch to use vmap, and use vfree to free both the vmalloc mapping and the page array, as well as dropping the references to each page. Link: https://lkml.kernel.org/r/20201002122204.1534411-7-hch@lst.de Signed-off-by: Christoph Hellwig Reviewed-by: Tvrtko Ursulin Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- drivers/gpu/drm/i915/gt/shmem_utils.c | 78 +++++------------------- 1 file changed, 19 insertions(+), 59 deletions(-) --- a/drivers/gpu/drm/i915/gt/shmem_utils.c~drm-i915-use-vmap-in-shmem_pin_map +++ a/drivers/gpu/drm/i915/gt/shmem_utils.c @@ -49,80 +49,40 @@ struct file *shmem_create_from_object(st return file; } -static size_t shmem_npte(struct file *file) -{ - return file->f_mapping->host->i_size >> PAGE_SHIFT; -} - -static void __shmem_unpin_map(struct file *file, void *ptr, size_t n_pte) -{ - unsigned long pfn; - - vunmap(ptr); - - for (pfn = 0; pfn < n_pte; pfn++) { - struct page *page; - - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, - GFP_KERNEL); - if (!WARN_ON(IS_ERR(page))) { - put_page(page); - put_page(page); - } - } -} - void *shmem_pin_map(struct file *file) { - const size_t n_pte = shmem_npte(file); - pte_t *stack[32], **ptes, **mem; - struct vm_struct *area; - unsigned long pfn; - - mem = stack; - if (n_pte > ARRAY_SIZE(stack)) { - mem = kvmalloc_array(n_pte, sizeof(*mem), GFP_KERNEL); - if (!mem) - return NULL; - } - - area = alloc_vm_area(n_pte << PAGE_SHIFT, mem); - if (!area) { - if (mem != stack) - kvfree(mem); + struct page **pages; + size_t n_pages, i; + void *vaddr; + + n_pages = file->f_mapping->host->i_size >> PAGE_SHIFT; + pages = kvmalloc_array(n_pages, sizeof(*pages), GFP_KERNEL); + if (!pages) return NULL; - } - ptes = mem; - for (pfn = 0; pfn < n_pte; pfn++) { - struct page *page; - - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, - GFP_KERNEL); - if (IS_ERR(page)) + for (i = 0; i < n_pages; i++) { + pages[i] = shmem_read_mapping_page_gfp(file->f_mapping, i, + GFP_KERNEL); + if (IS_ERR(pages[i])) goto err_page; - - **ptes++ = mk_pte(page, PAGE_KERNEL); } - if (mem != stack) - kvfree(mem); - + vaddr = vmap(pages, n_pages, VM_MAP_PUT_PAGES, PAGE_KERNEL); + if (!vaddr) + goto err_page; mapping_set_unevictable(file->f_mapping); - return area->addr; - + return vaddr; err_page: - if (mem != stack) - kvfree(mem); - - __shmem_unpin_map(file, area->addr, pfn); + while (--i >= 0) + put_page(pages[i]); + kvfree(pages); return NULL; } void shmem_unpin_map(struct file *file, void *ptr) { mapping_clear_unevictable(file->f_mapping); - __shmem_unpin_map(file, ptr, shmem_npte(file)); + vfree(ptr); } static int __shmem_rw(struct file *file, loff_t off, From patchwork Sat Oct 17 23:15:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843145 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0A4C14B5 for ; Sat, 17 Oct 2020 23:15:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6A05922203 for ; Sat, 17 Oct 2020 23:15:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="g4FQ3wuz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A05922203 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6D9796B0071; Sat, 17 Oct 2020 19:15:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6AF196B0072; Sat, 17 Oct 2020 19:15:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C66D6B0073; Sat, 17 Oct 2020 19:15:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 301716B0071 for ; Sat, 17 Oct 2020 19:15:28 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BECF2362B for ; Sat, 17 Oct 2020 23:15:27 +0000 (UTC) X-FDA: 77382975894.02.jelly41_55152ee27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 9AA52101E2D3A for ; Sat, 17 Oct 2020 23:15:27 +0000 (UTC) X-Spam-Summary: 1,0,0,89ab9a52120cbf3a,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1541:1711:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2693:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3352:3865:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:6737:7514:7576:8957:9025:9545:10004:11026:11658:11914:12043:12048:12294:12296:12297:12517:12519:12555:12679:12986:13069:13311:13357:13846:14181:14384:14721:21080:21451:21627:21939:21990:30054:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y8w19ifdpx5sr463k3xbp5uffqcypa57diu333p88n59d7gw6g4fsf6iyazee.ypktc5arrkmwm6ec9k6ifkxuk84qhobyjh53imfuuioazgyyo87c4cbqq5f5s4k.s-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules :0:0:0,L X-HE-Tag: jelly41_55152ee27229 X-Filterd-Recvd-Size: 3519 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:27 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 8AAC3216C4; Sat, 17 Oct 2020 23:15:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976526; bh=G6bNKV/FicSzR9QS63wElgYsdlKipWPVJtECbNSFAoU=; h=Date:From:To:Subject:In-Reply-To:From; b=g4FQ3wuzB1YKqNWI/pFZasnUJ5/kpfpPblFqKCJq6O51zl0KreoQUnBdShkKVCOtJ 8uttmi2Pa45KXNpzscOKDtiJ3jAU5aaqnh66LflC1QEicf9Ky8Vt0YMFPHv+8cJeai 2cPyQemKK2VMpy3fgtXjTpD3neIHkzZ6n0wfBG8E= Date: Sat, 17 Oct 2020 16:15:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 33/40] drm/i915: stop using kmap in i915_gem_object_map Message-ID: <20201017231525.AHOtu7_aI%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: drm/i915: stop using kmap in i915_gem_object_map kmap for !PageHighmem is just a convoluted way to say page_address, and kunmap is a no-op in that case. Link: https://lkml.kernel.org/r/20201002122204.1534411-8-hch@lst.de Signed-off-by: Christoph Hellwig Reviewed-by: Tvrtko Ursulin Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- drivers/gpu/drm/i915/gem/i915_gem_pages.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c~drm-i915-stop-using-kmap-in-i915_gem_object_map +++ a/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -162,8 +162,6 @@ static void unmap_object(struct drm_i915 { if (is_vmalloc_addr(ptr)) vunmap(ptr); - else - kunmap(kmap_to_page(ptr)); } struct sg_table * @@ -277,11 +275,10 @@ static void *i915_gem_object_map(struct * forever. * * So if the page is beyond the 32b boundary, make an explicit - * vmap. On 64b, this check will be optimised away as we can - * directly kmap any page on the system. + * vmap. */ if (!PageHighMem(page)) - return kmap(page); + return page_address(page); } mem = stack; From patchwork Sat Oct 17 23:15:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843147 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C5A01580 for ; Sat, 17 Oct 2020 23:15:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 06B5E21582 for ; Sat, 17 Oct 2020 23:15:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="HpM9t78L" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06B5E21582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0703F6B0072; Sat, 17 Oct 2020 19:15:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 045CF6B0073; Sat, 17 Oct 2020 19:15:31 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC6426B0074; Sat, 17 Oct 2020 19:15:31 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0213.hostedemail.com [216.40.44.213]) by kanga.kvack.org (Postfix) with ESMTP id C03626B0072 for ; Sat, 17 Oct 2020 19:15:31 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 63ED8181AEF07 for ; Sat, 17 Oct 2020 23:15:31 +0000 (UTC) X-FDA: 77382976062.09.fall73_5c117d727229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id 4BB2F180AD806 for ; Sat, 17 Oct 2020 23:15:31 +0000 (UTC) X-Spam-Summary: 1,0,0,7432d2fcd114f340,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:2:41:69:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1535:1605:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2914:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3865:3866:3867:3868:3870:3871:3872:3873:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4049:4119:4321:4385:4605:5007:6117:6119:6261:6653:6737:7514:7576:7875:8603:8957:9025:9036:9545:9592:10004:11026:11232:11473:11658:11914:12043:12048:12295:12296:12297:12438:12517:12519:12555:12679:12986:13141:13161:13229:13230:13846:21080:21451:21627:21939:21990:30054:30055:30064:30070,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrbjizby5p9g8wyuha5a7hab9tiypidjhe65q8oft388s8gm6ji8hiywjsaz4.djibjogn5a3i4hfhcr5tkr8a6ehppm4bcyxh6kh64un7yig744adryfx585wnd4.e-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesi an:0.5,0 X-HE-Tag: fall73_5c117d727229 X-Filterd-Recvd-Size: 8385 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:30 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 650BC21556; Sat, 17 Oct 2020 23:15:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976530; bh=AZXiCN6XJKeUjNzDLaqqrYCJ3Yx6ZP6Kc2v6SEIv1+k=; h=Date:From:To:Subject:In-Reply-To:From; b=HpM9t78LE51u+aWXBvBXNN6t/xSqfy3bxrsonMt2JJjU52h1/nX6E7PFMXXwbOr8F ELxynoRuE05YjDKE/TisfQHfZEYglYrSeBACz2/hL9t8iMEkMdRzMlJrkyd0Ff4hYf ZWM9ZMtowL/M5wBIl4TVNrjHPefzpSQ4zQuLmpD0= Date: Sat, 17 Oct 2020 16:15:28 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 34/40] drm/i915: use vmap in i915_gem_object_map Message-ID: <20201017231528.h_KjgaynS%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: drm/i915: use vmap in i915_gem_object_map i915_gem_object_map implements fairly low-level vmap functionality in a driver. Split it into two helpers, one for remapping kernel memory which can use vmap, and one for I/O memory that uses vmap_pfn. The only practical difference is that alloc_vm_area prefeaults the vmalloc area PTEs, which doesn't seem to be required here for the kernel memory case (and could be added to vmap using a flag if actually required). Link: https://lkml.kernel.org/r/20201002122204.1534411-9-hch@lst.de Signed-off-by: Christoph Hellwig Reviewed-by: Tvrtko Ursulin Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- drivers/gpu/drm/i915/Kconfig | 1 drivers/gpu/drm/i915/gem/i915_gem_pages.c | 129 +++++++++----------- 2 files changed, 61 insertions(+), 69 deletions(-) --- a/drivers/gpu/drm/i915/gem/i915_gem_pages.c~drm-i915-use-vmap-in-i915_gem_object_map +++ a/drivers/gpu/drm/i915/gem/i915_gem_pages.c @@ -232,34 +232,21 @@ unlock: return err; } -static inline pte_t iomap_pte(resource_size_t base, - dma_addr_t offset, - pgprot_t prot) -{ - return pte_mkspecial(pfn_pte((base + offset) >> PAGE_SHIFT, prot)); -} - /* The 'mapping' part of i915_gem_object_pin_map() below */ -static void *i915_gem_object_map(struct drm_i915_gem_object *obj, - enum i915_map_type type) +static void *i915_gem_object_map_page(struct drm_i915_gem_object *obj, + enum i915_map_type type) { - unsigned long n_pte = obj->base.size >> PAGE_SHIFT; - struct sg_table *sgt = obj->mm.pages; - pte_t *stack[32], **mem; - struct vm_struct *area; + unsigned long n_pages = obj->base.size >> PAGE_SHIFT, i; + struct page *stack[32], **pages = stack, *page; + struct sgt_iter iter; pgprot_t pgprot; + void *vaddr; - if (!i915_gem_object_has_struct_page(obj) && type != I915_MAP_WC) - return NULL; - - if (GEM_WARN_ON(type == I915_MAP_WC && - !static_cpu_has(X86_FEATURE_PAT))) - return NULL; - - /* A single page can always be kmapped */ - if (n_pte == 1 && type == I915_MAP_WB) { - struct page *page = sg_page(sgt->sgl); - + switch (type) { + default: + MISSING_CASE(type); + fallthrough; /* to use PAGE_KERNEL anyway */ + case I915_MAP_WB: /* * On 32b, highmem using a finite set of indirect PTE (i.e. * vmap) to provide virtual mappings of the high pages. @@ -277,30 +264,8 @@ static void *i915_gem_object_map(struct * So if the page is beyond the 32b boundary, make an explicit * vmap. */ - if (!PageHighMem(page)) - return page_address(page); - } - - mem = stack; - if (n_pte > ARRAY_SIZE(stack)) { - /* Too big for stack -- allocate temporary array instead */ - mem = kvmalloc_array(n_pte, sizeof(*mem), GFP_KERNEL); - if (!mem) - return NULL; - } - - area = alloc_vm_area(obj->base.size, mem); - if (!area) { - if (mem != stack) - kvfree(mem); - return NULL; - } - - switch (type) { - default: - MISSING_CASE(type); - fallthrough; /* to use PAGE_KERNEL anyway */ - case I915_MAP_WB: + if (n_pages == 1 && !PageHighMem(sg_page(obj->mm.pages->sgl))) + return page_address(sg_page(obj->mm.pages->sgl)); pgprot = PAGE_KERNEL; break; case I915_MAP_WC: @@ -308,30 +273,50 @@ static void *i915_gem_object_map(struct break; } - if (i915_gem_object_has_struct_page(obj)) { - struct sgt_iter iter; - struct page *page; - pte_t **ptes = mem; - - for_each_sgt_page(page, iter, sgt) - **ptes++ = mk_pte(page, pgprot); - } else { - resource_size_t iomap; - struct sgt_iter iter; - pte_t **ptes = mem; - dma_addr_t addr; + if (n_pages > ARRAY_SIZE(stack)) { + /* Too big for stack -- allocate temporary array instead */ + pages = kvmalloc_array(n_pages, sizeof(*pages), GFP_KERNEL); + if (!pages) + return NULL; + } - iomap = obj->mm.region->iomap.base; - iomap -= obj->mm.region->region.start; + i = 0; + for_each_sgt_page(page, iter, obj->mm.pages) + pages[i++] = page; + vaddr = vmap(pages, n_pages, 0, pgprot); + if (pages != stack) + kvfree(pages); + return vaddr; +} - for_each_sgt_daddr(addr, iter, sgt) - **ptes++ = iomap_pte(iomap, addr, pgprot); - } +static void *i915_gem_object_map_pfn(struct drm_i915_gem_object *obj, + enum i915_map_type type) +{ + resource_size_t iomap = obj->mm.region->iomap.base - + obj->mm.region->region.start; + unsigned long n_pfn = obj->base.size >> PAGE_SHIFT; + unsigned long stack[32], *pfns = stack, i; + struct sgt_iter iter; + dma_addr_t addr; + void *vaddr; - if (mem != stack) - kvfree(mem); + if (type != I915_MAP_WC) + return NULL; + + if (n_pfn > ARRAY_SIZE(stack)) { + /* Too big for stack -- allocate temporary array instead */ + pfns = kvmalloc_array(n_pfn, sizeof(*pfns), GFP_KERNEL); + if (!pfns) + return NULL; + } - return area->addr; + i = 0; + for_each_sgt_daddr(addr, iter, obj->mm.pages) + pfns[i++] = (iomap + addr) >> PAGE_SHIFT; + vaddr = vmap_pfn(pfns, n_pfn, pgprot_writecombine(PAGE_KERNEL_IO)); + if (pfns != stack) + kvfree(pfns); + return vaddr; } /* get, pin, and map the pages of the object into kernel space */ @@ -383,7 +368,13 @@ void *i915_gem_object_pin_map(struct drm } if (!ptr) { - ptr = i915_gem_object_map(obj, type); + if (GEM_WARN_ON(type == I915_MAP_WC && + !static_cpu_has(X86_FEATURE_PAT))) + ptr = NULL; + else if (i915_gem_object_has_struct_page(obj)) + ptr = i915_gem_object_map_page(obj, type); + else + ptr = i915_gem_object_map_pfn(obj, type); if (!ptr) { err = -ENOMEM; goto err_unpin; --- a/drivers/gpu/drm/i915/Kconfig~drm-i915-use-vmap-in-i915_gem_object_map +++ a/drivers/gpu/drm/i915/Kconfig @@ -25,6 +25,7 @@ config DRM_I915 select CRC32 select SND_HDA_I915 if SND_HDA_CORE select CEC_CORE if CEC_NOTIFIER + select VMAP_PFN help Choose this option if you have a system that has "Intel Graphics Media Accelerator" or "HD Graphics" integrated graphics, From patchwork Sat Oct 17 23:15:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843149 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 016D914B5 for ; Sat, 17 Oct 2020 23:15:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B87AC214DB for ; Sat, 17 Oct 2020 23:15:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="BuQ/WwOf" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B87AC214DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE0026B0073; Sat, 17 Oct 2020 19:15:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B900B6B0074; Sat, 17 Oct 2020 19:15:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA62B6B007D; Sat, 17 Oct 2020 19:15:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0095.hostedemail.com [216.40.44.95]) by kanga.kvack.org (Postfix) with ESMTP id 7D2306B0073 for ; Sat, 17 Oct 2020 19:15:35 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 181FA180AD807 for ; Sat, 17 Oct 2020 23:15:35 +0000 (UTC) X-FDA: 77382976230.06.river77_1913e2827229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin06.hostedemail.com (Postfix) with ESMTP id EC8EC101E2D39 for ; Sat, 17 Oct 2020 23:15:34 +0000 (UTC) X-Spam-Summary: 1,0,0,d2cda5c38a4582b6,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1543:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3353:3865:3866:3867:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:4605:5007:6119:6261:6653:6737:7514:7576:9025:9545:9592:10004:11026:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12986:13255:13846:14181:14721:21080:21451:21627:21939:21990:30054:30064:30075,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04y84yewjgmopksdgz5jknb6p1bhbocx9arqgk4yxgnwmpoujmbmj1or4hjbypb.m4q7ehp6ang3bkuwgwap7bsp79ebm7premc1y4wagbpyh1srzm9szg894ii14rn.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_ rules:0: X-HE-Tag: river77_1913e2827229 X-Filterd-Recvd-Size: 4936 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:34 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F285D21527; Sat, 17 Oct 2020 23:15:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976533; bh=8nZml5GP4AdSsnOb2lGUaLPGkA4Fb0vWkBge/ZUcH68=; h=Date:From:To:Subject:In-Reply-To:From; b=BuQ/WwOf2E4UxE8o9uOGymStfYTkpZk/KVwmjlA3R1RUK7DEUqUmFgoOKwlPDzH4S VUt30mRFrsth+6sOS6SqgLm6/qyaTTomGk6Em6E5xxrHMY7+aKW+SY5A71W4xO6G5B OTT+KpckPT1Sqg4/AFcbkq1BZ5x268t+jiDUibco= Date: Sat, 17 Oct 2020 16:15:32 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 35/40] xen/xenbus: use apply_to_page_range directly in xenbus_map_ring_pv Message-ID: <20201017231532.SrxvnQoK2%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: xen/xenbus: use apply_to_page_range directly in xenbus_map_ring_pv Replacing alloc_vm_area with get_vm_area_caller + apply_page_range allows to fill put the phys_addr values directly instead of doing another loop over all addresses. Link: https://lkml.kernel.org/r/20201002122204.1534411-10-hch@lst.de Signed-off-by: Christoph Hellwig Reviewed-by: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- drivers/xen/xenbus/xenbus_client.c | 30 ++++++++++++++------------- 1 file changed, 16 insertions(+), 14 deletions(-) --- a/drivers/xen/xenbus/xenbus_client.c~xen-xenbus-use-apply_to_page_range-directly-in-xenbus_map_ring_pv +++ a/drivers/xen/xenbus/xenbus_client.c @@ -73,16 +73,13 @@ struct map_ring_valloc { struct xenbus_map_node *node; /* Why do we need two arrays? See comment of __xenbus_map_ring */ - union { - unsigned long addrs[XENBUS_MAX_RING_GRANTS]; - pte_t *ptes[XENBUS_MAX_RING_GRANTS]; - }; + unsigned long addrs[XENBUS_MAX_RING_GRANTS]; phys_addr_t phys_addrs[XENBUS_MAX_RING_GRANTS]; struct gnttab_map_grant_ref map[XENBUS_MAX_RING_GRANTS]; struct gnttab_unmap_grant_ref unmap[XENBUS_MAX_RING_GRANTS]; - unsigned int idx; /* HVM only. */ + unsigned int idx; }; static DEFINE_SPINLOCK(xenbus_valloc_lock); @@ -686,6 +683,14 @@ int xenbus_unmap_ring_vfree(struct xenbu EXPORT_SYMBOL_GPL(xenbus_unmap_ring_vfree); #ifdef CONFIG_XEN_PV +static int map_ring_apply(pte_t *pte, unsigned long addr, void *data) +{ + struct map_ring_valloc *info = data; + + info->phys_addrs[info->idx++] = arbitrary_virt_to_machine(pte).maddr; + return 0; +} + static int xenbus_map_ring_pv(struct xenbus_device *dev, struct map_ring_valloc *info, grant_ref_t *gnt_refs, @@ -694,18 +699,15 @@ static int xenbus_map_ring_pv(struct xen { struct xenbus_map_node *node = info->node; struct vm_struct *area; - int err = GNTST_okay; - int i; - bool leaked; + bool leaked = false; + int err = -ENOMEM; - area = alloc_vm_area(XEN_PAGE_SIZE * nr_grefs, info->ptes); + area = get_vm_area(XEN_PAGE_SIZE * nr_grefs, VM_IOREMAP); if (!area) return -ENOMEM; - - for (i = 0; i < nr_grefs; i++) - info->phys_addrs[i] = - arbitrary_virt_to_machine(info->ptes[i]).maddr; - + if (apply_to_page_range(&init_mm, (unsigned long)area->addr, + XEN_PAGE_SIZE * nr_grefs, map_ring_apply, info)) + goto failed; err = __xenbus_map_ring(dev, gnt_refs, nr_grefs, node->handles, info, GNTMAP_host_map | GNTMAP_contains_pte, &leaked); From patchwork Sat Oct 17 23:15:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843151 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7E30E14B5 for ; Sat, 17 Oct 2020 23:15:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 40C3A21582 for ; Sat, 17 Oct 2020 23:15:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="uu7Vh0+Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 40C3A21582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 305C36B0074; Sat, 17 Oct 2020 19:15:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2B6E86B007D; Sat, 17 Oct 2020 19:15:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F39C6B0085; Sat, 17 Oct 2020 19:15:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id E7F2C6B0074 for ; Sat, 17 Oct 2020 19:15:38 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8E537362B for ; Sat, 17 Oct 2020 23:15:38 +0000 (UTC) X-FDA: 77382976356.15.month12_630f58827229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 6ABC31814B0C1 for ; Sat, 17 Oct 2020 23:15:38 +0000 (UTC) X-Spam-Summary: 1,0,0,181b944d9171334c,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3353:3865:3866:3867:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:6737:7514:7576:8660:9025:9545:9592:10004:11026:11658:11914:12043:12048:12291:12296:12297:12438:12517:12519:12555:12679:12986:13148:13230:13846:14096:14181:14721:21080:21451:21627:21939:21990:30012:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrdssxyhsdyqwsuwcfqsy8dbdmkoctg586otqinphz5p3wwr3g3afzuzbcxie.iq9y9pywoe65b4tgrjyibs78uqtyexcpt3kuak1b5s69tqqcmkpp3qykzbd5moj.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:n eutral,C X-HE-Tag: month12_630f58827229 X-Filterd-Recvd-Size: 4289 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:37 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 9282C20EDD; Sat, 17 Oct 2020 23:15:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976537; bh=qAFwAkNzaT30TPOdAwEkueK0qN8VetTiH9ic1zQT4ZA=; h=Date:From:To:Subject:In-Reply-To:From; b=uu7Vh0+QQcC63pan52uxirhz2MzIn08s4zfnHNqc4ptFsYEdIX+uzxJcnTQIvC7I4 rp1EhKCjG4zIFsratGnhq1D0bQWQhqdJfb4bl0/948f2iQTLK3DhHc9FScJYBsZ6rw uo2hXKOj3b12OjUTl5FFZPXSfUcafeAmZAhi1wYY= Date: Sat, 17 Oct 2020 16:15:36 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 36/40] x86/xen: open code alloc_vm_area in arch_gnttab_valloc Message-ID: <20201017231536.fECPtF46h%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: x86/xen: open code alloc_vm_area in arch_gnttab_valloc Replace the last call to alloc_vm_area with an open coded version using an iterator in struct gnttab_vm_area instead of the triple indirection magic in alloc_vm_area. Link: https://lkml.kernel.org/r/20201002122204.1534411-11-hch@lst.de Signed-off-by: Christoph Hellwig Reviewed-by: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- arch/x86/xen/grant-table.c | 27 ++++++++++++++++++++------- 1 file changed, 20 insertions(+), 7 deletions(-) --- a/arch/x86/xen/grant-table.c~x86-xen-open-code-alloc_vm_area-in-arch_gnttab_valloc +++ a/arch/x86/xen/grant-table.c @@ -25,6 +25,7 @@ static struct gnttab_vm_area { struct vm_struct *area; pte_t **ptes; + int idx; } gnttab_shared_vm_area, gnttab_status_vm_area; int arch_gnttab_map_shared(unsigned long *frames, unsigned long nr_gframes, @@ -90,19 +91,31 @@ void arch_gnttab_unmap(void *shared, uns } } +static int gnttab_apply(pte_t *pte, unsigned long addr, void *data) +{ + struct gnttab_vm_area *area = data; + + area->ptes[area->idx++] = pte; + return 0; +} + static int arch_gnttab_valloc(struct gnttab_vm_area *area, unsigned nr_frames) { area->ptes = kmalloc_array(nr_frames, sizeof(*area->ptes), GFP_KERNEL); if (area->ptes == NULL) return -ENOMEM; - - area->area = alloc_vm_area(PAGE_SIZE * nr_frames, area->ptes); - if (area->area == NULL) { - kfree(area->ptes); - return -ENOMEM; - } - + area->area = get_vm_area(PAGE_SIZE * nr_frames, VM_IOREMAP); + if (!area->area) + goto out_free_ptes; + if (apply_to_page_range(&init_mm, (unsigned long)area->area->addr, + PAGE_SIZE * nr_frames, gnttab_apply, area)) + goto out_free_vm_area; return 0; +out_free_vm_area: + free_vm_area(area->area); +out_free_ptes: + kfree(area->ptes); + return -ENOMEM; } static void arch_gnttab_vfree(struct gnttab_vm_area *area) From patchwork Sat Oct 17 23:15:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843153 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B52E14B5 for ; Sat, 17 Oct 2020 23:15:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 20D9621655 for ; Sat, 17 Oct 2020 23:15:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="EcfRqKbm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 20D9621655 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0701D6B0068; Sat, 17 Oct 2020 19:15:43 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 045386B007D; Sat, 17 Oct 2020 19:15:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EC4996B0085; Sat, 17 Oct 2020 19:15:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id C15346B0068 for ; Sat, 17 Oct 2020 19:15:42 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6F53D1EE6 for ; Sat, 17 Oct 2020 23:15:42 +0000 (UTC) X-FDA: 77382976524.03.dime52_0e080b927229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id 5260528A4E8 for ; Sat, 17 Oct 2020 23:15:42 +0000 (UTC) X-Spam-Summary: 1,0,0,dc3a4b5fd0240586,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:966:967:968:973:988:989:1260:1345:1359:1381:1431:1437:1535:1543:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2563:2682:2685:2859:2899:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3167:3355:3865:3866:3867:3868:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:4605:5007:6261:6653:6737:7514:7576:7875:8603:9025:9545:9592:10004:11026:11232:11473:11658:11914:12043:12048:12296:12297:12438:12517:12519:12555:12679:12683:12986:13846:14096:14181:14721:21080:21324:21451:21627:21939:21990:30003:30054:30064,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yfzfbjxyd6uu7cwgjxmg6tr69b5ypzwuyhk1dr8g7cbkouom3hz4d9idzhmpd.nhh8jmfjmi9s3gts5475cgzekodkckecos3j5z3kguno7d5yhne8d7nqpgqg8zi.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCach e:0,MSF: X-HE-Tag: dime52_0e080b927229 X-Filterd-Recvd-Size: 5380 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:41 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5247322202; Sat, 17 Oct 2020 23:15:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976541; bh=b/g7PZWr2XIH7jw8iEs5veTtxGNAQJ6dqDkXNF/nooY=; h=Date:From:To:Subject:In-Reply-To:From; b=EcfRqKbm7EpxHs1yM8zpICU2Tg0AE46f0EcOl5GMoKPqD2YlKQLmrvByucXgOMFZm 1fjUVPkafv7hvRQJ+NW9B15zSu+eaB+tuN9pMDy807e1SahtevW1GVmAtvDWXF/UxG oBESTxyh/nXkRq5am3Hc0oVe6Xt7J7pjkx4FkNv8= Date: Sat, 17 Oct 2020 16:15:39 -0700 From: Andrew Morton To: akpm@linux-foundation.org, boris.ostrovsky@oracle.com, chris@chris-wilson.co.uk, hch@lst.de, jani.nikula@linux.intel.com, jgross@suse.com, joonas.lahtinen@linux.intel.com, linux-mm@kvack.org, matthew.auld@intel.com, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, peterz@infradead.org, rodrigo.vivi@intel.com, sstabellini@kernel.org, torvalds@linux-foundation.org, tvrtko.ursulin@intel.com, urezki@gmail.com, willy@infradead.org Subject: [patch 37/40] mm: remove alloc_vm_area Message-ID: <20201017231539.rqXa9tuF3%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm: remove alloc_vm_area All users are gone now. Link: https://lkml.kernel.org/r/20201002122204.1534411-12-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Boris Ostrovsky Cc: Chris Wilson Cc: Jani Nikula Cc: Joonas Lahtinen Cc: Juergen Gross Cc: Matthew Auld Cc: "Matthew Wilcox (Oracle)" Cc: Minchan Kim Cc: Nitin Gupta Cc: Peter Zijlstra Cc: Rodrigo Vivi Cc: Stefano Stabellini Cc: Tvrtko Ursulin Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- include/linux/vmalloc.h | 5 --- mm/nommu.c | 7 ----- mm/vmalloc.c | 48 -------------------------------------- 3 files changed, 1 insertion(+), 59 deletions(-) --- a/include/linux/vmalloc.h~mm-remove-alloc_vm_area +++ a/include/linux/vmalloc.h @@ -169,6 +169,7 @@ extern struct vm_struct *__get_vm_area_c unsigned long flags, unsigned long start, unsigned long end, const void *caller); +void free_vm_area(struct vm_struct *area); extern struct vm_struct *remove_vm_area(const void *addr); extern struct vm_struct *find_vm_area(const void *addr); @@ -204,10 +205,6 @@ static inline void set_vm_flush_reset_pe } #endif -/* Allocate/destroy a 'vmalloc' VM area. */ -extern struct vm_struct *alloc_vm_area(size_t size, pte_t **ptes); -extern void free_vm_area(struct vm_struct *area); - /* for /dev/kmem */ extern long vread(char *buf, char *addr, unsigned long count); extern long vwrite(char *buf, char *addr, unsigned long count); --- a/mm/nommu.c~mm-remove-alloc_vm_area +++ a/mm/nommu.c @@ -354,13 +354,6 @@ void vm_unmap_aliases(void) } EXPORT_SYMBOL_GPL(vm_unmap_aliases); -struct vm_struct *alloc_vm_area(size_t size, pte_t **ptes) -{ - BUG(); - return NULL; -} -EXPORT_SYMBOL_GPL(alloc_vm_area); - void free_vm_area(struct vm_struct *area) { BUG(); --- a/mm/vmalloc.c~mm-remove-alloc_vm_area +++ a/mm/vmalloc.c @@ -3083,54 +3083,6 @@ int remap_vmalloc_range(struct vm_area_s } EXPORT_SYMBOL(remap_vmalloc_range); -static int f(pte_t *pte, unsigned long addr, void *data) -{ - pte_t ***p = data; - - if (p) { - *(*p) = pte; - (*p)++; - } - return 0; -} - -/** - * alloc_vm_area - allocate a range of kernel address space - * @size: size of the area - * @ptes: returns the PTEs for the address space - * - * Returns: NULL on failure, vm_struct on success - * - * This function reserves a range of kernel address space, and - * allocates pagetables to map that range. No actual mappings - * are created. - * - * If @ptes is non-NULL, pointers to the PTEs (in init_mm) - * allocated for the VM area are returned. - */ -struct vm_struct *alloc_vm_area(size_t size, pte_t **ptes) -{ - struct vm_struct *area; - - area = get_vm_area_caller(size, VM_IOREMAP, - __builtin_return_address(0)); - if (area == NULL) - return NULL; - - /* - * This ensures that page tables are constructed for this region - * of kernel virtual address space and mapped into init_mm. - */ - if (apply_to_page_range(&init_mm, (unsigned long)area->addr, - size, f, ptes ? &ptes : NULL)) { - free_vm_area(area); - return NULL; - } - - return area; -} -EXPORT_SYMBOL_GPL(alloc_vm_area); - void free_vm_area(struct vm_struct *area) { struct vm_struct *ret; From patchwork Sat Oct 17 23:15:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843155 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0EE71580 for ; Sat, 17 Oct 2020 23:15:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 606DB214DB for ; Sat, 17 Oct 2020 23:15:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="qX6HAqmB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 606DB214DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 32B4B6B0078; Sat, 17 Oct 2020 19:15:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2E0256B007B; Sat, 17 Oct 2020 19:15:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CB676B007D; Sat, 17 Oct 2020 19:15:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id E41B96B0078 for ; Sat, 17 Oct 2020 19:15:45 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 92AE6180AD807 for ; Sat, 17 Oct 2020 23:15:45 +0000 (UTC) X-FDA: 77382976650.30.farm75_5a04afc27229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 70CEA180B3C83 for ; Sat, 17 Oct 2020 23:15:45 +0000 (UTC) X-Spam-Summary: 1,0,0,9235075fe66b63bd,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:69:355:379:800:960:966:967:973:988:989:1260:1345:1359:1381:1431:1437:1534:1542:1711:1730:1747:1777:1792:2196:2199:2393:2525:2559:2564:2682:2685:2734:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3354:3865:3866:3867:3868:3870:3871:3872:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:4385:5007:6261:6653:7514:7576:9025:9036:9545:10004:11026:11473:11658:11914:12043:12048:12296:12297:12517:12519:12555:12679:12986:13161:13229:13846:14093:14096:14181:14721:21080:21451:21627:21939:30054,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrz7fmdp3uno1kwd56eao71ouzoopz9gfd89x8951ep4tfhht9ix7gt3fsjxk.aytkfrmy93uyg98tekyh4cynkgjt44k74pwyd1mokk9b68iskar5iewis4tc6rx.n-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_r ules:0:0 X-HE-Tag: farm75_5a04afc27229 X-Filterd-Recvd-Size: 4288 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:44 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 17D102158C; Sat, 17 Oct 2020 23:15:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976544; bh=BnAY0myCBCV3/m0EYMW5Y4ytQI+GeDCg4/z7VjOt1EU=; h=Date:From:To:Subject:In-Reply-To:From; b=qX6HAqmBrdy5lS7SWWm4jjAq3ScsRsXZuSi9fkMR1uRhvYoTU04MLMMnN24YWySRF Xf4iJvW572wnQMNZ56LX94T/0qhq2DtSNgu2cN5kCGKJRclk8xbxj0ue1sL484guoL sm5Br6ooADGCSM2QhyH9sAoLr3sFyQgMOh1Z4tQ4= Date: Sat, 17 Oct 2020 16:15:43 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, urezki@gmail.com Subject: [patch 38/40] mm: cleanup the gfp_mask handling in __vmalloc_area_node Message-ID: <20201017231543.A41g17t8J%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm: cleanup the gfp_mask handling in __vmalloc_area_node Patch series "two small vmalloc cleanups". This patch (of 2): __vmalloc_area_node currently has four different gfp_t variables to just express this simple logic: - use the passed in mask, plus __GFP_NOWARN and __GFP_HIGHMEM (if suitable) for the underlying page allocation - use just the reclaim flags from the passed in mask plus __GFP_ZERO for allocating the page array Simplify this down to just use the pre-existing nested_gfp as-is for the page array allocation, and just the passed in gfp_mask for the page allocation, after conditionally ORing __GFP_HIGHMEM into it. This also makes the allocation warning a little more correct. Also initialize two variables at the time of declaration while touching this area. Link: https://lkml.kernel.org/r/20201002124035.1539300-1-hch@lst.de Link: https://lkml.kernel.org/r/20201002124035.1539300-2-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- mm/vmalloc.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) --- a/mm/vmalloc.c~mm-cleanup-the-gfp_mask-handling-in-__vmalloc_area_node +++ a/mm/vmalloc.c @@ -2461,21 +2461,19 @@ EXPORT_SYMBOL_GPL(vmap_pfn); static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, pgprot_t prot, int node) { - struct page **pages; - unsigned int nr_pages, array_size, i; const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | __GFP_ZERO; - const gfp_t alloc_mask = gfp_mask | __GFP_NOWARN; - const gfp_t highmem_mask = (gfp_mask & (GFP_DMA | GFP_DMA32)) ? - 0 : - __GFP_HIGHMEM; + unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT; + unsigned int array_size = nr_pages * sizeof(struct page *), i; + struct page **pages; - nr_pages = get_vm_area_size(area) >> PAGE_SHIFT; - array_size = (nr_pages * sizeof(struct page *)); + gfp_mask |= __GFP_NOWARN; + if (!(gfp_mask & (GFP_DMA | GFP_DMA32))) + gfp_mask |= __GFP_HIGHMEM; /* Please note that the recursion is strictly bounded. */ if (array_size > PAGE_SIZE) { - pages = __vmalloc_node(array_size, 1, nested_gfp|highmem_mask, - node, area->caller); + pages = __vmalloc_node(array_size, 1, nested_gfp, node, + area->caller); } else { pages = kmalloc_node(array_size, nested_gfp, node); } @@ -2493,9 +2491,9 @@ static void *__vmalloc_area_node(struct struct page *page; if (node == NUMA_NO_NODE) - page = alloc_page(alloc_mask|highmem_mask); + page = alloc_page(gfp_mask); else - page = alloc_pages_node(node, alloc_mask|highmem_mask, 0); + page = alloc_pages_node(node, gfp_mask, 0); if (unlikely(!page)) { /* Successfully allocated i pages, free them in __vfree() */ From patchwork Sat Oct 17 23:15:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843157 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8EF0414B5 for ; Sat, 17 Oct 2020 23:15:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5844C21582 for ; Sat, 17 Oct 2020 23:15:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="BzvEL7k1" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5844C21582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2E0176B0070; Sat, 17 Oct 2020 19:15:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 28E936B007B; Sat, 17 Oct 2020 19:15:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A7D46B007D; Sat, 17 Oct 2020 19:15:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0109.hostedemail.com [216.40.44.109]) by kanga.kvack.org (Postfix) with ESMTP id E2FE66B0070 for ; Sat, 17 Oct 2020 19:15:48 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 87545181AEF07 for ; Sat, 17 Oct 2020 23:15:48 +0000 (UTC) X-FDA: 77382976776.17.dad79_1201ff527229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin17.hostedemail.com (Postfix) with ESMTP id 708E5180D0180 for ; Sat, 17 Oct 2020 23:15:48 +0000 (UTC) X-Spam-Summary: 1,0,0,7eae8ffc34c911a3,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:966:967:968:973:988:989:1260:1345:1359:1381:1431:1437:1534:1540:1711:1714:1730:1747:1777:1792:2194:2196:2199:2200:2393:2525:2538:2553:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3351:3865:3866:3870:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4250:4321:4385:5007:6261:6653:7514:7576:7903:9025:9545:10004:11658:11914:12043:12048:12297:12517:12519:12555:12679:12986:13069:13311:13357:13846:14181:14384:14721:21080:21451:21627:21939:30054:30067:30090,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04yrhrmeynao1pneaigiotfsudb8iocefuch5szz4uoni5xf3i3rtsmrw611bwh.1gjsuspig5r4aafc93sypw6jo4pedcnghhxjupcm8qqobz3mbjpk1bxdpyo51ut.k-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules: 0:0:0,LF X-HE-Tag: dad79_1201ff527229 X-Filterd-Recvd-Size: 2204 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:47 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1A4BE21655; Sat, 17 Oct 2020 23:15:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976547; bh=ai6/wwi6ZB6vRfjnPSuyHr3k6uaFaU2w0hKOQtRxEvE=; h=Date:From:To:Subject:In-Reply-To:From; b=BzvEL7k10+paYEqGTvkU6HpTXXkNKGQxiCiV2E9C44jHo1jcgLyoSzZaHngXDvABl Z4Pr8uyNr/mxU6jejpPRqpQ2vl3Bg8GBbRvSNJMLLqGPnS1xqQrJUHwo458jzFZuas uSL22Rr40RXJVB8lm3Gc7HaLfLEeyy/hSUtaxQ/E= Date: Sat, 17 Oct 2020 16:15:46 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, urezki@gmail.com Subject: [patch 39/40] mm: remove the filename in the top of file comment in vmalloc.c Message-ID: <20201017231546.XUjfSDW_s%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm: remove the filename in the top of file comment in vmalloc.c No point in having the filename inside the file. Link: https://lkml.kernel.org/r/20201002124035.1539300-3-hch@lst.de Signed-off-by: Christoph Hellwig Cc: Uladzislau Rezki (Sony) Signed-off-by: Andrew Morton --- mm/vmalloc.c | 2 -- 1 file changed, 2 deletions(-) --- a/mm/vmalloc.c~mm-remove-the-filename-in-the-top-of-file-comment-in-vmallocc +++ a/mm/vmalloc.c @@ -1,7 +1,5 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * linux/mm/vmalloc.c - * * Copyright (C) 1993 Linus Torvalds * Support of BIGMEM added by Gerhard Wichert, Siemens AG, July 1999 * SMP-safe vmalloc/vfree/ioremap, Tigran Aivazian , May 2000 From patchwork Sat Oct 17 23:15:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 11843159 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 834BD1580 for ; Sat, 17 Oct 2020 23:15:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 49B9E21582 for ; Sat, 17 Oct 2020 23:15:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="0vJowV5Q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 49B9E21582 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 32D436B007B; Sat, 17 Oct 2020 19:15:52 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2B5736B007D; Sat, 17 Oct 2020 19:15:52 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A4AE6B007E; Sat, 17 Oct 2020 19:15:52 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id E2D1C6B007B for ; Sat, 17 Oct 2020 19:15:51 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8A2A38249980 for ; Sat, 17 Oct 2020 23:15:51 +0000 (UTC) X-FDA: 77382976902.02.kite00_3f102c127229 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin02.hostedemail.com (Postfix) with ESMTP id 6A7B410096DF7 for ; Sat, 17 Oct 2020 23:15:51 +0000 (UTC) X-Spam-Summary: 1,0,0,7d9e79215cdd4a5d,d41d8cd98f00b204,akpm@linux-foundation.org,,RULES_HIT:41:355:379:800:960:967:968:973:988:989:1260:1345:1359:1381:1431:1437:1534:1539:1566:1711:1714:1730:1747:1777:1792:2393:2525:2559:2563:2682:2685:2859:2902:2933:2937:2939:2942:2945:2947:2951:2954:3022:3138:3139:3140:3141:3142:3865:3870:3871:3874:3934:3936:3938:3941:3944:3947:3950:3953:3956:3959:4321:5007:6261:6653:7576:9025:9391:9545:10004:11658:11914:12043:12048:12297:12517:12519:12555:12679:12986:13069:13311:13357:13846:14181:14384:14721:14777:21080:21433:21451:21627:21819:21939:30054,0,RBL:198.145.29.99:@linux-foundation.org:.lbl8.mailshell.net-62.2.0.100 64.100.201.201;04ygbx6po1ww1zggjtr8mqbw7yd8mycpsyoxt55ckd3z15keafg1oiq11itafnx.wdowprmofk858wyymkshui5witnrzpjotdmfjoi9istpbfcuegaympseri3j7tm.o-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: kite00_3f102c127229 X-Filterd-Recvd-Size: 2134 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Sat, 17 Oct 2020 23:15:51 +0000 (UTC) Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 12CD5214DB; Sat, 17 Oct 2020 23:15:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1602976550; bh=WW+QIOioUOhg8gLSnK8E45pkFkJmr2Apo31iYGEGcSc=; h=Date:From:To:Subject:In-Reply-To:From; b=0vJowV5QdYpPx/7YZP4Vegl3E2BHqrFif+NkHXVJRB3aEQmiz9XiLVtWCArv53LX9 gKJuNH/yMyKvOCIT3BXABemhno8ez4qfwqohl23ak7pots2B7dHky+XQfUOtF6m+sT ZLozsLpBWKroENTDvNW5mH5Ap1/eGU8dg3RdoYas= Date: Sat, 17 Oct 2020 16:15:49 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, tiantao6@hisilicon.com, torvalds@linux-foundation.org Subject: [patch 40/40] mm: remove duplicate include statement in mmu.c Message-ID: <20201017231549.ebZiVJavh%akpm@linux-foundation.org> In-Reply-To: <20201017161314.88890b87fae7446ccc13c902@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Tian Tao Subject: mm: remove duplicate include statement in mmu.c asm/sections.h is included more than once, Remove the one that isn't necessary. Link: https://lkml.kernel.org/r/1600088607-17327-1-git-send-email-tiantao6@hisilicon.com Signed-off-by: Tian Tao Reviewed-by: Mike Rapoport Signed-off-by: Andrew Morton --- arch/arm/mm/mmu.c | 1 - 1 file changed, 1 deletion(-) --- a/arch/arm/mm/mmu.c~mm-remove-duplicate-include-statement-in-mmuc +++ a/arch/arm/mm/mmu.c @@ -17,7 +17,6 @@ #include #include -#include #include #include #include