From patchwork Thu Jul 1 01:47:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5FF3C11F66 for ; Thu, 1 Jul 2021 01:47:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7A32661474 for ; Thu, 1 Jul 2021 01:47:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A32661474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3CDA8D01D4; Wed, 30 Jun 2021 21:47:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9EE9F8D01D0; Wed, 30 Jun 2021 21:47:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8408E8D01D4; Wed, 30 Jun 2021 21:47:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0141.hostedemail.com [216.40.44.141]) by kanga.kvack.org (Postfix) with ESMTP id 508C18D01D0 for ; Wed, 30 Jun 2021 21:47:05 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8D67625F5A for ; Thu, 1 Jul 2021 01:47:03 +0000 (UTC) X-FDA: 78312330726.13.F5A890A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id 25402F000123 for ; Thu, 1 Jul 2021 01:47:03 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1D72361466; Thu, 1 Jul 2021 01:47:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104022; bh=gdKVCw17VzX7/4YBu42nYIijFQlsmvL8FqmnKEDy848=; h=Date:From:To:Subject:In-Reply-To:From; b=YEoCaswA3CHQ/a91+jZ4Wcid6FOSHRDmS/C/2Cm6XPcHPXOKhOuJgkQQ0ku8ggkPI seOWwTv/zV60JcA8/0jCQiFaTCHKLd2QfpMXi3FO1v0GFXChEXqRn09trpCuI2ue6y JpYP9AhGuvMIJc8orzA4WjbwD1ZSXcc46OIEV+wQ= Date: Wed, 30 Jun 2021 18:47:00 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 001/192] mm: memory_hotplug: factor out bootmem core functions to bootmem_info.c Message-ID: <20210701014700.yBaTd77bR%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=YEoCaswA; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 9jyo1inuj33gfiwysu4xjt4et6wu7yy1 X-Rspamd-Queue-Id: 25402F000123 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104023-136035 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memory_hotplug: factor out bootmem core functions to bootmem_info.c Patch series "Free some vmemmap pages of HugeTLB page", v23. This patch series will free some vmemmap pages(struct page structures) associated with each HugeTLB page when preallocated to save memory. In order to reduce the difficulty of the first version of code review. In this version, we disable PMD/huge page mapping of vmemmap if this feature was enabled. This acutely eliminates a bunch of the complex code doing page table manipulation. When this patch series is solid, we cam add the code of vmemmap page table manipulation in the future. The struct page structures (page structs) are used to describe a physical page frame. By default, there is an one-to-one mapping from a page frame to it's corresponding page struct. The HugeTLB pages consist of multiple base page size pages and is supported by many architectures. See hugetlbpage.rst in the Documentation directory for more details. On the x86 architecture, HugeTLB pages of size 2MB and 1GB are currently supported. Since the base page size on x86 is 4KB, a 2MB HugeTLB page consists of 512 base pages and a 1GB HugeTLB page consists of 4096 base pages. For each base page, there is a corresponding page struct. Within the HugeTLB subsystem, only the first 4 page structs are used to contain unique information about a HugeTLB page. HUGETLB_CGROUP_MIN_ORDER provides this upper limit. The only 'useful' information in the remaining page structs is the compound_head field, and this field is the same for all tail pages. By removing redundant page structs for HugeTLB pages, memory can returned to the buddy allocator for other uses. When the system boot up, every 2M HugeTLB has 512 struct page structs which size is 8 pages(sizeof(struct page) * 512 / PAGE_SIZE). HugeTLB struct pages(8 pages) page frame(8 pages) +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ | | | 0 | -------------> | 0 | | | +-----------+ +-----------+ | | | 1 | -------------> | 1 | | | +-----------+ +-----------+ | | | 2 | -------------> | 2 | | | +-----------+ +-----------+ | | | 3 | -------------> | 3 | | | +-----------+ +-----------+ | | | 4 | -------------> | 4 | | 2MB | +-----------+ +-----------+ | | | 5 | -------------> | 5 | | | +-----------+ +-----------+ | | | 6 | -------------> | 6 | | | +-----------+ +-----------+ | | | 7 | -------------> | 7 | | | +-----------+ +-----------+ | | | | | | +-----------+ The value of page->compound_head is the same for all tail pages. The first page of page structs (page 0) associated with the HugeTLB page contains the 4 page structs necessary to describe the HugeTLB. The only use of the remaining pages of page structs (page 1 to page 7) is to point to page->compound_head. Therefore, we can remap pages 2 to 7 to page 1. Only 2 pages of page structs will be used for each HugeTLB page. This will allow us to free the remaining 6 pages to the buddy allocator. Here is how things look after remapping. HugeTLB struct pages(8 pages) page frame(8 pages) +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ | | | 0 | -------------> | 0 | | | +-----------+ +-----------+ | | | 1 | -------------> | 1 | | | +-----------+ +-----------+ | | | 2 | ----------------^ ^ ^ ^ ^ ^ | | +-----------+ | | | | | | | | 3 | ------------------+ | | | | | | +-----------+ | | | | | | | 4 | --------------------+ | | | | 2MB | +-----------+ | | | | | | 5 | ----------------------+ | | | | +-----------+ | | | | | 6 | ------------------------+ | | | +-----------+ | | | | 7 | --------------------------+ | | +-----------+ | | | | | | +-----------+ When a HugeTLB is freed to the buddy system, we should allocate 6 pages for vmemmap pages and restore the previous mapping relationship. Apart from 2MB HugeTLB page, we also have 1GB HugeTLB page. It is similar to the 2MB HugeTLB page. We also can use this approach to free the vmemmap pages. In this case, for the 1GB HugeTLB page, we can save 4094 pages. This is a very substantial gain. On our server, run some SPDK/QEMU applications which will use 1024GB HugeTLB page. With this feature enabled, we can save ~16GB (1G hugepage)/~12GB (2MB hugepage) memory. Because there are vmemmap page tables reconstruction on the freeing/allocating path, it increases some overhead. Here are some overhead analysis. 1) Allocating 10240 2MB HugeTLB pages. a) With this patch series applied: # time echo 10240 > /proc/sys/vm/nr_hugepages real 0m0.166s user 0m0.000s sys 0m0.166s # bpftrace -e 'kprobe:alloc_fresh_huge_page { @start[tid] = nsecs; } kretprobe:alloc_fresh_huge_page /@start[tid]/ { @latency = hist(nsecs - @start[tid]); delete(@start[tid]); }' Attaching 2 probes... @latency: [8K, 16K) 5476 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [16K, 32K) 4760 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ | [32K, 64K) 4 | | b) Without this patch series: # time echo 10240 > /proc/sys/vm/nr_hugepages real 0m0.067s user 0m0.000s sys 0m0.067s # bpftrace -e 'kprobe:alloc_fresh_huge_page { @start[tid] = nsecs; } kretprobe:alloc_fresh_huge_page /@start[tid]/ { @latency = hist(nsecs - @start[tid]); delete(@start[tid]); }' Attaching 2 probes... @latency: [4K, 8K) 10147 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [8K, 16K) 93 | | Summarize: this feature is about ~2x slower than before. 2) Freeing 10240 2MB HugeTLB pages. a) With this patch series applied: # time echo 0 > /proc/sys/vm/nr_hugepages real 0m0.213s user 0m0.000s sys 0m0.213s # bpftrace -e 'kprobe:free_pool_huge_page { @start[tid] = nsecs; } kretprobe:free_pool_huge_page /@start[tid]/ { @latency = hist(nsecs - @start[tid]); delete(@start[tid]); }' Attaching 2 probes... @latency: [8K, 16K) 6 | | [16K, 32K) 10227 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [32K, 64K) 7 | | b) Without this patch series: # time echo 0 > /proc/sys/vm/nr_hugepages real 0m0.081s user 0m0.000s sys 0m0.081s # bpftrace -e 'kprobe:free_pool_huge_page { @start[tid] = nsecs; } kretprobe:free_pool_huge_page /@start[tid]/ { @latency = hist(nsecs - @start[tid]); delete(@start[tid]); }' Attaching 2 probes... @latency: [4K, 8K) 6805 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@| [8K, 16K) 3427 |@@@@@@@@@@@@@@@@@@@@@@@@@@ | [16K, 32K) 8 | | Summary: The overhead of __free_hugepage is about ~2-3x slower than before. Although the overhead has increased, the overhead is not significant. Like Mike said, "However, remember that the majority of use cases create HugeTLB pages at or shortly after boot time and add them to the pool. So, additional overhead is at pool creation time. There is no change to 'normal run time' operations of getting a page from or returning a page to the pool (think page fault/unmap)". Despite the overhead and in addition to the memory gains from this series. The following data is obtained by Joao Martins. Very thanks to his effort. There's an additional benefit which is page (un)pinners will see an improvement and Joao presumes because there are fewer memmap pages and thus the tail/head pages are staying in cache more often. Out of the box Joao saw (when comparing linux-next against linux-next + this series) with gup_test and pinning a 16G HugeTLB file (with 1G pages): get_user_pages(): ~32k -> ~9k unpin_user_pages(): ~75k -> ~70k Usually any tight loop fetching compound_head(), or reading tail pages data (e.g. compound_head) benefit a lot. There's some unpinning inefficiencies Joao was fixing[2], but with that in added it shows even more: unpin_user_pages(): ~27k -> ~3.8k [1] https://lore.kernel.org/linux-mm/20210409205254.242291-1-mike.kravetz@oracle.com/ [2] https://lore.kernel.org/linux-mm/20210204202500.26474-1-joao.m.martins@oracle.com/ This patch (of 9): Move bootmem info registration common API to individual bootmem_info.c. And we will use {get,put}_page_bootmem() to initialize the page for the vmemmap pages or free the vmemmap pages to buddy in the later patch. So move them out of CONFIG_MEMORY_HOTPLUG_SPARSE. This is just code movement without any functional change. Link: https://lkml.kernel.org/r/20210510030027.56044-1-songmuchun@bytedance.com Link: https://lkml.kernel.org/r/20210510030027.56044-2-songmuchun@bytedance.com Signed-off-by: Muchun Song Acked-by: Mike Kravetz Reviewed-by: Oscar Salvador Reviewed-by: David Hildenbrand Reviewed-by: Miaohe Lin Tested-by: Chen Huang Tested-by: Bodeddula Balasubramaniam Cc: Jonathan Corbet Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: x86@kernel.org Cc: "H. Peter Anvin" Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: Alexander Viro Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Randy Dunlap Cc: Oliver Neukum Cc: Anshuman Khandual Cc: Joerg Roedel Cc: Mina Almasry Cc: David Rientjes Cc: Matthew Wilcox Cc: Michal Hocko Cc: Barry Song Cc: HORIGUCHI NAOYA Cc: Joao Martins Cc: Xiongchun Duan Cc: Balbir Singh Signed-off-by: Andrew Morton --- arch/sparc/mm/init_64.c | 1 arch/x86/mm/init_64.c | 3 include/linux/bootmem_info.h | 40 +++++++++ include/linux/memory_hotplug.h | 27 ------ mm/Makefile | 1 mm/bootmem_info.c | 127 +++++++++++++++++++++++++++++++ mm/memory_hotplug.c | 116 ---------------------------- mm/sparse.c | 1 8 files changed, 172 insertions(+), 144 deletions(-) --- a/arch/sparc/mm/init_64.c~mm-memory_hotplug-factor-out-bootmem-core-functions-to-bootmem_infoc +++ a/arch/sparc/mm/init_64.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include --- a/arch/x86/mm/init_64.c~mm-memory_hotplug-factor-out-bootmem-core-functions-to-bootmem_infoc +++ a/arch/x86/mm/init_64.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -1623,7 +1624,7 @@ int __meminit vmemmap_populate(unsigned return err; } -#if defined(CONFIG_MEMORY_HOTPLUG_SPARSE) && defined(CONFIG_HAVE_BOOTMEM_INFO_NODE) +#ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE void register_page_bootmem_memmap(unsigned long section_nr, struct page *start_page, unsigned long nr_pages) { --- /dev/null +++ a/include/linux/bootmem_info.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __LINUX_BOOTMEM_INFO_H +#define __LINUX_BOOTMEM_INFO_H + +#include + +/* + * Types for free bootmem stored in page->lru.next. These have to be in + * some random range in unsigned long space for debugging purposes. + */ +enum { + MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE = 12, + SECTION_INFO = MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE, + MIX_SECTION_INFO, + NODE_INFO, + MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE = NODE_INFO, +}; + +#ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE +void __init register_page_bootmem_info_node(struct pglist_data *pgdat); + +void get_page_bootmem(unsigned long info, struct page *page, + unsigned long type); +void put_page_bootmem(struct page *page); +#else +static inline void register_page_bootmem_info_node(struct pglist_data *pgdat) +{ +} + +static inline void put_page_bootmem(struct page *page) +{ +} + +static inline void get_page_bootmem(unsigned long info, struct page *page, + unsigned long type) +{ +} +#endif + +#endif /* __LINUX_BOOTMEM_INFO_H */ --- a/include/linux/memory_hotplug.h~mm-memory_hotplug-factor-out-bootmem-core-functions-to-bootmem_infoc +++ a/include/linux/memory_hotplug.h @@ -18,18 +18,6 @@ struct vmem_altmap; #ifdef CONFIG_MEMORY_HOTPLUG struct page *pfn_to_online_page(unsigned long pfn); -/* - * Types for free bootmem stored in page->lru.next. These have to be in - * some random range in unsigned long space for debugging purposes. - */ -enum { - MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE = 12, - SECTION_INFO = MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE, - MIX_SECTION_INFO, - NODE_INFO, - MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE = NODE_INFO, -}; - /* Types for control the zone type of onlined and offlined memory */ enum { /* Offline the memory. */ @@ -222,17 +210,6 @@ static inline void arch_refresh_nodedata #endif /* CONFIG_NUMA */ #endif /* CONFIG_HAVE_ARCH_NODEDATA_EXTENSION */ -#ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE -extern void __init register_page_bootmem_info_node(struct pglist_data *pgdat); -#else -static inline void register_page_bootmem_info_node(struct pglist_data *pgdat) -{ -} -#endif -extern void put_page_bootmem(struct page *page); -extern void get_page_bootmem(unsigned long ingo, struct page *page, - unsigned long type); - void get_online_mems(void); void put_online_mems(void); @@ -260,10 +237,6 @@ static inline void zone_span_writelock(s static inline void zone_span_writeunlock(struct zone *zone) {} static inline void zone_seqlock_init(struct zone *zone) {} -static inline void register_page_bootmem_info_node(struct pglist_data *pgdat) -{ -} - static inline int try_online_node(int nid) { return 0; --- /dev/null +++ a/mm/bootmem_info.c @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Bootmem core functions. + * + * Copyright (c) 2020, Bytedance. + * + * Author: Muchun Song + * + */ +#include +#include +#include +#include +#include + +void get_page_bootmem(unsigned long info, struct page *page, unsigned long type) +{ + page->freelist = (void *)type; + SetPagePrivate(page); + set_page_private(page, info); + page_ref_inc(page); +} + +void put_page_bootmem(struct page *page) +{ + unsigned long type; + + type = (unsigned long) page->freelist; + BUG_ON(type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE || + type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE); + + if (page_ref_dec_return(page) == 1) { + page->freelist = NULL; + ClearPagePrivate(page); + set_page_private(page, 0); + INIT_LIST_HEAD(&page->lru); + free_reserved_page(page); + } +} + +#ifndef CONFIG_SPARSEMEM_VMEMMAP +static void register_page_bootmem_info_section(unsigned long start_pfn) +{ + unsigned long mapsize, section_nr, i; + struct mem_section *ms; + struct page *page, *memmap; + struct mem_section_usage *usage; + + section_nr = pfn_to_section_nr(start_pfn); + ms = __nr_to_section(section_nr); + + /* Get section's memmap address */ + memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr); + + /* + * Get page for the memmap's phys address + * XXX: need more consideration for sparse_vmemmap... + */ + page = virt_to_page(memmap); + mapsize = sizeof(struct page) * PAGES_PER_SECTION; + mapsize = PAGE_ALIGN(mapsize) >> PAGE_SHIFT; + + /* remember memmap's page */ + for (i = 0; i < mapsize; i++, page++) + get_page_bootmem(section_nr, page, SECTION_INFO); + + usage = ms->usage; + page = virt_to_page(usage); + + mapsize = PAGE_ALIGN(mem_section_usage_size()) >> PAGE_SHIFT; + + for (i = 0; i < mapsize; i++, page++) + get_page_bootmem(section_nr, page, MIX_SECTION_INFO); + +} +#else /* CONFIG_SPARSEMEM_VMEMMAP */ +static void register_page_bootmem_info_section(unsigned long start_pfn) +{ + unsigned long mapsize, section_nr, i; + struct mem_section *ms; + struct page *page, *memmap; + struct mem_section_usage *usage; + + section_nr = pfn_to_section_nr(start_pfn); + ms = __nr_to_section(section_nr); + + memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr); + + register_page_bootmem_memmap(section_nr, memmap, PAGES_PER_SECTION); + + usage = ms->usage; + page = virt_to_page(usage); + + mapsize = PAGE_ALIGN(mem_section_usage_size()) >> PAGE_SHIFT; + + for (i = 0; i < mapsize; i++, page++) + get_page_bootmem(section_nr, page, MIX_SECTION_INFO); +} +#endif /* !CONFIG_SPARSEMEM_VMEMMAP */ + +void __init register_page_bootmem_info_node(struct pglist_data *pgdat) +{ + unsigned long i, pfn, end_pfn, nr_pages; + int node = pgdat->node_id; + struct page *page; + + nr_pages = PAGE_ALIGN(sizeof(struct pglist_data)) >> PAGE_SHIFT; + page = virt_to_page(pgdat); + + for (i = 0; i < nr_pages; i++, page++) + get_page_bootmem(node, page, NODE_INFO); + + pfn = pgdat->node_start_pfn; + end_pfn = pgdat_end_pfn(pgdat); + + /* register section info */ + for (; pfn < end_pfn; pfn += PAGES_PER_SECTION) { + /* + * Some platforms can assign the same pfn to multiple nodes - on + * node0 as well as nodeN. To avoid registering a pfn against + * multiple nodes we check that this pfn does not already + * reside in some other nodes. + */ + if (pfn_valid(pfn) && (early_pfn_to_nid(pfn) == node)) + register_page_bootmem_info_section(pfn); + } +} --- a/mm/Makefile~mm-memory_hotplug-factor-out-bootmem-core-functions-to-bootmem_infoc +++ a/mm/Makefile @@ -125,3 +125,4 @@ obj-$(CONFIG_MAPPING_DIRTY_HELPERS) += m obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o obj-$(CONFIG_IO_MAPPING) += io-mapping.o +obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o --- a/mm/memory_hotplug.c~mm-memory_hotplug-factor-out-bootmem-core-functions-to-bootmem_infoc +++ a/mm/memory_hotplug.c @@ -154,122 +154,6 @@ static void release_memory_resource(stru } #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE -void get_page_bootmem(unsigned long info, struct page *page, - unsigned long type) -{ - page->freelist = (void *)type; - SetPagePrivate(page); - set_page_private(page, info); - page_ref_inc(page); -} - -void put_page_bootmem(struct page *page) -{ - unsigned long type; - - type = (unsigned long) page->freelist; - BUG_ON(type < MEMORY_HOTPLUG_MIN_BOOTMEM_TYPE || - type > MEMORY_HOTPLUG_MAX_BOOTMEM_TYPE); - - if (page_ref_dec_return(page) == 1) { - page->freelist = NULL; - ClearPagePrivate(page); - set_page_private(page, 0); - INIT_LIST_HEAD(&page->lru); - free_reserved_page(page); - } -} - -#ifdef CONFIG_HAVE_BOOTMEM_INFO_NODE -#ifndef CONFIG_SPARSEMEM_VMEMMAP -static void register_page_bootmem_info_section(unsigned long start_pfn) -{ - unsigned long mapsize, section_nr, i; - struct mem_section *ms; - struct page *page, *memmap; - struct mem_section_usage *usage; - - section_nr = pfn_to_section_nr(start_pfn); - ms = __nr_to_section(section_nr); - - /* Get section's memmap address */ - memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr); - - /* - * Get page for the memmap's phys address - * XXX: need more consideration for sparse_vmemmap... - */ - page = virt_to_page(memmap); - mapsize = sizeof(struct page) * PAGES_PER_SECTION; - mapsize = PAGE_ALIGN(mapsize) >> PAGE_SHIFT; - - /* remember memmap's page */ - for (i = 0; i < mapsize; i++, page++) - get_page_bootmem(section_nr, page, SECTION_INFO); - - usage = ms->usage; - page = virt_to_page(usage); - - mapsize = PAGE_ALIGN(mem_section_usage_size()) >> PAGE_SHIFT; - - for (i = 0; i < mapsize; i++, page++) - get_page_bootmem(section_nr, page, MIX_SECTION_INFO); - -} -#else /* CONFIG_SPARSEMEM_VMEMMAP */ -static void register_page_bootmem_info_section(unsigned long start_pfn) -{ - unsigned long mapsize, section_nr, i; - struct mem_section *ms; - struct page *page, *memmap; - struct mem_section_usage *usage; - - section_nr = pfn_to_section_nr(start_pfn); - ms = __nr_to_section(section_nr); - - memmap = sparse_decode_mem_map(ms->section_mem_map, section_nr); - - register_page_bootmem_memmap(section_nr, memmap, PAGES_PER_SECTION); - - usage = ms->usage; - page = virt_to_page(usage); - - mapsize = PAGE_ALIGN(mem_section_usage_size()) >> PAGE_SHIFT; - - for (i = 0; i < mapsize; i++, page++) - get_page_bootmem(section_nr, page, MIX_SECTION_INFO); -} -#endif /* !CONFIG_SPARSEMEM_VMEMMAP */ - -void __init register_page_bootmem_info_node(struct pglist_data *pgdat) -{ - unsigned long i, pfn, end_pfn, nr_pages; - int node = pgdat->node_id; - struct page *page; - - nr_pages = PAGE_ALIGN(sizeof(struct pglist_data)) >> PAGE_SHIFT; - page = virt_to_page(pgdat); - - for (i = 0; i < nr_pages; i++, page++) - get_page_bootmem(node, page, NODE_INFO); - - pfn = pgdat->node_start_pfn; - end_pfn = pgdat_end_pfn(pgdat); - - /* register section info */ - for (; pfn < end_pfn; pfn += PAGES_PER_SECTION) { - /* - * Some platforms can assign the same pfn to multiple nodes - on - * node0 as well as nodeN. To avoid registering a pfn against - * multiple nodes we check that this pfn does not already - * reside in some other nodes. - */ - if (pfn_valid(pfn) && (early_pfn_to_nid(pfn) == node)) - register_page_bootmem_info_section(pfn); - } -} -#endif /* CONFIG_HAVE_BOOTMEM_INFO_NODE */ - static int check_pfn_span(unsigned long pfn, unsigned long nr_pages, const char *reason) { --- a/mm/sparse.c~mm-memory_hotplug-factor-out-bootmem-core-functions-to-bootmem_infoc +++ a/mm/sparse.c @@ -13,6 +13,7 @@ #include #include #include +#include #include "internal.h" #include From patchwork Thu Jul 1 01:47:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1E4AC11F6A for ; Thu, 1 Jul 2021 01:47:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6EBA36146B for ; Thu, 1 Jul 2021 01:47:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6EBA36146B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E34788D01D5; Wed, 30 Jun 2021 21:47:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE4A68D01D0; Wed, 30 Jun 2021 21:47:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C85118D01D5; Wed, 30 Jun 2021 21:47:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id 980028D01D0 for ; Wed, 30 Jun 2021 21:47:07 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 698CC25F59 for ; Thu, 1 Jul 2021 01:47:07 +0000 (UTC) X-FDA: 78312330894.02.2B0B521 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 04F9EF00008F for ; Thu, 1 Jul 2021 01:47:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4CD2D6120C; Thu, 1 Jul 2021 01:47:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104026; bh=B4h0Dd4FNioU/einNxt1eQR5e1CtQPB5UpPCpztf2sw=; h=Date:From:To:Subject:In-Reply-To:From; b=r5b/FvfAWk0OklAyrXfKpjaC1r4MvDyI7pqCNi5sayobVp+3CRk+zZ1tw0mYPDtZK sLwtd/fSupRHha2cRkBQ7/nhGhhzhdRsCWgtBVB2vFPoY9IW3Ol2mYi80ohZ+qpQsV 6LMHNzvVeqqdKwc34tDgPdz9HFtY7PjFlNWK0x+8= Date: Wed, 30 Jun 2021 18:47:04 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 002/192] mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP Message-ID: <20210701014704.sSHjBiLv9%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="r5b/FvfA"; dmarc=none; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: ygdsdqfyji3srtwxtdt1sf41gfe188r8 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 04F9EF00008F X-HE-Tag: 1625104026-875060 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP The option HUGETLB_PAGE_FREE_VMEMMAP allows for the freeing of some vmemmap pages associated with pre-allocated HugeTLB pages. For example, on X86_64 6 vmemmap pages of size 4KB each can be saved for each 2MB HugeTLB page. 4094 vmemmap pages of size 4KB each can be saved for each 1GB HugeTLB page. When a HugeTLB page is allocated or freed, the vmemmap array representing the range associated with the page will need to be remapped. When a page is allocated, vmemmap pages are freed after remapping. When a page is freed, previously discarded vmemmap pages must be allocated before remapping. The config option is introduced early so that supporting code can be written to depend on the option. The initial version of the code only provides support for x86-64. If config HAVE_BOOTMEM_INFO_NODE is enabled, the freeing vmemmap page code denpend on it to free vmemmap pages. Otherwise, just use free_reserved_page() to free vmemmmap pages. The routine register_page_bootmem_info() is used to register bootmem info. Therefore, make sure register_page_bootmem_info is enabled if HUGETLB_PAGE_FREE_VMEMMAP is defined. Link: https://lkml.kernel.org/r/20210510030027.56044-3-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador Acked-by: Mike Kravetz Reviewed-by: Miaohe Lin Tested-by: Chen Huang Tested-by: Bodeddula Balasubramaniam Reviewed-by: Balbir Singh Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Barry Song Cc: Borislav Petkov Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Michal Hocko Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- arch/x86/mm/init_64.c | 2 +- fs/Kconfig | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) --- a/arch/x86/mm/init_64.c~mm-hugetlb-introduce-a-new-config-hugetlb_page_free_vmemmap +++ a/arch/x86/mm/init_64.c @@ -1270,7 +1270,7 @@ static struct kcore_list kcore_vsyscall; static void __init register_page_bootmem_info(void) { -#ifdef CONFIG_NUMA +#if defined(CONFIG_NUMA) || defined(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP) int i; for_each_online_node(i) --- a/fs/Kconfig~mm-hugetlb-introduce-a-new-config-hugetlb_page_free_vmemmap +++ a/fs/Kconfig @@ -240,6 +240,11 @@ config HUGETLBFS config HUGETLB_PAGE def_bool HUGETLBFS +config HUGETLB_PAGE_FREE_VMEMMAP + def_bool HUGETLB_PAGE + depends on X86_64 + depends on SPARSEMEM_VMEMMAP + config MEMFD_CREATE def_bool TMPFS || HUGETLBFS From patchwork Thu Jul 1 01:47:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DCF0C11F66 for ; Thu, 1 Jul 2021 01:47:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AAF336146D for ; Thu, 1 Jul 2021 01:47:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAF336146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 29BAA8D01D6; Wed, 30 Jun 2021 21:47:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 24B9C8D01D0; Wed, 30 Jun 2021 21:47:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C51E8D01D6; Wed, 30 Jun 2021 21:47:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id D655E8D01D0 for ; Wed, 30 Jun 2021 21:47:11 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A7F93248AE for ; Thu, 1 Jul 2021 01:47:11 +0000 (UTC) X-FDA: 78312331062.08.6BC4EBD Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 4A5936001A84 for ; Thu, 1 Jul 2021 01:47:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 74ACB6146B; Thu, 1 Jul 2021 01:47:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104030; bh=2S3QwZiwEghGSp5nfLqq/cxKu5TcV4MtCCfYTL2Ebu8=; h=Date:From:To:Subject:In-Reply-To:From; b=1jSyBVYIBxgFjPvKAfRHe1iOEPcrbZbC3tkdpjtzNAI/ogMWL/jsxj1cmovuBomQ5 +3B8cV8eJ5X4GH6X3tUPDmOPO/HgTM1gC2ht0INW2KOc0MnlMaoyTrjWf4WfeSQKZ9 hKiNkzvl8x7g4aV3lz3sqzj81joBBy2lwGSlQwXU= Date: Wed, 30 Jun 2021 18:47:09 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 003/192] mm: hugetlb: gather discrete indexes of tail page Message-ID: <20210701014709.YmGkxqDfi%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=1jSyBVYI; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: 54yz59gxuhgzx8aancagpqh541zc35qi X-Rspamd-Queue-Id: 4A5936001A84 X-HE-Tag: 1625104031-256583 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: gather discrete indexes of tail page For HugeTLB page, there are more metadata to save in the struct page. But the head struct page cannot meet our needs, so we have to abuse other tail struct page to store the metadata. In order to avoid conflicts caused by subsequent use of more tail struct pages, we can gather these discrete indexes of tail struct page. In this case, it will be easier to add a new tail page index later. Link: https://lkml.kernel.org/r/20210510030027.56044-4-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador Reviewed-by: Miaohe Lin Tested-by: Chen Huang Tested-by: Bodeddula Balasubramaniam Acked-by: Michal Hocko Reviewed-by: Mike Kravetz Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Barry Song Cc: Borislav Petkov Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 21 +++++++++++++++++++-- include/linux/hugetlb_cgroup.h | 19 +++++++++++-------- 2 files changed, 30 insertions(+), 10 deletions(-) --- a/include/linux/hugetlb_cgroup.h~mm-hugetlb-gather-discrete-indexes-of-tail-page +++ a/include/linux/hugetlb_cgroup.h @@ -21,15 +21,16 @@ struct hugetlb_cgroup; struct resv_map; struct file_region; +#ifdef CONFIG_CGROUP_HUGETLB /* * Minimum page order trackable by hugetlb cgroup. * At least 4 pages are necessary for all the tracking information. - * The second tail page (hpage[2]) is the fault usage cgroup. - * The third tail page (hpage[3]) is the reservation usage cgroup. + * The second tail page (hpage[SUBPAGE_INDEX_CGROUP]) is the fault + * usage cgroup. The third tail page (hpage[SUBPAGE_INDEX_CGROUP_RSVD]) + * is the reservation usage cgroup. */ -#define HUGETLB_CGROUP_MIN_ORDER 2 +#define HUGETLB_CGROUP_MIN_ORDER order_base_2(__MAX_CGROUP_SUBPAGE_INDEX + 1) -#ifdef CONFIG_CGROUP_HUGETLB enum hugetlb_memory_event { HUGETLB_MAX, HUGETLB_NR_MEMORY_EVENTS, @@ -66,9 +67,9 @@ __hugetlb_cgroup_from_page(struct page * if (compound_order(page) < HUGETLB_CGROUP_MIN_ORDER) return NULL; if (rsvd) - return (struct hugetlb_cgroup *)page[3].private; + return (void *)page_private(page + SUBPAGE_INDEX_CGROUP_RSVD); else - return (struct hugetlb_cgroup *)page[2].private; + return (void *)page_private(page + SUBPAGE_INDEX_CGROUP); } static inline struct hugetlb_cgroup *hugetlb_cgroup_from_page(struct page *page) @@ -90,9 +91,11 @@ static inline int __set_hugetlb_cgroup(s if (compound_order(page) < HUGETLB_CGROUP_MIN_ORDER) return -1; if (rsvd) - page[3].private = (unsigned long)h_cg; + set_page_private(page + SUBPAGE_INDEX_CGROUP_RSVD, + (unsigned long)h_cg); else - page[2].private = (unsigned long)h_cg; + set_page_private(page + SUBPAGE_INDEX_CGROUP, + (unsigned long)h_cg); return 0; } --- a/include/linux/hugetlb.h~mm-hugetlb-gather-discrete-indexes-of-tail-page +++ a/include/linux/hugetlb.h @@ -29,6 +29,23 @@ typedef struct { unsigned long pd; } hug #include #include +/* + * For HugeTLB page, there are more metadata to save in the struct page. But + * the head struct page cannot meet our needs, so we have to abuse other tail + * struct page to store the metadata. In order to avoid conflicts caused by + * subsequent use of more tail struct pages, we gather these discrete indexes + * of tail struct page here. + */ +enum { + SUBPAGE_INDEX_SUBPOOL = 1, /* reuse page->private */ +#ifdef CONFIG_CGROUP_HUGETLB + SUBPAGE_INDEX_CGROUP, /* reuse page->private */ + SUBPAGE_INDEX_CGROUP_RSVD, /* reuse page->private */ + __MAX_CGROUP_SUBPAGE_INDEX = SUBPAGE_INDEX_CGROUP_RSVD, +#endif + __NR_USED_SUBPAGE, +}; + struct hugepage_subpool { spinlock_t lock; long count; @@ -635,13 +652,13 @@ extern unsigned int default_hstate_idx; */ static inline struct hugepage_subpool *hugetlb_page_subpool(struct page *hpage) { - return (struct hugepage_subpool *)(hpage+1)->private; + return (void *)page_private(hpage + SUBPAGE_INDEX_SUBPOOL); } static inline void hugetlb_set_page_subpool(struct page *hpage, struct hugepage_subpool *subpool) { - set_page_private(hpage+1, (unsigned long)subpool); + set_page_private(hpage + SUBPAGE_INDEX_SUBPOOL, (unsigned long)subpool); } static inline struct hstate *hstate_file(struct file *f) From patchwork Thu Jul 1 01:47:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F312C11F66 for ; Thu, 1 Jul 2021 01:47:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D09076120C for ; Thu, 1 Jul 2021 01:47:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D09076120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 55AF18D01D7; Wed, 30 Jun 2021 21:47:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 50A948D01D0; Wed, 30 Jun 2021 21:47:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35E9D8D01D7; Wed, 30 Jun 2021 21:47:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 063808D01D0 for ; Wed, 30 Jun 2021 21:47:15 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CB11B82E1AD5 for ; Thu, 1 Jul 2021 01:47:15 +0000 (UTC) X-FDA: 78312331230.39.9A263D7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 5BE395000097 for ; Thu, 1 Jul 2021 01:47:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A00E461469; Thu, 1 Jul 2021 01:47:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104034; bh=pq6jmN8yn9zJdB/eNYQYRDNtwFoxn5LPmjTX4YTwxpQ=; h=Date:From:To:Subject:In-Reply-To:From; b=mKxcrM5PxV52zyA5vILnC03YA9sPOHpE2XYJGTbGiYhqf6CNlX1D4AOrea5eFaB7U 4l2jw3Lo4s/BLJjGNjtXdkLFW0MIzxQ1kbD8q0m3rzZOsvuIHQA1VGIxLgkBaOVKwy Y4rmxOnvgHA/H/le4f/i7EjHYTiXy/SeRgwBfKHc= Date: Wed, 30 Jun 2021 18:47:13 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 004/192] mm: hugetlb: free the vmemmap pages associated with each HugeTLB page Message-ID: <20210701014713.b42x8Nm1B%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=mKxcrM5P; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: q97jrtt7durmwh5pg1zuf1gdmhqyock4 X-Rspamd-Queue-Id: 5BE395000097 X-HE-Tag: 1625104035-718568 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: free the vmemmap pages associated with each HugeTLB page Every HugeTLB has more than one struct page structure. We __know__ that we only use the first 4 (__NR_USED_SUBPAGE) struct page structures to store metadata associated with each HugeTLB. There are a lot of struct page structures associated with each HugeTLB page. For tail pages, the value of compound_head is the same. So we can reuse first page of tail page structures. We map the virtual addresses of the remaining pages of tail page structures to the first tail page struct, and then free these page frames. Therefore, we need to reserve two pages as vmemmap areas. When we allocate a HugeTLB page from the buddy, we can free some vmemmap pages associated with each HugeTLB page. It is more appropriate to do it in the prep_new_huge_page(). The free_vmemmap_pages_per_hpage(), which indicates how many vmemmap pages associated with a HugeTLB page can be freed, returns zero for now, which means the feature is disabled. We will enable it once all the infrastructure is there. [willy@infradead.org: fix documentation warning] Link: https://lkml.kernel.org/r/20210615200242.1716568-5-willy@infradead.org Link: https://lkml.kernel.org/r/20210510030027.56044-5-songmuchun@bytedance.com Signed-off-by: Muchun Song Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Oscar Salvador Tested-by: Chen Huang Tested-by: Bodeddula Balasubramaniam Acked-by: Michal Hocko Reviewed-by: Mike Kravetz Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Barry Song Cc: Borislav Petkov Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Miaohe Lin Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- include/linux/bootmem_info.h | 28 ++++ include/linux/mm.h | 3 mm/Makefile | 1 mm/hugetlb.c | 22 +-- mm/hugetlb_vmemmap.c | 218 +++++++++++++++++++++++++++++++++ mm/hugetlb_vmemmap.h | 20 +++ mm/sparse-vmemmap.c | 194 +++++++++++++++++++++++++++++ 7 files changed, 473 insertions(+), 13 deletions(-) --- a/include/linux/bootmem_info.h~mm-hugetlb-free-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/include/linux/bootmem_info.h @@ -2,7 +2,7 @@ #ifndef __LINUX_BOOTMEM_INFO_H #define __LINUX_BOOTMEM_INFO_H -#include +#include /* * Types for free bootmem stored in page->lru.next. These have to be in @@ -22,6 +22,27 @@ void __init register_page_bootmem_info_n void get_page_bootmem(unsigned long info, struct page *page, unsigned long type); void put_page_bootmem(struct page *page); + +/* + * Any memory allocated via the memblock allocator and not via the + * buddy will be marked reserved already in the memmap. For those + * pages, we can call this function to free it to buddy allocator. + */ +static inline void free_bootmem_page(struct page *page) +{ + unsigned long magic = (unsigned long)page->freelist; + + /* + * The reserve_bootmem_region sets the reserved flag on bootmem + * pages. + */ + VM_BUG_ON_PAGE(page_ref_count(page) != 2, page); + + if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) + put_page_bootmem(page); + else + VM_BUG_ON_PAGE(1, page); +} #else static inline void register_page_bootmem_info_node(struct pglist_data *pgdat) { @@ -35,6 +56,11 @@ static inline void get_page_bootmem(unsi unsigned long type) { } + +static inline void free_bootmem_page(struct page *page) +{ + free_reserved_page(page); +} #endif #endif /* __LINUX_BOOTMEM_INFO_H */ --- a/include/linux/mm.h~mm-hugetlb-free-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/include/linux/mm.h @@ -3076,6 +3076,9 @@ static inline void print_vma_addr(char * } #endif +void vmemmap_remap_free(unsigned long start, unsigned long end, + unsigned long reuse); + void *sparse_buffer_alloc(unsigned long size); struct page * __populate_section_memmap(unsigned long pfn, unsigned long nr_pages, int nid, struct vmem_altmap *altmap); --- a/mm/hugetlb.c~mm-hugetlb-free-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/hugetlb.c @@ -41,6 +41,7 @@ #include #include #include "internal.h" +#include "hugetlb_vmemmap.h" int hugetlb_max_hstate __read_mostly; unsigned int default_hstate_idx; @@ -1493,8 +1494,9 @@ static void __prep_account_new_huge_page h->nr_huge_pages_node[nid]++; } -static void __prep_new_huge_page(struct page *page) +static void __prep_new_huge_page(struct hstate *h, struct page *page) { + free_huge_page_vmemmap(h, page); INIT_LIST_HEAD(&page->lru); set_compound_page_dtor(page, HUGETLB_PAGE_DTOR); hugetlb_set_page_subpool(page, NULL); @@ -1504,7 +1506,7 @@ static void __prep_new_huge_page(struct static void prep_new_huge_page(struct hstate *h, struct page *page, int nid) { - __prep_new_huge_page(page); + __prep_new_huge_page(h, page); spin_lock_irq(&hugetlb_lock); __prep_account_new_huge_page(h, nid); spin_unlock_irq(&hugetlb_lock); @@ -2351,14 +2353,15 @@ static int alloc_and_dissolve_huge_page( /* * Before dissolving the page, we need to allocate a new one for the - * pool to remain stable. Using alloc_buddy_huge_page() allows us to - * not having to deal with prep_new_huge_page() and avoids dealing of any - * counters. This simplifies and let us do the whole thing under the - * lock. + * pool to remain stable. Here, we allocate the page and 'prep' it + * by doing everything but actually updating counters and adding to + * the pool. This simplifies and let us do most of the processing + * under the lock. */ new_page = alloc_buddy_huge_page(h, gfp_mask, nid, NULL, NULL); if (!new_page) return -ENOMEM; + __prep_new_huge_page(h, new_page); retry: spin_lock_irq(&hugetlb_lock); @@ -2397,14 +2400,9 @@ retry: remove_hugetlb_page(h, old_page, false); /* - * new_page needs to be initialized with the standard hugetlb - * state. This is normally done by prep_new_huge_page() but - * that takes hugetlb_lock which is already held so we need to - * open code it here. * Reference count trick is needed because allocator gives us * referenced page but the pool requires pages with 0 refcount. */ - __prep_new_huge_page(new_page); __prep_account_new_huge_page(h, nid); page_ref_dec(new_page); enqueue_huge_page(h, new_page); @@ -2420,7 +2418,7 @@ retry: free_new: spin_unlock_irq(&hugetlb_lock); - __free_pages(new_page, huge_page_order(h)); + update_and_free_page(h, new_page); return ret; } --- /dev/null +++ a/mm/hugetlb_vmemmap.c @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Free some vmemmap pages of HugeTLB + * + * Copyright (c) 2020, Bytedance. All rights reserved. + * + * Author: Muchun Song + * + * The struct page structures (page structs) are used to describe a physical + * page frame. By default, there is a one-to-one mapping from a page frame to + * it's corresponding page struct. + * + * HugeTLB pages consist of multiple base page size pages and is supported by + * many architectures. See hugetlbpage.rst in the Documentation directory for + * more details. On the x86-64 architecture, HugeTLB pages of size 2MB and 1GB + * are currently supported. Since the base page size on x86 is 4KB, a 2MB + * HugeTLB page consists of 512 base pages and a 1GB HugeTLB page consists of + * 4096 base pages. For each base page, there is a corresponding page struct. + * + * Within the HugeTLB subsystem, only the first 4 page structs are used to + * contain unique information about a HugeTLB page. __NR_USED_SUBPAGE provides + * this upper limit. The only 'useful' information in the remaining page structs + * is the compound_head field, and this field is the same for all tail pages. + * + * By removing redundant page structs for HugeTLB pages, memory can be returned + * to the buddy allocator for other uses. + * + * Different architectures support different HugeTLB pages. For example, the + * following table is the HugeTLB page size supported by x86 and arm64 + * architectures. Because arm64 supports 4k, 16k, and 64k base pages and + * supports contiguous entries, so it supports many kinds of sizes of HugeTLB + * page. + * + * +--------------+-----------+-----------------------------------------------+ + * | Architecture | Page Size | HugeTLB Page Size | + * +--------------+-----------+-----------+-----------+-----------+-----------+ + * | x86-64 | 4KB | 2MB | 1GB | | | + * +--------------+-----------+-----------+-----------+-----------+-----------+ + * | | 4KB | 64KB | 2MB | 32MB | 1GB | + * | +-----------+-----------+-----------+-----------+-----------+ + * | arm64 | 16KB | 2MB | 32MB | 1GB | | + * | +-----------+-----------+-----------+-----------+-----------+ + * | | 64KB | 2MB | 512MB | 16GB | | + * +--------------+-----------+-----------+-----------+-----------+-----------+ + * + * When the system boot up, every HugeTLB page has more than one struct page + * structs which size is (unit: pages): + * + * struct_size = HugeTLB_Size / PAGE_SIZE * sizeof(struct page) / PAGE_SIZE + * + * Where HugeTLB_Size is the size of the HugeTLB page. We know that the size + * of the HugeTLB page is always n times PAGE_SIZE. So we can get the following + * relationship. + * + * HugeTLB_Size = n * PAGE_SIZE + * + * Then, + * + * struct_size = n * PAGE_SIZE / PAGE_SIZE * sizeof(struct page) / PAGE_SIZE + * = n * sizeof(struct page) / PAGE_SIZE + * + * We can use huge mapping at the pud/pmd level for the HugeTLB page. + * + * For the HugeTLB page of the pmd level mapping, then + * + * struct_size = n * sizeof(struct page) / PAGE_SIZE + * = PAGE_SIZE / sizeof(pte_t) * sizeof(struct page) / PAGE_SIZE + * = sizeof(struct page) / sizeof(pte_t) + * = 64 / 8 + * = 8 (pages) + * + * Where n is how many pte entries which one page can contains. So the value of + * n is (PAGE_SIZE / sizeof(pte_t)). + * + * This optimization only supports 64-bit system, so the value of sizeof(pte_t) + * is 8. And this optimization also applicable only when the size of struct page + * is a power of two. In most cases, the size of struct page is 64 bytes (e.g. + * x86-64 and arm64). So if we use pmd level mapping for a HugeTLB page, the + * size of struct page structs of it is 8 page frames which size depends on the + * size of the base page. + * + * For the HugeTLB page of the pud level mapping, then + * + * struct_size = PAGE_SIZE / sizeof(pmd_t) * struct_size(pmd) + * = PAGE_SIZE / 8 * 8 (pages) + * = PAGE_SIZE (pages) + * + * Where the struct_size(pmd) is the size of the struct page structs of a + * HugeTLB page of the pmd level mapping. + * + * E.g.: A 2MB HugeTLB page on x86_64 consists in 8 page frames while 1GB + * HugeTLB page consists in 4096. + * + * Next, we take the pmd level mapping of the HugeTLB page as an example to + * show the internal implementation of this optimization. There are 8 pages + * struct page structs associated with a HugeTLB page which is pmd mapped. + * + * Here is how things look before optimization. + * + * HugeTLB struct pages(8 pages) page frame(8 pages) + * +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ + * | | | 0 | -------------> | 0 | + * | | +-----------+ +-----------+ + * | | | 1 | -------------> | 1 | + * | | +-----------+ +-----------+ + * | | | 2 | -------------> | 2 | + * | | +-----------+ +-----------+ + * | | | 3 | -------------> | 3 | + * | | +-----------+ +-----------+ + * | | | 4 | -------------> | 4 | + * | PMD | +-----------+ +-----------+ + * | level | | 5 | -------------> | 5 | + * | mapping | +-----------+ +-----------+ + * | | | 6 | -------------> | 6 | + * | | +-----------+ +-----------+ + * | | | 7 | -------------> | 7 | + * | | +-----------+ +-----------+ + * | | + * | | + * | | + * +-----------+ + * + * The value of page->compound_head is the same for all tail pages. The first + * page of page structs (page 0) associated with the HugeTLB page contains the 4 + * page structs necessary to describe the HugeTLB. The only use of the remaining + * pages of page structs (page 1 to page 7) is to point to page->compound_head. + * Therefore, we can remap pages 2 to 7 to page 1. Only 2 pages of page structs + * will be used for each HugeTLB page. This will allow us to free the remaining + * 6 pages to the buddy allocator. + * + * Here is how things look after remapping. + * + * HugeTLB struct pages(8 pages) page frame(8 pages) + * +-----------+ ---virt_to_page---> +-----------+ mapping to +-----------+ + * | | | 0 | -------------> | 0 | + * | | +-----------+ +-----------+ + * | | | 1 | -------------> | 1 | + * | | +-----------+ +-----------+ + * | | | 2 | ----------------^ ^ ^ ^ ^ ^ + * | | +-----------+ | | | | | + * | | | 3 | ------------------+ | | | | + * | | +-----------+ | | | | + * | | | 4 | --------------------+ | | | + * | PMD | +-----------+ | | | + * | level | | 5 | ----------------------+ | | + * | mapping | +-----------+ | | + * | | | 6 | ------------------------+ | + * | | +-----------+ | + * | | | 7 | --------------------------+ + * | | +-----------+ + * | | + * | | + * | | + * +-----------+ + * + * When a HugeTLB is freed to the buddy system, we should allocate 6 pages for + * vmemmap pages and restore the previous mapping relationship. + * + * For the HugeTLB page of the pud level mapping. It is similar to the former. + * We also can use this approach to free (PAGE_SIZE - 2) vmemmap pages. + * + * Apart from the HugeTLB page of the pmd/pud level mapping, some architectures + * (e.g. aarch64) provides a contiguous bit in the translation table entries + * that hints to the MMU to indicate that it is one of a contiguous set of + * entries that can be cached in a single TLB entry. + * + * The contiguous bit is used to increase the mapping size at the pmd and pte + * (last) level. So this type of HugeTLB page can be optimized only when its + * size of the struct page structs is greater than 2 pages. + */ +#include "hugetlb_vmemmap.h" + +/* + * There are a lot of struct page structures associated with each HugeTLB page. + * For tail pages, the value of compound_head is the same. So we can reuse first + * page of tail page structures. We map the virtual addresses of the remaining + * pages of tail page structures to the first tail page struct, and then free + * these page frames. Therefore, we need to reserve two pages as vmemmap areas. + */ +#define RESERVE_VMEMMAP_NR 2U +#define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) + +/* + * How many vmemmap pages associated with a HugeTLB page that can be freed + * to the buddy allocator. + * + * Todo: Returns zero for now, which means the feature is disabled. We will + * enable it once all the infrastructure is there. + */ +static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) +{ + return 0; +} + +static inline unsigned long free_vmemmap_pages_size_per_hpage(struct hstate *h) +{ + return (unsigned long)free_vmemmap_pages_per_hpage(h) << PAGE_SHIFT; +} + +void free_huge_page_vmemmap(struct hstate *h, struct page *head) +{ + unsigned long vmemmap_addr = (unsigned long)head; + unsigned long vmemmap_end, vmemmap_reuse; + + if (!free_vmemmap_pages_per_hpage(h)) + return; + + vmemmap_addr += RESERVE_VMEMMAP_SIZE; + vmemmap_end = vmemmap_addr + free_vmemmap_pages_size_per_hpage(h); + vmemmap_reuse = vmemmap_addr - PAGE_SIZE; + + /* + * Remap the vmemmap virtual address range [@vmemmap_addr, @vmemmap_end) + * to the page which @vmemmap_reuse is mapped to, then free the pages + * which the range [@vmemmap_addr, @vmemmap_end] is mapped to. + */ + vmemmap_remap_free(vmemmap_addr, vmemmap_end, vmemmap_reuse); +} --- /dev/null +++ a/mm/hugetlb_vmemmap.h @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Free some vmemmap pages of HugeTLB + * + * Copyright (c) 2020, Bytedance. All rights reserved. + * + * Author: Muchun Song + */ +#ifndef _LINUX_HUGETLB_VMEMMAP_H +#define _LINUX_HUGETLB_VMEMMAP_H +#include + +#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP +void free_huge_page_vmemmap(struct hstate *h, struct page *head); +#else +static inline void free_huge_page_vmemmap(struct hstate *h, struct page *head) +{ +} +#endif /* CONFIG_HUGETLB_PAGE_FREE_VMEMMAP */ +#endif /* _LINUX_HUGETLB_VMEMMAP_H */ --- a/mm/Makefile~mm-hugetlb-free-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/Makefile @@ -75,6 +75,7 @@ obj-$(CONFIG_FRONTSWAP) += frontswap.o obj-$(CONFIG_ZSWAP) += zswap.o obj-$(CONFIG_HAS_DMA) += dmapool.o obj-$(CONFIG_HUGETLBFS) += hugetlb.o +obj-$(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP) += hugetlb_vmemmap.o obj-$(CONFIG_NUMA) += mempolicy.o obj-$(CONFIG_SPARSEMEM) += sparse.o obj-$(CONFIG_SPARSEMEM_VMEMMAP) += sparse-vmemmap.o --- a/mm/sparse-vmemmap.c~mm-hugetlb-free-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/sparse-vmemmap.c @@ -27,8 +27,202 @@ #include #include #include +#include +#include + #include #include +#include + +/** + * struct vmemmap_remap_walk - walk vmemmap page table + * + * @remap_pte: called for each lowest-level entry (PTE). + * @reuse_page: the page which is reused for the tail vmemmap pages. + * @reuse_addr: the virtual address of the @reuse_page page. + * @vmemmap_pages: the list head of the vmemmap pages that can be freed. + */ +struct vmemmap_remap_walk { + void (*remap_pte)(pte_t *pte, unsigned long addr, + struct vmemmap_remap_walk *walk); + struct page *reuse_page; + unsigned long reuse_addr; + struct list_head *vmemmap_pages; +}; + +static void vmemmap_pte_range(pmd_t *pmd, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) +{ + pte_t *pte = pte_offset_kernel(pmd, addr); + + /* + * The reuse_page is found 'first' in table walk before we start + * remapping (which is calling @walk->remap_pte). + */ + if (!walk->reuse_page) { + walk->reuse_page = pte_page(*pte); + /* + * Because the reuse address is part of the range that we are + * walking, skip the reuse address range. + */ + addr += PAGE_SIZE; + pte++; + } + + for (; addr != end; addr += PAGE_SIZE, pte++) + walk->remap_pte(pte, addr, walk); +} + +static void vmemmap_pmd_range(pud_t *pud, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) +{ + pmd_t *pmd; + unsigned long next; + + pmd = pmd_offset(pud, addr); + do { + BUG_ON(pmd_leaf(*pmd)); + + next = pmd_addr_end(addr, end); + vmemmap_pte_range(pmd, addr, next, walk); + } while (pmd++, addr = next, addr != end); +} + +static void vmemmap_pud_range(p4d_t *p4d, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) +{ + pud_t *pud; + unsigned long next; + + pud = pud_offset(p4d, addr); + do { + next = pud_addr_end(addr, end); + vmemmap_pmd_range(pud, addr, next, walk); + } while (pud++, addr = next, addr != end); +} + +static void vmemmap_p4d_range(pgd_t *pgd, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) +{ + p4d_t *p4d; + unsigned long next; + + p4d = p4d_offset(pgd, addr); + do { + next = p4d_addr_end(addr, end); + vmemmap_pud_range(p4d, addr, next, walk); + } while (p4d++, addr = next, addr != end); +} + +static void vmemmap_remap_range(unsigned long start, unsigned long end, + struct vmemmap_remap_walk *walk) +{ + unsigned long addr = start; + unsigned long next; + pgd_t *pgd; + + VM_BUG_ON(!IS_ALIGNED(start, PAGE_SIZE)); + VM_BUG_ON(!IS_ALIGNED(end, PAGE_SIZE)); + + pgd = pgd_offset_k(addr); + do { + next = pgd_addr_end(addr, end); + vmemmap_p4d_range(pgd, addr, next, walk); + } while (pgd++, addr = next, addr != end); + + /* + * We only change the mapping of the vmemmap virtual address range + * [@start + PAGE_SIZE, end), so we only need to flush the TLB which + * belongs to the range. + */ + flush_tlb_kernel_range(start + PAGE_SIZE, end); +} + +/* + * Free a vmemmap page. A vmemmap page can be allocated from the memblock + * allocator or buddy allocator. If the PG_reserved flag is set, it means + * that it allocated from the memblock allocator, just free it via the + * free_bootmem_page(). Otherwise, use __free_page(). + */ +static inline void free_vmemmap_page(struct page *page) +{ + if (PageReserved(page)) + free_bootmem_page(page); + else + __free_page(page); +} + +/* Free a list of the vmemmap pages */ +static void free_vmemmap_page_list(struct list_head *list) +{ + struct page *page, *next; + + list_for_each_entry_safe(page, next, list, lru) { + list_del(&page->lru); + free_vmemmap_page(page); + } +} + +static void vmemmap_remap_pte(pte_t *pte, unsigned long addr, + struct vmemmap_remap_walk *walk) +{ + /* + * Remap the tail pages as read-only to catch illegal write operation + * to the tail pages. + */ + pgprot_t pgprot = PAGE_KERNEL_RO; + pte_t entry = mk_pte(walk->reuse_page, pgprot); + struct page *page = pte_page(*pte); + + list_add(&page->lru, walk->vmemmap_pages); + set_pte_at(&init_mm, addr, pte, entry); +} + +/** + * vmemmap_remap_free - remap the vmemmap virtual address range [@start, @end) + * to the page which @reuse is mapped to, then free vmemmap + * which the range are mapped to. + * @start: start address of the vmemmap virtual address range that we want + * to remap. + * @end: end address of the vmemmap virtual address range that we want to + * remap. + * @reuse: reuse address. + * + * Note: This function depends on vmemmap being base page mapped. Please make + * sure that we disable PMD mapping of vmemmap pages when calling this function. + */ +void vmemmap_remap_free(unsigned long start, unsigned long end, + unsigned long reuse) +{ + LIST_HEAD(vmemmap_pages); + struct vmemmap_remap_walk walk = { + .remap_pte = vmemmap_remap_pte, + .reuse_addr = reuse, + .vmemmap_pages = &vmemmap_pages, + }; + + /* + * In order to make remapping routine most efficient for the huge pages, + * the routine of vmemmap page table walking has the following rules + * (see more details from the vmemmap_pte_range()): + * + * - The range [@start, @end) and the range [@reuse, @reuse + PAGE_SIZE) + * should be continuous. + * - The @reuse address is part of the range [@reuse, @end) that we are + * walking which is passed to vmemmap_remap_range(). + * - The @reuse address is the first in the complete range. + * + * So we need to make sure that @start and @reuse meet the above rules. + */ + BUG_ON(start - reuse != PAGE_SIZE); + + vmemmap_remap_range(reuse, end, &walk); + free_vmemmap_page_list(&vmemmap_pages); +} /* * Allocate a block of memory to be used to back the virtual memory map From patchwork Thu Jul 1 01:47:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353069 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 386F5C11F6A for ; Thu, 1 Jul 2021 01:47:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D827C6146B for ; Thu, 1 Jul 2021 01:47:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D827C6146B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F6D68D01D8; Wed, 30 Jun 2021 21:47:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A7188D01D0; Wed, 30 Jun 2021 21:47:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4205A8D01D8; Wed, 30 Jun 2021 21:47:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220]) by kanga.kvack.org (Postfix) with ESMTP id 1507F8D01D0 for ; Wed, 30 Jun 2021 21:47:20 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D4E7A18086CD2 for ; Thu, 1 Jul 2021 01:47:19 +0000 (UTC) X-FDA: 78312331398.22.AB756C4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 807D9F0000AE for ; Thu, 1 Jul 2021 01:47:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B7B7A6120C; Thu, 1 Jul 2021 01:47:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104038; bh=FeUs2sFZLoSEwglpzrWDBHcLyHqfPjSOJ8xCVIMK774=; h=Date:From:To:Subject:In-Reply-To:From; b=iv5mpYCJqLAeylJ8W0++ZbPhimHAzh/A3AdlznqVy52OZjFdC6vPq3o2FIzijEUQl q5O0tD0MFkoVaCSRbdEWu16hF6Ppmxhed4wJBY6FxTOOgXdMSFpS9i9us7glDYP0I3 j0p8/yAptuwImxag1EgYXWYqMaDhzvK9xgmv4iuY= Date: Wed, 30 Jun 2021 18:47:17 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 005/192] mm: hugetlb: defer freeing of HugeTLB pages Message-ID: <20210701014717.qZklIT1OD%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 807D9F0000AE Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=iv5mpYCJ; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: uk1yy61d5m7nkj1rntbwy35mb1tgr5zo X-HE-Tag: 1625104039-778758 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: defer freeing of HugeTLB pages In the subsequent patch, we should allocate the vmemmap pages when freeing a HugeTLB page. But update_and_free_page() can be called under any context, so we cannot use GFP_KERNEL to allocate vmemmap pages. However, we can defer the actual freeing in a kworker to prevent from using GFP_ATOMIC to allocate the vmemmap pages. The __update_and_free_page() is where the call to allocate vmemmmap pages will be inserted. Link: https://lkml.kernel.org/r/20210510030027.56044-6-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Reviewed-by: Oscar Salvador Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Barry Song Cc: Bodeddula Balasubramaniam Cc: Borislav Petkov Cc: Chen Huang Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Miaohe Lin Cc: Michal Hocko Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- mm/hugetlb.c | 83 +++++++++++++++++++++++++++++++++++++---- mm/hugetlb_vmemmap.c | 12 ----- mm/hugetlb_vmemmap.h | 17 ++++++++ 3 files changed, 93 insertions(+), 19 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-defer-freeing-of-hugetlb-pages +++ a/mm/hugetlb.c @@ -1376,7 +1376,7 @@ static void remove_hugetlb_page(struct h h->nr_huge_pages_node[nid]--; } -static void update_and_free_page(struct hstate *h, struct page *page) +static void __update_and_free_page(struct hstate *h, struct page *page) { int i; struct page *subpage = page; @@ -1399,12 +1399,79 @@ static void update_and_free_page(struct } } +/* + * As update_and_free_page() can be called under any context, so we cannot + * use GFP_KERNEL to allocate vmemmap pages. However, we can defer the + * actual freeing in a workqueue to prevent from using GFP_ATOMIC to allocate + * the vmemmap pages. + * + * free_hpage_workfn() locklessly retrieves the linked list of pages to be + * freed and frees them one-by-one. As the page->mapping pointer is going + * to be cleared in free_hpage_workfn() anyway, it is reused as the llist_node + * structure of a lockless linked list of huge pages to be freed. + */ +static LLIST_HEAD(hpage_freelist); + +static void free_hpage_workfn(struct work_struct *work) +{ + struct llist_node *node; + + node = llist_del_all(&hpage_freelist); + + while (node) { + struct page *page; + struct hstate *h; + + page = container_of((struct address_space **)node, + struct page, mapping); + node = node->next; + page->mapping = NULL; + /* + * The VM_BUG_ON_PAGE(!PageHuge(page), page) in page_hstate() + * is going to trigger because a previous call to + * remove_hugetlb_page() will set_compound_page_dtor(page, + * NULL_COMPOUND_DTOR), so do not use page_hstate() directly. + */ + h = size_to_hstate(page_size(page)); + + __update_and_free_page(h, page); + + cond_resched(); + } +} +static DECLARE_WORK(free_hpage_work, free_hpage_workfn); + +static inline void flush_free_hpage_work(struct hstate *h) +{ + if (free_vmemmap_pages_per_hpage(h)) + flush_work(&free_hpage_work); +} + +static void update_and_free_page(struct hstate *h, struct page *page, + bool atomic) +{ + if (!free_vmemmap_pages_per_hpage(h) || !atomic) { + __update_and_free_page(h, page); + return; + } + + /* + * Defer freeing to avoid using GFP_ATOMIC to allocate vmemmap pages. + * + * Only call schedule_work() if hpage_freelist is previously + * empty. Otherwise, schedule_work() had been called but the workfn + * hasn't retrieved the list yet. + */ + if (llist_add((struct llist_node *)&page->mapping, &hpage_freelist)) + schedule_work(&free_hpage_work); +} + static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) { struct page *page, *t_page; list_for_each_entry_safe(page, t_page, list, lru) { - update_and_free_page(h, page); + update_and_free_page(h, page, false); cond_resched(); } } @@ -1471,12 +1538,12 @@ void free_huge_page(struct page *page) if (HPageTemporary(page)) { remove_hugetlb_page(h, page, false); spin_unlock_irqrestore(&hugetlb_lock, flags); - update_and_free_page(h, page); + update_and_free_page(h, page, true); } else if (h->surplus_huge_pages_node[nid]) { /* remove the page from active list */ remove_hugetlb_page(h, page, true); spin_unlock_irqrestore(&hugetlb_lock, flags); - update_and_free_page(h, page); + update_and_free_page(h, page, true); } else { arch_clear_hugepage_flags(page); enqueue_huge_page(h, page); @@ -1795,7 +1862,7 @@ retry: remove_hugetlb_page(h, head, false); h->max_huge_pages--; spin_unlock_irq(&hugetlb_lock); - update_and_free_page(h, head); + update_and_free_page(h, head, false); return 0; } out: @@ -2411,14 +2478,14 @@ retry: * Pages have been replaced, we can safely free the old one. */ spin_unlock_irq(&hugetlb_lock); - update_and_free_page(h, old_page); + update_and_free_page(h, old_page, false); } return ret; free_new: spin_unlock_irq(&hugetlb_lock); - update_and_free_page(h, new_page); + update_and_free_page(h, new_page, false); return ret; } @@ -2832,6 +2899,7 @@ static int set_max_huge_pages(struct hst * pages in hstate via the proc/sysfs interfaces. */ mutex_lock(&h->resize_lock); + flush_free_hpage_work(h); spin_lock_irq(&hugetlb_lock); /* @@ -2941,6 +3009,7 @@ static int set_max_huge_pages(struct hst /* free the pages after dropping lock */ spin_unlock_irq(&hugetlb_lock); update_and_free_pages_bulk(h, &page_list); + flush_free_hpage_work(h); spin_lock_irq(&hugetlb_lock); while (count < persistent_huge_pages(h)) { --- a/mm/hugetlb_vmemmap.c~mm-hugetlb-defer-freeing-of-hugetlb-pages +++ a/mm/hugetlb_vmemmap.c @@ -180,18 +180,6 @@ #define RESERVE_VMEMMAP_NR 2U #define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) -/* - * How many vmemmap pages associated with a HugeTLB page that can be freed - * to the buddy allocator. - * - * Todo: Returns zero for now, which means the feature is disabled. We will - * enable it once all the infrastructure is there. - */ -static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) -{ - return 0; -} - static inline unsigned long free_vmemmap_pages_size_per_hpage(struct hstate *h) { return (unsigned long)free_vmemmap_pages_per_hpage(h) << PAGE_SHIFT; --- a/mm/hugetlb_vmemmap.h~mm-hugetlb-defer-freeing-of-hugetlb-pages +++ a/mm/hugetlb_vmemmap.h @@ -12,9 +12,26 @@ #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP void free_huge_page_vmemmap(struct hstate *h, struct page *head); + +/* + * How many vmemmap pages associated with a HugeTLB page that can be freed + * to the buddy allocator. + * + * Todo: Returns zero for now, which means the feature is disabled. We will + * enable it once all the infrastructure is there. + */ +static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) +{ + return 0; +} #else static inline void free_huge_page_vmemmap(struct hstate *h, struct page *head) { } + +static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) +{ + return 0; +} #endif /* CONFIG_HUGETLB_PAGE_FREE_VMEMMAP */ #endif /* _LINUX_HUGETLB_VMEMMAP_H */ From patchwork Thu Jul 1 01:47:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353071 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82549C11F68 for ; Thu, 1 Jul 2021 01:47:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 261E361469 for ; Thu, 1 Jul 2021 01:47:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 261E361469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EFAA8D01D9; Wed, 30 Jun 2021 21:47:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 99F218D01D0; Wed, 30 Jun 2021 21:47:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F1CB8D01D9; Wed, 30 Jun 2021 21:47:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id 4F6828D01D0 for ; Wed, 30 Jun 2021 21:47:24 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 12B4725F58 for ; Thu, 1 Jul 2021 01:47:24 +0000 (UTC) X-FDA: 78312331608.39.3D08458 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id A37B8F000090 for ; Thu, 1 Jul 2021 01:47:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DD8156146B; Thu, 1 Jul 2021 01:47:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104042; bh=KnkQ08jqNgDkCd9Ck0dB0l+hnzKq6rBz45+4feK2hiw=; h=Date:From:To:Subject:In-Reply-To:From; b=lbIOUY8aGoiQNQWWIfTLeF4BeoI5wIrBiLDW6cjUr6ygjcXjA2q9jpW6TBmrCloUZ m918f3J41xfSYX/RvyOMeg/fqh25ShyfxRjM8r2co1wRSKPIh/OLf5+TAIb4xV/cUG 4rjCKd5py+dRoAdnTV352ZQX7FTpmBgmUOTm1Krc= Date: Wed, 30 Jun 2021 18:47:21 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 006/192] mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page Message-ID: <20210701014721.djRZcYPba%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lbIOUY8a; dmarc=none; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: x1m6dqd9k1nmgtrao98gr5ty7qn3ad34 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A37B8F000090 X-HE-Tag: 1625104043-124835 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page When we free a HugeTLB page to the buddy allocator, we need to allocate the vmemmap pages associated with it. However, we may not be able to allocate the vmemmap pages when the system is under memory pressure. In this case, we just refuse to free the HugeTLB page. This changes behavior in some corner cases as listed below: 1) Failing to free a huge page triggered by the user (decrease nr_pages). User needs to try again later. 2) Failing to free a surplus huge page when freed by the application. Try again later when freeing a huge page next time. 3) Failing to dissolve a free huge page on ZONE_MOVABLE via offline_pages(). This can happen when we have plenty of ZONE_MOVABLE memory, but not enough kernel memory to allocate vmemmmap pages. We may even be able to migrate huge page contents, but will not be able to dissolve the source huge page. This will prevent an offline operation and is unfortunate as memory offlining is expected to succeed on movable zones. Users that depend on memory hotplug to succeed for movable zones should carefully consider whether the memory savings gained from this feature are worth the risk of possibly not being able to offline memory in certain situations. 4) Failing to dissolve a huge page on CMA/ZONE_MOVABLE via alloc_contig_range() - once we have that handling in place. Mainly affects CMA and virtio-mem. Similar to 3). virito-mem will handle migration errors gracefully. CMA might be able to fallback on other free areas within the CMA region. Vmemmap pages are allocated from the page freeing context. In order for those allocations to be not disruptive (e.g. trigger oom killer) __GFP_NORETRY is used. hugetlb_lock is dropped for the allocation because a non sleeping allocation would be too fragile and it could fail too easily under memory pressure. GFP_ATOMIC or other modes to access memory reserves is not used because we want to prevent consuming reserves under heavy hugetlb freeing. [mike.kravetz@oracle.com: fix dissolve_free_huge_page use of tail/head page] Link: https://lkml.kernel.org/r/20210527231225.226987-1-mike.kravetz@oracle.com [willy@infradead.org: fix alloc_vmemmap_page_list documentation warning] Link: https://lkml.kernel.org/r/20210615200242.1716568-6-willy@infradead.org Link: https://lkml.kernel.org/r/20210510030027.56044-7-songmuchun@bytedance.com Signed-off-by: Muchun Song Signed-off-by: Mike Kravetz Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Mike Kravetz Reviewed-by: Oscar Salvador Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Barry Song Cc: Bodeddula Balasubramaniam Cc: Borislav Petkov Cc: Chen Huang Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Miaohe Lin Cc: Michal Hocko Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- Documentation/admin-guide/mm/hugetlbpage.rst | 8 + Documentation/admin-guide/mm/memory-hotplug.rst | 13 + include/linux/hugetlb.h | 3 include/linux/mm.h | 2 mm/hugetlb.c | 98 +++++++++++--- mm/hugetlb_vmemmap.c | 34 ++++ mm/hugetlb_vmemmap.h | 6 mm/migrate.c | 5 mm/sparse-vmemmap.c | 75 ++++++++++ 9 files changed, 227 insertions(+), 17 deletions(-) --- a/Documentation/admin-guide/mm/hugetlbpage.rst~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/Documentation/admin-guide/mm/hugetlbpage.rst @@ -60,6 +60,10 @@ HugePages_Surp the pool above the value in ``/proc/sys/vm/nr_hugepages``. The maximum number of surplus huge pages is controlled by ``/proc/sys/vm/nr_overcommit_hugepages``. + Note: When the feature of freeing unused vmemmap pages associated + with each hugetlb page is enabled, the number of surplus huge pages + may be temporarily larger than the maximum number of surplus huge + pages when the system is under memory pressure. Hugepagesize is the default hugepage size (in Kb). Hugetlb @@ -80,6 +84,10 @@ returned to the huge page pool when free privileges can dynamically allocate more or free some persistent huge pages by increasing or decreasing the value of ``nr_hugepages``. +Note: When the feature of freeing unused vmemmap pages associated with each +hugetlb page is enabled, we can fail to free the huge pages triggered by +the user when ths system is under memory pressure. Please try again later. + Pages that are used as huge pages are reserved inside the kernel and cannot be used for other purposes. Huge pages cannot be swapped out under memory pressure. --- a/Documentation/admin-guide/mm/memory-hotplug.rst~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/Documentation/admin-guide/mm/memory-hotplug.rst @@ -357,6 +357,19 @@ creates ZONE_MOVABLE as following. Unfortunately, there is no information to show which memory block belongs to ZONE_MOVABLE. This is TBD. + Memory offlining can fail when dissolving a free huge page on ZONE_MOVABLE + and the feature of freeing unused vmemmap pages associated with each hugetlb + page is enabled. + + This can happen when we have plenty of ZONE_MOVABLE memory, but not enough + kernel memory to allocate vmemmmap pages. We may even be able to migrate + huge page contents, but will not be able to dissolve the source huge page. + This will prevent an offline operation and is unfortunate as memory offlining + is expected to succeed on movable zones. Users that depend on memory hotplug + to succeed for movable zones should carefully consider whether the memory + savings gained from this feature are worth the risk of possibly not being + able to offline memory in certain situations. + .. note:: Techniques that rely on long-term pinnings of memory (especially, RDMA and vfio) are fundamentally problematic with ZONE_MOVABLE and, therefore, memory --- a/include/linux/hugetlb.h~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/include/linux/hugetlb.h @@ -532,12 +532,14 @@ unsigned long hugetlb_get_unmapped_area( * modifications require hugetlb_lock. * HPG_freed - Set when page is on the free lists. * Synchronization: hugetlb_lock held for examination and modification. + * HPG_vmemmap_optimized - Set when the vmemmap pages of the page are freed. */ enum hugetlb_page_flags { HPG_restore_reserve = 0, HPG_migratable, HPG_temporary, HPG_freed, + HPG_vmemmap_optimized, __NR_HPAGEFLAGS, }; @@ -583,6 +585,7 @@ HPAGEFLAG(RestoreReserve, restore_reserv HPAGEFLAG(Migratable, migratable) HPAGEFLAG(Temporary, temporary) HPAGEFLAG(Freed, freed) +HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) #ifdef CONFIG_HUGETLB_PAGE --- a/include/linux/mm.h~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/include/linux/mm.h @@ -3078,6 +3078,8 @@ static inline void print_vma_addr(char * void vmemmap_remap_free(unsigned long start, unsigned long end, unsigned long reuse); +int vmemmap_remap_alloc(unsigned long start, unsigned long end, + unsigned long reuse, gfp_t gfp_mask); void *sparse_buffer_alloc(unsigned long size); struct page * __populate_section_memmap(unsigned long pfn, --- a/mm/hugetlb.c~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/hugetlb.c @@ -1376,6 +1376,39 @@ static void remove_hugetlb_page(struct h h->nr_huge_pages_node[nid]--; } +static void add_hugetlb_page(struct hstate *h, struct page *page, + bool adjust_surplus) +{ + int zeroed; + int nid = page_to_nid(page); + + VM_BUG_ON_PAGE(!HPageVmemmapOptimized(page), page); + + lockdep_assert_held(&hugetlb_lock); + + INIT_LIST_HEAD(&page->lru); + h->nr_huge_pages++; + h->nr_huge_pages_node[nid]++; + + if (adjust_surplus) { + h->surplus_huge_pages++; + h->surplus_huge_pages_node[nid]++; + } + + set_compound_page_dtor(page, HUGETLB_PAGE_DTOR); + set_page_private(page, 0); + SetHPageVmemmapOptimized(page); + + /* + * This page is now managed by the hugetlb allocator and has + * no users -- drop the last reference. + */ + zeroed = put_page_testzero(page); + VM_BUG_ON_PAGE(!zeroed, page); + arch_clear_hugepage_flags(page); + enqueue_huge_page(h, page); +} + static void __update_and_free_page(struct hstate *h, struct page *page) { int i; @@ -1384,6 +1417,18 @@ static void __update_and_free_page(struc if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) return; + if (alloc_huge_page_vmemmap(h, page)) { + spin_lock_irq(&hugetlb_lock); + /* + * If we cannot allocate vmemmap pages, just refuse to free the + * page and put the page back on the hugetlb free list and treat + * as a surplus page. + */ + add_hugetlb_page(h, page, true); + spin_unlock_irq(&hugetlb_lock); + return; + } + for (i = 0; i < pages_per_huge_page(h); i++, subpage = mem_map_next(subpage, page, i)) { subpage->flags &= ~(1 << PG_locked | 1 << PG_error | @@ -1450,7 +1495,7 @@ static inline void flush_free_hpage_work static void update_and_free_page(struct hstate *h, struct page *page, bool atomic) { - if (!free_vmemmap_pages_per_hpage(h) || !atomic) { + if (!HPageVmemmapOptimized(page) || !atomic) { __update_and_free_page(h, page); return; } @@ -1806,10 +1851,14 @@ static struct page *remove_pool_huge_pag * nothing for in-use hugepages and non-hugepages. * This function returns values like below: * - * -EBUSY: failed to dissolved free hugepages or the hugepage is in-use - * (allocated or reserved.) - * 0: successfully dissolved free hugepages or the page is not a - * hugepage (considered as already dissolved) + * -ENOMEM: failed to allocate vmemmap pages to free the freed hugepages + * when the system is under memory pressure and the feature of + * freeing unused vmemmap pages associated with each hugetlb page + * is enabled. + * -EBUSY: failed to dissolved free hugepages or the hugepage is in-use + * (allocated or reserved.) + * 0: successfully dissolved free hugepages or the page is not a + * hugepage (considered as already dissolved) */ int dissolve_free_huge_page(struct page *page) { @@ -1851,19 +1900,38 @@ retry: goto retry; } - /* - * Move PageHWPoison flag from head page to the raw error page, - * which makes any subpages rather than the error page reusable. - */ - if (PageHWPoison(head) && page != head) { - SetPageHWPoison(page); - ClearPageHWPoison(head); - } remove_hugetlb_page(h, head, false); h->max_huge_pages--; spin_unlock_irq(&hugetlb_lock); - update_and_free_page(h, head, false); - return 0; + + /* + * Normally update_and_free_page will allocate required vmemmmap + * before freeing the page. update_and_free_page will fail to + * free the page if it can not allocate required vmemmap. We + * need to adjust max_huge_pages if the page is not freed. + * Attempt to allocate vmemmmap here so that we can take + * appropriate action on failure. + */ + rc = alloc_huge_page_vmemmap(h, head); + if (!rc) { + /* + * Move PageHWPoison flag from head page to the raw + * error page, which makes any subpages rather than + * the error page reusable. + */ + if (PageHWPoison(head) && page != head) { + SetPageHWPoison(page); + ClearPageHWPoison(head); + } + update_and_free_page(h, head, false); + } else { + spin_lock_irq(&hugetlb_lock); + add_hugetlb_page(h, head, false); + h->max_huge_pages++; + spin_unlock_irq(&hugetlb_lock); + } + + return rc; } out: spin_unlock_irq(&hugetlb_lock); --- a/mm/hugetlb_vmemmap.c~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/hugetlb_vmemmap.c @@ -185,6 +185,38 @@ static inline unsigned long free_vmemmap return (unsigned long)free_vmemmap_pages_per_hpage(h) << PAGE_SHIFT; } +/* + * Previously discarded vmemmap pages will be allocated and remapping + * after this function returns zero. + */ +int alloc_huge_page_vmemmap(struct hstate *h, struct page *head) +{ + int ret; + unsigned long vmemmap_addr = (unsigned long)head; + unsigned long vmemmap_end, vmemmap_reuse; + + if (!HPageVmemmapOptimized(head)) + return 0; + + vmemmap_addr += RESERVE_VMEMMAP_SIZE; + vmemmap_end = vmemmap_addr + free_vmemmap_pages_size_per_hpage(h); + vmemmap_reuse = vmemmap_addr - PAGE_SIZE; + /* + * The pages which the vmemmap virtual address range [@vmemmap_addr, + * @vmemmap_end) are mapped to are freed to the buddy allocator, and + * the range is mapped to the page which @vmemmap_reuse is mapped to. + * When a HugeTLB page is freed to the buddy allocator, previously + * discarded vmemmap pages must be allocated and remapping. + */ + ret = vmemmap_remap_alloc(vmemmap_addr, vmemmap_end, vmemmap_reuse, + GFP_KERNEL | __GFP_NORETRY | __GFP_THISNODE); + + if (!ret) + ClearHPageVmemmapOptimized(head); + + return ret; +} + void free_huge_page_vmemmap(struct hstate *h, struct page *head) { unsigned long vmemmap_addr = (unsigned long)head; @@ -203,4 +235,6 @@ void free_huge_page_vmemmap(struct hstat * which the range [@vmemmap_addr, @vmemmap_end] is mapped to. */ vmemmap_remap_free(vmemmap_addr, vmemmap_end, vmemmap_reuse); + + SetHPageVmemmapOptimized(head); } --- a/mm/hugetlb_vmemmap.h~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/hugetlb_vmemmap.h @@ -11,6 +11,7 @@ #include #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP +int alloc_huge_page_vmemmap(struct hstate *h, struct page *head); void free_huge_page_vmemmap(struct hstate *h, struct page *head); /* @@ -25,6 +26,11 @@ static inline unsigned int free_vmemmap_ return 0; } #else +static inline int alloc_huge_page_vmemmap(struct hstate *h, struct page *head) +{ + return 0; +} + static inline void free_huge_page_vmemmap(struct hstate *h, struct page *head) { } --- a/mm/migrate.c~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/migrate.c @@ -626,7 +626,10 @@ void migrate_page_states(struct page *ne if (PageSwapCache(page)) ClearPageSwapCache(page); ClearPagePrivate(page); - set_page_private(page, 0); + + /* page->private contains hugetlb specific flags */ + if (!PageHuge(page)) + set_page_private(page, 0); /* * If any waiters have accumulated on the new page then --- a/mm/sparse-vmemmap.c~mm-hugetlb-alloc-the-vmemmap-pages-associated-with-each-hugetlb-page +++ a/mm/sparse-vmemmap.c @@ -40,7 +40,8 @@ * @remap_pte: called for each lowest-level entry (PTE). * @reuse_page: the page which is reused for the tail vmemmap pages. * @reuse_addr: the virtual address of the @reuse_page page. - * @vmemmap_pages: the list head of the vmemmap pages that can be freed. + * @vmemmap_pages: the list head of the vmemmap pages that can be freed + * or is mapped from. */ struct vmemmap_remap_walk { void (*remap_pte)(pte_t *pte, unsigned long addr, @@ -224,6 +225,78 @@ void vmemmap_remap_free(unsigned long st free_vmemmap_page_list(&vmemmap_pages); } +static void vmemmap_restore_pte(pte_t *pte, unsigned long addr, + struct vmemmap_remap_walk *walk) +{ + pgprot_t pgprot = PAGE_KERNEL; + struct page *page; + void *to; + + BUG_ON(pte_page(*pte) != walk->reuse_page); + + page = list_first_entry(walk->vmemmap_pages, struct page, lru); + list_del(&page->lru); + to = page_to_virt(page); + copy_page(to, (void *)walk->reuse_addr); + + set_pte_at(&init_mm, addr, pte, mk_pte(page, pgprot)); +} + +static int alloc_vmemmap_page_list(unsigned long start, unsigned long end, + gfp_t gfp_mask, struct list_head *list) +{ + unsigned long nr_pages = (end - start) >> PAGE_SHIFT; + int nid = page_to_nid((struct page *)start); + struct page *page, *next; + + while (nr_pages--) { + page = alloc_pages_node(nid, gfp_mask, 0); + if (!page) + goto out; + list_add_tail(&page->lru, list); + } + + return 0; +out: + list_for_each_entry_safe(page, next, list, lru) + __free_pages(page, 0); + return -ENOMEM; +} + +/** + * vmemmap_remap_alloc - remap the vmemmap virtual address range [@start, end) + * to the page which is from the @vmemmap_pages + * respectively. + * @start: start address of the vmemmap virtual address range that we want + * to remap. + * @end: end address of the vmemmap virtual address range that we want to + * remap. + * @reuse: reuse address. + * @gfp_mask: GFP flag for allocating vmemmap pages. + */ +int vmemmap_remap_alloc(unsigned long start, unsigned long end, + unsigned long reuse, gfp_t gfp_mask) +{ + LIST_HEAD(vmemmap_pages); + struct vmemmap_remap_walk walk = { + .remap_pte = vmemmap_restore_pte, + .reuse_addr = reuse, + .vmemmap_pages = &vmemmap_pages, + }; + + /* See the comment in the vmemmap_remap_free(). */ + BUG_ON(start - reuse != PAGE_SIZE); + + might_sleep_if(gfpflags_allow_blocking(gfp_mask)); + + if (alloc_vmemmap_page_list(start, end, gfp_mask, &vmemmap_pages)) + return -ENOMEM; + + vmemmap_remap_range(reuse, end, &walk); + + return 0; +} + /* * Allocate a block of memory to be used to back the virtual memory map * or to back the page tables that are used to create the mapping. From patchwork Thu Jul 1 01:47:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353073 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B3DFC11F69 for ; Thu, 1 Jul 2021 01:47:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDE3A6120C for ; Thu, 1 Jul 2021 01:47:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDE3A6120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 756CE8D01DA; Wed, 30 Jun 2021 21:47:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 706218D01D0; Wed, 30 Jun 2021 21:47:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A7268D01DA; Wed, 30 Jun 2021 21:47:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id 2F4768D01D0 for ; Wed, 30 Jun 2021 21:47:28 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id E49A9180ABF62 for ; Thu, 1 Jul 2021 01:47:27 +0000 (UTC) X-FDA: 78312331734.03.E6775DE Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 891181983 for ; Thu, 1 Jul 2021 01:47:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C86A261469; Thu, 1 Jul 2021 01:47:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104046; bh=B/reZ1mSXT6NQEJmPCJE7MgtkRFoE/6D/4fnvFnQRoo=; h=Date:From:To:Subject:In-Reply-To:From; b=ktPjlGk8/Fo52jZ2zj6ErwyzCNpkW14fGRaEqrhGAXdvrDCLRZbnM0F7J7EwYFnS5 pV1DLJVoW3fjTP+rSznYH1pvgqxYUgHEjifzkYYaa1H++RQFRZfolJjacPtjGFW9Uh X/ZL3s6H2NL7RNEid/LJKI+6RTCk5NFtuJY0fobw= Date: Wed, 30 Jun 2021 18:47:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 007/192] mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap Message-ID: <20210701014725.0NAzb7_it%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 891181983 X-Stat-Signature: o33jx8rku5rqt5px8h65d7dca6c3gcfg Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ktPjlGk8; dmarc=none; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104047-750569 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap Add a kernel parameter hugetlb_free_vmemmap to enable the feature of freeing unused vmemmap pages associated with each hugetlb page on boot. We disable PMD mapping of vmemmap pages for x86-64 arch when this feature is enabled. Because vmemmap_remap_free() depends on vmemmap being base page mapped. Link: https://lkml.kernel.org/r/20210510030027.56044-8-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Oscar Salvador Reviewed-by: Barry Song Reviewed-by: Miaohe Lin Tested-by: Chen Huang Tested-by: Bodeddula Balasubramaniam Reviewed-by: Mike Kravetz Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Borislav Petkov Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Michal Hocko Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 17 +++++++++ Documentation/admin-guide/mm/hugetlbpage.rst | 3 + arch/x86/mm/init_64.c | 8 +++- include/linux/hugetlb.h | 19 +++++++++++ mm/hugetlb_vmemmap.c | 24 ++++++++++++++ 5 files changed, 69 insertions(+), 2 deletions(-) --- a/arch/x86/mm/init_64.c~mm-hugetlb-add-a-kernel-parameter-hugetlb_free_vmemmap +++ a/arch/x86/mm/init_64.c @@ -34,6 +34,7 @@ #include #include #include +#include #include #include @@ -1609,7 +1610,8 @@ int __meminit vmemmap_populate(unsigned VM_BUG_ON(!IS_ALIGNED(start, PAGE_SIZE)); VM_BUG_ON(!IS_ALIGNED(end, PAGE_SIZE)); - if (end - start < PAGES_PER_SECTION * sizeof(struct page)) + if ((is_hugetlb_free_vmemmap_enabled() && !altmap) || + end - start < PAGES_PER_SECTION * sizeof(struct page)) err = vmemmap_populate_basepages(start, end, node, NULL); else if (boot_cpu_has(X86_FEATURE_PSE)) err = vmemmap_populate_hugepages(start, end, node, altmap); @@ -1637,6 +1639,8 @@ void register_page_bootmem_memmap(unsign pmd_t *pmd; unsigned int nr_pmd_pages; struct page *page; + bool base_mapping = !boot_cpu_has(X86_FEATURE_PSE) || + is_hugetlb_free_vmemmap_enabled(); for (; addr < end; addr = next) { pte_t *pte = NULL; @@ -1662,7 +1666,7 @@ void register_page_bootmem_memmap(unsign } get_page_bootmem(section_nr, pud_page(*pud), MIX_SECTION_INFO); - if (!boot_cpu_has(X86_FEATURE_PSE)) { + if (base_mapping) { next = (addr + PAGE_SIZE) & PAGE_MASK; pmd = pmd_offset(pud, addr); if (pmd_none(*pmd)) --- a/Documentation/admin-guide/kernel-parameters.txt~mm-hugetlb-add-a-kernel-parameter-hugetlb_free_vmemmap +++ a/Documentation/admin-guide/kernel-parameters.txt @@ -1567,6 +1567,23 @@ Documentation/admin-guide/mm/hugetlbpage.rst. Format: size[KMG] + hugetlb_free_vmemmap= + [KNL] Reguires CONFIG_HUGETLB_PAGE_FREE_VMEMMAP + enabled. + Allows heavy hugetlb users to free up some more + memory (6 * PAGE_SIZE for each 2MB hugetlb page). + This feauture is not free though. Large page + tables are not used to back vmemmap pages which + can lead to a performance degradation for some + workloads. Also there will be memory allocation + required when hugetlb pages are freed from the + pool which can lead to corner cases under heavy + memory pressure. + Format: { on | off (default) } + + on: enable the feature + off: disable the feature + hung_task_panic= [KNL] Should the hung task detector generate panics. Format: 0 | 1 --- a/Documentation/admin-guide/mm/hugetlbpage.rst~mm-hugetlb-add-a-kernel-parameter-hugetlb_free_vmemmap +++ a/Documentation/admin-guide/mm/hugetlbpage.rst @@ -153,6 +153,9 @@ default_hugepagesz will all result in 256 2M huge pages being allocated. Valid default huge page size is architecture dependent. +hugetlb_free_vmemmap + When CONFIG_HUGETLB_PAGE_FREE_VMEMMAP is set, this enables freeing + unused vmemmap pages associated with each HugeTLB page. When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages`` indicates the current number of pre-allocated huge pages of the default size. --- a/include/linux/hugetlb.h~mm-hugetlb-add-a-kernel-parameter-hugetlb_free_vmemmap +++ a/include/linux/hugetlb.h @@ -892,6 +892,20 @@ static inline void huge_ptep_modify_prot } #endif +#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP +extern bool hugetlb_free_vmemmap_enabled; + +static inline bool is_hugetlb_free_vmemmap_enabled(void) +{ + return hugetlb_free_vmemmap_enabled; +} +#else +static inline bool is_hugetlb_free_vmemmap_enabled(void) +{ + return false; +} +#endif + #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; @@ -1046,6 +1060,11 @@ static inline void set_huge_swap_pte_at( pte_t *ptep, pte_t pte, unsigned long sz) { } + +static inline bool is_hugetlb_free_vmemmap_enabled(void) +{ + return false; +} #endif /* CONFIG_HUGETLB_PAGE */ static inline spinlock_t *huge_pte_lock(struct hstate *h, --- a/mm/hugetlb_vmemmap.c~mm-hugetlb-add-a-kernel-parameter-hugetlb_free_vmemmap +++ a/mm/hugetlb_vmemmap.c @@ -168,6 +168,8 @@ * (last) level. So this type of HugeTLB page can be optimized only when its * size of the struct page structs is greater than 2 pages. */ +#define pr_fmt(fmt) "HugeTLB: " fmt + #include "hugetlb_vmemmap.h" /* @@ -180,6 +182,28 @@ #define RESERVE_VMEMMAP_NR 2U #define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) +bool hugetlb_free_vmemmap_enabled; + +static int __init early_hugetlb_free_vmemmap_param(char *buf) +{ + /* We cannot optimize if a "struct page" crosses page boundaries. */ + if ((!is_power_of_2(sizeof(struct page)))) { + pr_warn("cannot free vmemmap pages because \"struct page\" crosses page boundaries\n"); + return 0; + } + + if (!buf) + return -EINVAL; + + if (!strcmp(buf, "on")) + hugetlb_free_vmemmap_enabled = true; + else if (strcmp(buf, "off")) + return -EINVAL; + + return 0; +} +early_param("hugetlb_free_vmemmap", early_hugetlb_free_vmemmap_param); + static inline unsigned long free_vmemmap_pages_size_per_hpage(struct hstate *h) { return (unsigned long)free_vmemmap_pages_per_hpage(h) << PAGE_SHIFT; From patchwork Thu Jul 1 01:47:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353075 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48D77C11F69 for ; Thu, 1 Jul 2021 01:47:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E91F161468 for ; Thu, 1 Jul 2021 01:47:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E91F161468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E1C48D01DB; Wed, 30 Jun 2021 21:47:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 690CF8D01D0; Wed, 30 Jun 2021 21:47:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 533F78D01DB; Wed, 30 Jun 2021 21:47:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 27C3E8D01D0 for ; Wed, 30 Jun 2021 21:47:32 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id ED91625F5B for ; Thu, 1 Jul 2021 01:47:31 +0000 (UTC) X-FDA: 78312331902.11.2743471 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 9D8FB30000A8 for ; Thu, 1 Jul 2021 01:47:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E8F8E6120C; Thu, 1 Jul 2021 01:47:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104050; bh=4YLZtCkgZIalRP/2sK4O76TGFUPJqbCxP4AmDrqm3fc=; h=Date:From:To:Subject:In-Reply-To:From; b=FDMNpy81/Rs/BtfPIAW1oaBs5V3+MZg1r5EmspWBNlcgzEU9caRB2XBkg3MkzT+X2 JDMmHmP+OHIGwD5EnFzj2pN5brzKBo2Esv47P5MW0dwmAOxSIIDtOFgoBHgLtIy9W0 A4V8zDAIo+Ci/C+GhkW4VwEucs0SlNB3qyBN7xLU= Date: Wed, 30 Jun 2021 18:47:29 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 008/192] mm: memory_hotplug: disable memmap_on_memory when hugetlb_free_vmemmap enabled Message-ID: <20210701014729.o3VEGZzKA%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 9D8FB30000A8 X-Stat-Signature: itsmoij7fpf1b7koxj9trgmckkqw9urj Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=FDMNpy81; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104051-968308 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memory_hotplug: disable memmap_on_memory when hugetlb_free_vmemmap enabled The parameter of memory_hotplug.memmap_on_memory is not compatible with hugetlb_free_vmemmap. So disable it when hugetlb_free_vmemmap is enabled. [akpm@linux-foundation.org: remove unneeded include, per Oscar] Link: https://lkml.kernel.org/r/20210510030027.56044-9-songmuchun@bytedance.com Signed-off-by: Muchun Song Acked-by: Mike Kravetz Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Barry Song Cc: Bodeddula Balasubramaniam Cc: Borislav Petkov Cc: Chen Huang Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Miaohe Lin Cc: Michal Hocko Cc: Mina Almasry Cc: Oliver Neukum Cc: Oscar Salvador Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++ mm/memory_hotplug.c | 1 + 2 files changed, 9 insertions(+) --- a/Documentation/admin-guide/kernel-parameters.txt~mm-memory_hotplug-disable-memmap_on_memory-when-hugetlb_free_vmemmap-enabled +++ a/Documentation/admin-guide/kernel-parameters.txt @@ -1584,6 +1584,10 @@ on: enable the feature off: disable the feature + This is not compatible with memory_hotplug.memmap_on_memory. + If both parameters are enabled, hugetlb_free_vmemmap takes + precedence over memory_hotplug.memmap_on_memory. + hung_task_panic= [KNL] Should the hung task detector generate panics. Format: 0 | 1 @@ -2850,6 +2854,10 @@ Note that even when enabled, there are a few cases where the feature is not effective. + This is not compatible with hugetlb_free_vmemmap. If + both parameters are enabled, hugetlb_free_vmemmap takes + precedence over memory_hotplug.memmap_on_memory. + memtest= [KNL,X86,ARM,PPC,RISCV] Enable memtest Format: default : 0 --- a/mm/memory_hotplug.c~mm-memory_hotplug-disable-memmap_on_memory-when-hugetlb_free_vmemmap-enabled +++ a/mm/memory_hotplug.c @@ -1056,6 +1056,7 @@ bool mhp_supports_memmap_on_memory(unsig * populate a single PMD. */ return memmap_on_memory && + !is_hugetlb_free_vmemmap_enabled() && IS_ENABLED(CONFIG_MHP_MEMMAP_ON_MEMORY) && size == memory_block_size_bytes() && IS_ALIGNED(vmemmap_size, PMD_SIZE) && From patchwork Thu Jul 1 01:47:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59955C11F68 for ; Thu, 1 Jul 2021 01:47:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 09E0D61468 for ; Thu, 1 Jul 2021 01:47:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09E0D61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 867658D01DC; Wed, 30 Jun 2021 21:47:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 817568D01D0; Wed, 30 Jun 2021 21:47:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B8F48D01DC; Wed, 30 Jun 2021 21:47:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 40BD58D01D0 for ; Wed, 30 Jun 2021 21:47:36 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1A7EE180B1A6C for ; Thu, 1 Jul 2021 01:47:36 +0000 (UTC) X-FDA: 78312332112.13.0B9B1EC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id B707430000A8 for ; Thu, 1 Jul 2021 01:47:35 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EEA7261474; Thu, 1 Jul 2021 01:47:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104055; bh=zcSzV+ro5reUHy0+ya+gJAmzSOTcyMWx+9b576OxDMw=; h=Date:From:To:Subject:In-Reply-To:From; b=XJMduY1zEj0VN4OxfH7d8CyVd1NBKFKZWdJ/sI2MKBSqeIV6a9ET5CGndNpQaUj3+ h7fMnXRFBvsC+CisS5JWs2PNPgzHcBgLlthS8KHnEu70MEMbRiEMuFa8gxlELnIyT5 IZ+i5R9z1YOB7fyWQ+CA21cMtwMTo+njSJDcvfDI= Date: Wed, 30 Jun 2021 18:47:33 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, anshuman.khandual@arm.com, bodeddub@amazon.com, bp@alien8.de, bsingharora@gmail.com, chenhuang5@huawei.com, corbet@lwn.net, dave.hansen@linux.intel.com, david@redhat.com, duanxiongchun@bytedance.com, hpa@zytor.com, joao.m.martins@oracle.com, jroedel@suse.de, linmiaohe@huawei.com, linux-mm@kvack.org, luto@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mingo@redhat.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, oneukum@suse.com, osalvador@suse.de, paulmck@kernel.org, pawan.kumar.gupta@linux.intel.com, peterz@infradead.org, rdunlap@infradead.org, rientjes@google.com, song.bao.hua@hisilicon.com, songmuchun@bytedance.com, tglx@linutronix.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 009/192] mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate Message-ID: <20210701014733.WWeLqTvdY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=XJMduY1z; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: emqc9qofut5r5rmojgp7zot6p3t9zkrj X-Rspamd-Queue-Id: B707430000A8 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104055-21306 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate All the infrastructure is ready, so we introduce nr_free_vmemmap_pages field in the hstate to indicate how many vmemmap pages associated with a HugeTLB page that can be freed to buddy allocator. And initialize it in the hugetlb_vmemmap_init(). This patch is actual enablement of the feature. There are only (RESERVE_VMEMMAP_SIZE / sizeof(struct page)) struct page structs that can be used when CONFIG_HUGETLB_PAGE_FREE_VMEMMAP, so add a BUILD_BUG_ON to catch invalid usage of the tail struct page. Link: https://lkml.kernel.org/r/20210510030027.56044-10-songmuchun@bytedance.com Signed-off-by: Muchun Song Acked-by: Mike Kravetz Reviewed-by: Oscar Salvador Reviewed-by: Miaohe Lin Tested-by: Chen Huang Tested-by: Bodeddula Balasubramaniam Cc: Alexander Viro Cc: Andy Lutomirski Cc: Anshuman Khandual Cc: Balbir Singh Cc: Barry Song Cc: Borislav Petkov Cc: Dave Hansen Cc: David Hildenbrand Cc: David Rientjes Cc: HORIGUCHI NAOYA Cc: "H. Peter Anvin" Cc: Ingo Molnar Cc: Joao Martins Cc: Joerg Roedel Cc: Jonathan Corbet Cc: Matthew Wilcox Cc: Michal Hocko Cc: Mina Almasry Cc: Oliver Neukum Cc: Paul E. McKenney Cc: Pawan Gupta Cc: Peter Zijlstra Cc: Randy Dunlap Cc: Thomas Gleixner Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 3 +++ mm/hugetlb.c | 1 + mm/hugetlb_vmemmap.c | 33 +++++++++++++++++++++++++++++++++ mm/hugetlb_vmemmap.h | 10 ++++++---- 4 files changed, 43 insertions(+), 4 deletions(-) --- a/include/linux/hugetlb.h~mm-hugetlb-introduce-nr_free_vmemmap_pages-in-the-struct-hstate +++ a/include/linux/hugetlb.h @@ -608,6 +608,9 @@ struct hstate { unsigned int nr_huge_pages_node[MAX_NUMNODES]; unsigned int free_huge_pages_node[MAX_NUMNODES]; unsigned int surplus_huge_pages_node[MAX_NUMNODES]; +#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP + unsigned int nr_free_vmemmap_pages; +#endif #ifdef CONFIG_CGROUP_HUGETLB /* cgroup control files */ struct cftype cgroup_files_dfl[7]; --- a/mm/hugetlb.c~mm-hugetlb-introduce-nr_free_vmemmap_pages-in-the-struct-hstate +++ a/mm/hugetlb.c @@ -3585,6 +3585,7 @@ void __init hugetlb_add_hstate(unsigned h->next_nid_to_free = first_memory_node; snprintf(h->name, HSTATE_NAME_LEN, "hugepages-%lukB", huge_page_size(h)/1024); + hugetlb_vmemmap_init(h); parsed_hstate = h; } --- a/mm/hugetlb_vmemmap.c~mm-hugetlb-introduce-nr_free_vmemmap_pages-in-the-struct-hstate +++ a/mm/hugetlb_vmemmap.c @@ -262,3 +262,36 @@ void free_huge_page_vmemmap(struct hstat SetHPageVmemmapOptimized(head); } + +void __init hugetlb_vmemmap_init(struct hstate *h) +{ + unsigned int nr_pages = pages_per_huge_page(h); + unsigned int vmemmap_pages; + + /* + * There are only (RESERVE_VMEMMAP_SIZE / sizeof(struct page)) struct + * page structs that can be used when CONFIG_HUGETLB_PAGE_FREE_VMEMMAP, + * so add a BUILD_BUG_ON to catch invalid usage of the tail struct page. + */ + BUILD_BUG_ON(__NR_USED_SUBPAGE >= + RESERVE_VMEMMAP_SIZE / sizeof(struct page)); + + if (!hugetlb_free_vmemmap_enabled) + return; + + vmemmap_pages = (nr_pages * sizeof(struct page)) >> PAGE_SHIFT; + /* + * The head page and the first tail page are not to be freed to buddy + * allocator, the other pages will map to the first tail page, so they + * can be freed. + * + * Could RESERVE_VMEMMAP_NR be greater than @vmemmap_pages? It is true + * on some architectures (e.g. aarch64). See Documentation/arm64/ + * hugetlbpage.rst for more details. + */ + if (likely(vmemmap_pages > RESERVE_VMEMMAP_NR)) + h->nr_free_vmemmap_pages = vmemmap_pages - RESERVE_VMEMMAP_NR; + + pr_info("can free %d vmemmap pages for %s\n", h->nr_free_vmemmap_pages, + h->name); +} --- a/mm/hugetlb_vmemmap.h~mm-hugetlb-introduce-nr_free_vmemmap_pages-in-the-struct-hstate +++ a/mm/hugetlb_vmemmap.h @@ -13,17 +13,15 @@ #ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP int alloc_huge_page_vmemmap(struct hstate *h, struct page *head); void free_huge_page_vmemmap(struct hstate *h, struct page *head); +void hugetlb_vmemmap_init(struct hstate *h); /* * How many vmemmap pages associated with a HugeTLB page that can be freed * to the buddy allocator. - * - * Todo: Returns zero for now, which means the feature is disabled. We will - * enable it once all the infrastructure is there. */ static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) { - return 0; + return h->nr_free_vmemmap_pages; } #else static inline int alloc_huge_page_vmemmap(struct hstate *h, struct page *head) @@ -35,6 +33,10 @@ static inline void free_huge_page_vmemma { } +static inline void hugetlb_vmemmap_init(struct hstate *h) +{ +} + static inline unsigned int free_vmemmap_pages_per_hpage(struct hstate *h) { return 0; From patchwork Thu Jul 1 01:47:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 561B7C11F68 for ; Thu, 1 Jul 2021 01:47:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0655A61468 for ; Thu, 1 Jul 2021 01:47:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0655A61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 819A18D01DD; Wed, 30 Jun 2021 21:47:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CA578D01D0; Wed, 30 Jun 2021 21:47:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66A3B8D01DD; Wed, 30 Jun 2021 21:47:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 3834A8D01D0 for ; Wed, 30 Jun 2021 21:47:39 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0077E181AEF3F for ; Thu, 1 Jul 2021 01:47:38 +0000 (UTC) X-FDA: 78312332238.19.DB30BAE Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 81C6B700009B for ; Thu, 1 Jul 2021 01:47:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9A98361469; Thu, 1 Jul 2021 01:47:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104057; bh=uHOr+PUatPdOBSNIiNb9bk8ywjouWaNlAxsMmCzmGDY=; h=Date:From:To:Subject:In-Reply-To:From; b=WqA/E5hbXxB/NKP787lWYg67I+4ftUo4rt67TcMgqWLBfZZL11KoDMI1Lt5mHEMhQ A9FFh92qbQQj3BVnBe+TrmGR321p7YY5s9GSOzIl2QqkSePdjAwwTbaV1uayK+vVfj Fw4tHLCYWk8N3SB1S4beQtfA21yK1U1k4WqozeNc= Date: Wed, 30 Jun 2021 18:47:37 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, linux-mm@kvack.org, liushixin2@huawei.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 010/192] mm/debug_vm_pgtable: move {pmd/pud}_huge_tests out of CONFIG_TRANSPARENT_HUGEPAGE Message-ID: <20210701014737.aHHVw2mHf%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="WqA/E5hb"; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 81C6B700009B X-Stat-Signature: tguhsshguf4fu19njmn9t3dynh8ogcx1 X-HE-Tag: 1625104058-666493 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shixin Liu Subject: mm/debug_vm_pgtable: move {pmd/pud}_huge_tests out of CONFIG_TRANSPARENT_HUGEPAGE The functions {pmd/pud}_set_huge and {pmd/pud}_clear_huge are not dependent on THP. Hence move {pmd/pud}_huge_tests out of CONFIG_TRANSPARENT_HUGEPAGE. Link: https://lkml.kernel.org/r/20210419071820.750217-1-liushixin2@huawei.com Signed-off-by: Shixin Liu Reviewed-by: Anshuman Khandual Signed-off-by: Andrew Morton --- mm/debug_vm_pgtable.c | 91 +++++++++++++++++----------------------- 1 file changed, 39 insertions(+), 52 deletions(-) --- a/mm/debug_vm_pgtable.c~mm-debug_vm_pgtable-move-pmd-pud_huge_tests-out-of-config_transparent_hugepage +++ a/mm/debug_vm_pgtable.c @@ -248,29 +248,6 @@ static void __init pmd_leaf_tests(unsign WARN_ON(!pmd_leaf(pmd)); } -#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP -static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) -{ - pmd_t pmd; - - if (!arch_vmap_pmd_supported(prot)) - return; - - pr_debug("Validating PMD huge\n"); - /* - * X86 defined pmd_set_huge() verifies that the given - * PMD is not a populated non-leaf entry. - */ - WRITE_ONCE(*pmdp, __pmd(0)); - WARN_ON(!pmd_set_huge(pmdp, __pfn_to_phys(pfn), prot)); - WARN_ON(!pmd_clear_huge(pmdp)); - pmd = READ_ONCE(*pmdp); - WARN_ON(!pmd_none(pmd)); -} -#else /* CONFIG_HAVE_ARCH_HUGE_VMAP */ -static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) { } -#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ - static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot) { pmd_t pmd; @@ -395,30 +372,6 @@ static void __init pud_leaf_tests(unsign pud = pud_mkhuge(pud); WARN_ON(!pud_leaf(pud)); } - -#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP -static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) -{ - pud_t pud; - - if (!arch_vmap_pud_supported(prot)) - return; - - pr_debug("Validating PUD huge\n"); - /* - * X86 defined pud_set_huge() verifies that the given - * PUD is not a populated non-leaf entry. - */ - WRITE_ONCE(*pudp, __pud(0)); - WARN_ON(!pud_set_huge(pudp, __pfn_to_phys(pfn), prot)); - WARN_ON(!pud_clear_huge(pudp)); - pud = READ_ONCE(*pudp); - WARN_ON(!pud_none(pud)); -} -#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ -static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { } -#endif /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ - #else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { } static void __init pud_advanced_tests(struct mm_struct *mm, @@ -428,9 +381,6 @@ static void __init pud_advanced_tests(st { } static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { } -static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) -{ -} #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ #else /* !CONFIG_TRANSPARENT_HUGEPAGE */ static void __init pmd_basic_tests(unsigned long pfn, int idx) { } @@ -449,14 +399,51 @@ static void __init pud_advanced_tests(st } static void __init pmd_leaf_tests(unsigned long pfn, pgprot_t prot) { } static void __init pud_leaf_tests(unsigned long pfn, pgprot_t prot) { } +static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot) { } +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + +#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) { + pmd_t pmd; + + if (!arch_vmap_pmd_supported(prot)) + return; + + pr_debug("Validating PMD huge\n"); + /* + * X86 defined pmd_set_huge() verifies that the given + * PMD is not a populated non-leaf entry. + */ + WRITE_ONCE(*pmdp, __pmd(0)); + WARN_ON(!pmd_set_huge(pmdp, __pfn_to_phys(pfn), prot)); + WARN_ON(!pmd_clear_huge(pmdp)); + pmd = READ_ONCE(*pmdp); + WARN_ON(!pmd_none(pmd)); } + static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { + pud_t pud; + + if (!arch_vmap_pud_supported(prot)) + return; + + pr_debug("Validating PUD huge\n"); + /* + * X86 defined pud_set_huge() verifies that the given + * PUD is not a populated non-leaf entry. + */ + WRITE_ONCE(*pudp, __pud(0)); + WARN_ON(!pud_set_huge(pudp, __pfn_to_phys(pfn), prot)); + WARN_ON(!pud_clear_huge(pudp)); + pud = READ_ONCE(*pudp); + WARN_ON(!pud_none(pud)); } -static void __init pmd_savedwrite_tests(unsigned long pfn, pgprot_t prot) { } -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +#else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ +static void __init pmd_huge_tests(pmd_t *pmdp, unsigned long pfn, pgprot_t prot) { } +static void __init pud_huge_tests(pud_t *pudp, unsigned long pfn, pgprot_t prot) { } +#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot) { From patchwork Thu Jul 1 01:47:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12748C11F68 for ; Thu, 1 Jul 2021 01:47:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B548861466 for ; Thu, 1 Jul 2021 01:47:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B548861466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 386428D01DE; Wed, 30 Jun 2021 21:47:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 273F18D01D0; Wed, 30 Jun 2021 21:47:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0EBC58D01DE; Wed, 30 Jun 2021 21:47:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0240.hostedemail.com [216.40.44.240]) by kanga.kvack.org (Postfix) with ESMTP id D86708D01D0 for ; Wed, 30 Jun 2021 21:47:41 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A3B14253BB for ; Thu, 1 Jul 2021 01:47:41 +0000 (UTC) X-FDA: 78312332322.15.D4C7939 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 635E5F0000AD for ; Thu, 1 Jul 2021 01:47:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7204D61468; Thu, 1 Jul 2021 01:47:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104060; bh=dNuUdCiICTfvtbB62jPvf/7gBHZvB+HwEYMvaen2rEk=; h=Date:From:To:Subject:In-Reply-To:From; b=y61KpBO41cxFhIPQGvy/ouH7DIoGCrWDMm4j+FgQvREK0yQZam0CqLJliVFIm368N u/RSxriYNpwhk7eUxwrRd3ZEszg74dhJhhUqlX0WhCE+zGV1Vv4oLp7luK0Nf8u+1F 9knbC7xCuObqT9xh3rvhegpHlTrh81DNFgPme+wo= Date: Wed, 30 Jun 2021 18:47:40 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, linux-mm@kvack.org, liushixin2@huawei.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 011/192] mm/debug_vm_pgtable: remove redundant pfn_{pmd/pte}() and fix one comment mistake Message-ID: <20210701014740.19O6xELEh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 635E5F0000AD Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=y61KpBO4; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 5k7ximoknn1gnre6164y4ak3ww36u583 X-HE-Tag: 1625104061-882312 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shixin Liu Subject: mm/debug_vm_pgtable: remove redundant pfn_{pmd/pte}() and fix one comment mistake Remove redundant pfn_{pmd/pte}() in {pmd/pte}_advanced_tests() and adjust pfn_pud() in pud_advanced_tests() to make it similar with other two functions. In addition, the branch condition should be CONFIG_TRANSPARENT_HUGEPAGE instead of CONFIG_ARCH_HAS_PTE_DEVMAP. Link: https://lkml.kernel.org/r/20210419071820.750217-2-liushixin2@huawei.com Signed-off-by: Shixin Liu Reviewed-by: Anshuman Khandual Signed-off-by: Andrew Morton --- mm/debug_vm_pgtable.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/mm/debug_vm_pgtable.c~mm-debug_vm_pgtable-remove-redundant-pfn_pmd-pte-and-fix-one-comment-mistake +++ a/mm/debug_vm_pgtable.c @@ -91,7 +91,7 @@ static void __init pte_advanced_tests(st unsigned long pfn, unsigned long vaddr, pgprot_t prot) { - pte_t pte = pfn_pte(pfn, prot); + pte_t pte; /* * Architectures optimize set_pte_at by avoiding TLB flush. @@ -778,12 +778,12 @@ static void __init pmd_swap_soft_dirty_t WARN_ON(!pmd_swp_soft_dirty(pmd_swp_mksoft_dirty(pmd))); WARN_ON(pmd_swp_soft_dirty(pmd_swp_clear_soft_dirty(pmd))); } -#else /* !CONFIG_ARCH_HAS_PTE_DEVMAP */ +#else /* !CONFIG_TRANSPARENT_HUGEPAGE */ static void __init pmd_soft_dirty_tests(unsigned long pfn, pgprot_t prot) { } static void __init pmd_swap_soft_dirty_tests(unsigned long pfn, pgprot_t prot) { } -#endif /* CONFIG_ARCH_HAS_PTE_DEVMAP */ +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ static void __init pte_swap_tests(unsigned long pfn, pgprot_t prot) { From patchwork Thu Jul 1 01:47:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353083 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 745EBC11F68 for ; Thu, 1 Jul 2021 01:47:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 21F3C6120C for ; Thu, 1 Jul 2021 01:47:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21F3C6120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9793D8D01DF; Wed, 30 Jun 2021 21:47:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9017B8D01D0; Wed, 30 Jun 2021 21:47:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77BB68D01DF; Wed, 30 Jun 2021 21:47:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id 4BF088D01D0 for ; Wed, 30 Jun 2021 21:47:45 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 117A8181AEF3F for ; Thu, 1 Jul 2021 01:47:45 +0000 (UTC) X-FDA: 78312332490.38.F5F9273 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id CCDE930002A4 for ; Thu, 1 Jul 2021 01:47:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 82C2A6146D; Thu, 1 Jul 2021 01:47:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104064; bh=NGvdmXgGOwZKRPlNpAaYgirBshPSUNJ54JosTn6l5RA=; h=Date:From:To:Subject:In-Reply-To:From; b=vaGM7L1oGOJE6HO2/onIxlgBpoVd8Lrb/S8nlKqFiKUHgfyMcsBAp1CCLP0RAtMqk CCvSWKJmbv0bWa21+wMbWfqk2KsbZ8N0pZI8oiCfPokBG+m3e0SjDyRrB0DCPhQZZ1 AIFS75PS2DaWbsX8YvzArP8920X5+8kw4a79NJOU= Date: Wed, 30 Jun 2021 18:47:43 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, anshuman.khandual@arm.com, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, riel@surriel.com, shy828301@gmail.com, songliubraving@fb.com, torvalds@linux-foundation.org, william.kucharski@oracle.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 012/192] mm/huge_memory.c: remove dedicated macro HPAGE_CACHE_INDEX_MASK Message-ID: <20210701014743.FQGc4cdmS%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=vaGM7L1o; spf=pass (imf09.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 193qrxrscyjrh1w8wh7the6kq9qejpx8 X-Rspamd-Queue-Id: CCDE930002A4 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104064-120048 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/huge_memory.c: remove dedicated macro HPAGE_CACHE_INDEX_MASK Patch series "Cleanup and fixup for huge_memory:, v3. This series contains cleanups to remove dedicated macro and remove unnecessary tlb_remove_page_size() for huge zero pmd. Also this adds missing read-only THP checking for transparent_hugepage_enabled() and avoids discarding hugepage if other processes are mapping it. More details can be found in the respective changelogs. Thi patch (of 5): Rewrite the pgoff checking logic to remove macro HPAGE_CACHE_INDEX_MASK which is only used here to simplify the code. Link: https://lkml.kernel.org/r/20210511134857.1581273-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210511134857.1581273-2-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Yang Shi Reviewed-by: Anshuman Khandual Reviewed-by: David Hildenbrand Cc: Zi Yan Cc: William Kucharski Cc: Matthew Wilcox Cc: "Aneesh Kumar K . V" Cc: Ralph Campbell Cc: Song Liu Cc: Kirill A. Shutemov Cc: Rik van Riel Cc: Johannes Weiner Cc: Minchan Kim Cc: Hugh Dickins Cc: Alexey Dobriyan Cc: Mike Kravetz Signed-off-by: Andrew Morton --- include/linux/huge_mm.h | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/include/linux/huge_mm.h~mm-huge_memoryc-remove-dedicated-macro-hpage_cache_index_mask +++ a/include/linux/huge_mm.h @@ -152,15 +152,13 @@ static inline bool __transparent_hugepag bool transparent_hugepage_enabled(struct vm_area_struct *vma); -#define HPAGE_CACHE_INDEX_MASK (HPAGE_PMD_NR - 1) - static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, unsigned long haddr) { /* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous(vma)) { - if (((vma->vm_start >> PAGE_SHIFT) & HPAGE_CACHE_INDEX_MASK) != - (vma->vm_pgoff & HPAGE_CACHE_INDEX_MASK)) + if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, + HPAGE_PMD_NR)) return false; } From patchwork Thu Jul 1 01:47:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353085 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC62DC11F6B for ; Thu, 1 Jul 2021 01:47:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8BEE261468 for ; Thu, 1 Jul 2021 01:47:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8BEE261468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0ED848D01E0; Wed, 30 Jun 2021 21:47:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 076258D01D0; Wed, 30 Jun 2021 21:47:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E590B8D01E0; Wed, 30 Jun 2021 21:47:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id BA0B28D01D0 for ; Wed, 30 Jun 2021 21:47:48 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8E07982DAC36 for ; Thu, 1 Jul 2021 01:47:48 +0000 (UTC) X-FDA: 78312332616.01.0F18700 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id 4AF40900009E for ; Thu, 1 Jul 2021 01:47:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E388F6120C; Thu, 1 Jul 2021 01:47:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104067; bh=0Y9TreA1BTIYY2tKgHs7riBKdzT/8e6/D20+LtH+EJA=; h=Date:From:To:Subject:In-Reply-To:From; b=PjIh2ONn1dtCeX+SgTUQXoa/6i9HVhxTJZcWrmkDE3n0S4iwuD/eKe9dz99T00Lz1 FcP0jWpyZKKHte2tJiqf74D4CXhXm+PbsyshiKL30eHYf+Ujz59+rAkCJvGvIXbToP SOcLeiSa/5scRyuV6lhcoo3ufWZHUsY569A0smj8= Date: Wed, 30 Jun 2021 18:47:46 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, anshuman.khandual@arm.com, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, riel@surriel.com, shy828301@gmail.com, songliubraving@fb.com, torvalds@linux-foundation.org, william.kucharski@oracle.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 013/192] mm/huge_memory.c: use page->deferred_list Message-ID: <20210701014746.deARj1PP8%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4AF40900009E X-Stat-Signature: 4k1ppb6k3i13s634dqsacpwnybteh7ah Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=PjIh2ONn; dmarc=none; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104068-260483 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/huge_memory.c: use page->deferred_list Now that we can represent the location of ->deferred_list instead of ->mapping + ->index, make use of it to improve readability. Link: https://lkml.kernel.org/r/20210511134857.1581273-3-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Yang Shi Reviewed-by: David Hildenbrand Cc: Alexey Dobriyan Cc: "Aneesh Kumar K . V" Cc: Anshuman Khandual Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kirill A. Shutemov Cc: Matthew Wilcox Cc: Minchan Kim Cc: Ralph Campbell Cc: Rik van Riel Cc: Song Liu Cc: William Kucharski Cc: Zi Yan Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/huge_memory.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/huge_memory.c~mm-huge_memoryc-use-page-deferred_list +++ a/mm/huge_memory.c @@ -2870,7 +2870,7 @@ static unsigned long deferred_split_scan spin_lock_irqsave(&ds_queue->split_queue_lock, flags); /* Take pin on all head pages to avoid freeing them under us */ list_for_each_safe(pos, next, &ds_queue->split_queue) { - page = list_entry((void *)pos, struct page, mapping); + page = list_entry((void *)pos, struct page, deferred_list); page = compound_head(page); if (get_page_unless_zero(page)) { list_move(page_deferred_list(page), &list); @@ -2885,7 +2885,7 @@ static unsigned long deferred_split_scan spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); list_for_each_safe(pos, next, &list) { - page = list_entry((void *)pos, struct page, mapping); + page = list_entry((void *)pos, struct page, deferred_list); if (!trylock_page(page)) goto next; /* split_huge_page() removes page from list on success */ From patchwork Thu Jul 1 01:47:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70956C11F6A for ; Thu, 1 Jul 2021 01:47:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 26DF56120C for ; Thu, 1 Jul 2021 01:47:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26DF56120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9EE238D01E1; Wed, 30 Jun 2021 21:47:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 99D5D8D01D0; Wed, 30 Jun 2021 21:47:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83E7F8D01E1; Wed, 30 Jun 2021 21:47:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id 577748D01D0 for ; Wed, 30 Jun 2021 21:47:52 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1DFC3181AEF3F for ; Thu, 1 Jul 2021 01:47:52 +0000 (UTC) X-FDA: 78312332784.25.44FDAE3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id B849EB00009A for ; Thu, 1 Jul 2021 01:47:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7146961468; Thu, 1 Jul 2021 01:47:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104071; bh=Gejd49g/3PZP7EBCE7UH0eF7UTw+4D8cMEdYvAsz9Fs=; h=Date:From:To:Subject:In-Reply-To:From; b=necjXPqNSulMiNg7K01BJXyjFxDkvWABAWhZsAWm9wP+/2EKRLvtW6T2NeLf6GtEm VOJZpXxf2TB5QLkvkKolufV4y/OAudUVGXnPPwceKzN7EPaVnREBfu48ki8kIfqp7z zDuJXcEW2/ljwaUHpihLmLZEtoSMia+sSP4dJa/w= Date: Wed, 30 Jun 2021 18:47:50 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, anshuman.khandual@arm.com, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, riel@surriel.com, shy828301@gmail.com, songliubraving@fb.com, torvalds@linux-foundation.org, william.kucharski@oracle.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 014/192] mm/huge_memory.c: add missing read-only THP checking in transparent_hugepage_enabled() Message-ID: <20210701014750.ckJ1rS67A%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=necjXPqN; dmarc=none; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: wc896k357kuppnpawpodoupehzaoai8a X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B849EB00009A X-HE-Tag: 1625104071-325435 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/huge_memory.c: add missing read-only THP checking in transparent_hugepage_enabled() Since commit 99cb0dbd47a1 ("mm,thp: add read-only THP support for (non-shmem) FS"), read-only THP file mapping is supported. But it forgot to add checking for it in transparent_hugepage_enabled(). To fix it, we add checking for read-only THP file mapping and also introduce helper transhuge_vma_enabled() to check whether thp is enabled for specified vma to reduce duplicated code. We rename transparent_hugepage_enabled to transparent_hugepage_active to make the code easier to follow as suggested by David Hildenbrand. [linmiaohe@huawei.com: define transhuge_vma_enabled next to transhuge_vma_suitable] Link: https://lkml.kernel.org/r/20210514093007.4117906-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210511134857.1581273-4-linmiaohe@huawei.com Fixes: 99cb0dbd47a1 ("mm,thp: add read-only THP support for (non-shmem) FS") Signed-off-by: Miaohe Lin Reviewed-by: Yang Shi Cc: Alexey Dobriyan Cc: "Aneesh Kumar K . V" Cc: Anshuman Khandual Cc: David Hildenbrand Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kirill A. Shutemov Cc: Matthew Wilcox Cc: Minchan Kim Cc: Ralph Campbell Cc: Rik van Riel Cc: Song Liu Cc: William Kucharski Cc: Zi Yan Cc: Mike Kravetz Signed-off-by: Andrew Morton --- fs/proc/task_mmu.c | 2 - include/linux/huge_mm.h | 57 +++++++++++++++++++++++--------------- mm/huge_memory.c | 11 ++++++- mm/khugepaged.c | 4 -- mm/shmem.c | 3 -- 5 files changed, 48 insertions(+), 29 deletions(-) --- a/fs/proc/task_mmu.c~mm-huge_memoryc-add-missing-read-only-thp-checking-in-transparent_hugepage_enabled +++ a/fs/proc/task_mmu.c @@ -832,7 +832,7 @@ static int show_smap(struct seq_file *m, __show_smap(m, &mss, false); seq_printf(m, "THPeligible: %d\n", - transparent_hugepage_enabled(vma)); + transparent_hugepage_active(vma)); if (arch_pkeys_enabled()) seq_printf(m, "ProtectionKey: %8u\n", vma_pkey(vma)); --- a/include/linux/huge_mm.h~mm-huge_memoryc-add-missing-read-only-thp-checking-in-transparent_hugepage_enabled +++ a/include/linux/huge_mm.h @@ -115,9 +115,34 @@ extern struct kobj_attribute shmem_enabl extern unsigned long transparent_hugepage_flags; +static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, + unsigned long haddr) +{ + /* Don't have to check pgoff for anonymous vma */ + if (!vma_is_anonymous(vma)) { + if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, + HPAGE_PMD_NR)) + return false; + } + + if (haddr < vma->vm_start || haddr + HPAGE_PMD_SIZE > vma->vm_end) + return false; + return true; +} + +static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + /* Explicitly disabled through madvise. */ + if ((vm_flags & VM_NOHUGEPAGE) || + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + return false; + return true; +} + /* * to be used on vmas which are known to support THP. - * Use transparent_hugepage_enabled otherwise + * Use transparent_hugepage_active otherwise */ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) { @@ -128,15 +153,12 @@ static inline bool __transparent_hugepag if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) return false; - if (vma->vm_flags & VM_NOHUGEPAGE) + if (!transhuge_vma_enabled(vma, vma->vm_flags)) return false; if (vma_is_temporary_stack(vma)) return false; - if (test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) - return false; - if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) return true; @@ -150,22 +172,7 @@ static inline bool __transparent_hugepag return false; } -bool transparent_hugepage_enabled(struct vm_area_struct *vma); - -static inline bool transhuge_vma_suitable(struct vm_area_struct *vma, - unsigned long haddr) -{ - /* Don't have to check pgoff for anonymous vma */ - if (!vma_is_anonymous(vma)) { - if (!IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, - HPAGE_PMD_NR)) - return false; - } - - if (haddr < vma->vm_start || haddr + HPAGE_PMD_SIZE > vma->vm_end) - return false; - return true; -} +bool transparent_hugepage_active(struct vm_area_struct *vma); #define transparent_hugepage_use_zero_page() \ (transparent_hugepage_flags & \ @@ -352,7 +359,7 @@ static inline bool __transparent_hugepag return false; } -static inline bool transparent_hugepage_enabled(struct vm_area_struct *vma) +static inline bool transparent_hugepage_active(struct vm_area_struct *vma) { return false; } @@ -362,6 +369,12 @@ static inline bool transhuge_vma_suitabl { return false; } + +static inline bool transhuge_vma_enabled(struct vm_area_struct *vma, + unsigned long vm_flags) +{ + return false; +} static inline void prep_transhuge_page(struct page *page) {} --- a/mm/huge_memory.c~mm-huge_memoryc-add-missing-read-only-thp-checking-in-transparent_hugepage_enabled +++ a/mm/huge_memory.c @@ -64,7 +64,14 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly = ~0UL; -bool transparent_hugepage_enabled(struct vm_area_struct *vma) +static inline bool file_thp_enabled(struct vm_area_struct *vma) +{ + return transhuge_vma_enabled(vma, vma->vm_flags) && vma->vm_file && + !inode_is_open_for_write(vma->vm_file->f_inode) && + (vma->vm_flags & VM_EXEC); +} + +bool transparent_hugepage_active(struct vm_area_struct *vma) { /* The addr is used to check if the vma size fits */ unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE; @@ -75,6 +82,8 @@ bool transparent_hugepage_enabled(struct return __transparent_hugepage_enabled(vma); if (vma_is_shmem(vma)) return shmem_huge_enabled(vma); + if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) + return file_thp_enabled(vma); return false; } --- a/mm/khugepaged.c~mm-huge_memoryc-add-missing-read-only-thp-checking-in-transparent_hugepage_enabled +++ a/mm/khugepaged.c @@ -442,9 +442,7 @@ static inline int khugepaged_test_exit(s static bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_flags) { - /* Explicitly disabled through madvise. */ - if ((vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + if (!transhuge_vma_enabled(vma, vm_flags)) return false; /* Enabled via shmem mount options or sysfs settings. */ --- a/mm/shmem.c~mm-huge_memoryc-add-missing-read-only-thp-checking-in-transparent_hugepage_enabled +++ a/mm/shmem.c @@ -4040,8 +4040,7 @@ bool shmem_huge_enabled(struct vm_area_s loff_t i_size; pgoff_t off; - if ((vma->vm_flags & VM_NOHUGEPAGE) || - test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags)) + if (!transhuge_vma_enabled(vma, vma->vm_flags)) return false; if (shmem_huge == SHMEM_HUGE_FORCE) return true; From patchwork Thu Jul 1 01:47:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9514C11F69 for ; Thu, 1 Jul 2021 01:47:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 90A4661468 for ; Thu, 1 Jul 2021 01:47:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 90A4661468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1631A8D01E2; Wed, 30 Jun 2021 21:47:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 112A48D01D0; Wed, 30 Jun 2021 21:47:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF5938D01E2; Wed, 30 Jun 2021 21:47:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0210.hostedemail.com [216.40.44.210]) by kanga.kvack.org (Postfix) with ESMTP id C60CA8D01D0 for ; Wed, 30 Jun 2021 21:47:55 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9C43D25F5D for ; Thu, 1 Jul 2021 01:47:55 +0000 (UTC) X-FDA: 78312332910.04.4F873A0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id 5EAFD6001A82 for ; Thu, 1 Jul 2021 01:47:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DDA9C6120C; Thu, 1 Jul 2021 01:47:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104074; bh=ktGpswfKHgSeup/foAAxXRZQdBofcRyy+/naJSEVGhg=; h=Date:From:To:Subject:In-Reply-To:From; b=SHNX+oE57T0UsFLB58lLtaKDo+5zrXJtw6RSTJSWDyQl2DBymN994o+rVbmSuFM/+ pvYrqO9TEDsxt79teQFtnMS0ghIzDUxAZCddfGMTc3tR76gMB5t+SClN9atVXdvZPq HQuJJXxSI8IYcxfjBr/kH3C726V+TvnJyDsvJrjI= Date: Wed, 30 Jun 2021 18:47:53 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, anshuman.khandual@arm.com, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, riel@surriel.com, shy828301@gmail.com, songliubraving@fb.com, torvalds@linux-foundation.org, william.kucharski@oracle.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 015/192] mm/huge_memory.c: remove unnecessary tlb_remove_page_size() for huge zero pmd Message-ID: <20210701014753.FkgXLukHD%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=SHNX+oE5; spf=pass (imf10.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5EAFD6001A82 X-Stat-Signature: mssjdk85r8rrfsieigqstc9qsdiw3r4w X-HE-Tag: 1625104075-284543 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/huge_memory.c: remove unnecessary tlb_remove_page_size() for huge zero pmd Commit aa88b68c3b1d ("thp: keep huge zero page pinned until tlb flush") introduced tlb_remove_page() for huge zero page to keep it pinned until flush is complete and prevents the page from being split under us. But huge zero page is kept pinned until all relevant mm_users reach zero since the commit 6fcb52a56ff6 ("thp: reduce usage of huge zero page's atomic counter"). So tlb_remove_page_size() for huge zero pmd is unnecessary now. Link: https://lkml.kernel.org/r/20210511134857.1581273-5-linmiaohe@huawei.com Reviewed-by: Yang Shi Acked-by: David Hildenbrand Signed-off-by: Miaohe Lin Cc: Alexey Dobriyan Cc: "Aneesh Kumar K . V" Cc: Anshuman Khandual Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kirill A. Shutemov Cc: Matthew Wilcox Cc: Minchan Kim Cc: Ralph Campbell Cc: Rik van Riel Cc: Song Liu Cc: William Kucharski Cc: Zi Yan Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/huge_memory.c | 3 --- 1 file changed, 3 deletions(-) --- a/mm/huge_memory.c~mm-huge_memoryc-remove-unnecessary-tlb_remove_page_size-for-huge-zero-pmd +++ a/mm/huge_memory.c @@ -1686,12 +1686,9 @@ int zap_huge_pmd(struct mmu_gather *tlb, if (arch_needs_pgtable_deposit()) zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); - if (is_huge_zero_pmd(orig_pmd)) - tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE); } else if (is_huge_zero_pmd(orig_pmd)) { zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); - tlb_remove_page_size(tlb, pmd_page(orig_pmd), HPAGE_PMD_SIZE); } else { struct page *page = NULL; int flush_needed = 1; From patchwork Thu Jul 1 01:47:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CE8CC11F69 for ; Thu, 1 Jul 2021 01:48:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E5B9D61090 for ; Thu, 1 Jul 2021 01:47:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5B9D61090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E0AD8D01E3; Wed, 30 Jun 2021 21:47:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 690528D01D0; Wed, 30 Jun 2021 21:47:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 531258D01E3; Wed, 30 Jun 2021 21:47:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id 28DC38D01D0 for ; Wed, 30 Jun 2021 21:47:59 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 056FF25F5A for ; Thu, 1 Jul 2021 01:47:59 +0000 (UTC) X-FDA: 78312333078.31.736F9FC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id AD3AFE003050 for ; Thu, 1 Jul 2021 01:47:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5ED9061468; Thu, 1 Jul 2021 01:47:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104078; bh=CWbtpEeKSCBvywFsbct1J9BSrRL6Ri6SyG74X8TBURA=; h=Date:From:To:Subject:In-Reply-To:From; b=Zsf4cdI376IqMUe4nNxrjAyWoDIu3BwiRBNCkl+8f9addegcFSAQAacClf3l4Tjrv oeTS6qWA3rfRcd9YQx9D3AsbbrBLKNnn44aIBgP1kGSvoLcLbY+1bh3Ak0Aic7ytww iP4c327T11qq3v4N1yZ4j1h1dqjgncBAQ1lpAIQs= Date: Wed, 30 Jun 2021 18:47:57 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, anshuman.khandual@arm.com, david@redhat.com, hannes@cmpxchg.org, hughd@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, rcampbell@nvidia.com, riel@surriel.com, shy828301@gmail.com, songliubraving@fb.com, torvalds@linux-foundation.org, william.kucharski@oracle.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 016/192] mm/huge_memory.c: don't discard hugepage if other processes are mapping it Message-ID: <20210701014757.SdeAPP5pY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Zsf4cdI3; spf=pass (imf30.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 8hpd67d9hhf3djouu6jkptg4omif15yk X-Rspamd-Queue-Id: AD3AFE003050 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104078-978107 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/huge_memory.c: don't discard hugepage if other processes are mapping it If other processes are mapping any other subpages of the hugepage, i.e. in pte-mapped thp case, page_mapcount() will return 1 incorrectly. Then we would discard the page while other processes are still mapping it. Fix it by using total_mapcount() which can tell whether other processes are still mapping it. Link: https://lkml.kernel.org/r/20210511134857.1581273-6-linmiaohe@huawei.com Fixes: b8d3c4c3009d ("mm/huge_memory.c: don't split THP page when MADV_FREE syscall is called") Reviewed-by: Yang Shi Signed-off-by: Miaohe Lin Cc: Alexey Dobriyan Cc: "Aneesh Kumar K . V" Cc: Anshuman Khandual Cc: David Hildenbrand Cc: Hugh Dickins Cc: Johannes Weiner Cc: Kirill A. Shutemov Cc: Matthew Wilcox Cc: Minchan Kim Cc: Ralph Campbell Cc: Rik van Riel Cc: Song Liu Cc: William Kucharski Cc: Zi Yan Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/huge_memory.c~mm-huge_memoryc-dont-discard-hugepage-if-other-processes-are-mapping-it +++ a/mm/huge_memory.c @@ -1613,7 +1613,7 @@ bool madvise_free_huge_pmd(struct mmu_ga * If other processes are mapping this page, we couldn't discard * the page unless they all do MADV_FREE so let's skip the page. */ - if (page_mapcount(page) != 1) + if (total_mapcount(page) != 1) goto out; if (!trylock_page(page)) From patchwork Thu Jul 1 01:48:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EDEFC11F68 for ; Thu, 1 Jul 2021 01:48:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34B3A61090 for ; Thu, 1 Jul 2021 01:48:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34B3A61090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD7DD8D01E4; Wed, 30 Jun 2021 21:48:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AAEE18D01D0; Wed, 30 Jun 2021 21:48:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 976B78D01E4; Wed, 30 Jun 2021 21:48:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0221.hostedemail.com [216.40.44.221]) by kanga.kvack.org (Postfix) with ESMTP id 69E2B8D01D0 for ; Wed, 30 Jun 2021 21:48:02 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3D0AE22030 for ; Thu, 1 Jul 2021 01:48:02 +0000 (UTC) X-FDA: 78312333204.34.DE92129 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id D9F031985 for ; Thu, 1 Jul 2021 01:48:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C02A661469; Thu, 1 Jul 2021 01:48:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104081; bh=IQEN/M12f0PORBwYj/FDdHYtEI7KhRyewLQGoOKbkvQ=; h=Date:From:To:Subject:In-Reply-To:From; b=bl4klmpcpfmNTJY6Huyl0fat4vD/SgoEuXWcYNYPPW80vZA9XFB4CFR/yKXSxXDFT fg0lfMd7HqM+/uKdt5TTUuCkbMa7OcGnKCrGY/LxOH/CIlo4m3XZ00B7CMzUqkEGte OxbpAdzUFh47s/2eiLmEvUYCMwDcz4fdbrQMux+8= Date: Wed, 30 Jun 2021 18:48:00 -0700 From: Andrew Morton To: akpm@linux-foundation.org, benh@kernel.crashing.org, christophe.leroy@csgroup.eu, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, paulus@samba.org, rppt@kernel.org, torvalds@linux-foundation.org, uladzislau.rezki@sony.com Subject: [patch 017/192] mm/hugetlb: change parameters of arch_make_huge_pte() Message-ID: <20210701014800.hyS9J3TRY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D9F031985 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=bl4klmpc; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 131mf4gwgnqqdfe5aimzx87ohtuhhu78 X-HE-Tag: 1625104081-177034 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christophe Leroy Subject: mm/hugetlb: change parameters of arch_make_huge_pte() Patch series "Subject: [PATCH v2 0/5] Implement huge VMAP and VMALLOC on powerpc 8xx", v2. This series implements huge VMAP and VMALLOC on powerpc 8xx. Powerpc 8xx has 4 page sizes: - 4k - 16k - 512k - 8M At the time being, vmalloc and vmap only support huge pages which are leaf at PMD level. Here the PMD level is 4M, it doesn't correspond to any supported page size. For now, implement use of 16k and 512k pages which is done at PTE level. Support of 8M pages will be implemented later, it requires use of hugepd tables. To allow this, the architecture provides two functions: - arch_vmap_pte_range_map_size() which tells vmap_pte_range() what page size to use. A stub returning PAGE_SIZE is provided when the architecture doesn't provide this function. - arch_vmap_pte_supported_shift() which tells __vmalloc_node_range() what page shift to use for a given area size. A stub returning PAGE_SHIFT is provided when the architecture doesn't provide this function. This patch (of 5): At the time being, arch_make_huge_pte() has the following prototype: pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, struct page *page, int writable); vma is used to get the pages shift or size. vma is also used on Sparc to get vm_flags. page is not used. writable is not used. In order to use this function without a vma, replace vma by shift and flags. Also remove the used parameters. Link: https://lkml.kernel.org/r/cover.1620795204.git.christophe.leroy@csgroup.eu Link: https://lkml.kernel.org/r/f4633ac6a7da2f22f31a04a89e0a7026bb78b15b.1620795204.git.christophe.leroy@csgroup.eu Signed-off-by: Christophe Leroy Acked-by: Mike Kravetz Cc: Nicholas Piggin Cc: Mike Kravetz Cc: Mike Rapoport Cc: Michael Ellerman Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Uladzislau Rezki Signed-off-by: Andrew Morton --- arch/arm64/include/asm/hugetlb.h | 3 +-- arch/arm64/mm/hugetlbpage.c | 5 ++--- arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h | 5 ++--- arch/sparc/include/asm/pgtable_64.h | 3 +-- arch/sparc/mm/hugetlbpage.c | 6 ++---- include/linux/hugetlb.h | 4 ++-- mm/hugetlb.c | 6 ++++-- mm/migrate.c | 4 +++- 8 files changed, 17 insertions(+), 19 deletions(-) --- a/arch/arm64/include/asm/hugetlb.h~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/arch/arm64/include/asm/hugetlb.h @@ -23,8 +23,7 @@ static inline void arch_clear_hugepage_f } #define arch_clear_hugepage_flags arch_clear_hugepage_flags -extern pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable); +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte #define __HAVE_ARCH_HUGE_SET_HUGE_PTE_AT extern void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, --- a/arch/arm64/mm/hugetlbpage.c~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/arch/arm64/mm/hugetlbpage.c @@ -339,10 +339,9 @@ pte_t *huge_pte_offset(struct mm_struct return NULL; } -pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable) +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { - size_t pagesize = huge_page_size(hstate_vma(vma)); + size_t pagesize = 1UL << shift; if (pagesize == CONT_PTE_SIZE) { entry = pte_mkcont(entry); --- a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/arch/powerpc/include/asm/nohash/32/hugetlb-8xx.h @@ -66,10 +66,9 @@ static inline void huge_ptep_set_wrprote } #ifdef CONFIG_PPC_4K_PAGES -static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable) +static inline pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { - size_t size = huge_page_size(hstate_vma(vma)); + size_t size = 1UL << shift; if (size == SZ_16K) return __pte(pte_val(entry) & ~_PAGE_HUGE); --- a/arch/sparc/include/asm/pgtable_64.h~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/arch/sparc/include/asm/pgtable_64.h @@ -377,8 +377,7 @@ static inline pgprot_t pgprot_noncached( #define pgprot_noncached pgprot_noncached #if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE) -extern pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable); +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags); #define arch_make_huge_pte arch_make_huge_pte static inline unsigned long __pte_default_huge_mask(void) { --- a/arch/sparc/mm/hugetlbpage.c~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/arch/sparc/mm/hugetlbpage.c @@ -177,10 +177,8 @@ static pte_t hugepage_shift_to_tte(pte_t return sun4u_hugepage_shift_to_tte(entry, shift); } -pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writeable) +pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, vm_flags_t flags) { - unsigned int shift = huge_page_shift(hstate_vma(vma)); pte_t pte; pte = hugepage_shift_to_tte(entry, shift); @@ -188,7 +186,7 @@ pte_t arch_make_huge_pte(pte_t entry, st #ifdef CONFIG_SPARC64 /* If this vma has ADI enabled on it, turn on TTE.mcd */ - if (vma->vm_flags & VM_SPARC_ADI) + if (flags & VM_SPARC_ADI) return pte_mkmcd(pte); else return pte_mknotmcd(pte); --- a/include/linux/hugetlb.h~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/include/linux/hugetlb.h @@ -741,8 +741,8 @@ static inline void arch_clear_hugepage_f #endif #ifndef arch_make_huge_pte -static inline pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma, - struct page *page, int writable) +static inline pte_t arch_make_huge_pte(pte_t entry, unsigned int shift, + vm_flags_t flags) { return entry; } --- a/mm/hugetlb.c~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/mm/hugetlb.c @@ -4060,6 +4060,7 @@ static pte_t make_huge_pte(struct vm_are int writable) { pte_t entry; + unsigned int shift = huge_page_shift(hstate_vma(vma)); if (writable) { entry = huge_pte_mkwrite(huge_pte_mkdirty(mk_huge_pte(page, @@ -4070,7 +4071,7 @@ static pte_t make_huge_pte(struct vm_are } entry = pte_mkyoung(entry); entry = pte_mkhuge(entry); - entry = arch_make_huge_pte(entry, vma, page, writable); + entry = arch_make_huge_pte(entry, shift, vma->vm_flags); return entry; } @@ -5468,10 +5469,11 @@ unsigned long hugetlb_change_protection( } if (!huge_pte_none(pte)) { pte_t old_pte; + unsigned int shift = huge_page_shift(hstate_vma(vma)); old_pte = huge_ptep_modify_prot_start(vma, address, ptep); pte = pte_mkhuge(huge_pte_modify(old_pte, newprot)); - pte = arch_make_huge_pte(pte, vma, NULL, 0); + pte = arch_make_huge_pte(pte, shift, vma->vm_flags); huge_ptep_modify_prot_commit(vma, address, ptep, old_pte, pte); pages++; } --- a/mm/migrate.c~mm-hugetlb-change-parameters-of-arch_make_huge_pte +++ a/mm/migrate.c @@ -226,8 +226,10 @@ static bool remove_migration_pte(struct #ifdef CONFIG_HUGETLB_PAGE if (PageHuge(new)) { + unsigned int shift = huge_page_shift(hstate_vma(vma)); + pte = pte_mkhuge(pte); - pte = arch_make_huge_pte(pte, vma, new, 0); + pte = arch_make_huge_pte(pte, shift, vma->vm_flags); set_huge_pte_at(vma->vm_mm, pvmw.address, pvmw.pte, pte); if (PageAnon(new)) hugepage_add_anon_rmap(new, vma, pvmw.address); From patchwork Thu Jul 1 01:48:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5310C11F6B for ; Thu, 1 Jul 2021 01:48:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5E66261468 for ; Thu, 1 Jul 2021 01:48:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E66261468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D4A548D01E5; Wed, 30 Jun 2021 21:48:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD2178D01D0; Wed, 30 Jun 2021 21:48:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4B838D01E5; Wed, 30 Jun 2021 21:48:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id 8994A8D01D0 for ; Wed, 30 Jun 2021 21:48:05 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4E23A252A0 for ; Thu, 1 Jul 2021 01:48:05 +0000 (UTC) X-FDA: 78312333330.03.6B2A833 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id F0A6EF0000AC for ; Thu, 1 Jul 2021 01:48:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E298E61090; Thu, 1 Jul 2021 01:48:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104084; bh=W04BZQyHI8Y3cnY8wTCOhMWigds5hckFeLoTe4oVIRQ=; h=Date:From:To:Subject:In-Reply-To:From; b=BK1wRMW7+OMItAfYulA4yPOaEw/LjA9PjA/jNEYhfJolzoN8Kbihvuo6Wq0wvjSOD Jl1+nsFvYYshMML9f/05LnEuCUDCoKzoxxY489fb/qzqYjGPEu8t6zjYXQXxYJHN5+ zPt1vBwuYYaf3/PMD7bXToiCmy2FR0x3rRKYJCIM= Date: Wed, 30 Jun 2021 18:48:03 -0700 From: Andrew Morton To: akpm@linux-foundation.org, benh@kernel.crashing.org, christophe.leroy@csgroup.eu, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, naresh.kamboju@linaro.org, npiggin@gmail.com, paulus@samba.org, rppt@kernel.org, torvalds@linux-foundation.org, uladzislau.rezki@sony.com Subject: [patch 018/192] mm/pgtable: add stubs for {pmd/pub}_{set/clear}_huge Message-ID: <20210701014803.NuCUAlJTH%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=BK1wRMW7; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: yinkx3h6caximfkxzzrachfcy93m8mbu X-Rspamd-Queue-Id: F0A6EF0000AC X-Rspamd-Server: rspam06 X-HE-Tag: 1625104084-287546 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christophe Leroy Subject: mm/pgtable: add stubs for {pmd/pub}_{set/clear}_huge For architectures with no PMD and/or no PUD, add stubs similar to what we have for architectures without P4D. [christophe.leroy@csgroup.eu: arm64: define only {pud/pmd}_{set/clear}_huge when useful] Link: https://lkml.kernel.org/r/73ec95f40cafbbb69bdfb43a7f53876fd845b0ce.1620990479.git.christophe.leroy@csgroup.eu [christophe.leroy@csgroup.eu: x86: define only {pud/pmd}_{set/clear}_huge when useful] Link: https://lkml.kernel.org/r/7fbf1b6bc3e15c07c24fa45278d57064f14c896b.1620930415.git.christophe.leroy@csgroup.eu Link: https://lkml.kernel.org/r/5ac5976419350e8e048d463a64cae449eb3ba4b0.1620795204.git.christophe.leroy@csgroup.eu Signed-off-by: Christophe Leroy Cc: Benjamin Herrenschmidt Cc: Michael Ellerman Cc: Mike Kravetz Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Paul Mackerras Cc: Uladzislau Rezki Cc: Naresh Kamboju Signed-off-by: Andrew Morton --- arch/arm64/mm/mmu.c | 20 ++++++++++++-------- arch/x86/mm/pgtable.c | 34 +++++++++++++++++++--------------- include/linux/pgtable.h | 26 +++++++++++++++++++++++++- 3 files changed, 56 insertions(+), 24 deletions(-) --- a/arch/arm64/mm/mmu.c~mm-pgtable-add-stubs-for-pmd-pub_set-clear_huge +++ a/arch/arm64/mm/mmu.c @@ -1338,6 +1338,7 @@ void *__init fixmap_remap_fdt(phys_addr_ return dt_virt; } +#if CONFIG_PGTABLE_LEVELS > 3 int pud_set_huge(pud_t *pudp, phys_addr_t phys, pgprot_t prot) { pud_t new_pud = pfn_pud(__phys_to_pfn(phys), mk_pud_sect_prot(prot)); @@ -1352,6 +1353,16 @@ int pud_set_huge(pud_t *pudp, phys_addr_ return 1; } +int pud_clear_huge(pud_t *pudp) +{ + if (!pud_sect(READ_ONCE(*pudp))) + return 0; + pud_clear(pudp); + return 1; +} +#endif + +#if CONFIG_PGTABLE_LEVELS > 2 int pmd_set_huge(pmd_t *pmdp, phys_addr_t phys, pgprot_t prot) { pmd_t new_pmd = pfn_pmd(__phys_to_pfn(phys), mk_pmd_sect_prot(prot)); @@ -1366,14 +1377,6 @@ int pmd_set_huge(pmd_t *pmdp, phys_addr_ return 1; } -int pud_clear_huge(pud_t *pudp) -{ - if (!pud_sect(READ_ONCE(*pudp))) - return 0; - pud_clear(pudp); - return 1; -} - int pmd_clear_huge(pmd_t *pmdp) { if (!pmd_sect(READ_ONCE(*pmdp))) @@ -1381,6 +1384,7 @@ int pmd_clear_huge(pmd_t *pmdp) pmd_clear(pmdp); return 1; } +#endif int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) { --- a/arch/x86/mm/pgtable.c~mm-pgtable-add-stubs-for-pmd-pub_set-clear_huge +++ a/arch/x86/mm/pgtable.c @@ -682,6 +682,7 @@ int p4d_clear_huge(p4d_t *p4d) } #endif +#if CONFIG_PGTABLE_LEVELS > 3 /** * pud_set_huge - setup kernel PUD mapping * @@ -721,6 +722,23 @@ int pud_set_huge(pud_t *pud, phys_addr_t } /** + * pud_clear_huge - clear kernel PUD mapping when it is set + * + * Returns 1 on success and 0 on failure (no PUD map is found). + */ +int pud_clear_huge(pud_t *pud) +{ + if (pud_large(*pud)) { + pud_clear(pud); + return 1; + } + + return 0; +} +#endif + +#if CONFIG_PGTABLE_LEVELS > 2 +/** * pmd_set_huge - setup kernel PMD mapping * * See text over pud_set_huge() above. @@ -751,21 +769,6 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t } /** - * pud_clear_huge - clear kernel PUD mapping when it is set - * - * Returns 1 on success and 0 on failure (no PUD map is found). - */ -int pud_clear_huge(pud_t *pud) -{ - if (pud_large(*pud)) { - pud_clear(pud); - return 1; - } - - return 0; -} - -/** * pmd_clear_huge - clear kernel PMD mapping when it is set * * Returns 1 on success and 0 on failure (no PMD map is found). @@ -779,6 +782,7 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +#endif #ifdef CONFIG_X86_64 /** --- a/include/linux/pgtable.h~mm-pgtable-add-stubs-for-pmd-pub_set-clear_huge +++ a/include/linux/pgtable.h @@ -1379,10 +1379,34 @@ static inline int p4d_clear_huge(p4d_t * } #endif /* !__PAGETABLE_P4D_FOLDED */ +#ifndef __PAGETABLE_PUD_FOLDED int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); -int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); +#else +static inline int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) +{ + return 0; +} +static inline int pud_clear_huge(pud_t *pud) +{ + return 0; +} +#endif /* !__PAGETABLE_PUD_FOLDED */ + +#ifndef __PAGETABLE_PMD_FOLDED +int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pmd_clear_huge(pmd_t *pmd); +#else +static inline int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot) +{ + return 0; +} +static inline int pmd_clear_huge(pmd_t *pmd) +{ + return 0; +} +#endif /* !__PAGETABLE_PMD_FOLDED */ + int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); From patchwork Thu Jul 1 01:48:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C230CC11F69 for ; Thu, 1 Jul 2021 01:48:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 773E361466 for ; Thu, 1 Jul 2021 01:48:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 773E361466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 002788D01E6; Wed, 30 Jun 2021 21:48:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EF4C38D01D0; Wed, 30 Jun 2021 21:48:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBC3A8D01E6; Wed, 30 Jun 2021 21:48:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id B1BF78D01D0 for ; Wed, 30 Jun 2021 21:48:08 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8FBCB808E8EA for ; Thu, 1 Jul 2021 01:48:08 +0000 (UTC) X-FDA: 78312333456.31.0A65966 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 3FD919000328 for ; Thu, 1 Jul 2021 01:48:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0CF2B61468; Thu, 1 Jul 2021 01:48:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104087; bh=1Tnqcjpn1lxKVF+SS8i3MXAK1LI4r9vPnrvG0YG50bU=; h=Date:From:To:Subject:In-Reply-To:From; b=eRAJ6jO4Dfq2bd49M1OixD0/l5ALmKgdgEItqVOWav1ps2Qy4D62p5c+SO4Jmr8pU 0I18UYZWGPJOkvMRYiR1d+OUfwfOTUASJeRTNzKRThmGle5+mIaBEpDeA77nT9k9br gCED/CyONM6+jeOnevamhtGUg5NUOU9MuOOF3yG4= Date: Wed, 30 Jun 2021 18:48:06 -0700 From: Andrew Morton To: akpm@linux-foundation.org, benh@kernel.crashing.org, christophe.leroy@csgroup.eu, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, paulus@samba.org, rppt@kernel.org, torvalds@linux-foundation.org, uladzislau.rezki@sony.com Subject: [patch 019/192] mm/vmalloc: enable mapping of huge pages at pte level in vmap Message-ID: <20210701014806.AhSMhR7Z2%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3FD919000328 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=eRAJ6jO4; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: ryws98i88hhkfpjoh57kryz7gew76mdx X-HE-Tag: 1625104088-94376 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christophe Leroy Subject: mm/vmalloc: enable mapping of huge pages at pte level in vmap On some architectures like powerpc, there are huge pages that are mapped at pte level. Enable it in vmap. For that, architectures can provide arch_vmap_pte_range_map_size() that returns the size of pages to map at pte level. Link: https://lkml.kernel.org/r/fb3ccc73377832ac6708181ec419128a2f98ce36.1620795204.git.christophe.leroy@csgroup.eu Signed-off-by: Christophe Leroy Cc: Benjamin Herrenschmidt Cc: Michael Ellerman Cc: Mike Kravetz Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Paul Mackerras Cc: Uladzislau Rezki Signed-off-by: Andrew Morton --- include/linux/vmalloc.h | 8 ++++++++ mm/vmalloc.c | 21 ++++++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) --- a/include/linux/vmalloc.h~mm-vmalloc-enable-mapping-of-huge-pages-at-pte-level-in-vmap +++ a/include/linux/vmalloc.h @@ -104,6 +104,14 @@ static inline bool arch_vmap_pmd_support } #endif +#ifndef arch_vmap_pte_range_map_size +static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr, unsigned long end, + u64 pfn, unsigned int max_page_shift) +{ + return PAGE_SIZE; +} +#endif + /* * Highlevel APIs for driver use */ --- a/mm/vmalloc.c~mm-vmalloc-enable-mapping-of-huge-pages-at-pte-level-in-vmap +++ a/mm/vmalloc.c @@ -36,6 +36,7 @@ #include #include #include +#include #include #include @@ -83,10 +84,11 @@ static void free_work(struct work_struct /*** Page table manipulation functions ***/ static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot, - pgtbl_mod_mask *mask) + unsigned int max_page_shift, pgtbl_mod_mask *mask) { pte_t *pte; u64 pfn; + unsigned long size = PAGE_SIZE; pfn = phys_addr >> PAGE_SHIFT; pte = pte_alloc_kernel_track(pmd, addr, mask); @@ -94,9 +96,22 @@ static int vmap_pte_range(pmd_t *pmd, un return -ENOMEM; do { BUG_ON(!pte_none(*pte)); + +#ifdef CONFIG_HUGETLB_PAGE + size = arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift); + if (size != PAGE_SIZE) { + pte_t entry = pfn_pte(pfn, prot); + + entry = pte_mkhuge(entry); + entry = arch_make_huge_pte(entry, ilog2(size), 0); + set_huge_pte_at(&init_mm, addr, pte, entry); + pfn += PFN_DOWN(size); + continue; + } +#endif set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); pfn++; - } while (pte++, addr += PAGE_SIZE, addr != end); + } while (pte += PFN_DOWN(size), addr += size, addr != end); *mask |= PGTBL_PTE_MODIFIED; return 0; } @@ -145,7 +160,7 @@ static int vmap_pmd_range(pud_t *pud, un continue; } - if (vmap_pte_range(pmd, addr, next, phys_addr, prot, mask)) + if (vmap_pte_range(pmd, addr, next, phys_addr, prot, max_page_shift, mask)) return -ENOMEM; } while (pmd++, phys_addr += (next - addr), addr = next, addr != end); return 0; From patchwork Thu Jul 1 01:48:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9CE88C11F6A for ; Thu, 1 Jul 2021 01:48:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 540F961474 for ; Thu, 1 Jul 2021 01:48:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 540F961474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD3D68D01E7; Wed, 30 Jun 2021 21:48:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C83978D01D0; Wed, 30 Jun 2021 21:48:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4B968D01E7; Wed, 30 Jun 2021 21:48:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 8D4E88D01D0 for ; Wed, 30 Jun 2021 21:48:11 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6C10420408 for ; Thu, 1 Jul 2021 01:48:11 +0000 (UTC) X-FDA: 78312333582.38.CB88A39 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 322127000081 for ; Thu, 1 Jul 2021 01:48:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2DF0261466; Thu, 1 Jul 2021 01:48:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104090; bh=YsFm77K8nv/9Df5K9SwD+Opx/GDtruwXJE9noDhJd8Q=; h=Date:From:To:Subject:In-Reply-To:From; b=CbacaMLHBIPgIXnvkevOzQ0DnbimY2zmo4bqkKzISJ0fUt6RAbFBiKrU89Sv7dLve Fwjzg2TAThGX8X+5k2AaPqeNKPrulw6q5pkL+mOqwcWb+PJJkkhqxjZCXnespWu8nb fXwGY9cxxKwvtaMb6hjkdUtABvfwd/JDnxgC4wyE= Date: Wed, 30 Jun 2021 18:48:09 -0700 From: Andrew Morton To: akpm@linux-foundation.org, benh@kernel.crashing.org, christophe.leroy@csgroup.eu, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, paulus@samba.org, rppt@kernel.org, torvalds@linux-foundation.org, uladzislau.rezki@sony.com Subject: [patch 020/192] mm/vmalloc: enable mapping of huge pages at pte level in vmalloc Message-ID: <20210701014809.lGVB3ZolV%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 322127000081 X-Stat-Signature: 3oowzerotrhu8s9bhurckjqpfqd6ojz7 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=CbacaMLH; dmarc=none; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104091-606394 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christophe Leroy Subject: mm/vmalloc: enable mapping of huge pages at pte level in vmalloc On some architectures like powerpc, there are huge pages that are mapped at pte level. Enable it in vmalloc. For that, architectures can provide arch_vmap_pte_supported_shift() that returns the shift for pages to map at pte level. Link: https://lkml.kernel.org/r/2c717e3b1fba1894d890feb7669f83025bfa314d.1620795204.git.christophe.leroy@csgroup.eu Signed-off-by: Christophe Leroy Cc: Benjamin Herrenschmidt Cc: Michael Ellerman Cc: Mike Kravetz Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Paul Mackerras Cc: Uladzislau Rezki Signed-off-by: Andrew Morton --- include/linux/vmalloc.h | 7 +++++++ mm/vmalloc.c | 13 +++++++------ 2 files changed, 14 insertions(+), 6 deletions(-) --- a/include/linux/vmalloc.h~mm-vmalloc-enable-mapping-of-huge-pages-at-pte-level-in-vmalloc +++ a/include/linux/vmalloc.h @@ -112,6 +112,13 @@ static inline unsigned long arch_vmap_pt } #endif +#ifndef arch_vmap_pte_supported_shift +static inline int arch_vmap_pte_supported_shift(unsigned long size) +{ + return PAGE_SHIFT; +} +#endif + /* * Highlevel APIs for driver use */ --- a/mm/vmalloc.c~mm-vmalloc-enable-mapping-of-huge-pages-at-pte-level-in-vmalloc +++ a/mm/vmalloc.c @@ -2927,8 +2927,7 @@ void *__vmalloc_node_range(unsigned long return NULL; } - if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP) && - arch_vmap_pmd_supported(prot)) { + if (vmap_allow_huge && !(vm_flags & VM_NO_HUGE_VMAP)) { unsigned long size_per_node; /* @@ -2941,11 +2940,13 @@ void *__vmalloc_node_range(unsigned long size_per_node = size; if (node == NUMA_NO_NODE) size_per_node /= num_online_nodes(); - if (size_per_node >= PMD_SIZE) { + if (arch_vmap_pmd_supported(prot) && size_per_node >= PMD_SIZE) shift = PMD_SHIFT; - align = max(real_align, 1UL << shift); - size = ALIGN(real_size, 1UL << shift); - } + else + shift = arch_vmap_pte_supported_shift(size_per_node); + + align = max(real_align, 1UL << shift); + size = ALIGN(real_size, 1UL << shift); } again: From patchwork Thu Jul 1 01:48:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353101 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1682EC11F69 for ; Thu, 1 Jul 2021 01:48:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BA37961090 for ; Thu, 1 Jul 2021 01:48:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA37961090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3D3268D01E8; Wed, 30 Jun 2021 21:48:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A9E58D01D0; Wed, 30 Jun 2021 21:48:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2719C8D01E8; Wed, 30 Jun 2021 21:48:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id F3C668D01D0 for ; Wed, 30 Jun 2021 21:48:14 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C66C9808E8EA for ; Thu, 1 Jul 2021 01:48:14 +0000 (UTC) X-FDA: 78312333708.01.FE90C85 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 78278B000099 for ; Thu, 1 Jul 2021 01:48:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4FB7B61474; Thu, 1 Jul 2021 01:48:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104093; bh=nsFe5IGqEtRPPNthLsJQfTBDVMzXOIkTejzq3hr86ko=; h=Date:From:To:Subject:In-Reply-To:From; b=b9OKVP+TrPX0RTpRbgGlo/3bPUYlolXzarUb260sldCec0F9OQuCnnpwVa3rbiM75 dVvaB8DJIgoaDSheqcBB0F5AjUXul3NsmioDpwnkrcQ0aNoBzSlv1i/tTaUkV6yp2Y +StMmT7NpxXHLTr+98qvcZR5X8i08OUF6AtyqUPc= Date: Wed, 30 Jun 2021 18:48:12 -0700 From: Andrew Morton To: akpm@linux-foundation.org, benh@kernel.crashing.org, christophe.leroy@csgroup.eu, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, paulus@samba.org, rppt@kernel.org, torvalds@linux-foundation.org, uladzislau.rezki@sony.com Subject: [patch 021/192] powerpc/8xx: add support for huge pages on VMAP and VMALLOC Message-ID: <20210701014812.fzJjD0Pwv%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=b9OKVP+T; dmarc=none; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: t6zdyn9figy9z6xtqug91mzqbu1gs6nn X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 78278B000099 X-HE-Tag: 1625104094-197467 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christophe Leroy Subject: powerpc/8xx: add support for huge pages on VMAP and VMALLOC powerpc 8xx has 4 page sizes: - 4k - 16k - 512k - 8M At the time being, vmalloc and vmap only support huge pages which are leaf at PMD level. Here the PMD level is 4M, it doesn't correspond to any supported page size. For now, implement use of 16k and 512k pages which is done at PTE level. Support of 8M pages will be implemented later, it requires vmalloc to support hugepd tables. Link: https://lkml.kernel.org/r/8b972f1c03fb6bd59953035f0a3e4d26659de4f8.1620795204.git.christophe.leroy@csgroup.eu Signed-off-by: Christophe Leroy Cc: Benjamin Herrenschmidt Cc: Michael Ellerman Cc: Mike Kravetz Cc: Mike Rapoport Cc: Nicholas Piggin Cc: Paul Mackerras Cc: Uladzislau Rezki Signed-off-by: Andrew Morton --- arch/powerpc/Kconfig | 2 arch/powerpc/include/asm/nohash/32/mmu-8xx.h | 43 +++++++++++++++++ 2 files changed, 44 insertions(+), 1 deletion(-) --- a/arch/powerpc/include/asm/nohash/32/mmu-8xx.h~powerpc-8xx-add-support-for-huge-pages-on-vmap-and-vmalloc +++ a/arch/powerpc/include/asm/nohash/32/mmu-8xx.h @@ -178,6 +178,7 @@ #ifndef __ASSEMBLY__ #include +#include void mmu_pin_tlb(unsigned long top, bool readonly); @@ -225,6 +226,48 @@ static inline unsigned int mmu_psize_to_ BUG(); } +static inline bool arch_vmap_try_size(unsigned long addr, unsigned long end, u64 pfn, + unsigned int max_page_shift, unsigned long size) +{ + if (end - addr < size) + return false; + + if ((1UL << max_page_shift) < size) + return false; + + if (!IS_ALIGNED(addr, size)) + return false; + + if (!IS_ALIGNED(PFN_PHYS(pfn), size)) + return false; + + return true; +} + +static inline unsigned long arch_vmap_pte_range_map_size(unsigned long addr, unsigned long end, + u64 pfn, unsigned int max_page_shift) +{ + if (arch_vmap_try_size(addr, end, pfn, max_page_shift, SZ_512K)) + return SZ_512K; + if (PAGE_SIZE == SZ_16K) + return SZ_16K; + if (arch_vmap_try_size(addr, end, pfn, max_page_shift, SZ_16K)) + return SZ_16K; + return PAGE_SIZE; +} +#define arch_vmap_pte_range_map_size arch_vmap_pte_range_map_size + +static inline int arch_vmap_pte_supported_shift(unsigned long size) +{ + if (size >= SZ_512K) + return 19; + else if (size >= SZ_16K) + return 14; + else + return PAGE_SHIFT; +} +#define arch_vmap_pte_supported_shift arch_vmap_pte_supported_shift + /* patch sites */ extern s32 patch__itlbmiss_exit_1, patch__dtlbmiss_exit_1; extern s32 patch__itlbmiss_perf, patch__dtlbmiss_perf; --- a/arch/powerpc/Kconfig~powerpc-8xx-add-support-for-huge-pages-on-vmap-and-vmalloc +++ a/arch/powerpc/Kconfig @@ -187,7 +187,7 @@ config PPC select GENERIC_VDSO_TIME_NS select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_HUGE_VMALLOC if HAVE_ARCH_HUGE_VMAP - select HAVE_ARCH_HUGE_VMAP if PPC_BOOK3S_64 && PPC_RADIX_MMU + select HAVE_ARCH_HUGE_VMAP if PPC_RADIX_MMU || PPC_8xx select HAVE_ARCH_JUMP_LABEL select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if PPC32 && PPC_PAGE_SHIFT <= 14 From patchwork Thu Jul 1 01:48:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19A4FC11F68 for ; Thu, 1 Jul 2021 01:48:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C22A461466 for ; Thu, 1 Jul 2021 01:48:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C22A461466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 421088D01E9; Wed, 30 Jun 2021 21:48:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3D0BB8D01D0; Wed, 30 Jun 2021 21:48:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C04A8D01E9; Wed, 30 Jun 2021 21:48:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F32BD8D01D0 for ; Wed, 30 Jun 2021 21:48:17 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BEC861690B for ; Thu, 1 Jul 2021 01:48:17 +0000 (UTC) X-FDA: 78312333834.28.1071767 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id 741D5900009A for ; Thu, 1 Jul 2021 01:48:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6D1EE61090; Thu, 1 Jul 2021 01:48:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104096; bh=8lusK2P5LmnMksZCoFyiQWTexn57PKUElwekH74Hi+s=; h=Date:From:To:Subject:In-Reply-To:From; b=c7iipkKW2ylRcoflYEWZxpgTYZ3WkZWEtQsbV0ImgTBFmuIb1z7XudeC2axcc2xgh wv7ZRmbgjgCp1zuPihqLlke1GQQXW8AGLtsnUEKLd7glsE/7iYtmxISkatZpkZVP9H ip9IWpptoj0cxCHfRgI71EgTDJ48q2/ektW8DKHo= Date: Wed, 30 Jun 2021 18:48:16 -0700 From: Andrew Morton To: akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, shuah@kernel.org, sunnanyong@huawei.com, torvalds@linux-foundation.org, wangkefeng.wang@huawei.com, yang.shi@linux.alibaba.com, ziy@nvidia.com Subject: [patch 022/192] khugepaged: selftests: remove debug_cow Message-ID: <20210701014816.GLW58rvLa%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=c7iipkKW; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: cp1yw3fa14fzm8tzu4z99tqd11qsfxxx X-Rspamd-Queue-Id: 741D5900009A X-Rspamd-Server: rspam06 X-HE-Tag: 1625104097-365067 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nanyong Sun Subject: khugepaged: selftests: remove debug_cow The debug_cow attribute had been removed since commit 4958e4d86ecb01 ("mm: thp: remove debug_cow switch"), so remove it in selftest code too, otherwise the khugepaged test will fail. Link: https://lkml.kernel.org/r/20210430051117.400189-1-sunnanyong@huawei.com Fixes: 4958e4d86ecb01 ("mm: thp: remove debug_cow switch") Signed-off-by: Nanyong Sun Cc: Yang Shi Cc: Zi Yan Cc: Kirill A. Shutemov Cc: Shuah Khan Cc: Kefeng Wang Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/khugepaged.c | 4 ---- 1 file changed, 4 deletions(-) --- a/tools/testing/selftests/vm/khugepaged.c~khugepaged-selftests-remove-debug_cow +++ a/tools/testing/selftests/vm/khugepaged.c @@ -86,7 +86,6 @@ struct settings { enum thp_enabled thp_enabled; enum thp_defrag thp_defrag; enum shmem_enabled shmem_enabled; - bool debug_cow; bool use_zero_page; struct khugepaged_settings khugepaged; }; @@ -95,7 +94,6 @@ static struct settings default_settings .thp_enabled = THP_MADVISE, .thp_defrag = THP_DEFRAG_ALWAYS, .shmem_enabled = SHMEM_NEVER, - .debug_cow = 0, .use_zero_page = 0, .khugepaged = { .defrag = 1, @@ -268,7 +266,6 @@ static void write_settings(struct settin write_string("defrag", thp_defrag_strings[settings->thp_defrag]); write_string("shmem_enabled", shmem_enabled_strings[settings->shmem_enabled]); - write_num("debug_cow", settings->debug_cow); write_num("use_zero_page", settings->use_zero_page); write_num("khugepaged/defrag", khugepaged->defrag); @@ -304,7 +301,6 @@ static void save_settings(void) .thp_defrag = read_string("defrag", thp_defrag_strings), .shmem_enabled = read_string("shmem_enabled", shmem_enabled_strings), - .debug_cow = read_num("debug_cow"), .use_zero_page = read_num("use_zero_page"), }; saved_settings.khugepaged = (struct khugepaged_settings) { From patchwork Thu Jul 1 01:48:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353105 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 436D4C11F68 for ; Thu, 1 Jul 2021 01:48:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E427061090 for ; Thu, 1 Jul 2021 01:48:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E427061090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E0DA8D01EA; Wed, 30 Jun 2021 21:48:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B7EB8D01D0; Wed, 30 Jun 2021 21:48:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57ED48D01EA; Wed, 30 Jun 2021 21:48:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0043.hostedemail.com [216.40.44.43]) by kanga.kvack.org (Postfix) with ESMTP id 2E1D08D01D0 for ; Wed, 30 Jun 2021 21:48:21 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F2B021801E690 for ; Thu, 1 Jul 2021 01:48:20 +0000 (UTC) X-FDA: 78312333960.05.015A3EC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 9A8875000BC6 for ; Thu, 1 Jul 2021 01:48:20 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7CAA261466; Thu, 1 Jul 2021 01:48:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104099; bh=m6Locb0iCyr9Wynhmp1bB56/oBZ9LXuJ/1Sl8nbRdjI=; h=Date:From:To:Subject:In-Reply-To:From; b=yGidbD0Ub/VqMWvfP4welJWOh0CTGrv+70K+o7ZevIgUrGKjFPGFo4SL25S0tDAYD /DTckn1gZB/vSmX98Yjmx/BnuAUc2aClUbxtOIr1a6Ehn3S8YgjUArDPFAX1neWbmT ruCUNejXBjT6lidoHfoVoRfa839dYEG6q0NENM5M= Date: Wed, 30 Jun 2021 18:48:19 -0700 From: Andrew Morton To: akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, peterx@redhat.com, torvalds@linux-foundation.org, yuehaibing@huawei.com Subject: [patch 023/192] mm, hugetlb: fix racy resv_huge_pages underflow on UFFDIO_COPY Message-ID: <20210701014819.Vm-gaPGHW%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=yGidbD0U; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 9A8875000BC6 X-Stat-Signature: md7mscpdwwp4u5eac5j8uszu6c4tkb7x X-HE-Tag: 1625104100-190631 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mina Almasry Subject: mm, hugetlb: fix racy resv_huge_pages underflow on UFFDIO_COPY On UFFDIO_COPY, if we fail to copy the page contents while holding the hugetlb_fault_mutex, we will drop the mutex and return to the caller after allocating a page that consumed a reservation. In this case there may be a fault that double consumes the reservation. To handle this, we free the allocated page, fix the reservations, and allocate a temporary hugetlb page and return that to the caller. When the caller does the copy outside of the lock, we again check the cache, and allocate a page consuming the reservation, and copy over the contents. Test: Hacked the code locally such that resv_huge_pages underflows produce a warning and the copy_huge_page_from_user() always fails, then: ./tools/testing/selftests/vm/userfaultfd hugetlb_shared 10 2 /tmp/kokonut_test/huge/userfaultfd_test && echo test success ./tools/testing/selftests/vm/userfaultfd hugetlb 10 2 /tmp/kokonut_test/huge/userfaultfd_test && echo test success Both tests succeed and produce no warnings. After the test runs number of free/resv hugepages is correct. [yuehaibing@huawei.com: remove set but not used variable 'vm_alloc_shared'] Link: https://lkml.kernel.org/r/20210601141610.28332-1-yuehaibing@huawei.com [almasrymina@google.com: fix allocation error check and copy func name] Link: https://lkml.kernel.org/r/20210605010626.1459873-1-almasrymina@google.com Link: https://lkml.kernel.org/r/20210528005029.88088-1-almasrymina@google.com Signed-off-by: Mina Almasry Signed-off-by: YueHaibing Cc: Axel Rasmussen Cc: Peter Xu Cc: Mike Kravetz Signed-off-by: Andrew Morton --- include/linux/migrate.h | 4 +++ mm/hugetlb.c | 48 +++++++++++++++++++++++++++++------- mm/migrate.c | 2 - mm/userfaultfd.c | 50 -------------------------------------- 4 files changed, 45 insertions(+), 59 deletions(-) --- a/include/linux/migrate.h~mm-hugetlb-fix-racy-resv_huge_pages-underflow-on-uffdio_copy +++ a/include/linux/migrate.h @@ -51,6 +51,7 @@ extern int migrate_huge_page_move_mappin struct page *newpage, struct page *page); extern int migrate_page_move_mapping(struct address_space *mapping, struct page *newpage, struct page *page, int extra_count); +extern void copy_huge_page(struct page *dst, struct page *src); #else static inline void putback_movable_pages(struct list_head *l) {} @@ -77,6 +78,9 @@ static inline int migrate_huge_page_move return -ENOSYS; } +static inline void copy_huge_page(struct page *dst, struct page *src) +{ +} #endif /* CONFIG_MIGRATION */ #ifdef CONFIG_COMPACTION --- a/mm/hugetlb.c~mm-hugetlb-fix-racy-resv_huge_pages-underflow-on-uffdio_copy +++ a/mm/hugetlb.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include @@ -5076,20 +5077,17 @@ int hugetlb_mcopy_atomic_pte(struct mm_s struct page **pagep) { bool is_continue = (mode == MCOPY_ATOMIC_CONTINUE); - struct address_space *mapping; - pgoff_t idx; + struct hstate *h = hstate_vma(dst_vma); + struct address_space *mapping = dst_vma->vm_file->f_mapping; + pgoff_t idx = vma_hugecache_offset(h, dst_vma, dst_addr); unsigned long size; int vm_shared = dst_vma->vm_flags & VM_SHARED; - struct hstate *h = hstate_vma(dst_vma); pte_t _dst_pte; spinlock_t *ptl; - int ret; + int ret = -ENOMEM; struct page *page; int writable; - mapping = dst_vma->vm_file->f_mapping; - idx = vma_hugecache_offset(h, dst_vma, dst_addr); - if (is_continue) { ret = -EFAULT; page = find_lock_page(mapping, idx); @@ -5118,12 +5116,44 @@ int hugetlb_mcopy_atomic_pte(struct mm_s /* fallback to copy_from_user outside mmap_lock */ if (unlikely(ret)) { ret = -ENOENT; + /* Free the allocated page which may have + * consumed a reservation. + */ + restore_reserve_on_error(h, dst_vma, dst_addr, page); + put_page(page); + + /* Allocate a temporary page to hold the copied + * contents. + */ + page = alloc_huge_page_vma(h, dst_vma, dst_addr); + if (!page) { + ret = -ENOMEM; + goto out; + } *pagep = page; - /* don't free the page */ + /* Set the outparam pagep and return to the caller to + * copy the contents outside the lock. Don't free the + * page. + */ goto out; } } else { - page = *pagep; + if (vm_shared && + hugetlbfs_pagecache_present(h, dst_vma, dst_addr)) { + put_page(*pagep); + ret = -EEXIST; + *pagep = NULL; + goto out; + } + + page = alloc_huge_page(dst_vma, dst_addr, 0); + if (IS_ERR(page)) { + ret = -ENOMEM; + *pagep = NULL; + goto out; + } + copy_huge_page(page, *pagep); + put_page(*pagep); *pagep = NULL; } --- a/mm/migrate.c~mm-hugetlb-fix-racy-resv_huge_pages-underflow-on-uffdio_copy +++ a/mm/migrate.c @@ -553,7 +553,7 @@ static void __copy_gigantic_page(struct } } -static void copy_huge_page(struct page *dst, struct page *src) +void copy_huge_page(struct page *dst, struct page *src) { int i; int nr_pages; --- a/mm/userfaultfd.c~mm-hugetlb-fix-racy-resv_huge_pages-underflow-on-uffdio_copy +++ a/mm/userfaultfd.c @@ -209,7 +209,6 @@ static __always_inline ssize_t __mcopy_a unsigned long len, enum mcopy_atomic_mode mode) { - int vm_alloc_shared = dst_vma->vm_flags & VM_SHARED; int vm_shared = dst_vma->vm_flags & VM_SHARED; ssize_t err; pte_t *dst_pte; @@ -308,7 +307,6 @@ retry: mutex_unlock(&hugetlb_fault_mutex_table[hash]); i_mmap_unlock_read(mapping); - vm_alloc_shared = vm_shared; cond_resched(); @@ -346,54 +344,8 @@ retry: out_unlock: mmap_read_unlock(dst_mm); out: - if (page) { - /* - * We encountered an error and are about to free a newly - * allocated huge page. - * - * Reservation handling is very subtle, and is different for - * private and shared mappings. See the routine - * restore_reserve_on_error for details. Unfortunately, we - * can not call restore_reserve_on_error now as it would - * require holding mmap_lock. - * - * If a reservation for the page existed in the reservation - * map of a private mapping, the map was modified to indicate - * the reservation was consumed when the page was allocated. - * We clear the HPageRestoreReserve flag now so that the global - * reserve count will not be incremented in free_huge_page. - * The reservation map will still indicate the reservation - * was consumed and possibly prevent later page allocation. - * This is better than leaking a global reservation. If no - * reservation existed, it is still safe to clear - * HPageRestoreReserve as no adjustments to reservation counts - * were made during allocation. - * - * The reservation map for shared mappings indicates which - * pages have reservations. When a huge page is allocated - * for an address with a reservation, no change is made to - * the reserve map. In this case HPageRestoreReserve will be - * set to indicate that the global reservation count should be - * incremented when the page is freed. This is the desired - * behavior. However, when a huge page is allocated for an - * address without a reservation a reservation entry is added - * to the reservation map, and HPageRestoreReserve will not be - * set. When the page is freed, the global reserve count will - * NOT be incremented and it will appear as though we have - * leaked reserved page. In this case, set HPageRestoreReserve - * so that the global reserve count will be incremented to - * match the reservation map entry which was created. - * - * Note that vm_alloc_shared is based on the flags of the vma - * for which the page was originally allocated. dst_vma could - * be different or NULL on error. - */ - if (vm_alloc_shared) - SetHPageRestoreReserve(page); - else - ClearHPageRestoreReserve(page); + if (page) put_page(page); - } BUG_ON(copied < 0); BUG_ON(err > 0); BUG_ON(!copied && !err); From patchwork Thu Jul 1 01:48:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353107 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BD2BC11F69 for ; Thu, 1 Jul 2021 01:48:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D76161469 for ; Thu, 1 Jul 2021 01:48:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D76161469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 88D838D01EB; Wed, 30 Jun 2021 21:48:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8643D8D01D0; Wed, 30 Jun 2021 21:48:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 705988D01EB; Wed, 30 Jun 2021 21:48:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id 454708D01D0 for ; Wed, 30 Jun 2021 21:48:24 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0D67418087344 for ; Thu, 1 Jul 2021 01:48:24 +0000 (UTC) X-FDA: 78312334128.36.3D83F6D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 9EF32801AB09 for ; Thu, 1 Jul 2021 01:48:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 90CE861468; Thu, 1 Jul 2021 01:48:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104102; bh=4MU/Msm7LnOqKmYg7eViBHzZqbZMGhhTSbDyH9B8U4k=; h=Date:From:To:Subject:In-Reply-To:From; b=1zuCK5WgCt6IiWqEz0WV77TsD++hC7qQhYnI8LJTSi+4L0NC7iSPp9JPntXaeJWSV R45BsyGfbt/UEiy2oQ47twEncIk5DZkSushJ502N+dpy7knPI/ifktPXRdU6mSH3aq 4ahZ2EaCYbCcJEfqFjvSxWqBkPF4f/CwrmdcWB5w= Date: Wed, 30 Jun 2021 18:48:22 -0700 From: Andrew Morton To: akpm@linux-foundation.org, chenhuang5@huawei.com, corbet@lwn.net, david@redhat.com, duanxiongchun@bytedance.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org Subject: [patch 024/192] mm: sparsemem: split the huge PMD mapping of vmemmap pages Message-ID: <20210701014822.0jt6gWNGd%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9EF32801AB09 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=1zuCK5Wg; spf=pass (imf06.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: zf8r4nr34s8z87kw6ck1cdh6ebbmetri X-HE-Tag: 1625104103-886110 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: sparsemem: split the huge PMD mapping of vmemmap pages Patch series "Split huge PMD mapping of vmemmap pages", v4. In order to reduce the difficulty of code review in series[1]. We disable huge PMD mapping of vmemmap pages when that feature is enabled. In this series, we do not disable huge PMD mapping of vmemmap pages anymore. We will split huge PMD mapping when needed. When HugeTLB pages are freed from the pool we do not attempt coalasce and move back to a PMD mapping because it is much more complex. [1] https://lore.kernel.org/linux-doc/20210510030027.56044-1-songmuchun@bytedance.com/ This patch (of 3): In [1], PMD mappings of vmemmap pages were disabled if the the feature hugetlb_free_vmemmap was enabled. This was done to simplify the initial implementation of vmmemap freeing for hugetlb pages. Now, remove this simplification by allowing PMD mapping and switching to PTE mappings as needed for allocated hugetlb pages. When a hugetlb page is allocated, the vmemmap page tables are walked to free vmemmap pages. During this walk, split huge PMD mappings to PTE mappings as required. In the unlikely case PTE pages can not be allocated, return error(ENOMEM) and do not optimize vmemmap of the hugetlb page. When HugeTLB pages are freed from the pool, we do not attempt to coalesce and move back to a PMD mapping because it is much more complex. [1] https://lkml.kernel.org/r/20210510030027.56044-8-songmuchun@bytedance.com Link: https://lkml.kernel.org/r/20210616094915.34432-1-songmuchun@bytedance.com Link: https://lkml.kernel.org/r/20210616094915.34432-2-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Cc: Oscar Salvador Cc: Michal Hocko Cc: David Hildenbrand Cc: Chen Huang Cc: Jonathan Corbet Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- include/linux/mm.h | 4 - mm/hugetlb_vmemmap.c | 5 - mm/sparse-vmemmap.c | 163 +++++++++++++++++++++++++++++++---------- 3 files changed, 129 insertions(+), 43 deletions(-) --- a/include/linux/mm.h~mm-sparsemem-split-the-huge-pmd-mapping-of-vmemmap-pages +++ a/include/linux/mm.h @@ -3076,8 +3076,8 @@ static inline void print_vma_addr(char * } #endif -void vmemmap_remap_free(unsigned long start, unsigned long end, - unsigned long reuse); +int vmemmap_remap_free(unsigned long start, unsigned long end, + unsigned long reuse); int vmemmap_remap_alloc(unsigned long start, unsigned long end, unsigned long reuse, gfp_t gfp_mask); --- a/mm/hugetlb_vmemmap.c~mm-sparsemem-split-the-huge-pmd-mapping-of-vmemmap-pages +++ a/mm/hugetlb_vmemmap.c @@ -258,9 +258,8 @@ void free_huge_page_vmemmap(struct hstat * to the page which @vmemmap_reuse is mapped to, then free the pages * which the range [@vmemmap_addr, @vmemmap_end] is mapped to. */ - vmemmap_remap_free(vmemmap_addr, vmemmap_end, vmemmap_reuse); - - SetHPageVmemmapOptimized(head); + if (!vmemmap_remap_free(vmemmap_addr, vmemmap_end, vmemmap_reuse)) + SetHPageVmemmapOptimized(head); } void __init hugetlb_vmemmap_init(struct hstate *h) --- a/mm/sparse-vmemmap.c~mm-sparsemem-split-the-huge-pmd-mapping-of-vmemmap-pages +++ a/mm/sparse-vmemmap.c @@ -38,6 +38,7 @@ * struct vmemmap_remap_walk - walk vmemmap page table * * @remap_pte: called for each lowest-level entry (PTE). + * @nr_walked: the number of walked pte. * @reuse_page: the page which is reused for the tail vmemmap pages. * @reuse_addr: the virtual address of the @reuse_page page. * @vmemmap_pages: the list head of the vmemmap pages that can be freed @@ -46,11 +47,44 @@ struct vmemmap_remap_walk { void (*remap_pte)(pte_t *pte, unsigned long addr, struct vmemmap_remap_walk *walk); + unsigned long nr_walked; struct page *reuse_page; unsigned long reuse_addr; struct list_head *vmemmap_pages; }; +static int split_vmemmap_huge_pmd(pmd_t *pmd, unsigned long start, + struct vmemmap_remap_walk *walk) +{ + pmd_t __pmd; + int i; + unsigned long addr = start; + struct page *page = pmd_page(*pmd); + pte_t *pgtable = pte_alloc_one_kernel(&init_mm); + + if (!pgtable) + return -ENOMEM; + + pmd_populate_kernel(&init_mm, &__pmd, pgtable); + + for (i = 0; i < PMD_SIZE / PAGE_SIZE; i++, addr += PAGE_SIZE) { + pte_t entry, *pte; + pgprot_t pgprot = PAGE_KERNEL; + + entry = mk_pte(page + i, pgprot); + pte = pte_offset_kernel(&__pmd, addr); + set_pte_at(&init_mm, addr, pte, entry); + } + + /* Make pte visible before pmd. See comment in __pte_alloc(). */ + smp_wmb(); + pmd_populate_kernel(&init_mm, pmd, pgtable); + + flush_tlb_kernel_range(start, start + PMD_SIZE); + + return 0; +} + static void vmemmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct vmemmap_remap_walk *walk) @@ -69,58 +103,80 @@ static void vmemmap_pte_range(pmd_t *pmd */ addr += PAGE_SIZE; pte++; + walk->nr_walked++; } - for (; addr != end; addr += PAGE_SIZE, pte++) + for (; addr != end; addr += PAGE_SIZE, pte++) { walk->remap_pte(pte, addr, walk); + walk->nr_walked++; + } } -static void vmemmap_pmd_range(pud_t *pud, unsigned long addr, - unsigned long end, - struct vmemmap_remap_walk *walk) +static int vmemmap_pmd_range(pud_t *pud, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) { pmd_t *pmd; unsigned long next; pmd = pmd_offset(pud, addr); do { - BUG_ON(pmd_leaf(*pmd)); + if (pmd_leaf(*pmd)) { + int ret; + ret = split_vmemmap_huge_pmd(pmd, addr & PMD_MASK, walk); + if (ret) + return ret; + } next = pmd_addr_end(addr, end); vmemmap_pte_range(pmd, addr, next, walk); } while (pmd++, addr = next, addr != end); + + return 0; } -static void vmemmap_pud_range(p4d_t *p4d, unsigned long addr, - unsigned long end, - struct vmemmap_remap_walk *walk) +static int vmemmap_pud_range(p4d_t *p4d, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) { pud_t *pud; unsigned long next; pud = pud_offset(p4d, addr); do { + int ret; + next = pud_addr_end(addr, end); - vmemmap_pmd_range(pud, addr, next, walk); + ret = vmemmap_pmd_range(pud, addr, next, walk); + if (ret) + return ret; } while (pud++, addr = next, addr != end); + + return 0; } -static void vmemmap_p4d_range(pgd_t *pgd, unsigned long addr, - unsigned long end, - struct vmemmap_remap_walk *walk) +static int vmemmap_p4d_range(pgd_t *pgd, unsigned long addr, + unsigned long end, + struct vmemmap_remap_walk *walk) { p4d_t *p4d; unsigned long next; p4d = p4d_offset(pgd, addr); do { + int ret; + next = p4d_addr_end(addr, end); - vmemmap_pud_range(p4d, addr, next, walk); + ret = vmemmap_pud_range(p4d, addr, next, walk); + if (ret) + return ret; } while (p4d++, addr = next, addr != end); + + return 0; } -static void vmemmap_remap_range(unsigned long start, unsigned long end, - struct vmemmap_remap_walk *walk) +static int vmemmap_remap_range(unsigned long start, unsigned long end, + struct vmemmap_remap_walk *walk) { unsigned long addr = start; unsigned long next; @@ -131,8 +187,12 @@ static void vmemmap_remap_range(unsigned pgd = pgd_offset_k(addr); do { + int ret; + next = pgd_addr_end(addr, end); - vmemmap_p4d_range(pgd, addr, next, walk); + ret = vmemmap_p4d_range(pgd, addr, next, walk); + if (ret) + return ret; } while (pgd++, addr = next, addr != end); /* @@ -141,6 +201,8 @@ static void vmemmap_remap_range(unsigned * belongs to the range. */ flush_tlb_kernel_range(start + PAGE_SIZE, end); + + return 0; } /* @@ -179,10 +241,27 @@ static void vmemmap_remap_pte(pte_t *pte pte_t entry = mk_pte(walk->reuse_page, pgprot); struct page *page = pte_page(*pte); - list_add(&page->lru, walk->vmemmap_pages); + list_add_tail(&page->lru, walk->vmemmap_pages); set_pte_at(&init_mm, addr, pte, entry); } +static void vmemmap_restore_pte(pte_t *pte, unsigned long addr, + struct vmemmap_remap_walk *walk) +{ + pgprot_t pgprot = PAGE_KERNEL; + struct page *page; + void *to; + + BUG_ON(pte_page(*pte) != walk->reuse_page); + + page = list_first_entry(walk->vmemmap_pages, struct page, lru); + list_del(&page->lru); + to = page_to_virt(page); + copy_page(to, (void *)walk->reuse_addr); + + set_pte_at(&init_mm, addr, pte, mk_pte(page, pgprot)); +} + /** * vmemmap_remap_free - remap the vmemmap virtual address range [@start, @end) * to the page which @reuse is mapped to, then free vmemmap @@ -193,12 +272,12 @@ static void vmemmap_remap_pte(pte_t *pte * remap. * @reuse: reuse address. * - * Note: This function depends on vmemmap being base page mapped. Please make - * sure that we disable PMD mapping of vmemmap pages when calling this function. + * Return: %0 on success, negative error code otherwise. */ -void vmemmap_remap_free(unsigned long start, unsigned long end, - unsigned long reuse) +int vmemmap_remap_free(unsigned long start, unsigned long end, + unsigned long reuse) { + int ret; LIST_HEAD(vmemmap_pages); struct vmemmap_remap_walk walk = { .remap_pte = vmemmap_remap_pte, @@ -221,25 +300,31 @@ void vmemmap_remap_free(unsigned long st */ BUG_ON(start - reuse != PAGE_SIZE); - vmemmap_remap_range(reuse, end, &walk); - free_vmemmap_page_list(&vmemmap_pages); -} + mmap_write_lock(&init_mm); + ret = vmemmap_remap_range(reuse, end, &walk); + mmap_write_downgrade(&init_mm); -static void vmemmap_restore_pte(pte_t *pte, unsigned long addr, - struct vmemmap_remap_walk *walk) -{ - pgprot_t pgprot = PAGE_KERNEL; - struct page *page; - void *to; + if (ret && walk.nr_walked) { + end = reuse + walk.nr_walked * PAGE_SIZE; + /* + * vmemmap_pages contains pages from the previous + * vmemmap_remap_range call which failed. These + * are pages which were removed from the vmemmap. + * They will be restored in the following call. + */ + walk = (struct vmemmap_remap_walk) { + .remap_pte = vmemmap_restore_pte, + .reuse_addr = reuse, + .vmemmap_pages = &vmemmap_pages, + }; - BUG_ON(pte_page(*pte) != walk->reuse_page); + vmemmap_remap_range(reuse, end, &walk); + } + mmap_read_unlock(&init_mm); - page = list_first_entry(walk->vmemmap_pages, struct page, lru); - list_del(&page->lru); - to = page_to_virt(page); - copy_page(to, (void *)walk->reuse_addr); + free_vmemmap_page_list(&vmemmap_pages); - set_pte_at(&init_mm, addr, pte, mk_pte(page, pgprot)); + return ret; } static int alloc_vmemmap_page_list(unsigned long start, unsigned long end, @@ -273,6 +358,8 @@ out: * remap. * @reuse: reuse address. * @gfp_mask: GFP flag for allocating vmemmap pages. + * + * Return: %0 on success, negative error code otherwise. */ int vmemmap_remap_alloc(unsigned long start, unsigned long end, unsigned long reuse, gfp_t gfp_mask) @@ -287,12 +374,12 @@ int vmemmap_remap_alloc(unsigned long st /* See the comment in the vmemmap_remap_free(). */ BUG_ON(start - reuse != PAGE_SIZE); - might_sleep_if(gfpflags_allow_blocking(gfp_mask)); - if (alloc_vmemmap_page_list(start, end, gfp_mask, &vmemmap_pages)) return -ENOMEM; + mmap_read_lock(&init_mm); vmemmap_remap_range(reuse, end, &walk); + mmap_read_unlock(&init_mm); return 0; } From patchwork Thu Jul 1 01:48:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353109 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E032C11F69 for ; Thu, 1 Jul 2021 01:48:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1BEA161469 for ; Thu, 1 Jul 2021 01:48:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1BEA161469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8DBD38D01EC; Wed, 30 Jun 2021 21:48:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B2578D01D0; Wed, 30 Jun 2021 21:48:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 752CD8D01EC; Wed, 30 Jun 2021 21:48:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0167.hostedemail.com [216.40.44.167]) by kanga.kvack.org (Postfix) with ESMTP id 4BDC88D01D0 for ; Wed, 30 Jun 2021 21:48:27 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2559C1690B for ; Thu, 1 Jul 2021 01:48:27 +0000 (UTC) X-FDA: 78312334254.30.9DA4EB6 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id C9B08E00304F for ; Thu, 1 Jul 2021 01:48:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A7AA46120C; Thu, 1 Jul 2021 01:48:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104106; bh=XAuYIAdAZhyletHJMmbYp5F9FarmAoFJaDBgj2EVx3Q=; h=Date:From:To:Subject:In-Reply-To:From; b=mCUkG+SeCq+9LKiSD8u3Grex5wRyt73d6mcivb2tUbjSND9GljKqHcLVJrKRwzfLl uTG2TUejZdNJaMQMpeWN/DjtzPOU+s+Ts6duAMF8EwUCO4lGXwp6tolJBkb9K7BVtb UjfuBuFP3jXuk9fS1nPpXyZ412UbmBL+E3zfBl0M= Date: Wed, 30 Jun 2021 18:48:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, chenhuang5@huawei.com, corbet@lwn.net, david@redhat.com, duanxiongchun@bytedance.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org Subject: [patch 025/192] mm: sparsemem: use huge PMD mapping for vmemmap pages Message-ID: <20210701014825.6-QDiD-kW%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=mCUkG+Se; dmarc=none; spf=pass (imf30.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: x9ab6fdb5gwfqrhfnidtfjw5yew3a9pr X-Rspamd-Queue-Id: C9B08E00304F X-HE-Tag: 1625104106-743108 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: sparsemem: use huge PMD mapping for vmemmap pages The preparation of splitting huge PMD mapping of vmemmap pages is ready, so switch the mapping from PTE to PMD. Link: https://lkml.kernel.org/r/20210616094915.34432-3-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Mike Kravetz Cc: Chen Huang Cc: David Hildenbrand Cc: Jonathan Corbet Cc: Michal Hocko Cc: Oscar Salvador Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 7 --- arch/x86/mm/init_64.c | 8 +--- include/linux/hugetlb.h | 25 +++----------- mm/memory_hotplug.c | 2 - 4 files changed, 9 insertions(+), 33 deletions(-) --- a/arch/x86/mm/init_64.c~mm-sparsemem-use-huge-pmd-mapping-for-vmemmap-pages +++ a/arch/x86/mm/init_64.c @@ -34,7 +34,6 @@ #include #include #include -#include #include #include @@ -1610,8 +1609,7 @@ int __meminit vmemmap_populate(unsigned VM_BUG_ON(!IS_ALIGNED(start, PAGE_SIZE)); VM_BUG_ON(!IS_ALIGNED(end, PAGE_SIZE)); - if ((is_hugetlb_free_vmemmap_enabled() && !altmap) || - end - start < PAGES_PER_SECTION * sizeof(struct page)) + if (end - start < PAGES_PER_SECTION * sizeof(struct page)) err = vmemmap_populate_basepages(start, end, node, NULL); else if (boot_cpu_has(X86_FEATURE_PSE)) err = vmemmap_populate_hugepages(start, end, node, altmap); @@ -1639,8 +1637,6 @@ void register_page_bootmem_memmap(unsign pmd_t *pmd; unsigned int nr_pmd_pages; struct page *page; - bool base_mapping = !boot_cpu_has(X86_FEATURE_PSE) || - is_hugetlb_free_vmemmap_enabled(); for (; addr < end; addr = next) { pte_t *pte = NULL; @@ -1666,7 +1662,7 @@ void register_page_bootmem_memmap(unsign } get_page_bootmem(section_nr, pud_page(*pud), MIX_SECTION_INFO); - if (base_mapping) { + if (!boot_cpu_has(X86_FEATURE_PSE)) { next = (addr + PAGE_SIZE) & PAGE_MASK; pmd = pmd_offset(pud, addr); if (pmd_none(*pmd)) --- a/Documentation/admin-guide/kernel-parameters.txt~mm-sparsemem-use-huge-pmd-mapping-for-vmemmap-pages +++ a/Documentation/admin-guide/kernel-parameters.txt @@ -1572,13 +1572,6 @@ enabled. Allows heavy hugetlb users to free up some more memory (6 * PAGE_SIZE for each 2MB hugetlb page). - This feauture is not free though. Large page - tables are not used to back vmemmap pages which - can lead to a performance degradation for some - workloads. Also there will be memory allocation - required when hugetlb pages are freed from the - pool which can lead to corner cases under heavy - memory pressure. Format: { on | off (default) } on: enable the feature --- a/include/linux/hugetlb.h~mm-sparsemem-use-huge-pmd-mapping-for-vmemmap-pages +++ a/include/linux/hugetlb.h @@ -895,20 +895,6 @@ static inline void huge_ptep_modify_prot } #endif -#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP -extern bool hugetlb_free_vmemmap_enabled; - -static inline bool is_hugetlb_free_vmemmap_enabled(void) -{ - return hugetlb_free_vmemmap_enabled; -} -#else -static inline bool is_hugetlb_free_vmemmap_enabled(void) -{ - return false; -} -#endif - #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; @@ -1063,13 +1049,14 @@ static inline void set_huge_swap_pte_at( pte_t *ptep, pte_t pte, unsigned long sz) { } - -static inline bool is_hugetlb_free_vmemmap_enabled(void) -{ - return false; -} #endif /* CONFIG_HUGETLB_PAGE */ +#ifdef CONFIG_HUGETLB_PAGE_FREE_VMEMMAP +extern bool hugetlb_free_vmemmap_enabled; +#else +#define hugetlb_free_vmemmap_enabled false +#endif + static inline spinlock_t *huge_pte_lock(struct hstate *h, struct mm_struct *mm, pte_t *pte) { --- a/mm/memory_hotplug.c~mm-sparsemem-use-huge-pmd-mapping-for-vmemmap-pages +++ a/mm/memory_hotplug.c @@ -1056,7 +1056,7 @@ bool mhp_supports_memmap_on_memory(unsig * populate a single PMD. */ return memmap_on_memory && - !is_hugetlb_free_vmemmap_enabled() && + !hugetlb_free_vmemmap_enabled && IS_ENABLED(CONFIG_MHP_MEMMAP_ON_MEMORY) && size == memory_block_size_bytes() && IS_ALIGNED(vmemmap_size, PMD_SIZE) && From patchwork Thu Jul 1 01:48:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353111 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F3A3C11F69 for ; Thu, 1 Jul 2021 01:48:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 28CBC6120C for ; Thu, 1 Jul 2021 01:48:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 28CBC6120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3AFE8D01ED; Wed, 30 Jun 2021 21:48:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A11ED8D01D0; Wed, 30 Jun 2021 21:48:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DA348D01ED; Wed, 30 Jun 2021 21:48:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0071.hostedemail.com [216.40.44.71]) by kanga.kvack.org (Postfix) with ESMTP id 668038D01D0 for ; Wed, 30 Jun 2021 21:48:30 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 36FB8181AEF3F for ; Thu, 1 Jul 2021 01:48:30 +0000 (UTC) X-FDA: 78312334380.39.07269EE Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id C8B709000328 for ; Thu, 1 Jul 2021 01:48:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C137561090; Thu, 1 Jul 2021 01:48:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104109; bh=YsizUCViGmidYVNh6PTUlSBYfdXMUnBWhuzIEpIGP1s=; h=Date:From:To:Subject:In-Reply-To:From; b=2g+uLhRv1Idq9bTFvCwwgI08/TLCrT9Li5PCrgEaN0MjTkbThZ4G7e6FH6ed8dct1 K8suOEMPxskCmLjZJQXYID8ZcIMs3Y2JTyJtyUV/DOQR/fEwN6z9Steox3SdXPlDF1 eaw/Z4Uhu9zZ9AHmmzo4iw9ZNEpABKjeevdRtSjs= Date: Wed, 30 Jun 2021 18:48:28 -0700 From: Andrew Morton To: akpm@linux-foundation.org, chenhuang5@huawei.com, corbet@lwn.net, david@redhat.com, duanxiongchun@bytedance.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org Subject: [patch 026/192] mm: hugetlb: introduce CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON Message-ID: <20210701014828.73mtzQZj9%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=2g+uLhRv; dmarc=none; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: oc8nox5zh8c979gw48wwrmenwycne91n X-Rspamd-Queue-Id: C8B709000328 X-HE-Tag: 1625104109-880128 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: hugetlb: introduce CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON When using HUGETLB_PAGE_FREE_VMEMMAP, the freeing unused vmemmap pages associated with each HugeTLB page is default off. Now the vmemmap is PMD mapped. So there is no side effect when this feature is enabled with no HugeTLB pages in the system. Someone may want to enable this feature in the compiler time instead of using boot command line. So add a config to make it default on when someone do not want to enable it via command line. Link: https://lkml.kernel.org/r/20210616094915.34432-4-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Chen Huang Cc: David Hildenbrand Cc: Jonathan Corbet Cc: Michal Hocko Cc: Mike Kravetz Cc: Oscar Salvador Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 3 +++ fs/Kconfig | 10 ++++++++++ mm/hugetlb_vmemmap.c | 6 ++++-- 3 files changed, 17 insertions(+), 2 deletions(-) --- a/Documentation/admin-guide/kernel-parameters.txt~mm-hugetlb-introduce-config_hugetlb_page_free_vmemmap_default_on +++ a/Documentation/admin-guide/kernel-parameters.txt @@ -1577,6 +1577,9 @@ on: enable the feature off: disable the feature + Built with CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON=y, + the default is on. + This is not compatible with memory_hotplug.memmap_on_memory. If both parameters are enabled, hugetlb_free_vmemmap takes precedence over memory_hotplug.memmap_on_memory. --- a/fs/Kconfig~mm-hugetlb-introduce-config_hugetlb_page_free_vmemmap_default_on +++ a/fs/Kconfig @@ -245,6 +245,16 @@ config HUGETLB_PAGE_FREE_VMEMMAP depends on X86_64 depends on SPARSEMEM_VMEMMAP +config HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON + bool "Default freeing vmemmap pages of HugeTLB to on" + default n + depends on HUGETLB_PAGE_FREE_VMEMMAP + help + When using HUGETLB_PAGE_FREE_VMEMMAP, the freeing unused vmemmap + pages associated with each HugeTLB page is default off. Say Y here + to enable freeing vmemmap pages of HugeTLB by default. It can then + be disabled on the command line via hugetlb_free_vmemmap=off. + config MEMFD_CREATE def_bool TMPFS || HUGETLBFS --- a/mm/hugetlb_vmemmap.c~mm-hugetlb-introduce-config_hugetlb_page_free_vmemmap_default_on +++ a/mm/hugetlb_vmemmap.c @@ -182,7 +182,7 @@ #define RESERVE_VMEMMAP_NR 2U #define RESERVE_VMEMMAP_SIZE (RESERVE_VMEMMAP_NR << PAGE_SHIFT) -bool hugetlb_free_vmemmap_enabled; +bool hugetlb_free_vmemmap_enabled = IS_ENABLED(CONFIG_HUGETLB_PAGE_FREE_VMEMMAP_DEFAULT_ON); static int __init early_hugetlb_free_vmemmap_param(char *buf) { @@ -197,7 +197,9 @@ static int __init early_hugetlb_free_vme if (!strcmp(buf, "on")) hugetlb_free_vmemmap_enabled = true; - else if (strcmp(buf, "off")) + else if (!strcmp(buf, "off")) + hugetlb_free_vmemmap_enabled = false; + else return -EINVAL; return 0; From patchwork Thu Jul 1 01:48:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8685C11F69 for ; Thu, 1 Jul 2021 01:48:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6C12C61466 for ; Thu, 1 Jul 2021 01:48:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C12C61466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E59C68D01EE; Wed, 30 Jun 2021 21:48:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E30D78D01D0; Wed, 30 Jun 2021 21:48:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF8B78D01EE; Wed, 30 Jun 2021 21:48:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0250.hostedemail.com [216.40.44.250]) by kanga.kvack.org (Postfix) with ESMTP id A52AE8D01D0 for ; Wed, 30 Jun 2021 21:48:33 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7081E1801968D for ; Thu, 1 Jul 2021 01:48:33 +0000 (UTC) X-FDA: 78312334506.15.05CCCAB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 1BEC8F0000AC for ; Thu, 1 Jul 2021 01:48:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EAFB46120C; Thu, 1 Jul 2021 01:48:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104112; bh=rAbut2H0J4tPh0ojPJzCigwOzOtlCtjx4TO9qUXi+oM=; h=Date:From:To:Subject:In-Reply-To:From; b=A8Bjeuk77DzSFM4CDKFDbxIa6/ci6eZ4Nw69vqV6ALlcFJ+Vn/c9am8xPGoFOzMq8 Aj4/KiU+Z9PMt0R8+Cx0JQZr95+vjkEMhJuSy3jK/47EmAUmty9eDrbJAoFTJynMd2 g5EA/zCNyUNWDu0t8zej/EOLSZDAFtgp4pfakuq4= Date: Wed, 30 Jun 2021 18:48:31 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, jack@suse.cz, jannh@google.com, jhubbard@nvidia.com, kirill@shutemov.name, linux-mm@kvack.org, mhocko@kernel.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org, youquan.song@intel.com Subject: [patch 027/192] hugetlb: remove prep_compound_huge_page cleanup Message-ID: <20210701014831.SWwKpeGh-%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=A8Bjeuk7; dmarc=none; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: 1x9ssgmq5byitt71i7imzfh987trypey X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1BEC8F0000AC X-HE-Tag: 1625104113-855867 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: remove prep_compound_huge_page cleanup Patch series "Fix prep_compound_gigantic_page ref count adjustment". These patches address the possible race between prep_compound_gigantic_page and __page_cache_add_speculative as described by Jann Horn in [1]. The first patch simply removes the unnecessary/obsolete helper routine prep_compound_huge_page to make the actual fix a little simpler. The second patch is the actual fix and has a detailed explanation in the commit message. This potential issue has existed for almost 10 years and I am unaware of anyone actually hitting the race. I did not cc stable, but would be happy to squash the patches and send to stable if anyone thinks that is a good idea. [1] https://lore.kernel.org/linux-mm/CAG48ez23q0Jy9cuVnwAe7t_fdhMk2S7N5Hdi-GLcCeq5bsfLxw@mail.gmail.com/ This patch (of 2): I could not think of a reliable way to recreate the issue for testing. Rather, I 'simulated errors' to exercise all the error paths. The routine prep_compound_huge_page is a simple wrapper to call either prep_compound_gigantic_page or prep_compound_page. However, it is only called from gather_bootmem_prealloc which only processes gigantic pages. Eliminate the routine and call prep_compound_gigantic_page directly. Link: https://lkml.kernel.org/r/20210622021423.154662-1-mike.kravetz@oracle.com Link: https://lkml.kernel.org/r/20210622021423.154662-2-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Cc: Andrea Arcangeli Cc: Jan Kara Cc: Jann Horn Cc: John Hubbard Cc: "Kirill A . Shutemov" Cc: Matthew Wilcox Cc: Michal Hocko Cc: Youquan Song Cc: Muchun Song Signed-off-by: Andrew Morton --- mm/hugetlb.c | 29 ++++++++++------------------- 1 file changed, 10 insertions(+), 19 deletions(-) --- a/mm/hugetlb.c~hugetlb-remove-prep_compound_huge_page-cleanup +++ a/mm/hugetlb.c @@ -1320,8 +1320,6 @@ static struct page *alloc_gigantic_page( return alloc_contig_pages(nr_pages, gfp_mask, nid, nodemask); } -static void prep_new_huge_page(struct hstate *h, struct page *page, int nid); -static void prep_compound_gigantic_page(struct page *page, unsigned int order); #else /* !CONFIG_CONTIG_ALLOC */ static struct page *alloc_gigantic_page(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nodemask) @@ -2759,16 +2757,10 @@ found: return 1; } -static void __init prep_compound_huge_page(struct page *page, - unsigned int order) -{ - if (unlikely(order > (MAX_ORDER - 1))) - prep_compound_gigantic_page(page, order); - else - prep_compound_page(page, order); -} - -/* Put bootmem huge pages into the standard lists after mem_map is up */ +/* + * Put bootmem huge pages into the standard lists after mem_map is up. + * Note: This only applies to gigantic (order > MAX_ORDER) pages. + */ static void __init gather_bootmem_prealloc(void) { struct huge_bootmem_page *m; @@ -2777,20 +2769,19 @@ static void __init gather_bootmem_preall struct page *page = virt_to_page(m); struct hstate *h = m->hstate; + VM_BUG_ON(!hstate_is_gigantic(h)); WARN_ON(page_count(page) != 1); - prep_compound_huge_page(page, huge_page_order(h)); + prep_compound_gigantic_page(page, huge_page_order(h)); WARN_ON(PageReserved(page)); prep_new_huge_page(h, page, page_to_nid(page)); put_page(page); /* free it into the hugepage allocator */ /* - * If we had gigantic hugepages allocated at boot time, we need - * to restore the 'stolen' pages to totalram_pages in order to - * fix confusing memory reports from free(1) and another - * side-effects, like CommitLimit going negative. + * We need to restore the 'stolen' pages to totalram_pages + * in order to fix confusing memory reports from free(1) and + * other side-effects, like CommitLimit going negative. */ - if (hstate_is_gigantic(h)) - adjust_managed_page_count(page, pages_per_huge_page(h)); + adjust_managed_page_count(page, pages_per_huge_page(h)); cond_resched(); } } From patchwork Thu Jul 1 01:48:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CDF8C11F69 for ; Thu, 1 Jul 2021 01:48:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AFA4761466 for ; Thu, 1 Jul 2021 01:48:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFA4761466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 35D1B8D01EF; Wed, 30 Jun 2021 21:48:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3345F8D01D0; Wed, 30 Jun 2021 21:48:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1FD698D01EF; Wed, 30 Jun 2021 21:48:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E8D3D8D01D0 for ; Wed, 30 Jun 2021 21:48:36 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id BA8AD21969 for ; Thu, 1 Jul 2021 01:48:36 +0000 (UTC) X-FDA: 78312334632.25.1F5310B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 66546D000099 for ; Thu, 1 Jul 2021 01:48:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 443FE61090; Thu, 1 Jul 2021 01:48:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104115; bh=fqoDSDrqLeZnrvI23duIKA3jLaSkuts4WzxUu7vYMGQ=; h=Date:From:To:Subject:In-Reply-To:From; b=meXK1BlBAB9ve5pTvcJWefWj39RKdXicP5DQ56dFZTBZSopNMw8qZ7W7G3u1oUMnJ g7gnSQ6RWYz35R0RPSyjydlf8eUGoSvWpkjyx4YzDC+l7eVLgN/MVLhbk0CYOyZzNS bl4enn0YorYcKeJxOPeVrc16YASV8aWYrwhaE/Ls= Date: Wed, 30 Jun 2021 18:48:34 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, jack@suse.cz, jannh@google.com, jhubbard@nvidia.com, kirill@shutemov.name, linux-mm@kvack.org, mhocko@kernel.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org, youquan.song@intel.com Subject: [patch 028/192] hugetlb: address ref count racing in prep_compound_gigantic_page Message-ID: <20210701014834.7_vR9PvR7%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 66546D000099 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=meXK1BlB; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: ztkbuudrkde8w86wt5tpcnfagk4gu733 X-HE-Tag: 1625104116-42808 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: address ref count racing in prep_compound_gigantic_page In [1], Jann Horn points out a possible race between prep_compound_gigantic_page and __page_cache_add_speculative. The root cause of the possible race is prep_compound_gigantic_page uncondittionally setting the ref count of pages to zero. It does this because prep_compound_gigantic_page is handed a 'group' of pages from an allocator and needs to convert that group of pages to a compound page. The ref count of each page in this 'group' is one as set by the allocator. However, the ref count of compound page tail pages must be zero. The potential race comes about when ref counted pages are returned from the allocator. When this happens, other mm code could also take a reference on the page. __page_cache_add_speculative is one such example. Therefore, prep_compound_gigantic_page can not just set the ref count of pages to zero as it does today. Doing so would lose the reference taken by any other code. This would lead to BUGs in code checking ref counts and could possibly even lead to memory corruption. There are two possible ways to address this issue. 1) Make all allocators of gigantic groups of pages be able to return a properly constructed compound page. 2) Make prep_compound_gigantic_page be more careful when constructing a compound page. This patch takes approach 2. In prep_compound_gigantic_page, use cmpxchg to only set ref count to zero if it is one. If the cmpxchg fails, call synchronize_rcu() in the hope that the extra ref count will be driopped during a rcu grace period. This is not a performance critical code path and the wait should be accceptable. If the ref count is still inflated after the grace period, then undo any modifications made and return an error. Currently prep_compound_gigantic_page is type void and does not return errors. Modify the two callers to check for and handle error returns. On error, the caller must free the 'group' of pages as they can not be used to form a gigantic page. After freeing pages, the runtime caller (alloc_fresh_huge_page) will retry the allocation once. Boot time allocations can not be retried. The routine prep_compound_page also unconditionally sets the ref count of compound page tail pages to zero. However, in this case the buddy allocator is constructing a compound page from freshly allocated pages. The ref count on those freshly allocated pages is already zero, so the set_page_count(p, 0) is unnecessary and could lead to confusion. Just remove it. [1] https://lore.kernel.org/linux-mm/CAG48ez23q0Jy9cuVnwAe7t_fdhMk2S7N5Hdi-GLcCeq5bsfLxw@mail.gmail.com/ Link: https://lkml.kernel.org/r/20210622021423.154662-3-mike.kravetz@oracle.com Fixes: 58a84aa92723 ("thp: set compound tail page _count to zero") Signed-off-by: Mike Kravetz Reported-by: Jann Horn Cc: Youquan Song Cc: Andrea Arcangeli Cc: Jan Kara Cc: John Hubbard Cc: "Kirill A . Shutemov" Cc: Matthew Wilcox Cc: Michal Hocko Cc: Muchun Song Signed-off-by: Andrew Morton --- mm/hugetlb.c | 72 ++++++++++++++++++++++++++++++++++++++++------ mm/page_alloc.c | 1 2 files changed, 64 insertions(+), 9 deletions(-) --- a/mm/hugetlb.c~hugetlb-address-ref-count-racing-in-prep_compound_gigantic_page +++ a/mm/hugetlb.c @@ -1623,9 +1623,9 @@ static void prep_new_huge_page(struct hs spin_unlock_irq(&hugetlb_lock); } -static void prep_compound_gigantic_page(struct page *page, unsigned int order) +static bool prep_compound_gigantic_page(struct page *page, unsigned int order) { - int i; + int i, j; int nr_pages = 1 << order; struct page *p = page + 1; @@ -1647,11 +1647,48 @@ static void prep_compound_gigantic_page( * after get_user_pages(). */ __ClearPageReserved(p); + /* + * Subtle and very unlikely + * + * Gigantic 'page allocators' such as memblock or cma will + * return a set of pages with each page ref counted. We need + * to turn this set of pages into a compound page with tail + * page ref counts set to zero. Code such as speculative page + * cache adding could take a ref on a 'to be' tail page. + * We need to respect any increased ref count, and only set + * the ref count to zero if count is currently 1. If count + * is not 1, we call synchronize_rcu in the hope that a rcu + * grace period will cause ref count to drop and then retry. + * If count is still inflated on retry we return an error and + * must discard the pages. + */ + if (!page_ref_freeze(p, 1)) { + pr_info("HugeTLB unexpected inflated ref count on freshly allocated page\n"); + synchronize_rcu(); + if (!page_ref_freeze(p, 1)) + goto out_error; + } set_page_count(p, 0); set_compound_head(p, page); } atomic_set(compound_mapcount_ptr(page), -1); atomic_set(compound_pincount_ptr(page), 0); + return true; + +out_error: + /* undo tail page modifications made above */ + p = page + 1; + for (j = 1; j < i; j++, p = mem_map_next(p, page, j)) { + clear_compound_head(p); + set_page_refcounted(p); + } + /* need to clear PG_reserved on remaining tail pages */ + for (; j < nr_pages; j++, p = mem_map_next(p, page, j)) + __ClearPageReserved(p); + set_compound_order(page, 0); + page[1].compound_nr = 0; + __ClearPageHead(page); + return false; } /* @@ -1771,7 +1808,9 @@ static struct page *alloc_fresh_huge_pag nodemask_t *node_alloc_noretry) { struct page *page; + bool retry = false; +retry: if (hstate_is_gigantic(h)) page = alloc_gigantic_page(h, gfp_mask, nid, nmask); else @@ -1780,8 +1819,21 @@ static struct page *alloc_fresh_huge_pag if (!page) return NULL; - if (hstate_is_gigantic(h)) - prep_compound_gigantic_page(page, huge_page_order(h)); + if (hstate_is_gigantic(h)) { + if (!prep_compound_gigantic_page(page, huge_page_order(h))) { + /* + * Rare failure to convert pages to compound page. + * Free pages and try again - ONCE! + */ + free_gigantic_page(page, huge_page_order(h)); + if (!retry) { + retry = true; + goto retry; + } + pr_warn("HugeTLB page can not be used due to unexpected inflated ref count\n"); + return NULL; + } + } prep_new_huge_page(h, page, page_to_nid(page)); return page; @@ -2771,10 +2823,14 @@ static void __init gather_bootmem_preall VM_BUG_ON(!hstate_is_gigantic(h)); WARN_ON(page_count(page) != 1); - prep_compound_gigantic_page(page, huge_page_order(h)); - WARN_ON(PageReserved(page)); - prep_new_huge_page(h, page, page_to_nid(page)); - put_page(page); /* free it into the hugepage allocator */ + if (prep_compound_gigantic_page(page, huge_page_order(h))) { + WARN_ON(PageReserved(page)); + prep_new_huge_page(h, page, page_to_nid(page)); + put_page(page); /* add to the hugepage allocator */ + } else { + free_gigantic_page(page, huge_page_order(h)); + pr_warn("HugeTLB page can not be used due to unexpected inflated ref count\n"); + } /* * We need to restore the 'stolen' pages to totalram_pages --- a/mm/page_alloc.c~hugetlb-address-ref-count-racing-in-prep_compound_gigantic_page +++ a/mm/page_alloc.c @@ -754,7 +754,6 @@ void prep_compound_page(struct page *pag __SetPageHead(page); for (i = 1; i < nr_pages; i++) { struct page *p = page + i; - set_page_count(p, 0); p->mapping = TAIL_MAPPING; set_compound_head(p, page); } From patchwork Thu Jul 1 01:48:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21BBAC11F69 for ; Thu, 1 Jul 2021 01:48:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CDEED61468 for ; Thu, 1 Jul 2021 01:48:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDEED61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 559438D01F0; Wed, 30 Jun 2021 21:48:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 530A58D01D0; Wed, 30 Jun 2021 21:48:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F88F8D01F0; Wed, 30 Jun 2021 21:48:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0092.hostedemail.com [216.40.44.92]) by kanga.kvack.org (Postfix) with ESMTP id 153898D01D0 for ; Wed, 30 Jun 2021 21:48:40 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DABCF180177F0 for ; Thu, 1 Jul 2021 01:48:39 +0000 (UTC) X-FDA: 78312334758.23.6E6B60A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 801C2D000090 for ; Thu, 1 Jul 2021 01:48:39 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6AB3B6120C; Thu, 1 Jul 2021 01:48:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104118; bh=nQ0gbeppwSM962tSymacgiCh73iB7vJGWKuZvbkVpjM=; h=Date:From:To:Subject:In-Reply-To:From; b=apJgT3a1t2kdPZxfua7IXAUpj0yqiIxcuTcHrxH4avYdsBuCOVlgEkmuwDDXV7apg jiCvdjRSxAW8vXNT7HTzICGSOzdsBEPQAWg1swCsSDTUab6GFgalnJK1VdFwk3L+/f 9rJf2FiZYg1CpmNtvQSqXQx1pHGxdA2w9j+0KoqI= Date: Wed, 30 Jun 2021 18:48:38 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, osalvador@suse.de, torvalds@linux-foundation.org Subject: [patch 029/192] mm/hwpoison: disable pcp for page_handle_poison() Message-ID: <20210701014838.TSZ9CNH3C%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=apJgT3a1; dmarc=none; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: soxm837pw8w5gjjw81roc1giot1sqs5h X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 801C2D000090 X-HE-Tag: 1625104119-366341 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Naoya Horiguchi Subject: mm/hwpoison: disable pcp for page_handle_poison() Recent changes by patch "mm/page_alloc: allow high-order pages to be stored on the per-cpu lists" makes kernels determine whether to use pcp by pcp_allowed_order(), which breaks soft-offline for hugetlb pages. Soft-offline dissolves a migration source page, then removes it from buddy free list, so it's assumed that any subpage of the soft-offlined hugepage are recognized as a buddy page just after returning from dissolve_free_huge_page(). pcp_allowed_order() returns true for hugetlb, so this assumption is no longer true. So disable pcp during dissolve_free_huge_page() and take_page_off_buddy() to prevent soft-offlined hugepages from linking to pcp lists. Soft-offline should not be common events so the impact on performance should be minimal. And I think that the optimization of Mel's patch could benefit to hugetlb so zone_pcp_disable() is called only in hwpoison context. Link: https://lkml.kernel.org/r/20210617092626.291006-1-nao.horiguchi@gmail.com Signed-off-by: Naoya Horiguchi Acked-by: Mel Gorman Cc: Mike Kravetz Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memory-failure.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) --- a/mm/memory-failure.c~mm-hwpoison-disable-pcp-for-page_handle_poison +++ a/mm/memory-failure.c @@ -66,6 +66,19 @@ int sysctl_memory_failure_recovery __rea atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0); +static bool __page_handle_poison(struct page *page) +{ + bool ret; + + zone_pcp_disable(page_zone(page)); + ret = dissolve_free_huge_page(page); + if (!ret) + ret = take_page_off_buddy(page); + zone_pcp_enable(page_zone(page)); + + return ret; +} + static bool page_handle_poison(struct page *page, bool hugepage_or_freepage, bool release) { if (hugepage_or_freepage) { @@ -73,7 +86,7 @@ static bool page_handle_poison(struct pa * Doing this check for free pages is also fine since dissolve_free_huge_page * returns 0 for non-hugetlb pages as well. */ - if (dissolve_free_huge_page(page) || !take_page_off_buddy(page)) + if (!__page_handle_poison(page)) /* * We could fail to take off the target page from buddy * for example due to racy page allocation, but that's @@ -985,7 +998,7 @@ static int me_huge_page(struct page *p, */ if (PageAnon(hpage)) put_page(hpage); - if (!dissolve_free_huge_page(p) && take_page_off_buddy(p)) { + if (__page_handle_poison(p)) { page_ref_inc(p); res = MF_RECOVERED; } @@ -1446,7 +1459,7 @@ static int memory_failure_hugetlb(unsign } unlock_page(head); res = MF_FAILED; - if (!dissolve_free_huge_page(p) && take_page_off_buddy(p)) { + if (__page_handle_poison(p)) { page_ref_inc(p); res = MF_RECOVERED; } From patchwork Thu Jul 1 01:48:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C5E8C11F69 for ; Thu, 1 Jul 2021 01:48:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34D8961090 for ; Thu, 1 Jul 2021 01:48:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34D8961090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B4C488D01F1; Wed, 30 Jun 2021 21:48:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFCA28D01D0; Wed, 30 Jun 2021 21:48:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C3F28D01F1; Wed, 30 Jun 2021 21:48:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 751E48D01D0 for ; Wed, 30 Jun 2021 21:48:43 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4B302181AEF3F for ; Thu, 1 Jul 2021 01:48:43 +0000 (UTC) X-FDA: 78312334926.09.F94B761 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 00C7C30002A7 for ; Thu, 1 Jul 2021 01:48:42 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9E89A61468; Thu, 1 Jul 2021 01:48:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104122; bh=T//CElXVE7xBFxBiOkfV6NqZ/Uq66JZRJynSpcZPrWg=; h=Date:From:To:Subject:In-Reply-To:From; b=GRh3Y7umnWDBY8ssFfANHqhlKyzcH6/pIH7N8Nhl2nQL+o/xDHXAJscWv05yP6zIP NF1vQxhMuaRNVXgcaEU6J/3z4YrgJhciEyxIhzT633cFvcCrUno9neLxObVgKa8VG5 Wltlqbe0cusl1dhMxKcBxsjq8yp2jP20p+sA+HOw= Date: Wed, 30 Jun 2021 18:48:41 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 030/192] userfaultfd/selftests: use user mode only Message-ID: <20210701014841.w-4MhufHn%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 00C7C30002A7 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=GRh3Y7um; spf=pass (imf09.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: q14rfub6fwfjc5e6k1ckwbc6h8o31xnu X-HE-Tag: 1625104122-106346 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: userfaultfd/selftests: use user mode only Patch series "userfaultfd/selftests: A few cleanups", v2. I wanted to cleanup userfaultfd.c fault handling for a long time. If it's not cleaned, when the new code grows the file it'll also grow the size that needs to be cleaned... This is my attempt to cleanup the userfaultfd selftest on fault handling, to use an err() macro instead of either fprintf() or perror() then another exit() call. The huge cleanup is done in the last patch. The first 4 patches are some other standalone cleanups for the same file, so I put them together. This patch (of 5): Userfaultfd selftest does not need to handle kernel initiated fault. Set user mode so it can be run even if unprivileged_userfaultfd=0 (which is the default). Link: https://lkml.kernel.org/r/20210412232753.1012412-2-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen Cc: Andrea Arcangeli Cc: Mike Rapoport Cc: Alexander Viro Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-use-user-mode-only +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -831,7 +831,7 @@ static int userfaultfd_open_ext(uint64_t { struct uffdio_api uffdio_api; - uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK); + uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY); if (uffd < 0) { fprintf(stderr, "userfaultfd syscall not available in this kernel\n"); From patchwork Thu Jul 1 01:48:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40870C11F68 for ; Thu, 1 Jul 2021 01:48:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EADA661466 for ; Thu, 1 Jul 2021 01:48:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EADA661466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7068D8D01F2; Wed, 30 Jun 2021 21:48:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DD748D01D0; Wed, 30 Jun 2021 21:48:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A6008D01F2; Wed, 30 Jun 2021 21:48:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0055.hostedemail.com [216.40.44.55]) by kanga.kvack.org (Postfix) with ESMTP id 31EDC8D01D0 for ; Wed, 30 Jun 2021 21:48:47 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 04616181AEF3F for ; Thu, 1 Jul 2021 01:48:47 +0000 (UTC) X-FDA: 78312335094.20.282EE6E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id ABA99100009D for ; Thu, 1 Jul 2021 01:48:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3F58A6120C; Thu, 1 Jul 2021 01:48:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104125; bh=YOOzhUTqf7KWYxyFvUMrP7gImFxTqWLWGXQUYUpxmeA=; h=Date:From:To:Subject:In-Reply-To:From; b=EJHnh/B3d/0pOYMZqS6aopctm3R0gDAN9d5JEQoG3t9Z5lb5svMvm/qPEaqLoPqf9 HhYpYuXNcBvhLgwWl1HDPUBg/MWQM6FRBq8ftpB2V+mMSriNq0bpm7LHG4gLG8X+qm Kz+fEWYL71ZAlLuiXM98ZfQTdNyvt6ZJiMZLZxpY= Date: Wed, 30 Jun 2021 18:48:44 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 031/192] userfaultfd/selftests: remove the time() check on delayed uffd Message-ID: <20210701014844.ASqKQfeHh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: ABA99100009D X-Stat-Signature: qf5dqbwwx1cgkh91atjkuc6ipekmj644 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="EJHnh/B3"; dmarc=none; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104126-542919 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: userfaultfd/selftests: remove the time() check on delayed uffd There seems to have no guarantee that time() will return the same for the two calls even if there's no delay, e.g. when a fault is accidentally crossing the changing of a second. Meanwhile, this message is also not helping that much since delay could happen with a lot of reasons, e.g., schedule latency of resolving thread. It may not mean an issue with uffd. Neither do I saw this error triggered either in the past runs. Even if it triggers, it'll be drown in all the rest of test logs. Remove it. Link: https://lkml.kernel.org/r/20210412232753.1012412-3-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 8 -------- 1 file changed, 8 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-remove-the-time-check-on-delayed-uffd +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -395,7 +395,6 @@ static void *locking_thread(void *arg) unsigned long long count; char randstate[64]; unsigned int seed; - time_t start; if (bounces & BOUNCE_RANDOM) { seed = (unsigned int) time(NULL) - bounces; @@ -432,7 +431,6 @@ static void *locking_thread(void *arg) page_nr += 1; page_nr %= nr_pages; - start = time(NULL); if (bounces & BOUNCE_VERIFY) { count = *area_count(area_dst, page_nr); if (!count) { @@ -495,12 +493,6 @@ static void *locking_thread(void *arg) count++; *area_count(area_dst, page_nr) = count_verify[page_nr] = count; pthread_mutex_unlock(area_mutex(area_dst, page_nr)); - - if (time(NULL) - start > 1) - fprintf(stderr, - "userfault too slow %ld " - "possible false positive with overcommit\n", - time(NULL) - start); } return NULL; From patchwork Thu Jul 1 01:48:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAC4DC11F68 for ; Thu, 1 Jul 2021 01:48:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70EFE6120C for ; Thu, 1 Jul 2021 01:48:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70EFE6120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ECEEF8D01F3; Wed, 30 Jun 2021 21:48:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA5B98D01D0; Wed, 30 Jun 2021 21:48:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6E638D01F3; Wed, 30 Jun 2021 21:48:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0225.hostedemail.com [216.40.44.225]) by kanga.kvack.org (Postfix) with ESMTP id ADD328D01D0 for ; Wed, 30 Jun 2021 21:48:50 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 82E8725F58 for ; Thu, 1 Jul 2021 01:48:50 +0000 (UTC) X-FDA: 78312335220.05.85A1C67 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 339257001A08 for ; Thu, 1 Jul 2021 01:48:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CA17F61466; Thu, 1 Jul 2021 01:48:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104129; bh=N1LQqUiDSomKhABEtMR/OUZHtMe9w8viZXVhfQJPVQE=; h=Date:From:To:Subject:In-Reply-To:From; b=YAvyqLJnC/cTWxtwxWig2iDPEvC1JrEZAqvKkEvChT006g6EAm9phO+Q0eHkldYUY blrUqV9sohddGRIzw3cPUzB/AKbii9DiXl5m2RgF+SFUqCkyM50sBXDFm5Is72FnYE TaH+n1QDuc2qWTRyr1jnlJ1uCyxUwWQXGs+7GJtw= Date: Wed, 30 Jun 2021 18:48:48 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 032/192] userfaultfd/selftests: dropping VERIFY check in locking_thread Message-ID: <20210701014848.i6f-Ezd9w%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 339257001A08 X-Stat-Signature: egh9afretco936zcfp476y8rxggdfn53 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=YAvyqLJn; dmarc=none; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104130-610189 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: userfaultfd/selftests: dropping VERIFY check in locking_thread It tries to check against all zeros and looped for quite a few times. However after that we'll verify the same page with count_verify, while count_verify can never be zero. So it means if it's a zero page we'll detect it anyways with below code. There's yet another place we conditionally check the fault flag - just do it unconditionally. Link: https://lkml.kernel.org/r/20210412232753.1012412-4-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 55 --------------------- 1 file changed, 1 insertion(+), 54 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-dropping-verify-check-in-locking_thread +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -430,58 +430,6 @@ static void *locking_thread(void *arg) } else page_nr += 1; page_nr %= nr_pages; - - if (bounces & BOUNCE_VERIFY) { - count = *area_count(area_dst, page_nr); - if (!count) { - fprintf(stderr, - "page_nr %lu wrong count %Lu %Lu\n", - page_nr, count, - count_verify[page_nr]); - exit(1); - } - - - /* - * We can't use bcmp (or memcmp) because that - * returns 0 erroneously if the memory is - * changing under it (even if the end of the - * page is never changing and always - * different). - */ -#if 1 - if (!my_bcmp(area_dst + page_nr * page_size, zeropage, - page_size)) { - fprintf(stderr, - "my_bcmp page_nr %lu wrong count %Lu %Lu\n", - page_nr, count, count_verify[page_nr]); - exit(1); - } -#else - unsigned long loops; - - loops = 0; - /* uncomment the below line to test with mutex */ - /* pthread_mutex_lock(area_mutex(area_dst, page_nr)); */ - while (!bcmp(area_dst + page_nr * page_size, zeropage, - page_size)) { - loops += 1; - if (loops > 10) - break; - } - /* uncomment below line to test with mutex */ - /* pthread_mutex_unlock(area_mutex(area_dst, page_nr)); */ - if (loops) { - fprintf(stderr, - "page_nr %lu all zero thread %lu %p %lu\n", - page_nr, cpu, area_dst + page_nr * page_size, - loops); - if (loops > 10) - exit(1); - } -#endif - } - pthread_mutex_lock(area_mutex(area_dst, page_nr)); count = *area_count(area_dst, page_nr); if (count != count_verify[page_nr]) { @@ -613,8 +561,7 @@ static void uffd_handle_page_fault(struc stats->minor_faults++; } else { /* Missing page faults */ - if (bounces & BOUNCE_VERIFY && - msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) { + if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) { fprintf(stderr, "unexpected write fault\n"); exit(1); } From patchwork Thu Jul 1 01:48:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48322C11F69 for ; Thu, 1 Jul 2021 01:48:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F3C9E61090 for ; Thu, 1 Jul 2021 01:48:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3C9E61090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C9E88D01F4; Wed, 30 Jun 2021 21:48:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A0EC8D01D0; Wed, 30 Jun 2021 21:48:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6693E8D01F4; Wed, 30 Jun 2021 21:48:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id 3E9CA8D01D0 for ; Wed, 30 Jun 2021 21:48:54 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 189FA1801A832 for ; Thu, 1 Jul 2021 01:48:54 +0000 (UTC) X-FDA: 78312335388.11.A3AD7CF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id CFE4B30000A5 for ; Thu, 1 Jul 2021 01:48:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E9066120C; Thu, 1 Jul 2021 01:48:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104133; bh=HRL8VpJjpH2u4FSHjGZdMu7W0aM7i2zTf2sdnsmrkcs=; h=Date:From:To:Subject:In-Reply-To:From; b=MDoGtfY/oaCp/PaGRzUUk4DfhJWBfMDGppD1Ji27maKXOoMCFmgwBNqGhTfpjaEaR MDGgcVFw8qQsUiNF8oO41GM/5LARD5aq/Xhj8qexcZM1Gh/MEmiVXXQDIwNZAg77qC 6wQNsfCgKlB/Bz6239H4swpyRZcMOD9ZOiEhY+qk= Date: Wed, 30 Jun 2021 18:48:52 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 033/192] userfaultfd/selftests: only dump counts if mode enabled Message-ID: <20210701014852.RuvkWyfWK%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CFE4B30000A5 X-Stat-Signature: o8488rdo8q3q9mkpd77h4ufwunmcpkar Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="MDoGtfY/"; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104133-163413 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: userfaultfd/selftests: only dump counts if mode enabled WP and MINOR modes are conditionally enabled on specific memory types. This patch avoids dumping tons of zeros for those cases when the modes are not supported at all. Link: https://lkml.kernel.org/r/20210412232753.1012412-5-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 30 ++++++++++++++------- 1 file changed, 20 insertions(+), 10 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-only-dump-counts-if-mode-enabled +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -171,16 +171,26 @@ static void uffd_stats_report(struct uff minor_total += stats[i].minor_faults; } - printf("userfaults: %llu missing (", miss_total); - for (i = 0; i < n_cpus; i++) - printf("%lu+", stats[i].missing_faults); - printf("\b), %llu wp (", wp_total); - for (i = 0; i < n_cpus; i++) - printf("%lu+", stats[i].wp_faults); - printf("\b), %llu minor (", minor_total); - for (i = 0; i < n_cpus; i++) - printf("%lu+", stats[i].minor_faults); - printf("\b)\n"); + printf("userfaults: "); + if (miss_total) { + printf("%llu missing (", miss_total); + for (i = 0; i < n_cpus; i++) + printf("%lu+", stats[i].missing_faults); + printf("\b) "); + } + if (wp_total) { + printf("%llu wp (", wp_total); + for (i = 0; i < n_cpus; i++) + printf("%lu+", stats[i].wp_faults); + printf("\b) "); + } + if (minor_total) { + printf("%llu minor (", minor_total); + for (i = 0; i < n_cpus; i++) + printf("%lu+", stats[i].minor_faults); + printf("\b)"); + } + printf("\n"); } static int anon_release_pages(char *rel_area) From patchwork Thu Jul 1 01:48:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36FA3C11F68 for ; Thu, 1 Jul 2021 01:48:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C0A1F6120C for ; Thu, 1 Jul 2021 01:48:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0A1F6120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 495968D01F5; Wed, 30 Jun 2021 21:48:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4457A8D01D0; Wed, 30 Jun 2021 21:48:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 270B58D01F5; Wed, 30 Jun 2021 21:48:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0100.hostedemail.com [216.40.44.100]) by kanga.kvack.org (Postfix) with ESMTP id EABA18D01D0 for ; Wed, 30 Jun 2021 21:48:57 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B49362488D for ; Thu, 1 Jul 2021 01:48:57 +0000 (UTC) X-FDA: 78312335514.11.EA1CD4E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 55BC8D0000A0 for ; Thu, 1 Jul 2021 01:48:57 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0300061090; Thu, 1 Jul 2021 01:48:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104136; bh=prN5qK548hl5QuLjo2bY5nWhxPqUUkWBYp993JsMSp0=; h=Date:From:To:Subject:In-Reply-To:From; b=i77MjbG8jWhe/ghRgVoUYfEau/wrggWTjKutYc41adWNeDMFSlM+5pGe8cS26tMU+ 2SR/3g3GTEnmr90r2Vd4cwWJVHRzSVZpbe6VjddM8e+cjzyPYgBAu/QqyGV/MNbgGh fDjnTkIBgHKaBCWWIXjvWAz4x21u7ZyBnhbh5iSY= Date: Wed, 30 Jun 2021 18:48:55 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 034/192] userfaultfd/selftests: unify error handling Message-ID: <20210701014855._Z9WTu8lZ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 55BC8D0000A0 X-Stat-Signature: htcpu5q9e16r4jky3xhgyew439x3a7rq Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=i77MjbG8; dmarc=none; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104137-884048 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: userfaultfd/selftests: unify error handling Introduce err()/_err() and replace all the different ways to fail the program, mostly "fprintf" and "perror" with tons of exit() calls. Always stop the test program at any failure. Link: https://lkml.kernel.org/r/20210412232753.1012412-6-peterx@redhat.com Signed-off-by: Peter Xu Reviewed-by: Axel Rasmussen Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 556 +++++++-------------- 1 file changed, 187 insertions(+), 369 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-unify-error-handling +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -140,11 +140,18 @@ static void usage(void) exit(1); } -#define uffd_error(code, fmt, ...) \ - do { \ - fprintf(stderr, fmt, ##__VA_ARGS__); \ - fprintf(stderr, ": %" PRId64 "\n", (int64_t)(code)); \ - exit(1); \ +#define _err(fmt, ...) \ + do { \ + int ret = errno; \ + fprintf(stderr, "ERROR: " fmt, ##__VA_ARGS__); \ + fprintf(stderr, " (errno=%d, line=%d)\n", \ + ret, __LINE__); \ + } while (0) + +#define err(fmt, ...) \ + do { \ + _err(fmt, ##__VA_ARGS__); \ + exit(1); \ } while (0) static void uffd_stats_reset(struct uffd_stats *uffd_stats, @@ -193,44 +200,28 @@ static void uffd_stats_report(struct uff printf("\n"); } -static int anon_release_pages(char *rel_area) +static void anon_release_pages(char *rel_area) { - int ret = 0; - - if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED)) { - perror("madvise"); - ret = 1; - } - - return ret; + if (madvise(rel_area, nr_pages * page_size, MADV_DONTNEED)) + err("madvise(MADV_DONTNEED) failed"); } static void anon_allocate_area(void **alloc_area) { - if (posix_memalign(alloc_area, page_size, nr_pages * page_size)) { - fprintf(stderr, "out of memory\n"); - *alloc_area = NULL; - } + if (posix_memalign(alloc_area, page_size, nr_pages * page_size)) + err("posix_memalign() failed"); } static void noop_alias_mapping(__u64 *start, size_t len, unsigned long offset) { } -/* HugeTLB memory */ -static int hugetlb_release_pages(char *rel_area) +static void hugetlb_release_pages(char *rel_area) { - int ret = 0; - if (fallocate(huge_fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, - rel_area == huge_fd_off0 ? 0 : - nr_pages * page_size, - nr_pages * page_size)) { - perror("fallocate"); - ret = 1; - } - - return ret; + rel_area == huge_fd_off0 ? 0 : nr_pages * page_size, + nr_pages * page_size)) + err("fallocate() failed"); } static void hugetlb_allocate_area(void **alloc_area) @@ -243,20 +234,16 @@ static void hugetlb_allocate_area(void * MAP_HUGETLB, huge_fd, *alloc_area == area_src ? 0 : nr_pages * page_size); - if (*alloc_area == MAP_FAILED) { - perror("mmap of hugetlbfs file failed"); - goto fail; - } + if (*alloc_area == MAP_FAILED) + err("mmap of hugetlbfs file failed"); if (map_shared) { area_alias = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_HUGETLB, huge_fd, *alloc_area == area_src ? 0 : nr_pages * page_size); - if (area_alias == MAP_FAILED) { - perror("mmap of hugetlb file alias failed"); - goto fail_munmap; - } + if (area_alias == MAP_FAILED) + err("mmap of hugetlb file alias failed"); } if (*alloc_area == area_src) { @@ -267,16 +254,6 @@ static void hugetlb_allocate_area(void * } if (area_alias) *alloc_area_alias = area_alias; - - return; - -fail_munmap: - if (munmap(*alloc_area, nr_pages * page_size) < 0) { - perror("hugetlb munmap"); - exit(1); - } -fail: - *alloc_area = NULL; } static void hugetlb_alias_mapping(__u64 *start, size_t len, unsigned long offset) @@ -292,33 +269,24 @@ static void hugetlb_alias_mapping(__u64 *start = (unsigned long) area_dst_alias + offset; } -/* Shared memory */ -static int shmem_release_pages(char *rel_area) +static void shmem_release_pages(char *rel_area) { - int ret = 0; - - if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE)) { - perror("madvise"); - ret = 1; - } - - return ret; + if (madvise(rel_area, nr_pages * page_size, MADV_REMOVE)) + err("madvise(MADV_REMOVE) failed"); } static void shmem_allocate_area(void **alloc_area) { *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_SHARED, -1, 0); - if (*alloc_area == MAP_FAILED) { - fprintf(stderr, "shared memory mmap failed\n"); - *alloc_area = NULL; - } + if (*alloc_area == MAP_FAILED) + err("mmap of memfd failed"); } struct uffd_test_ops { unsigned long expected_ioctls; void (*allocate_area)(void **alloc_area); - int (*release_pages)(char *rel_area); + void (*release_pages)(char *rel_area); void (*alias_mapping)(__u64 *start, size_t len, unsigned long offset); }; @@ -373,11 +341,8 @@ static void wp_range(int ufd, __u64 star /* Undo write-protect, do wakeup after that */ prms.mode = wp ? UFFDIO_WRITEPROTECT_MODE_WP : 0; - if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms)) { - fprintf(stderr, "clear WP failed for address 0x%" PRIx64 "\n", - (uint64_t)start); - exit(1); - } + if (ioctl(ufd, UFFDIO_WRITEPROTECT, &prms)) + err("clear WP failed: address=0x%"PRIx64, (uint64_t)start); } static void continue_range(int ufd, __u64 start, __u64 len) @@ -388,12 +353,9 @@ static void continue_range(int ufd, __u6 req.range.len = len; req.mode = 0; - if (ioctl(ufd, UFFDIO_CONTINUE, &req)) { - fprintf(stderr, - "UFFDIO_CONTINUE failed for address 0x%" PRIx64 "\n", - (uint64_t)start); - exit(1); - } + if (ioctl(ufd, UFFDIO_CONTINUE, &req)) + err("UFFDIO_CONTINUE failed for address 0x%" PRIx64, + (uint64_t)start); } static void *locking_thread(void *arg) @@ -412,10 +374,8 @@ static void *locking_thread(void *arg) seed += cpu; bzero(&rand, sizeof(rand)); bzero(&randstate, sizeof(randstate)); - if (initstate_r(seed, randstate, sizeof(randstate), &rand)) { - fprintf(stderr, "srandom_r error\n"); - exit(1); - } + if (initstate_r(seed, randstate, sizeof(randstate), &rand)) + err("initstate_r failed"); } else { page_nr = -bounces; if (!(bounces & BOUNCE_RACINGFAULTS)) @@ -424,16 +384,12 @@ static void *locking_thread(void *arg) while (!finished) { if (bounces & BOUNCE_RANDOM) { - if (random_r(&rand, &rand_nr)) { - fprintf(stderr, "random_r 1 error\n"); - exit(1); - } + if (random_r(&rand, &rand_nr)) + err("random_r failed"); page_nr = rand_nr; if (sizeof(page_nr) > sizeof(rand_nr)) { - if (random_r(&rand, &rand_nr)) { - fprintf(stderr, "random_r 2 error\n"); - exit(1); - } + if (random_r(&rand, &rand_nr)) + err("random_r failed"); page_nr |= (((unsigned long) rand_nr) << 16) << 16; } @@ -442,12 +398,9 @@ static void *locking_thread(void *arg) page_nr %= nr_pages; pthread_mutex_lock(area_mutex(area_dst, page_nr)); count = *area_count(area_dst, page_nr); - if (count != count_verify[page_nr]) { - fprintf(stderr, - "page_nr %lu memory corruption %Lu %Lu\n", - page_nr, count, - count_verify[page_nr]); exit(1); - } + if (count != count_verify[page_nr]) + err("page_nr %lu memory corruption %llu %llu", + page_nr, count, count_verify[page_nr]); count++; *area_count(area_dst, page_nr) = count_verify[page_nr] = count; pthread_mutex_unlock(area_mutex(area_dst, page_nr)); @@ -464,22 +417,21 @@ static void retry_copy_page(int ufd, str offset); if (ioctl(ufd, UFFDIO_COPY, uffdio_copy)) { /* real retval in ufdio_copy.copy */ - if (uffdio_copy->copy != -EEXIST) { - uffd_error(uffdio_copy->copy, - "UFFDIO_COPY retry error"); - } - } else - uffd_error(uffdio_copy->copy, "UFFDIO_COPY retry unexpected"); + if (uffdio_copy->copy != -EEXIST) + err("UFFDIO_COPY retry error: %"PRId64, + (int64_t)uffdio_copy->copy); + } else { + err("UFFDIO_COPY retry unexpected: %"PRId64, + (int64_t)uffdio_copy->copy); + } } static int __copy_page(int ufd, unsigned long offset, bool retry) { struct uffdio_copy uffdio_copy; - if (offset >= nr_pages * page_size) { - fprintf(stderr, "unexpected offset %lu\n", offset); - exit(1); - } + if (offset >= nr_pages * page_size) + err("unexpected offset %lu\n", offset); uffdio_copy.dst = (unsigned long) area_dst + offset; uffdio_copy.src = (unsigned long) area_src + offset; uffdio_copy.len = page_size; @@ -491,9 +443,10 @@ static int __copy_page(int ufd, unsigned if (ioctl(ufd, UFFDIO_COPY, &uffdio_copy)) { /* real retval in ufdio_copy.copy */ if (uffdio_copy.copy != -EEXIST) - uffd_error(uffdio_copy.copy, "UFFDIO_COPY error"); + err("UFFDIO_COPY error: %"PRId64, + (int64_t)uffdio_copy.copy); } else if (uffdio_copy.copy != page_size) { - uffd_error(uffdio_copy.copy, "UFFDIO_COPY unexpected copy"); + err("UFFDIO_COPY error: %"PRId64, (int64_t)uffdio_copy.copy); } else { if (test_uffdio_copy_eexist && retry) { test_uffdio_copy_eexist = false; @@ -522,11 +475,10 @@ static int uffd_read_msg(int ufd, struct if (ret < 0) { if (errno == EAGAIN) return 1; - perror("blocking read error"); + err("blocking read error"); } else { - fprintf(stderr, "short read\n"); + err("short read"); } - exit(1); } return 0; @@ -537,10 +489,8 @@ static void uffd_handle_page_fault(struc { unsigned long offset; - if (msg->event != UFFD_EVENT_PAGEFAULT) { - fprintf(stderr, "unexpected msg event %u\n", msg->event); - exit(1); - } + if (msg->event != UFFD_EVENT_PAGEFAULT) + err("unexpected msg event %u", msg->event); if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WP) { /* Write protect page faults */ @@ -571,10 +521,8 @@ static void uffd_handle_page_fault(struc stats->minor_faults++; } else { /* Missing page faults */ - if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) { - fprintf(stderr, "unexpected write fault\n"); - exit(1); - } + if (msg->arg.pagefault.flags & UFFD_PAGEFAULT_FLAG_WRITE) + err("unexpected write fault"); offset = (char *)(unsigned long)msg->arg.pagefault.address - area_dst; offset &= ~(page_size-1); @@ -601,32 +549,20 @@ static void *uffd_poll_thread(void *arg) for (;;) { ret = poll(pollfd, 2, -1); - if (!ret) { - fprintf(stderr, "poll error %d\n", ret); - exit(1); - } - if (ret < 0) { - perror("poll"); - exit(1); - } + if (ret <= 0) + err("poll error: %d", ret); if (pollfd[1].revents & POLLIN) { - if (read(pollfd[1].fd, &tmp_chr, 1) != 1) { - fprintf(stderr, "read pipefd error\n"); - exit(1); - } + if (read(pollfd[1].fd, &tmp_chr, 1) != 1) + err("read pipefd error"); break; } - if (!(pollfd[0].revents & POLLIN)) { - fprintf(stderr, "pollfd[0].revents %d\n", - pollfd[0].revents); - exit(1); - } + if (!(pollfd[0].revents & POLLIN)) + err("pollfd[0].revents %d", pollfd[0].revents); if (uffd_read_msg(uffd, &msg)) continue; switch (msg.event) { default: - fprintf(stderr, "unexpected msg event %u\n", - msg.event); exit(1); + err("unexpected msg event %u\n", msg.event); break; case UFFD_EVENT_PAGEFAULT: uffd_handle_page_fault(&msg, stats); @@ -640,10 +576,8 @@ static void *uffd_poll_thread(void *arg) uffd_reg.range.start = msg.arg.remove.start; uffd_reg.range.len = msg.arg.remove.end - msg.arg.remove.start; - if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range)) { - fprintf(stderr, "remove failure\n"); - exit(1); - } + if (ioctl(uffd, UFFDIO_UNREGISTER, &uffd_reg.range)) + err("remove failure"); break; case UFFD_EVENT_REMAP: area_dst = (char *)(unsigned long)msg.arg.remap.to; @@ -746,9 +680,7 @@ static int stress(struct uffd_stats *uff * UFFDIO_COPY without writing zero pages into area_dst * because the background threads already completed). */ - if (uffd_test_ops->release_pages(area_src)) - return 1; - + uffd_test_ops->release_pages(area_src); finished = 1; for (cpu = 0; cpu < nr_cpus; cpu++) @@ -758,10 +690,8 @@ static int stress(struct uffd_stats *uff for (cpu = 0; cpu < nr_cpus; cpu++) { char c; if (bounces & BOUNCE_POLL) { - if (write(pipefd[cpu*2+1], &c, 1) != 1) { - fprintf(stderr, "pipefd write error\n"); - return 1; - } + if (write(pipefd[cpu*2+1], &c, 1) != 1) + err("pipefd write error"); if (pthread_join(uffd_threads[cpu], (void *)&uffd_stats[cpu])) return 1; @@ -861,10 +791,8 @@ static int faulting_process(int signal_t memset(&act, 0, sizeof(act)); act.sa_sigaction = sighndl; act.sa_flags = SA_SIGINFO; - if (sigaction(SIGBUS, &act, 0)) { - perror("sigaction"); - return 1; - } + if (sigaction(SIGBUS, &act, 0)) + err("sigaction"); lastnr = (unsigned long)-1; } @@ -874,10 +802,8 @@ static int faulting_process(int signal_t if (signal_test) { if (sigsetjmp(*sigbuf, 1) != 0) { - if (steps == 1 && nr == lastnr) { - fprintf(stderr, "Signal repeated\n"); - return 1; - } + if (steps == 1 && nr == lastnr) + err("Signal repeated"); lastnr = nr; if (signal_test == 1) { @@ -902,12 +828,9 @@ static int faulting_process(int signal_t } count = *area_count(area_dst, nr); - if (count != count_verify[nr]) { - fprintf(stderr, - "nr %lu memory corruption %Lu %Lu\n", - nr, count, - count_verify[nr]); - } + if (count != count_verify[nr]) + err("nr %lu memory corruption %llu %llu\n", + nr, count, count_verify[nr]); /* * Trigger write protection if there is by writing * the same value back. @@ -923,18 +846,14 @@ static int faulting_process(int signal_t area_dst = mremap(area_dst, nr_pages * page_size, nr_pages * page_size, MREMAP_MAYMOVE | MREMAP_FIXED, area_src); - if (area_dst == MAP_FAILED) { - perror("mremap"); - exit(1); - } + if (area_dst == MAP_FAILED) + err("mremap"); for (; nr < nr_pages; nr++) { count = *area_count(area_dst, nr); if (count != count_verify[nr]) { - fprintf(stderr, - "nr %lu memory corruption %Lu %Lu\n", - nr, count, - count_verify[nr]); exit(1); + err("nr %lu memory corruption %llu %llu\n", + nr, count, count_verify[nr]); } /* * Trigger write protection if there is by writing @@ -943,15 +862,11 @@ static int faulting_process(int signal_t *area_count(area_dst, nr) = count; } - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); - for (nr = 0; nr < nr_pages; nr++) { - if (my_bcmp(area_dst + nr * page_size, zeropage, page_size)) { - fprintf(stderr, "nr %lu is not zero\n", nr); - exit(1); - } - } + for (nr = 0; nr < nr_pages; nr++) + if (my_bcmp(area_dst + nr * page_size, zeropage, page_size)) + err("nr %lu is not zero", nr); return 0; } @@ -964,13 +879,12 @@ static void retry_uffdio_zeropage(int uf uffdio_zeropage->range.len, offset); if (ioctl(ufd, UFFDIO_ZEROPAGE, uffdio_zeropage)) { - if (uffdio_zeropage->zeropage != -EEXIST) { - uffd_error(uffdio_zeropage->zeropage, - "UFFDIO_ZEROPAGE retry error"); - } + if (uffdio_zeropage->zeropage != -EEXIST) + err("UFFDIO_ZEROPAGE error: %"PRId64, + (int64_t)uffdio_zeropage->zeropage); } else { - uffd_error(uffdio_zeropage->zeropage, - "UFFDIO_ZEROPAGE retry unexpected"); + err("UFFDIO_ZEROPAGE error: %"PRId64, + (int64_t)uffdio_zeropage->zeropage); } } @@ -983,10 +897,8 @@ static int __uffdio_zeropage(int ufd, un has_zeropage = uffd_test_ops->expected_ioctls & (1 << _UFFDIO_ZEROPAGE); - if (offset >= nr_pages * page_size) { - fprintf(stderr, "unexpected offset %lu\n", offset); - exit(1); - } + if (offset >= nr_pages * page_size) + err("unexpected offset %lu", offset); uffdio_zeropage.range.start = (unsigned long) area_dst + offset; uffdio_zeropage.range.len = page_size; uffdio_zeropage.mode = 0; @@ -994,14 +906,13 @@ static int __uffdio_zeropage(int ufd, un res = uffdio_zeropage.zeropage; if (ret) { /* real retval in ufdio_zeropage.zeropage */ - if (has_zeropage) { - uffd_error(res, "UFFDIO_ZEROPAGE %s", - res == -EEXIST ? "-EEXIST" : "error"); - } else if (res != -EINVAL) - uffd_error(res, "UFFDIO_ZEROPAGE not -EINVAL"); + if (has_zeropage) + err("UFFDIO_ZEROPAGE error: %"PRId64, (int64_t)res); + else if (res != -EINVAL) + err("UFFDIO_ZEROPAGE not -EINVAL"); } else if (has_zeropage) { if (res != page_size) { - uffd_error(res, "UFFDIO_ZEROPAGE unexpected"); + err("UFFDIO_ZEROPAGE unexpected size"); } else { if (test_uffdio_zeropage_eexist && retry) { test_uffdio_zeropage_eexist = false; @@ -1011,7 +922,7 @@ static int __uffdio_zeropage(int ufd, un return 1; } } else - uffd_error(res, "UFFDIO_ZEROPAGE succeeded"); + err("UFFDIO_ZEROPAGE succeeded"); return 0; } @@ -1030,8 +941,7 @@ static int userfaultfd_zeropage_test(voi printf("testing UFFDIO_ZEROPAGE: "); fflush(stdout); - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); if (userfaultfd_open(0)) return 1; @@ -1040,25 +950,16 @@ static int userfaultfd_zeropage_test(voi uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING; if (test_uffdio_wp) uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP; - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) { - fprintf(stderr, "register failure\n"); - exit(1); - } + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failure"); expected_ioctls = uffd_test_ops->expected_ioctls; - if ((uffdio_register.ioctls & expected_ioctls) != - expected_ioctls) { - fprintf(stderr, - "unexpected missing ioctl for anon memory\n"); - exit(1); - } + if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) + err("unexpected missing ioctl for anon memory"); - if (uffdio_zeropage(uffd, 0)) { - if (my_bcmp(area_dst, zeropage, page_size)) { - fprintf(stderr, "zeropage is not zero\n"); - exit(1); - } - } + if (uffdio_zeropage(uffd, 0)) + if (my_bcmp(area_dst, zeropage, page_size)) + err("zeropage is not zero"); close(uffd); printf("done.\n"); @@ -1078,8 +979,7 @@ static int userfaultfd_events_test(void) printf("testing events (fork, remap, remove): "); fflush(stdout); - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); features = UFFD_FEATURE_EVENT_FORK | UFFD_FEATURE_EVENT_REMAP | UFFD_FEATURE_EVENT_REMOVE; @@ -1092,41 +992,28 @@ static int userfaultfd_events_test(void) uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING; if (test_uffdio_wp) uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP; - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) { - fprintf(stderr, "register failure\n"); - exit(1); - } + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failure"); expected_ioctls = uffd_test_ops->expected_ioctls; - if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) { - fprintf(stderr, "unexpected missing ioctl for anon memory\n"); - exit(1); - } + if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) + err("unexpected missing ioctl for anon memory"); - if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats)) { - perror("uffd_poll_thread create"); - exit(1); - } + if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats)) + err("uffd_poll_thread create"); pid = fork(); - if (pid < 0) { - perror("fork"); - exit(1); - } + if (pid < 0) + err("fork"); if (!pid) exit(faulting_process(0)); waitpid(pid, &err, 0); - if (err) { - fprintf(stderr, "faulting process failed\n"); - exit(1); - } - - if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) { - perror("pipe write"); - exit(1); - } + if (err) + err("faulting process failed"); + if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) + err("pipe write"); if (pthread_join(uffd_mon, NULL)) return 1; @@ -1151,8 +1038,7 @@ static int userfaultfd_sig_test(void) printf("testing signal delivery: "); fflush(stdout); - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); features = UFFD_FEATURE_EVENT_FORK|UFFD_FEATURE_SIGBUS; if (userfaultfd_open(features)) @@ -1164,57 +1050,41 @@ static int userfaultfd_sig_test(void) uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING; if (test_uffdio_wp) uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP; - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) { - fprintf(stderr, "register failure\n"); - exit(1); - } + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failure"); expected_ioctls = uffd_test_ops->expected_ioctls; - if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) { - fprintf(stderr, "unexpected missing ioctl for anon memory\n"); - exit(1); - } + if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) + err("unexpected missing ioctl for anon memory"); - if (faulting_process(1)) { - fprintf(stderr, "faulting process failed\n"); - exit(1); - } + if (faulting_process(1)) + err("faulting process failed"); - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); - if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats)) { - perror("uffd_poll_thread create"); - exit(1); - } + if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats)) + err("uffd_poll_thread create"); pid = fork(); - if (pid < 0) { - perror("fork"); - exit(1); - } + if (pid < 0) + err("fork"); if (!pid) exit(faulting_process(2)); waitpid(pid, &err, 0); - if (err) { - fprintf(stderr, "faulting process failed\n"); - exit(1); - } - - if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) { - perror("pipe write"); - exit(1); - } + if (err) + err("faulting process failed"); + if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) + err("pipe write"); if (pthread_join(uffd_mon, (void **)&userfaults)) return 1; printf("done.\n"); if (userfaults) - fprintf(stderr, "Signal test failed, userfaults: %ld\n", - userfaults); + err("Signal test failed, userfaults: %ld", userfaults); close(uffd); + return userfaults != 0; } @@ -1236,8 +1106,7 @@ static int userfaultfd_minor_test(void) printf("testing minor faults: "); fflush(stdout); - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); if (userfaultfd_open_ext(&features)) return 1; @@ -1251,17 +1120,13 @@ static int userfaultfd_minor_test(void) uffdio_register.range.start = (unsigned long)area_dst_alias; uffdio_register.range.len = nr_pages * page_size; uffdio_register.mode = UFFDIO_REGISTER_MODE_MINOR; - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) { - fprintf(stderr, "register failure\n"); - exit(1); - } + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failure"); expected_ioctls = uffd_test_ops->expected_ioctls; expected_ioctls |= 1 << _UFFDIO_CONTINUE; - if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) { - fprintf(stderr, "unexpected missing ioctl(s)\n"); - exit(1); - } + if ((uffdio_register.ioctls & expected_ioctls) != expected_ioctls) + err("unexpected missing ioctl(s)"); /* * After registering with UFFD, populate the non-UFFD-registered side of @@ -1272,10 +1137,8 @@ static int userfaultfd_minor_test(void) page_size); } - if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats)) { - perror("uffd_poll_thread create"); - exit(1); - } + if (pthread_create(&uffd_mon, &attr, uffd_poll_thread, &stats)) + err("uffd_poll_thread create"); /* * Read each of the pages back using the UFFD-registered mapping. We @@ -1284,26 +1147,19 @@ static int userfaultfd_minor_test(void) * page's contents, and then issuing a CONTINUE ioctl. */ - if (posix_memalign(&expected_page, page_size, page_size)) { - fprintf(stderr, "out of memory\n"); - return 1; - } + if (posix_memalign(&expected_page, page_size, page_size)) + err("out of memory"); for (p = 0; p < nr_pages; ++p) { expected_byte = ~((uint8_t)(p % ((uint8_t)-1))); memset(expected_page, expected_byte, page_size); if (my_bcmp(expected_page, area_dst_alias + (p * page_size), - page_size)) { - fprintf(stderr, - "unexpected page contents after minor fault\n"); - exit(1); - } + page_size)) + err("unexpected page contents after minor fault"); } - if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) { - perror("pipe write"); - exit(1); - } + if (write(pipefd[1], &c, sizeof(c)) != sizeof(c)) + err("pipe write"); if (pthread_join(uffd_mon, NULL)) return 1; @@ -1321,7 +1177,6 @@ static int userfaultfd_stress(void) unsigned long nr; struct uffdio_register uffdio_register; unsigned long cpu; - int err; struct uffd_stats uffd_stats[nr_cpus]; uffd_test_ops->allocate_area((void **)&area_src); @@ -1366,10 +1221,8 @@ static int userfaultfd_stress(void) } } - if (posix_memalign(&area, page_size, page_size)) { - fprintf(stderr, "out of memory\n"); - return 1; - } + if (posix_memalign(&area, page_size, page_size)) + err("out of memory"); zeropage = area; bzero(zeropage, page_size); @@ -1378,7 +1231,6 @@ static int userfaultfd_stress(void) pthread_attr_init(&attr); pthread_attr_setstacksize(&attr, 16*1024*1024); - err = 0; while (bounces--) { unsigned long expected_ioctls; @@ -1407,25 +1259,18 @@ static int userfaultfd_stress(void) uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING; if (test_uffdio_wp) uffdio_register.mode |= UFFDIO_REGISTER_MODE_WP; - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) { - fprintf(stderr, "register failure\n"); - return 1; - } + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failure"); expected_ioctls = uffd_test_ops->expected_ioctls; if ((uffdio_register.ioctls & expected_ioctls) != - expected_ioctls) { - fprintf(stderr, - "unexpected missing ioctl for anon memory\n"); - return 1; - } + expected_ioctls) + err("unexpected missing ioctl for anon memory"); if (area_dst_alias) { uffdio_register.range.start = (unsigned long) area_dst_alias; - if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) { - fprintf(stderr, "register failure alias\n"); - return 1; - } + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failure alias"); } /* @@ -1452,8 +1297,7 @@ static int userfaultfd_stress(void) * MADV_DONTNEED only after the UFFDIO_REGISTER, so it's * required to MADV_DONTNEED here. */ - if (uffd_test_ops->release_pages(area_dst)) - return 1; + uffd_test_ops->release_pages(area_dst); uffd_stats_reset(uffd_stats, nr_cpus); @@ -1467,33 +1311,22 @@ static int userfaultfd_stress(void) nr_pages * page_size, false); /* unregister */ - if (ioctl(uffd, UFFDIO_UNREGISTER, &uffdio_register.range)) { - fprintf(stderr, "unregister failure\n"); - return 1; - } + if (ioctl(uffd, UFFDIO_UNREGISTER, &uffdio_register.range)) + err("unregister failure"); if (area_dst_alias) { uffdio_register.range.start = (unsigned long) area_dst; if (ioctl(uffd, UFFDIO_UNREGISTER, - &uffdio_register.range)) { - fprintf(stderr, "unregister failure alias\n"); - return 1; - } + &uffdio_register.range)) + err("unregister failure alias"); } /* verification */ - if (bounces & BOUNCE_VERIFY) { - for (nr = 0; nr < nr_pages; nr++) { - if (*area_count(area_dst, nr) != count_verify[nr]) { - fprintf(stderr, - "error area_count %Lu %Lu %lu\n", - *area_count(area_src, nr), - count_verify[nr], - nr); - err = 1; - bounces = 0; - } - } - } + if (bounces & BOUNCE_VERIFY) + for (nr = 0; nr < nr_pages; nr++) + if (*area_count(area_dst, nr) != count_verify[nr]) + err("error area_count %llu %llu %lu\n", + *area_count(area_src, nr), + count_verify[nr], nr); /* prepare next bounce */ tmp_area = area_src; @@ -1507,9 +1340,6 @@ static int userfaultfd_stress(void) uffd_stats_report(uffd_stats, nr_cpus); } - if (err) - return err; - close(uffd); return userfaultfd_zeropage_test() || userfaultfd_sig_test() || userfaultfd_events_test() || userfaultfd_minor_test(); @@ -1560,7 +1390,7 @@ static void set_test_type(const char *ty test_type = TEST_SHMEM; uffd_test_ops = &shmem_uffd_test_ops; } else { - fprintf(stderr, "Unknown test type: %s\n", type); exit(1); + err("Unknown test type: %s", type); } if (test_type == TEST_HUGETLB) @@ -1568,15 +1398,11 @@ static void set_test_type(const char *ty else page_size = sysconf(_SC_PAGE_SIZE); - if (!page_size) { - fprintf(stderr, "Unable to determine page size\n"); - exit(2); - } + if (!page_size) + err("Unable to determine page size"); if ((unsigned long) area_count(NULL, 0) + sizeof(unsigned long long) * 2 - > page_size) { - fprintf(stderr, "Impossible to run this test\n"); - exit(2); - } + > page_size) + err("Impossible to run this test"); } static void sigalrm(int sig) @@ -1593,10 +1419,8 @@ int main(int argc, char **argv) if (argc < 4) usage(); - if (signal(SIGALRM, sigalrm) == SIG_ERR) { - fprintf(stderr, "failed to arm SIGALRM"); - exit(1); - } + if (signal(SIGALRM, sigalrm) == SIG_ERR) + err("failed to arm SIGALRM"); alarm(ALARM_INTERVAL_SECS); set_test_type(argv[1]); @@ -1605,13 +1429,13 @@ int main(int argc, char **argv) nr_pages_per_cpu = atol(argv[2]) * 1024*1024 / page_size / nr_cpus; if (!nr_pages_per_cpu) { - fprintf(stderr, "invalid MiB\n"); + _err("invalid MiB"); usage(); } bounces = atoi(argv[3]); if (bounces <= 0) { - fprintf(stderr, "invalid bounces\n"); + _err("invalid bounces"); usage(); } nr_pages = nr_pages_per_cpu * nr_cpus; @@ -1620,16 +1444,10 @@ int main(int argc, char **argv) if (argc < 5) usage(); huge_fd = open(argv[4], O_CREAT | O_RDWR, 0755); - if (huge_fd < 0) { - fprintf(stderr, "Open of %s failed", argv[3]); - perror("open"); - exit(1); - } - if (ftruncate(huge_fd, 0)) { - fprintf(stderr, "ftruncate %s to size 0 failed", argv[3]); - perror("ftruncate"); - exit(1); - } + if (huge_fd < 0) + err("Open of %s failed", argv[4]); + if (ftruncate(huge_fd, 0)) + err("ftruncate %s to size 0 failed", argv[4]); } printf("nr_pages: %lu, nr_pages_per_cpu: %lu\n", nr_pages, nr_pages_per_cpu); From patchwork Thu Jul 1 01:48:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5B18C11F69 for ; Thu, 1 Jul 2021 01:49:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7E83461090 for ; Thu, 1 Jul 2021 01:49:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7E83461090 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 076DB8D01F6; Wed, 30 Jun 2021 21:49:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 04E3E8D01D0; Wed, 30 Jun 2021 21:49:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E57378D01F6; Wed, 30 Jun 2021 21:49:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id BE8938D01D0 for ; Wed, 30 Jun 2021 21:49:01 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 945C2252D7 for ; Thu, 1 Jul 2021 01:49:01 +0000 (UTC) X-FDA: 78312335682.31.D3251F1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 235DEF0000AF for ; Thu, 1 Jul 2021 01:49:00 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A29D261469; Thu, 1 Jul 2021 01:48:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104140; bh=iUaA8MK1PDOj7bGi7HEjZogXJ88oDseJdbLzFpj1Kk0=; h=Date:From:To:Subject:In-Reply-To:From; b=HB0nxXAZf/Vex0k5PM5fZzrXsGpoxLS9JPLUUh9xUaG1ahydLGl+6aeHj337dusVQ Mj6rs2zdot6lrA7D8AR/6oRlN2WoixQmvcjIvid9KSKy8G35onH3D2Gnc0KDUztY2Z IJWmDHgSSw4+bbX8X+603rE8d+QtaryFaWV5sLpQ= Date: Wed, 30 Jun 2021 18:48:59 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 035/192] mm/thp: simplify copying of huge zero page pmd when fork Message-ID: <20210701014859.DK9n7jkks%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 235DEF0000AF X-Stat-Signature: r67pmde3ei38cp3kqad6g8tb4gp94d1b Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=HB0nxXAZ; dmarc=none; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104140-812845 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: mm/thp: simplify copying of huge zero page pmd when fork Patch series "mm/uffd: Misc fix for uffd-wp and one more test". This series tries to fix some corner case bugs for uffd-wp on either thp or fork(). Then it introduced a new test with pagemap/pageout. Patch layout: Patch 1: cleanup for THP, it'll slightly simplify the follow up patches Patch 2-4: misc fixes for uffd-wp here and there; please refer to each patch Patch 5: add pagemap support for uffd-wp Patch 6: add pagemap/pageout test for uffd-wp The last test introduced can also verify some of the fixes in previous patches, as the test will fail without the fixes. However it's not easy to verify all the changes in patch 2-4, but hopefully they can still be properly reviewed. Note that if considering the ongoing uffd-wp shmem & hugetlbfs work, patch 5 will be incomplete as it's missing e.g. hugetlbfs part or the special swap pte detection. However that's not needed in this series, and since that series is still during review, this series does not depend on that one (the last test only runs with anonymous memory, not file-backed). So this series can be merged even before that series. This patch (of 6): Huge zero page is handled in a special path in copy_huge_pmd(), however it should share most codes with a normal thp page. Trying to share more code with it by removing the special path. The only leftover so far is the huge zero page refcounting (mm_get_huge_zero_page()), because that's separately done with a global counter. This prepares for a future patch to modify the huge pmd to be installed, so that we don't need to duplicate it explicitly into huge zero page case too. Link: https://lkml.kernel.org/r/20210428225030.9708-1-peterx@redhat.com Link: https://lkml.kernel.org/r/20210428225030.9708-2-peterx@redhat.com Signed-off-by: Peter Xu Cc: Kirill A. Shutemov Cc: Mike Kravetz , peterx@redhat.com Cc: Mike Rapoport Cc: Axel Rasmussen Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Jerome Glisse Cc: Alexander Viro Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Joe Perches Cc: Lokesh Gidra Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- mm/huge_memory.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) --- a/mm/huge_memory.c~mm-thp-simplify-copying-of-huge-zero-page-pmd-when-fork +++ a/mm/huge_memory.c @@ -1088,17 +1088,13 @@ int copy_huge_pmd(struct mm_struct *dst_ * a page table. */ if (is_huge_zero_pmd(pmd)) { - struct page *zero_page; /* * get_huge_zero_page() will never allocate a new page here, * since we already have a zero page to copy. It just takes a * reference. */ - zero_page = mm_get_huge_zero_page(dst_mm); - set_huge_zero_page(pgtable, dst_mm, vma, addr, dst_pmd, - zero_page); - ret = 0; - goto out_unlock; + mm_get_huge_zero_page(dst_mm); + goto out_zero_page; } src_page = pmd_page(pmd); @@ -1122,6 +1118,7 @@ int copy_huge_pmd(struct mm_struct *dst_ get_page(src_page); page_dup_rmap(src_page, true); add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); +out_zero_page: mm_inc_nr_ptes(dst_mm); pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); From patchwork Thu Jul 1 01:49:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353131 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C3A6C11F68 for ; Thu, 1 Jul 2021 01:49:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CB35761466 for ; Thu, 1 Jul 2021 01:49:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB35761466 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 531748D01F7; Wed, 30 Jun 2021 21:49:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 508A68D01D0; Wed, 30 Jun 2021 21:49:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D0E78D01F7; Wed, 30 Jun 2021 21:49:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0136.hostedemail.com [216.40.44.136]) by kanga.kvack.org (Postfix) with ESMTP id 15CC78D01D0 for ; Wed, 30 Jun 2021 21:49:05 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DD09C252B3 for ; Thu, 1 Jul 2021 01:49:04 +0000 (UTC) X-FDA: 78312335808.34.27C917D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 82CF130000A4 for ; Thu, 1 Jul 2021 01:49:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 46BD461090; Thu, 1 Jul 2021 01:49:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104143; bh=wwDsjJobF16k+dzZRIgPHnLBhryvHR1rHOeXHDZpOpU=; h=Date:From:To:Subject:In-Reply-To:From; b=eHqFjBAO1M5px+St3X7TFE/uZyFLZKKAgbbn+99/pJtFspL+e/63bLF0+Ks+GRQ+8 4maMBBZmAI6F8rRNMSCpNctzNPM4+L8532CEhR+ZfTn005pLlU9Q2hfECXxTvQNeGZ JAzQwYeATHKopJIugqqv6TUHyUEMDWR2Q6ab+UTE= Date: Wed, 30 Jun 2021 18:49:02 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 036/192] mm/userfaultfd: fix uffd-wp special cases for fork() Message-ID: <20210701014902.262nd0-z4%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 82CF130000A4 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=eHqFjBAO; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 9hhgdfgio5o3fcmz7snj47jpgs6se67n X-HE-Tag: 1625104144-768294 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: mm/userfaultfd: fix uffd-wp special cases for fork() We tried to do something similar in b569a1760782 ("userfaultfd: wp: drop _PAGE_UFFD_WP properly when fork") previously, but it's not doing it all right.. A few fixes around the code path: 1. We were referencing VM_UFFD_WP vm_flags on the _old_ vma rather than the new vma. That's overlooked in b569a1760782, so it won't work as expected. Thanks to the recent rework on fork code (7a4830c380f3a8b3), we can easily get the new vma now, so switch the checks to that. 2. Dropping the uffd-wp bit in copy_huge_pmd() could be wrong if the huge pmd is a migration huge pmd. When it happens, instead of using pmd_uffd_wp(), we should use pmd_swp_uffd_wp(). The fix is simply to handle them separately. 3. Forget to carry over uffd-wp bit for a write migration huge pmd entry. This also happens in copy_huge_pmd(), where we converted a write huge migration entry into a read one. 4. In copy_nonpresent_pte(), drop uffd-wp if necessary for swap ptes. 5. In copy_present_page() when COW is enforced when fork(), we also need to pass over the uffd-wp bit if VM_UFFD_WP is armed on the new vma, and when the pte to be copied has uffd-wp bit set. Remove the comment in copy_present_pte() about this. It won't help a huge lot to only comment there, but comment everywhere would be an overkill. Let's assume the commit messages would help. [peterx@redhat.com: fix a few thp pmd missing uffd-wp bit] Link: https://lkml.kernel.org/r/20210428225030.9708-4-peterx@redhat.com Link: https://lkml.kernel.org/r/20210428225030.9708-3-peterx@redhat.com Fixes: b569a1760782f ("userfaultfd: wp: drop _PAGE_UFFD_WP properly when fork") Signed-off-by: Peter Xu Cc: Jerome Glisse Cc: Mike Rapoport Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Axel Rasmussen Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- include/linux/huge_mm.h | 2 +- include/linux/swapops.h | 2 ++ mm/huge_memory.c | 27 ++++++++++++++------------- mm/memory.c | 25 +++++++++++++------------ 4 files changed, 30 insertions(+), 26 deletions(-) --- a/include/linux/huge_mm.h~mm-userfaultfd-fix-uffd-wp-special-cases-for-fork +++ a/include/linux/huge_mm.h @@ -10,7 +10,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf); int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, - struct vm_area_struct *vma); + struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma); void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd); int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, pud_t *dst_pud, pud_t *src_pud, unsigned long addr, --- a/include/linux/swapops.h~mm-userfaultfd-fix-uffd-wp-special-cases-for-fork +++ a/include/linux/swapops.h @@ -265,6 +265,8 @@ static inline swp_entry_t pmd_to_swp_ent if (pmd_swp_soft_dirty(pmd)) pmd = pmd_swp_clear_soft_dirty(pmd); + if (pmd_swp_uffd_wp(pmd)) + pmd = pmd_swp_clear_uffd_wp(pmd); arch_entry = __pmd_to_swp_entry(pmd); return swp_entry(__swp_type(arch_entry), __swp_offset(arch_entry)); } --- a/mm/huge_memory.c~mm-userfaultfd-fix-uffd-wp-special-cases-for-fork +++ a/mm/huge_memory.c @@ -1026,7 +1026,7 @@ struct page *follow_devmap_pmd(struct vm int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, - struct vm_area_struct *vma) + struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma) { spinlock_t *dst_ptl, *src_ptl; struct page *src_page; @@ -1035,7 +1035,7 @@ int copy_huge_pmd(struct mm_struct *dst_ int ret = -ENOMEM; /* Skip if can be re-fill on fault */ - if (!vma_is_anonymous(vma)) + if (!vma_is_anonymous(dst_vma)) return 0; pgtable = pte_alloc_one(dst_mm); @@ -1049,14 +1049,6 @@ int copy_huge_pmd(struct mm_struct *dst_ ret = -EAGAIN; pmd = *src_pmd; - /* - * Make sure the _PAGE_UFFD_WP bit is cleared if the new VMA - * does not have the VM_UFFD_WP, which means that the uffd - * fork event is not enabled. - */ - if (!(vma->vm_flags & VM_UFFD_WP)) - pmd = pmd_clear_uffd_wp(pmd); - #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION if (unlikely(is_swap_pmd(pmd))) { swp_entry_t entry = pmd_to_swp_entry(pmd); @@ -1067,11 +1059,15 @@ int copy_huge_pmd(struct mm_struct *dst_ pmd = swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*src_pmd)) pmd = pmd_swp_mksoft_dirty(pmd); + if (pmd_swp_uffd_wp(*src_pmd)) + pmd = pmd_swp_mkuffd_wp(pmd); set_pmd_at(src_mm, addr, src_pmd, pmd); } add_mm_counter(dst_mm, MM_ANONPAGES, HPAGE_PMD_NR); mm_inc_nr_ptes(dst_mm); pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); + if (!userfaultfd_wp(dst_vma)) + pmd = pmd_swp_clear_uffd_wp(pmd); set_pmd_at(dst_mm, addr, dst_pmd, pmd); ret = 0; goto out_unlock; @@ -1107,11 +1103,11 @@ int copy_huge_pmd(struct mm_struct *dst_ * best effort that the pinned pages won't be replaced by another * random page during the coming copy-on-write. */ - if (unlikely(page_needs_cow_for_dma(vma, src_page))) { + if (unlikely(page_needs_cow_for_dma(src_vma, src_page))) { pte_free(dst_mm, pgtable); spin_unlock(src_ptl); spin_unlock(dst_ptl); - __split_huge_pmd(vma, src_pmd, addr, false, NULL); + __split_huge_pmd(src_vma, src_pmd, addr, false, NULL); return -EAGAIN; } @@ -1121,8 +1117,9 @@ int copy_huge_pmd(struct mm_struct *dst_ out_zero_page: mm_inc_nr_ptes(dst_mm); pgtable_trans_huge_deposit(dst_mm, dst_pmd, pgtable); - pmdp_set_wrprotect(src_mm, addr, src_pmd); + if (!userfaultfd_wp(dst_vma)) + pmd = pmd_clear_uffd_wp(pmd); pmd = pmd_mkold(pmd_wrprotect(pmd)); set_pmd_at(dst_mm, addr, dst_pmd, pmd); @@ -1835,6 +1832,8 @@ int change_huge_pmd(struct vm_area_struc newpmd = swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*pmd)) newpmd = pmd_swp_mksoft_dirty(newpmd); + if (pmd_swp_uffd_wp(*pmd)) + newpmd = pmd_swp_mkuffd_wp(newpmd); set_pmd_at(mm, addr, pmd, newpmd); } goto unlock; @@ -3245,6 +3244,8 @@ void remove_migration_pmd(struct page_vm pmde = pmd_mksoft_dirty(pmde); if (is_write_migration_entry(entry)) pmde = maybe_pmd_mkwrite(pmde, vma); + if (pmd_swp_uffd_wp(*pvmw->pmd)) + pmde = pmd_wrprotect(pmd_mkuffd_wp(pmde)); flush_cache_range(vma, mmun_start, mmun_start + HPAGE_PMD_SIZE); if (PageAnon(new)) --- a/mm/memory.c~mm-userfaultfd-fix-uffd-wp-special-cases-for-fork +++ a/mm/memory.c @@ -707,10 +707,10 @@ out: static unsigned long copy_nonpresent_pte(struct mm_struct *dst_mm, struct mm_struct *src_mm, - pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *vma, - unsigned long addr, int *rss) + pte_t *dst_pte, pte_t *src_pte, struct vm_area_struct *dst_vma, + struct vm_area_struct *src_vma, unsigned long addr, int *rss) { - unsigned long vm_flags = vma->vm_flags; + unsigned long vm_flags = dst_vma->vm_flags; pte_t pte = *src_pte; struct page *page; swp_entry_t entry = pte_to_swp_entry(pte); @@ -779,6 +779,8 @@ copy_nonpresent_pte(struct mm_struct *ds set_pte_at(src_mm, addr, src_pte, pte); } } + if (!userfaultfd_wp(dst_vma)) + pte = pte_swp_clear_uffd_wp(pte); set_pte_at(dst_mm, addr, dst_pte, pte); return 0; } @@ -844,6 +846,9 @@ copy_present_page(struct vm_area_struct /* All done, just insert the new page copy in the child */ pte = mk_pte(new_page, dst_vma->vm_page_prot); pte = maybe_mkwrite(pte_mkdirty(pte), dst_vma); + if (userfaultfd_pte_wp(dst_vma, *src_pte)) + /* Uffd-wp needs to be delivered to dest pte as well */ + pte = pte_wrprotect(pte_mkuffd_wp(pte)); set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); return 0; } @@ -893,12 +898,7 @@ copy_present_pte(struct vm_area_struct * pte = pte_mkclean(pte); pte = pte_mkold(pte); - /* - * Make sure the _PAGE_UFFD_WP bit is cleared if the new VMA - * does not have the VM_UFFD_WP, which means that the uffd - * fork event is not enabled. - */ - if (!(vm_flags & VM_UFFD_WP)) + if (!userfaultfd_wp(dst_vma)) pte = pte_clear_uffd_wp(pte); set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); @@ -973,7 +973,8 @@ again: if (unlikely(!pte_present(*src_pte))) { entry.val = copy_nonpresent_pte(dst_mm, src_mm, dst_pte, src_pte, - src_vma, addr, rss); + dst_vma, src_vma, + addr, rss); if (entry.val) break; progress += 8; @@ -1050,8 +1051,8 @@ copy_pmd_range(struct vm_area_struct *ds || pmd_devmap(*src_pmd)) { int err; VM_BUG_ON_VMA(next-addr != HPAGE_PMD_SIZE, src_vma); - err = copy_huge_pmd(dst_mm, src_mm, - dst_pmd, src_pmd, addr, src_vma); + err = copy_huge_pmd(dst_mm, src_mm, dst_pmd, src_pmd, + addr, dst_vma, src_vma); if (err == -ENOMEM) return -ENOMEM; if (!err) From patchwork Thu Jul 1 01:49:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A84EAC11F69 for ; Thu, 1 Jul 2021 01:49:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 61C9D6120C for ; Thu, 1 Jul 2021 01:49:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 61C9D6120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DBCCB8D01F8; Wed, 30 Jun 2021 21:49:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D93E68D01D0; Wed, 30 Jun 2021 21:49:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C350F8D01F8; Wed, 30 Jun 2021 21:49:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0092.hostedemail.com [216.40.44.92]) by kanga.kvack.org (Postfix) with ESMTP id 9F8E48D01D0 for ; Wed, 30 Jun 2021 21:49:08 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7728181C21F8 for ; Thu, 1 Jul 2021 01:49:08 +0000 (UTC) X-FDA: 78312335976.01.0BD4098 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 35C917001A08 for ; Thu, 1 Jul 2021 01:49:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C0A4E61469; Thu, 1 Jul 2021 01:49:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104147; bh=zdOaIlOLdN22SKbBP9VrmQ4d6AMoLYE4qirfKewrXKc=; h=Date:From:To:Subject:In-Reply-To:From; b=Heowlqhpeb8YspTqqkVik2xtlG3jAr1W1DMbr2FeCAkITy3ypMv7+PAtBDPYs7OYQ b6wk2DifsuSMGhWvct+5uRPYwd0p1QB1ZsdY/pEiGZy1WKkjB6wCW8j6ipcfNvPJuk tonQMa/TmkQkdKYPoVXzQLX8i/RGCkkcfvPe7GBY= Date: Wed, 30 Jun 2021 18:49:06 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 037/192] mm/userfaultfd: fail uffd-wp registration if not supported Message-ID: <20210701014906.8qOxw6Ozk%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 35C917001A08 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Heowlqhp; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: pytc9ir4zmppo9ze97pk7u1kk6psat5j X-HE-Tag: 1625104148-856635 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: mm/userfaultfd: fail uffd-wp registration if not supported We should fail uffd-wp registration immediately if the arch does not even have CONFIG_HAVE_ARCH_USERFAULTFD_WP defined. That'll block also relevant ioctls on e.g. UFFDIO_WRITEPROTECT because that'll check against VM_UFFD_WP, which can only be applied with a success registration. Remove the WP feature bit too for those archs when handling UFFDIO_API ioctl. Link: https://lkml.kernel.org/r/20210428225030.9708-5-peterx@redhat.com Signed-off-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Axel Rasmussen Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- fs/userfaultfd.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) --- a/fs/userfaultfd.c~mm-userfaultfd-fail-uffd-wp-registeration-if-not-supported +++ a/fs/userfaultfd.c @@ -1304,8 +1304,12 @@ static int userfaultfd_register(struct u vm_flags = 0; if (uffdio_register.mode & UFFDIO_REGISTER_MODE_MISSING) vm_flags |= VM_UFFD_MISSING; - if (uffdio_register.mode & UFFDIO_REGISTER_MODE_WP) + if (uffdio_register.mode & UFFDIO_REGISTER_MODE_WP) { +#ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP + goto out; +#endif vm_flags |= VM_UFFD_WP; + } if (uffdio_register.mode & UFFDIO_REGISTER_MODE_MINOR) { #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR goto out; @@ -1943,6 +1947,9 @@ static int userfaultfd_api(struct userfa #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR uffdio_api.features &= ~UFFD_FEATURE_MINOR_HUGETLBFS; #endif +#ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP + uffdio_api.features &= ~UFFD_FEATURE_PAGEFAULT_FLAG_WP; +#endif uffdio_api.ioctls = UFFD_API_IOCTLS; ret = -EFAULT; if (copy_to_user(buf, &uffdio_api, sizeof(uffdio_api))) From patchwork Thu Jul 1 01:49:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5675CC11F66 for ; Thu, 1 Jul 2021 01:49:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0558D61468 for ; Thu, 1 Jul 2021 01:49:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0558D61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 817C18D01F9; Wed, 30 Jun 2021 21:49:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7EE3A8D01D0; Wed, 30 Jun 2021 21:49:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6B6258D01F9; Wed, 30 Jun 2021 21:49:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 44E998D01D0 for ; Wed, 30 Jun 2021 21:49:12 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1D22018012CF9 for ; Thu, 1 Jul 2021 01:49:12 +0000 (UTC) X-FDA: 78312336144.10.D88203D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id B043C900009D for ; Thu, 1 Jul 2021 01:49:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 43AA06120C; Thu, 1 Jul 2021 01:49:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104151; bh=aIaGo794mMGbuHL6tP7rcsrhT06V8WvV9A6oUDVluKg=; h=Date:From:To:Subject:In-Reply-To:From; b=RUtEl2hme+VDBY3VdflPgZTQYJX9lVbbF0eJOf3nQOda/m+1FwRP2Gfv46Zpqa9w+ oStBEU7QZEf5jkaWssg1EhN8J/K56GhbIaZhW7bqMupTVUvcZzEO164WIg23T9SjBk CLwFcLga156VPA4A6pqxxsNNczIjg0pgZmZJhBTQ= Date: Wed, 30 Jun 2021 18:49:09 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 038/192] mm/pagemap: export uffd-wp protection information Message-ID: <20210701014909.GbTe4JDyQ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=RUtEl2hm; dmarc=none; spf=pass (imf23.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: mymf9gki14aqktohsor6444ffdakcuj9 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B043C900009D X-HE-Tag: 1625104151-573774 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: mm/pagemap: export uffd-wp protection information Export the PTE/PMD status of uffd-wp to pagemap too. Link: https://lkml.kernel.org/r/20210428225030.9708-6-peterx@redhat.com Signed-off-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Axel Rasmussen Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- Documentation/admin-guide/mm/pagemap.rst | 2 ++ fs/proc/task_mmu.c | 9 +++++++++ 2 files changed, 11 insertions(+) --- a/Documentation/admin-guide/mm/pagemap.rst~mm-pagemap-export-uffd-wp-protection-information +++ a/Documentation/admin-guide/mm/pagemap.rst @@ -21,6 +21,8 @@ There are four components to pagemap: * Bit 55 pte is soft-dirty (see :ref:`Documentation/admin-guide/mm/soft-dirty.rst `) * Bit 56 page exclusively mapped (since 4.2) + * Bit 57 pte is uffd-wp write-protected (since 5.13) (see + :ref:`Documentation/admin-guide/mm/userfaultfd.rst `) * Bits 57-60 zero * Bit 61 page is file-page or shared-anon (since 3.5) * Bit 62 page swapped --- a/fs/proc/task_mmu.c~mm-pagemap-export-uffd-wp-protection-information +++ a/fs/proc/task_mmu.c @@ -1302,6 +1302,7 @@ struct pagemapread { #define PM_PFRAME_MASK GENMASK_ULL(PM_PFRAME_BITS - 1, 0) #define PM_SOFT_DIRTY BIT_ULL(55) #define PM_MMAP_EXCLUSIVE BIT_ULL(56) +#define PM_UFFD_WP BIT_ULL(57) #define PM_FILE BIT_ULL(61) #define PM_SWAP BIT_ULL(62) #define PM_PRESENT BIT_ULL(63) @@ -1375,10 +1376,14 @@ static pagemap_entry_t pte_to_pagemap_en page = vm_normal_page(vma, addr, pte); if (pte_soft_dirty(pte)) flags |= PM_SOFT_DIRTY; + if (pte_uffd_wp(pte)) + flags |= PM_UFFD_WP; } else if (is_swap_pte(pte)) { swp_entry_t entry; if (pte_swp_soft_dirty(pte)) flags |= PM_SOFT_DIRTY; + if (pte_swp_uffd_wp(pte)) + flags |= PM_UFFD_WP; entry = pte_to_swp_entry(pte); if (pm->show_pfn) frame = swp_type(entry) | @@ -1426,6 +1431,8 @@ static int pagemap_pmd_range(pmd_t *pmdp flags |= PM_PRESENT; if (pmd_soft_dirty(pmd)) flags |= PM_SOFT_DIRTY; + if (pmd_uffd_wp(pmd)) + flags |= PM_UFFD_WP; if (pm->show_pfn) frame = pmd_pfn(pmd) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); @@ -1444,6 +1451,8 @@ static int pagemap_pmd_range(pmd_t *pmdp flags |= PM_SWAP; if (pmd_swp_soft_dirty(pmd)) flags |= PM_SOFT_DIRTY; + if (pmd_swp_uffd_wp(pmd)) + flags |= PM_UFFD_WP; VM_BUG_ON(!is_pmd_migration_entry(pmd)); page = migration_entry_to_page(entry); } From patchwork Thu Jul 1 01:49:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB327C11F68 for ; Thu, 1 Jul 2021 01:49:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9DA986120C for ; Thu, 1 Jul 2021 01:49:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DA986120C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 24B2E8D01FA; Wed, 30 Jun 2021 21:49:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 223CF8D01D0; Wed, 30 Jun 2021 21:49:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0748C8D01FA; Wed, 30 Jun 2021 21:49:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0024.hostedemail.com [216.40.44.24]) by kanga.kvack.org (Postfix) with ESMTP id D86EA8D01D0 for ; Wed, 30 Jun 2021 21:49:15 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id AB115252D7 for ; Thu, 1 Jul 2021 01:49:15 +0000 (UTC) X-FDA: 78312336270.09.BD8A761 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 52055F0000AC for ; Thu, 1 Jul 2021 01:49:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E507C61466; Thu, 1 Jul 2021 01:49:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104154; bh=5XBSRVRXfFJDd5DpJNwandjUuEDDnsRSeJOE67UUtaA=; h=Date:From:To:Subject:In-Reply-To:From; b=WOv8Pddnuo5DfQf7Y741usywRwZmpD8f3PT0eqEd5XmlAwxcHctvDxwUghM5jW+Tg UIW7ahfLGueCIu/lcGqv0OcVTGuRmz7/uKHAI6bA3FbcnuLS+3yy3gqhwRTyXICiq6 h7fXzDlsX5ZOOKwgy1axiYnJlv500QQq4MBStItM= Date: Wed, 30 Jun 2021 18:49:13 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 039/192] userfaultfd/selftests: add pagemap uffd-wp test Message-ID: <20210701014913.l05nEifZW%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 52055F0000AC X-Stat-Signature: shamuyszxnq5n7tat9dq6pa41ugoofxf Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=WOv8Pddn; dmarc=none; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104155-668479 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Peter Xu Subject: userfaultfd/selftests: add pagemap uffd-wp test Add one anonymous specific test to start using pagemap. With pagemap support, we can directly read the uffd-wp bit from pgtable without triggering any fault, so it's easier to do sanity checks in unit tests. Meanwhile this test also leverages the newly introduced MADV_PAGEOUT madvise function to test swap ptes with uffd-wp bit set, and across fork()s. Link: https://lkml.kernel.org/r/20210428225030.9708-7-peterx@redhat.com Signed-off-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Axel Rasmussen Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 154 +++++++++++++++++++++ 1 file changed, 154 insertions(+) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-add-pagemap-uffd-wp-test +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -1170,6 +1170,144 @@ static int userfaultfd_minor_test(void) return stats.missing_faults != 0 || stats.minor_faults != nr_pages; } +#define BIT_ULL(nr) (1ULL << (nr)) +#define PM_SOFT_DIRTY BIT_ULL(55) +#define PM_MMAP_EXCLUSIVE BIT_ULL(56) +#define PM_UFFD_WP BIT_ULL(57) +#define PM_FILE BIT_ULL(61) +#define PM_SWAP BIT_ULL(62) +#define PM_PRESENT BIT_ULL(63) + +static int pagemap_open(void) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + + if (fd < 0) + err("open pagemap"); + + return fd; +} + +static uint64_t pagemap_read_vaddr(int fd, void *vaddr) +{ + uint64_t value; + int ret; + + ret = pread(fd, &value, sizeof(uint64_t), + ((uint64_t)vaddr >> 12) * sizeof(uint64_t)); + if (ret != sizeof(uint64_t)) + err("pread() on pagemap failed"); + + return value; +} + +/* This macro let __LINE__ works in err() */ +#define pagemap_check_wp(value, wp) do { \ + if (!!(value & PM_UFFD_WP) != wp) \ + err("pagemap uffd-wp bit error: 0x%"PRIx64, value); \ + } while (0) + +static int pagemap_test_fork(bool present) +{ + pid_t child = fork(); + uint64_t value; + int fd, result; + + if (!child) { + /* Open the pagemap fd of the child itself */ + fd = pagemap_open(); + value = pagemap_read_vaddr(fd, area_dst); + /* + * After fork() uffd-wp bit should be gone as long as we're + * without UFFD_FEATURE_EVENT_FORK + */ + pagemap_check_wp(value, false); + /* Succeed */ + exit(0); + } + waitpid(child, &result, 0); + return result; +} + +static void userfaultfd_pagemap_test(unsigned int test_pgsize) +{ + struct uffdio_register uffdio_register; + int pagemap_fd; + uint64_t value; + + /* Pagemap tests uffd-wp only */ + if (!test_uffdio_wp) + return; + + /* Not enough memory to test this page size */ + if (test_pgsize > nr_pages * page_size) + return; + + printf("testing uffd-wp with pagemap (pgsize=%u): ", test_pgsize); + /* Flush so it doesn't flush twice in parent/child later */ + fflush(stdout); + + uffd_test_ops->release_pages(area_dst); + + if (test_pgsize > page_size) { + /* This is a thp test */ + if (madvise(area_dst, nr_pages * page_size, MADV_HUGEPAGE)) + err("madvise(MADV_HUGEPAGE) failed"); + } else if (test_pgsize == page_size) { + /* This is normal page test; force no thp */ + if (madvise(area_dst, nr_pages * page_size, MADV_NOHUGEPAGE)) + err("madvise(MADV_NOHUGEPAGE) failed"); + } + + if (userfaultfd_open(0)) + err("userfaultfd_open"); + + uffdio_register.range.start = (unsigned long) area_dst; + uffdio_register.range.len = nr_pages * page_size; + uffdio_register.mode = UFFDIO_REGISTER_MODE_WP; + if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register)) + err("register failed"); + + pagemap_fd = pagemap_open(); + + /* Touch the page */ + *area_dst = 1; + wp_range(uffd, (uint64_t)area_dst, test_pgsize, true); + value = pagemap_read_vaddr(pagemap_fd, area_dst); + pagemap_check_wp(value, true); + /* Make sure uffd-wp bit dropped when fork */ + if (pagemap_test_fork(true)) + err("Detected stall uffd-wp bit in child"); + + /* Exclusive required or PAGEOUT won't work */ + if (!(value & PM_MMAP_EXCLUSIVE)) + err("multiple mapping detected: 0x%"PRIx64, value); + + if (madvise(area_dst, test_pgsize, MADV_PAGEOUT)) + err("madvise(MADV_PAGEOUT) failed"); + + /* Uffd-wp should persist even swapped out */ + value = pagemap_read_vaddr(pagemap_fd, area_dst); + pagemap_check_wp(value, true); + /* Make sure uffd-wp bit dropped when fork */ + if (pagemap_test_fork(false)) + err("Detected stall uffd-wp bit in child"); + + /* Unprotect; this tests swap pte modifications */ + wp_range(uffd, (uint64_t)area_dst, page_size, false); + value = pagemap_read_vaddr(pagemap_fd, area_dst); + pagemap_check_wp(value, false); + + /* Fault in the page from disk */ + *area_dst = 2; + value = pagemap_read_vaddr(pagemap_fd, area_dst); + pagemap_check_wp(value, false); + + close(pagemap_fd); + close(uffd); + printf("done\n"); +} + static int userfaultfd_stress(void) { void *area; @@ -1341,6 +1479,22 @@ static int userfaultfd_stress(void) } close(uffd); + + if (test_type == TEST_ANON) { + /* + * shmem/hugetlb won't be able to run since they have different + * behavior on fork() (file-backed memory normally drops ptes + * directly when fork), meanwhile the pagemap test will verify + * pgtable entry of fork()ed child. + */ + userfaultfd_pagemap_test(page_size); + /* + * Hard-code for x86_64 for now for 2M THP, as x86_64 is + * currently the only one that supports uffd-wp + */ + userfaultfd_pagemap_test(page_size * 512); + } + return userfaultfd_zeropage_test() || userfaultfd_sig_test() || userfaultfd_events_test() || userfaultfd_minor_test(); } From patchwork Thu Jul 1 01:49:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91F02C11F69 for ; Thu, 1 Jul 2021 01:49:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3344D61469 for ; Thu, 1 Jul 2021 01:49:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3344D61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A3AFC8D01FB; Wed, 30 Jun 2021 21:49:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A11688D01D0; Wed, 30 Jun 2021 21:49:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8D8C18D01FB; Wed, 30 Jun 2021 21:49:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 65C918D01D0 for ; Wed, 30 Jun 2021 21:49:19 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 32D4D18024AFD for ; Thu, 1 Jul 2021 01:49:19 +0000 (UTC) X-FDA: 78312336438.16.6FE6A4F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id DBC677000081 for ; Thu, 1 Jul 2021 01:49:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8987261090; Thu, 1 Jul 2021 01:49:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104158; bh=DT/+NfgrdZQOYp4/8qPtS9HhRkjG+rqHo6nhOV5Ir2w=; h=Date:From:To:Subject:In-Reply-To:From; b=N2FV295DAJsUOKRLcLBjHMAaxBifxfthi49Sx7zK3aZKMiJmimCKA0gJ5d293v7RB /Pdz9eW+LtmjaeMz3jd83XI8EXg/QFz9RLiNzQe71U0sX6lnibYFYwW/M6d5jIl2uT 57M9kHIkFwZdjU3+0+uPTKHTHmeB4wov0WEZWIBw= Date: Wed, 30 Jun 2021 18:49:17 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 040/192] userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte Message-ID: <20210701014917.RD2G3TGCN%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=N2FV295D; dmarc=none; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: ziwh1if9yqe4mjy1bdp86kc5padym1pt X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: DBC677000081 X-HE-Tag: 1625104158-317448 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/shmem: combine shmem_{mcopy_atomic,mfill_zeropage}_pte Patch series "userfaultfd: add minor fault handling for shmem", v6. Overview ======== See the series which added minor faults for hugetlbfs [3] for a detailed overview of minor fault handling in general. This series adds the same support for shmem-backed areas. This series is structured as follows: - Commits 1 and 2 are cleanups. - Commits 3 and 4 implement the new feature (minor fault handling for shmem). - Commit 5 advertises that the feature is now available since at this point it's fully implemented. - Commit 6 is a final cleanup, modifying an existing code path to re-use a new helper we've introduced. - Commits 7, 8, 9, 10 update the userfaultfd selftest to exercise the feature. Use Case ======== In some cases it is useful to have VM memory backed by tmpfs instead of hugetlbfs. So, this feature will be used to support the same VM live migration use case described in my original series. Additionally, Android folks (Lokesh Gidra ) hope to optimize the Android Runtime garbage collector using this feature: "The plan is to use userfaultfd for concurrently compacting the heap. With this feature, the heap can be shared-mapped at another location where the GC-thread(s) could continue the compaction operation without the need to invoke userfault ioctl(UFFDIO_COPY) each time. OTOH, if and when Java threads get faults on the heap, UFFDIO_CONTINUE can be used to resume execution. Furthermore, this feature enables updating references in the 'non-moving' portion of the heap efficiently. Without this feature, uneccessary page copying (ioctl(UFFDIO_COPY)) would be required." [1] https://lore.kernel.org/patchwork/cover/1388144/ [2] https://lore.kernel.org/patchwork/patch/1408161/ [3] https://lore.kernel.org/linux-fsdevel/20210301222728.176417-1-axelrasmussen@google.com/T/#t This patch (of 9): Previously, we did a dance where we had one calling path in userfaultfd.c (mfill_atomic_pte), but then we split it into two in shmem_fs.h (shmem_{mcopy_atomic,mfill_zeropage}_pte), and then rejoined into a single shared function in shmem.c (shmem_mfill_atomic_pte). This is all a bit overly complex. Just call the single combined shmem function directly, allowing us to clean up various branches, boilerplate, etc. While we're touching this function, two other small cleanup changes: - offset is equivalent to pgoff, so we can get rid of offset entirely. - Split two VM_BUG_ON cases into two statements. This means the line number reported when the BUG is hit specifies exactly which condition was true. Link: https://lkml.kernel.org/r/20210503180737.2487560-1-axelrasmussen@google.com Link: https://lkml.kernel.org/r/20210503180737.2487560-3-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Peter Xu Acked-by: Hugh Dickins Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- include/linux/shmem_fs.h | 19 +++++-------- mm/shmem.c | 52 +++++++++++-------------------------- mm/userfaultfd.c | 10 ++----- 3 files changed, 27 insertions(+), 54 deletions(-) --- a/include/linux/shmem_fs.h~userfaultfd-shmem-combine-shmem_mcopy_atomicmfill_zeropage_pte +++ a/include/linux/shmem_fs.h @@ -122,21 +122,18 @@ static inline bool shmem_file(struct fil extern bool shmem_charge(struct inode *inode, long pages); extern void shmem_uncharge(struct inode *inode, long pages); +#ifdef CONFIG_USERFAULTFD #ifdef CONFIG_SHMEM -extern int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, +extern int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, unsigned long dst_addr, unsigned long src_addr, + bool zeropage, struct page **pagep); -extern int shmem_mfill_zeropage_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr); -#else -#define shmem_mcopy_atomic_pte(dst_mm, dst_pte, dst_vma, dst_addr, \ - src_addr, pagep) ({ BUG(); 0; }) -#define shmem_mfill_zeropage_pte(dst_mm, dst_pmd, dst_vma, \ - dst_addr) ({ BUG(); 0; }) -#endif +#else /* !CONFIG_SHMEM */ +#define shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, \ + src_addr, zeropage, pagep) ({ BUG(); 0; }) +#endif /* CONFIG_SHMEM */ +#endif /* CONFIG_USERFAULTFD */ #endif --- a/mm/shmem.c~userfaultfd-shmem-combine-shmem_mcopy_atomicmfill_zeropage_pte +++ a/mm/shmem.c @@ -2352,13 +2352,14 @@ static struct inode *shmem_get_inode(str return inode; } -static int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - bool zeropage, - struct page **pagep) +#ifdef CONFIG_USERFAULTFD +int shmem_mfill_atomic_pte(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + bool zeropage, + struct page **pagep) { struct inode *inode = file_inode(dst_vma->vm_file); struct shmem_inode_info *info = SHMEM_I(inode); @@ -2370,7 +2371,7 @@ static int shmem_mfill_atomic_pte(struct struct page *page; pte_t _dst_pte, *dst_pte; int ret; - pgoff_t offset, max_off; + pgoff_t max_off; ret = -ENOMEM; if (!shmem_inode_acct_block(inode, 1)) { @@ -2391,7 +2392,7 @@ static int shmem_mfill_atomic_pte(struct if (!page) goto out_unacct_blocks; - if (!zeropage) { /* mcopy_atomic */ + if (!zeropage) { /* COPY */ page_kaddr = kmap_atomic(page); ret = copy_from_user(page_kaddr, (const void __user *)src_addr, @@ -2405,7 +2406,7 @@ static int shmem_mfill_atomic_pte(struct /* don't free the page */ return -ENOENT; } - } else { /* mfill_zeropage_atomic */ + } else { /* ZEROPAGE */ clear_highpage(page); } } else { @@ -2413,15 +2414,15 @@ static int shmem_mfill_atomic_pte(struct *pagep = NULL; } - VM_BUG_ON(PageLocked(page) || PageSwapBacked(page)); + VM_BUG_ON(PageLocked(page)); + VM_BUG_ON(PageSwapBacked(page)); __SetPageLocked(page); __SetPageSwapBacked(page); __SetPageUptodate(page); ret = -EFAULT; - offset = linear_page_index(dst_vma, dst_addr); max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); - if (unlikely(offset >= max_off)) + if (unlikely(pgoff >= max_off)) goto out_release; ret = shmem_add_to_page_cache(page, mapping, pgoff, NULL, @@ -2447,7 +2448,7 @@ static int shmem_mfill_atomic_pte(struct ret = -EFAULT; max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); - if (unlikely(offset >= max_off)) + if (unlikely(pgoff >= max_off)) goto out_release_unlock; ret = -EEXIST; @@ -2484,28 +2485,7 @@ out_unacct_blocks: shmem_inode_unacct_blocks(inode, 1); goto out; } - -int shmem_mcopy_atomic_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - struct page **pagep) -{ - return shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, - dst_addr, src_addr, false, pagep); -} - -int shmem_mfill_zeropage_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr) -{ - struct page *page = NULL; - - return shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, - dst_addr, 0, true, &page); -} +#endif /* CONFIG_USERFAULTFD */ #ifdef CONFIG_TMPFS static const struct inode_operations shmem_symlink_inode_operations; --- a/mm/userfaultfd.c~userfaultfd-shmem-combine-shmem_mcopy_atomicmfill_zeropage_pte +++ a/mm/userfaultfd.c @@ -392,13 +392,9 @@ static __always_inline ssize_t mfill_ato dst_vma, dst_addr); } else { VM_WARN_ON_ONCE(wp_copy); - if (!zeropage) - err = shmem_mcopy_atomic_pte(dst_mm, dst_pmd, - dst_vma, dst_addr, - src_addr, page); - else - err = shmem_mfill_zeropage_pte(dst_mm, dst_pmd, - dst_vma, dst_addr); + err = shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, + dst_addr, src_addr, zeropage, + page); } return err; From patchwork Thu Jul 1 01:49:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0675EC11F6B for ; Thu, 1 Jul 2021 01:49:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B23DB61241 for ; Thu, 1 Jul 2021 01:49:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B23DB61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 383A48D01FC; Wed, 30 Jun 2021 21:49:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 35A498D01D0; Wed, 30 Jun 2021 21:49:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 221ED8D01FC; Wed, 30 Jun 2021 21:49:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id EE85E8D01D0 for ; Wed, 30 Jun 2021 21:49:22 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C4EE5181AEF3F for ; Thu, 1 Jul 2021 01:49:22 +0000 (UTC) X-FDA: 78312336564.26.C530795 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 7981ED000097 for ; Thu, 1 Jul 2021 01:49:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2BEB561469; Thu, 1 Jul 2021 01:49:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104161; bh=5xrnHgGUjf4XtwpufsBBiTLZuJ2GgK4jx5DkhGxwBt0=; h=Date:From:To:Subject:In-Reply-To:From; b=n40a4q3AMSr7kNo5+1xB7i889mu5RHYyzNZpq6LWMtJV5tcvfOQN+mbObp/kWpij5 k8mDA/CvD01Cj3oD67M5NSe1xA6ZTYvd6vMCGl6j/kbFirWZwm8C94/FTBsyrR6IP8 k9YRfHKDJscsgjyhyV83FFijsetzlfgcLw5rgj7w= Date: Wed, 30 Jun 2021 18:49:20 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 041/192] userfaultfd/shmem: support minor fault registration for shmem Message-ID: <20210701014920.bfUNqkkPY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=n40a4q3A; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 7981ED000097 X-Stat-Signature: sock6a993jrnsp5yu3epbh1c13p7s99u X-HE-Tag: 1625104162-403041 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/shmem: support minor fault registration for shmem This patch allows shmem-backed VMAs to be registered for minor faults. Minor faults are appropriately relayed to userspace in the fault path, for VMAs with the relevant flag. This commit doesn't hook up the UFFDIO_CONTINUE ioctl for shmem-backed minor faults, though, so userspace doesn't yet have a way to resolve such faults. Because of this, we also don't yet advertise this as a supported feature. That will be done in a separate commit when the feature is fully implemented. Link: https://lkml.kernel.org/r/20210503180737.2487560-4-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Acked-by: Peter Xu Acked-by: Hugh Dickins Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- fs/userfaultfd.c | 3 +-- mm/memory.c | 8 +++++--- mm/shmem.c | 12 +++++++++++- 3 files changed, 17 insertions(+), 6 deletions(-) --- a/fs/userfaultfd.c~userfaultfd-shmem-support-minor-fault-registration-for-shmem +++ a/fs/userfaultfd.c @@ -1267,8 +1267,7 @@ static inline bool vma_can_userfault(str } if (vm_flags & VM_UFFD_MINOR) { - /* FIXME: Add minor fault interception for shmem. */ - if (!is_vm_hugetlb_page(vma)) + if (!(is_vm_hugetlb_page(vma) || vma_is_shmem(vma))) return false; } --- a/mm/memory.c~userfaultfd-shmem-support-minor-fault-registration-for-shmem +++ a/mm/memory.c @@ -4026,9 +4026,11 @@ static vm_fault_t do_read_fault(struct v * something). */ if (vma->vm_ops->map_pages && fault_around_bytes >> PAGE_SHIFT > 1) { - ret = do_fault_around(vmf); - if (ret) - return ret; + if (likely(!userfaultfd_minor(vmf->vma))) { + ret = do_fault_around(vmf); + if (ret) + return ret; + } } ret = __do_fault(vmf); --- a/mm/shmem.c~userfaultfd-shmem-support-minor-fault-registration-for-shmem +++ a/mm/shmem.c @@ -1797,7 +1797,7 @@ unlock: * vm. If we swap it in we mark it dirty since we also free the swap * entry since a page cannot live in both the swap and page cache. * - * vmf and fault_type are only supplied by shmem_fault: + * vma, vmf, and fault_type are only supplied by shmem_fault: * otherwise they are NULL. */ static int shmem_getpage_gfp(struct inode *inode, pgoff_t index, @@ -1832,6 +1832,16 @@ repeat: page = pagecache_get_page(mapping, index, FGP_ENTRY | FGP_HEAD | FGP_LOCK, 0); + + if (page && vma && userfaultfd_minor(vma)) { + if (!xa_is_value(page)) { + unlock_page(page); + put_page(page); + } + *fault_type = handle_userfault(vmf, VM_UFFD_MINOR); + return 0; + } + if (xa_is_value(page)) { error = shmem_swapin_page(inode, index, &page, sgp, gfp, vma, fault_type); From patchwork Thu Jul 1 01:49:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD3C5C11F6A for ; Thu, 1 Jul 2021 01:49:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5F7D961260 for ; Thu, 1 Jul 2021 01:49:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5F7D961260 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DBD0F8D01FD; Wed, 30 Jun 2021 21:49:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D932A8D01D0; Wed, 30 Jun 2021 21:49:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5B0F8D01FD; Wed, 30 Jun 2021 21:49:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0235.hostedemail.com [216.40.44.235]) by kanga.kvack.org (Postfix) with ESMTP id 9E75D8D01D0 for ; Wed, 30 Jun 2021 21:49:26 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7382D181AEF3F for ; Thu, 1 Jul 2021 01:49:26 +0000 (UTC) X-FDA: 78312336732.07.36337A1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 17B877000081 for ; Thu, 1 Jul 2021 01:49:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BC31E6135C; Thu, 1 Jul 2021 01:49:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104165; bh=U9dX1TEc6LZqxnWJopjHn++zZFHkt57eTj96WPCXfGM=; h=Date:From:To:Subject:In-Reply-To:From; b=aNKPzJbuCGwN6cWQtLk5CeuJ/7W7bwAizp0NDwU7ELzNRGmutBhjGlPZxbkIQciLT 3xbCiEEu8uY/1dEI4DhqvQmgt5GDxnsx17c6HSDFI82hBCf6L7e9MTBgDljWL/7DqZ aUa8ttAqcJZsQSuQUS4f+D/jXRA3A6KZH+qT5tGs= Date: Wed, 30 Jun 2021 18:49:24 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 042/192] userfaultfd/shmem: support UFFDIO_CONTINUE for shmem Message-ID: <20210701014924.hFrtt40Cb%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=aNKPzJbu; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: zdnbocrazjcddaaqnfiksjizr6txc3hq X-Rspamd-Queue-Id: 17B877000081 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104165-477232 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/shmem: support UFFDIO_CONTINUE for shmem With this change, userspace can resolve a minor fault within a shmem-backed area with a UFFDIO_CONTINUE ioctl. The semantics for this match those for hugetlbfs - we look up the existing page in the page cache, and install a PTE for it. This commit introduces a new helper: mfill_atomic_install_pte. Why handle UFFDIO_CONTINUE for shmem in mm/userfaultfd.c, instead of in shmem.c? The existing userfault implementation only relies on shmem.c for VM_SHARED VMAs. However, minor fault handling / CONTINUE work just fine for !VM_SHARED VMAs as well. We'd prefer to handle CONTINUE for shmem in one place, regardless of shared/private (to reduce code duplication). Why add a new mfill_atomic_install_pte helper? A problem we have with continue is that shmem_mfill_atomic_pte() and mcopy_atomic_pte() are *close* to what we want, but not exactly. We do want to setup the PTEs in a CONTINUE operation, but we don't want to e.g. allocate a new page, charge it (e.g. to the shmem inode), manipulate various flags, etc. Also we have the problem stated above: shmem_mfill_atomic_pte() and mcopy_atomic_pte() both handle one-half of the problem (shared / private) continue cares about. So, introduce mcontinue_atomic_pte(), to handle all of the shmem continue cases. Introduce the helper so it doesn't duplicate code with mcopy_atomic_pte(). In a future commit, shmem_mfill_atomic_pte() will also be modified to use this new helper. However, since this is a bigger refactor, it seems most clear to do it as a separate change. Link: https://lkml.kernel.org/r/20210503180737.2487560-5-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Acked-by: Hugh Dickins Acked-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- mm/userfaultfd.c | 172 +++++++++++++++++++++++++++++++++------------ 1 file changed, 127 insertions(+), 45 deletions(-) --- a/mm/userfaultfd.c~userfaultfd-shmem-support-uffdio_continue-for-shmem +++ a/mm/userfaultfd.c @@ -48,6 +48,83 @@ struct vm_area_struct *find_dst_vma(stru return dst_vma; } +/* + * Install PTEs, to map dst_addr (within dst_vma) to page. + * + * This function handles MCOPY_ATOMIC_CONTINUE (which is always file-backed), + * whether or not dst_vma is VM_SHARED. It also handles the more general + * MCOPY_ATOMIC_NORMAL case, when dst_vma is *not* VM_SHARED (it may be file + * backed, or not). + * + * Note that MCOPY_ATOMIC_NORMAL for a VM_SHARED dst_vma is handled by + * shmem_mcopy_atomic_pte instead. + */ +static int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, struct page *page, + bool newly_allocated, bool wp_copy) +{ + int ret; + pte_t _dst_pte, *dst_pte; + bool writable = dst_vma->vm_flags & VM_WRITE; + bool vm_shared = dst_vma->vm_flags & VM_SHARED; + bool page_in_cache = page->mapping; + spinlock_t *ptl; + struct inode *inode; + pgoff_t offset, max_off; + + _dst_pte = mk_pte(page, dst_vma->vm_page_prot); + if (page_in_cache && !vm_shared) + writable = false; + if (writable || !page_in_cache) + _dst_pte = pte_mkdirty(_dst_pte); + if (writable) { + if (wp_copy) + _dst_pte = pte_mkuffd_wp(_dst_pte); + else + _dst_pte = pte_mkwrite(_dst_pte); + } + + dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); + + if (vma_is_shmem(dst_vma)) { + /* serialize against truncate with the page table lock */ + inode = dst_vma->vm_file->f_inode; + offset = linear_page_index(dst_vma, dst_addr); + max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); + ret = -EFAULT; + if (unlikely(offset >= max_off)) + goto out_unlock; + } + + ret = -EEXIST; + if (!pte_none(*dst_pte)) + goto out_unlock; + + if (page_in_cache) + page_add_file_rmap(page, false); + else + page_add_new_anon_rmap(page, dst_vma, dst_addr, false); + + /* + * Must happen after rmap, as mm_counter() checks mapping (via + * PageAnon()), which is set by __page_set_anon_rmap(). + */ + inc_mm_counter(dst_mm, mm_counter(page)); + + if (newly_allocated) + lru_cache_add_inactive_or_unevictable(page, dst_vma); + + set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); + + /* No need to invalidate - it was non-present before */ + update_mmu_cache(dst_vma, dst_addr, dst_pte); + ret = 0; +out_unlock: + pte_unmap_unlock(dst_pte, ptl); + return ret; +} + static int mcopy_atomic_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, struct vm_area_struct *dst_vma, @@ -56,13 +133,9 @@ static int mcopy_atomic_pte(struct mm_st struct page **pagep, bool wp_copy) { - pte_t _dst_pte, *dst_pte; - spinlock_t *ptl; void *page_kaddr; int ret; struct page *page; - pgoff_t offset, max_off; - struct inode *inode; if (!*pagep) { ret = -ENOMEM; @@ -99,43 +172,12 @@ static int mcopy_atomic_pte(struct mm_st if (mem_cgroup_charge(page, dst_mm, GFP_KERNEL)) goto out_release; - _dst_pte = pte_mkdirty(mk_pte(page, dst_vma->vm_page_prot)); - if (dst_vma->vm_flags & VM_WRITE) { - if (wp_copy) - _dst_pte = pte_mkuffd_wp(_dst_pte); - else - _dst_pte = pte_mkwrite(_dst_pte); - } - - dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); - if (dst_vma->vm_file) { - /* the shmem MAP_PRIVATE case requires checking the i_size */ - inode = dst_vma->vm_file->f_inode; - offset = linear_page_index(dst_vma, dst_addr); - max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); - ret = -EFAULT; - if (unlikely(offset >= max_off)) - goto out_release_uncharge_unlock; - } - ret = -EEXIST; - if (!pte_none(*dst_pte)) - goto out_release_uncharge_unlock; - - inc_mm_counter(dst_mm, MM_ANONPAGES); - page_add_new_anon_rmap(page, dst_vma, dst_addr, false); - lru_cache_add_inactive_or_unevictable(page, dst_vma); - - set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); - - /* No need to invalidate - it was non-present before */ - update_mmu_cache(dst_vma, dst_addr, dst_pte); - - pte_unmap_unlock(dst_pte, ptl); - ret = 0; + ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr, + page, true, wp_copy); + if (ret) + goto out_release; out: return ret; -out_release_uncharge_unlock: - pte_unmap_unlock(dst_pte, ptl); out_release: put_page(page); goto out; @@ -176,6 +218,41 @@ out_unlock: return ret; } +/* Handles UFFDIO_CONTINUE for all shmem VMAs (shared or private). */ +static int mcontinue_atomic_pte(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + bool wp_copy) +{ + struct inode *inode = file_inode(dst_vma->vm_file); + pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); + struct page *page; + int ret; + + ret = shmem_getpage(inode, pgoff, &page, SGP_READ); + if (ret) + goto out; + if (!page) { + ret = -EFAULT; + goto out; + } + + ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr, + page, false, wp_copy); + if (ret) + goto out_release; + + unlock_page(page); + ret = 0; +out: + return ret; +out_release: + unlock_page(page); + put_page(page); + goto out; +} + static pmd_t *mm_alloc_pmd(struct mm_struct *mm, unsigned long address) { pgd_t *pgd; @@ -367,11 +444,16 @@ static __always_inline ssize_t mfill_ato unsigned long dst_addr, unsigned long src_addr, struct page **page, - bool zeropage, + enum mcopy_atomic_mode mode, bool wp_copy) { ssize_t err; + if (mode == MCOPY_ATOMIC_CONTINUE) { + return mcontinue_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, + wp_copy); + } + /* * The normal page fault path for a shmem will invoke the * fault, fill the hole in the file and COW it right away. The @@ -383,7 +465,7 @@ static __always_inline ssize_t mfill_ato * and not in the radix tree. */ if (!(dst_vma->vm_flags & VM_SHARED)) { - if (!zeropage) + if (mode == MCOPY_ATOMIC_NORMAL) err = mcopy_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, src_addr, page, wp_copy); @@ -393,7 +475,8 @@ static __always_inline ssize_t mfill_ato } else { VM_WARN_ON_ONCE(wp_copy); err = shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, - dst_addr, src_addr, zeropage, + dst_addr, src_addr, + mode != MCOPY_ATOMIC_NORMAL, page); } @@ -415,7 +498,6 @@ static __always_inline ssize_t __mcopy_a long copied; struct page *page; bool wp_copy; - bool zeropage = (mcopy_mode == MCOPY_ATOMIC_ZEROPAGE); /* * Sanitize the command parameters: @@ -478,7 +560,7 @@ retry: if (!vma_is_anonymous(dst_vma) && !vma_is_shmem(dst_vma)) goto out_unlock; - if (mcopy_mode == MCOPY_ATOMIC_CONTINUE) + if (!vma_is_shmem(dst_vma) && mcopy_mode == MCOPY_ATOMIC_CONTINUE) goto out_unlock; /* @@ -526,7 +608,7 @@ retry: BUG_ON(pmd_trans_huge(*dst_pmd)); err = mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - src_addr, &page, zeropage, wp_copy); + src_addr, &page, mcopy_mode, wp_copy); cond_resched(); if (unlikely(err == -ENOENT)) { From patchwork Thu Jul 1 01:49:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F8F1C11F6B for ; Thu, 1 Jul 2021 01:49:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 06B086135C for ; Thu, 1 Jul 2021 01:49:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 06B086135C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 83E3B8D01FE; Wed, 30 Jun 2021 21:49:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7ED7D8D01D0; Wed, 30 Jun 2021 21:49:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 667C28D01FE; Wed, 30 Jun 2021 21:49:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0224.hostedemail.com [216.40.44.224]) by kanga.kvack.org (Postfix) with ESMTP id 424378D01D0 for ; Wed, 30 Jun 2021 21:49:30 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1649925F58 for ; Thu, 1 Jul 2021 01:49:30 +0000 (UTC) X-FDA: 78312336900.02.4CA10E3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id B30F4200170A for ; Thu, 1 Jul 2021 01:49:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4EB6A61260; Thu, 1 Jul 2021 01:49:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104169; bh=XQVdbklUcY3Y4OBn3/SpERoYTvVfSwjiwcqOrf76sLk=; h=Date:From:To:Subject:In-Reply-To:From; b=1RSpofW/p0HP7tFRQ86Dt4q243E7xOOucv3o64+zFPiyOrC2syJxlo8Q7ICqiNbM3 FnfdaER7zstjYJJLRXXvr4S1VhxQAL5l6/UA8ZHfrU4Ix/5988FDbaSicqI24He8m0 5ZLPxEuryhdGDL9xeaz1C1D1cjgh95oa2rJCj7aY= Date: Wed, 30 Jun 2021 18:49:27 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 043/192] userfaultfd/shmem: advertise shmem minor fault support Message-ID: <20210701014927.FnCDjrQ5E%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: B30F4200170A Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="1RSpofW/"; spf=pass (imf26.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: gpuauk66wh9yae8q4cg8jage5cf9rudh X-HE-Tag: 1625104169-12209 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/shmem: advertise shmem minor fault support Now that the feature is fully implemented (the faulting path hooks exist so userspace is notified, and the ioctl to resolve such faults is available), advertise this as a supported feature. Link: https://lkml.kernel.org/r/20210503180737.2487560-6-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Acked-by: Hugh Dickins Acked-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- Documentation/admin-guide/mm/userfaultfd.rst | 3 ++- fs/userfaultfd.c | 3 ++- include/uapi/linux/userfaultfd.h | 7 ++++++- 3 files changed, 10 insertions(+), 3 deletions(-) --- a/Documentation/admin-guide/mm/userfaultfd.rst~userfaultfd-shmem-advertise-shmem-minor-fault-support +++ a/Documentation/admin-guide/mm/userfaultfd.rst @@ -77,7 +77,8 @@ events, except page fault notifications, - ``UFFD_FEATURE_MINOR_HUGETLBFS`` indicates that the kernel supports ``UFFDIO_REGISTER_MODE_MINOR`` registration for hugetlbfs virtual memory - areas. + areas. ``UFFD_FEATURE_MINOR_SHMEM`` is the analogous feature indicating + support for shmem virtual memory areas. The userland application should set the feature flags it intends to use when invoking the ``UFFDIO_API`` ioctl, to request that those features be --- a/fs/userfaultfd.c~userfaultfd-shmem-advertise-shmem-minor-fault-support +++ a/fs/userfaultfd.c @@ -1944,7 +1944,8 @@ static int userfaultfd_api(struct userfa /* report all available features and ioctls to userland */ uffdio_api.features = UFFD_API_FEATURES; #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_MINOR - uffdio_api.features &= ~UFFD_FEATURE_MINOR_HUGETLBFS; + uffdio_api.features &= + ~(UFFD_FEATURE_MINOR_HUGETLBFS | UFFD_FEATURE_MINOR_SHMEM); #endif #ifndef CONFIG_HAVE_ARCH_USERFAULTFD_WP uffdio_api.features &= ~UFFD_FEATURE_PAGEFAULT_FLAG_WP; --- a/include/uapi/linux/userfaultfd.h~userfaultfd-shmem-advertise-shmem-minor-fault-support +++ a/include/uapi/linux/userfaultfd.h @@ -31,7 +31,8 @@ UFFD_FEATURE_MISSING_SHMEM | \ UFFD_FEATURE_SIGBUS | \ UFFD_FEATURE_THREAD_ID | \ - UFFD_FEATURE_MINOR_HUGETLBFS) + UFFD_FEATURE_MINOR_HUGETLBFS | \ + UFFD_FEATURE_MINOR_SHMEM) #define UFFD_API_IOCTLS \ ((__u64)1 << _UFFDIO_REGISTER | \ (__u64)1 << _UFFDIO_UNREGISTER | \ @@ -185,6 +186,9 @@ struct uffdio_api { * UFFD_FEATURE_MINOR_HUGETLBFS indicates that minor faults * can be intercepted (via REGISTER_MODE_MINOR) for * hugetlbfs-backed pages. + * + * UFFD_FEATURE_MINOR_SHMEM indicates the same support as + * UFFD_FEATURE_MINOR_HUGETLBFS, but for shmem-backed pages instead. */ #define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0) #define UFFD_FEATURE_EVENT_FORK (1<<1) @@ -196,6 +200,7 @@ struct uffdio_api { #define UFFD_FEATURE_SIGBUS (1<<7) #define UFFD_FEATURE_THREAD_ID (1<<8) #define UFFD_FEATURE_MINOR_HUGETLBFS (1<<9) +#define UFFD_FEATURE_MINOR_SHMEM (1<<10) __u64 features; __u64 ioctls; From patchwork Thu Jul 1 01:49:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD8C4C11F6A for ; Thu, 1 Jul 2021 01:49:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6223D61260 for ; Thu, 1 Jul 2021 01:49:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6223D61260 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E0B048D01FF; Wed, 30 Jun 2021 21:49:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DE0F48D01D0; Wed, 30 Jun 2021 21:49:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C821E8D01FF; Wed, 30 Jun 2021 21:49:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0033.hostedemail.com [216.40.44.33]) by kanga.kvack.org (Postfix) with ESMTP id A62C98D01D0 for ; Wed, 30 Jun 2021 21:49:33 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7DF03815A79C for ; Thu, 1 Jul 2021 01:49:33 +0000 (UTC) X-FDA: 78312337026.31.39AD833 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 1C2117001A08 for ; Thu, 1 Jul 2021 01:49:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BB6046135C; Thu, 1 Jul 2021 01:49:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104172; bh=5i1ZbkF7O2Ww0Cthjyad1KJsaV31GfRDz8sohDiopZI=; h=Date:From:To:Subject:In-Reply-To:From; b=hsTZuH4JaWLTh47DEvNx4jcDTFAXQRz80KMzry9Mwq3C9ByGAioSyJ6iJ8Hgre7QS Jo5t1aexs1VC0g9xUtf7veOUSOqCpfF9LotHmdyrVHy3N+wr3KYQpXPLEtPCUiYPCR iecQrkG8WlSd/jj6b1EpcnV00yKnZyU4TCe3oeE8= Date: Wed, 30 Jun 2021 18:49:31 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 044/192] userfaultfd/shmem: modify shmem_mfill_atomic_pte to use install_pte() Message-ID: <20210701014931.siUl7ScLk%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=hsTZuH4J; dmarc=none; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: eo15e4aydxhufcj38uqjkh4hsgojpn3w X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1C2117001A08 X-HE-Tag: 1625104172-159988 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/shmem: modify shmem_mfill_atomic_pte to use install_pte() In a previous commit, we added the mfill_atomic_install_pte() helper. This helper does the job of setting up PTEs for an existing page, to map it into a given VMA. It deals with both the anon and shmem cases, as well as the shared and private cases. In other words, shmem_mfill_atomic_pte() duplicates a case it already handles. So, expose it, and let shmem_mfill_atomic_pte() use it directly, to reduce code duplication. This requires that we refactor shmem_mfill_atomic_pte() a bit: Instead of doing accounting (shmem_recalc_inode() et al) part-way through the PTE setup, do it afterward. This frees up mfill_atomic_install_pte() from having to care about this accounting, and means we don't need to e.g. shmem_uncharge() in the error path. A side effect is this switches shmem_mfill_atomic_pte() to use lru_cache_add_inactive_or_unevictable() instead of just lru_cache_add(). This wrapper does some extra accounting in an exceptional case, if appropriate, so it's actually the more correct thing to use. Link: https://lkml.kernel.org/r/20210503180737.2487560-7-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Peter Xu Acked-by: Hugh Dickins Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- include/linux/userfaultfd_k.h | 5 ++ mm/shmem.c | 58 ++++++-------------------------- mm/userfaultfd.c | 17 +++------ 3 files changed, 23 insertions(+), 57 deletions(-) --- a/include/linux/userfaultfd_k.h~userfaultfd-shmem-modify-shmem_mfill_atomic_pte-to-use-install_pte +++ a/include/linux/userfaultfd_k.h @@ -53,6 +53,11 @@ enum mcopy_atomic_mode { MCOPY_ATOMIC_CONTINUE, }; +extern int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, struct page *page, + bool newly_allocated, bool wp_copy); + extern ssize_t mcopy_atomic(struct mm_struct *dst_mm, unsigned long dst_start, unsigned long src_start, unsigned long len, bool *mmap_changing, __u64 mode); --- a/mm/shmem.c~userfaultfd-shmem-modify-shmem_mfill_atomic_pte-to-use-install_pte +++ a/mm/shmem.c @@ -2376,14 +2376,11 @@ int shmem_mfill_atomic_pte(struct mm_str struct address_space *mapping = inode->i_mapping; gfp_t gfp = mapping_gfp_mask(mapping); pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); - spinlock_t *ptl; void *page_kaddr; struct page *page; - pte_t _dst_pte, *dst_pte; int ret; pgoff_t max_off; - ret = -ENOMEM; if (!shmem_inode_acct_block(inode, 1)) { /* * We may have got a page, returned -ENOENT triggering a retry, @@ -2394,10 +2391,11 @@ int shmem_mfill_atomic_pte(struct mm_str put_page(*pagep); *pagep = NULL; } - goto out; + return -ENOMEM; } if (!*pagep) { + ret = -ENOMEM; page = shmem_alloc_page(gfp, info, pgoff); if (!page) goto out_unacct_blocks; @@ -2412,9 +2410,9 @@ int shmem_mfill_atomic_pte(struct mm_str /* fallback to copy_from_user outside mmap_lock */ if (unlikely(ret)) { *pagep = page; - shmem_inode_unacct_blocks(inode, 1); + ret = -ENOENT; /* don't free the page */ - return -ENOENT; + goto out_unacct_blocks; } } else { /* ZEROPAGE */ clear_highpage(page); @@ -2440,32 +2438,10 @@ int shmem_mfill_atomic_pte(struct mm_str if (ret) goto out_release; - _dst_pte = mk_pte(page, dst_vma->vm_page_prot); - if (dst_vma->vm_flags & VM_WRITE) - _dst_pte = pte_mkwrite(pte_mkdirty(_dst_pte)); - else { - /* - * We don't set the pte dirty if the vma has no - * VM_WRITE permission, so mark the page dirty or it - * could be freed from under us. We could do it - * unconditionally before unlock_page(), but doing it - * only if VM_WRITE is not set is faster. - */ - set_page_dirty(page); - } - - dst_pte = pte_offset_map_lock(dst_mm, dst_pmd, dst_addr, &ptl); - - ret = -EFAULT; - max_off = DIV_ROUND_UP(i_size_read(inode), PAGE_SIZE); - if (unlikely(pgoff >= max_off)) - goto out_release_unlock; - - ret = -EEXIST; - if (!pte_none(*dst_pte)) - goto out_release_unlock; - - lru_cache_add(page); + ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr, + page, true, false); + if (ret) + goto out_delete_from_cache; spin_lock_irq(&info->lock); info->alloced++; @@ -2473,27 +2449,17 @@ int shmem_mfill_atomic_pte(struct mm_str shmem_recalc_inode(inode); spin_unlock_irq(&info->lock); - inc_mm_counter(dst_mm, mm_counter_file(page)); - page_add_file_rmap(page, false); - set_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte); - - /* No need to invalidate - it was non-present before */ - update_mmu_cache(dst_vma, dst_addr, dst_pte); - pte_unmap_unlock(dst_pte, ptl); + SetPageDirty(page); unlock_page(page); - ret = 0; -out: - return ret; -out_release_unlock: - pte_unmap_unlock(dst_pte, ptl); - ClearPageDirty(page); + return 0; +out_delete_from_cache: delete_from_page_cache(page); out_release: unlock_page(page); put_page(page); out_unacct_blocks: shmem_inode_unacct_blocks(inode, 1); - goto out; + return ret; } #endif /* CONFIG_USERFAULTFD */ --- a/mm/userfaultfd.c~userfaultfd-shmem-modify-shmem_mfill_atomic_pte-to-use-install_pte +++ a/mm/userfaultfd.c @@ -51,18 +51,13 @@ struct vm_area_struct *find_dst_vma(stru /* * Install PTEs, to map dst_addr (within dst_vma) to page. * - * This function handles MCOPY_ATOMIC_CONTINUE (which is always file-backed), - * whether or not dst_vma is VM_SHARED. It also handles the more general - * MCOPY_ATOMIC_NORMAL case, when dst_vma is *not* VM_SHARED (it may be file - * backed, or not). - * - * Note that MCOPY_ATOMIC_NORMAL for a VM_SHARED dst_vma is handled by - * shmem_mcopy_atomic_pte instead. + * This function handles both MCOPY_ATOMIC_NORMAL and _CONTINUE for both shmem + * and anon, and for both shared and private VMAs. */ -static int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, struct page *page, - bool newly_allocated, bool wp_copy) +int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, struct page *page, + bool newly_allocated, bool wp_copy) { int ret; pte_t _dst_pte, *dst_pte; From patchwork Thu Jul 1 01:49:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353149 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CCB8C11F69 for ; Thu, 1 Jul 2021 01:49:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1482B6105A for ; Thu, 1 Jul 2021 01:49:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1482B6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9287C8D0200; Wed, 30 Jun 2021 21:49:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8FF288D01D0; Wed, 30 Jun 2021 21:49:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A03B8D0200; Wed, 30 Jun 2021 21:49:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0064.hostedemail.com [216.40.44.64]) by kanga.kvack.org (Postfix) with ESMTP id 57C938D01D0 for ; Wed, 30 Jun 2021 21:49:37 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 02A62252D7 for ; Thu, 1 Jul 2021 01:49:36 +0000 (UTC) X-FDA: 78312337194.01.586212E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 9461AD000081 for ; Thu, 1 Jul 2021 01:49:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2C76461467; Thu, 1 Jul 2021 01:49:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104175; bh=aEErmlUA/YhJIlCXvj9fzODhqv6FtEJErjkFemSYRzI=; h=Date:From:To:Subject:In-Reply-To:From; b=jAAE8IYPS2CL7KCXpxbqH4jGp/Geyhi1NKcbey9Ixi7A4Bxof2ExvBbp6CNdMDum/ ZkXypVnSGBjNFrdN9mNeYuhm8ZiKFI00o34fGzcMxJktiwdXikn3m/iL3/+cDaVu+0 lnUAKmMOrGoMm6fKamNIA0+rld7MkhyeQlsTN11w= Date: Wed, 30 Jun 2021 18:49:34 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 045/192] userfaultfd/selftests: use memfd_create for shmem test type Message-ID: <20210701014934.ysD_Zomj8%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9461AD000081 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=jAAE8IYP; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: sofssm5n77zaiq9pfrj3kedjhg4g6yex X-HE-Tag: 1625104176-770919 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/selftests: use memfd_create for shmem test type This is a preparatory commit. In the future, we want to be able to setup alias mappings for area_src and area_dst in the shmem test, like we do in the hugetlb_shared test. With a VMA obtained via mmap(MAP_ANONYMOUS | MAP_SHARED), it isn't clear how to do this. So, mmap() with an fd, so we can create alias mappings. Use memfd_create instead of actually passing in a tmpfs path like hugetlb does, since it's more convenient / simpler to run, and works just as well. Future commits will: 1. Setup the alias mappings. 2. Extend our tests to actually take advantage of this, to test new userfaultfd behavior being introduced in this series. Also, a small fix in the area we're changing: when the hugetlb setup fails in main(), pass in the right argv[] so we actually print out the hugetlb file path. Link: https://lkml.kernel.org/r/20210503180737.2487560-8-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-use-memfd_create-for-shmem-test-type +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -85,6 +85,7 @@ static bool test_uffdio_wp = false; static bool test_uffdio_minor = false; static bool map_shared; +static int shm_fd; static int huge_fd; static char *huge_fd_off0; static unsigned long long *count_verify; @@ -277,8 +278,11 @@ static void shmem_release_pages(char *re static void shmem_allocate_area(void **alloc_area) { + unsigned long offset = + alloc_area == (void **)&area_src ? 0 : nr_pages * page_size; + *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE, - MAP_ANONYMOUS | MAP_SHARED, -1, 0); + MAP_SHARED, shm_fd, offset); if (*alloc_area == MAP_FAILED) err("mmap of memfd failed"); } @@ -1602,6 +1606,16 @@ int main(int argc, char **argv) err("Open of %s failed", argv[4]); if (ftruncate(huge_fd, 0)) err("ftruncate %s to size 0 failed", argv[4]); + } else if (test_type == TEST_SHMEM) { + shm_fd = memfd_create(argv[0], 0); + if (shm_fd < 0) + err("memfd_create"); + if (ftruncate(shm_fd, nr_pages * page_size * 2)) + err("ftruncate"); + if (fallocate(shm_fd, + FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, 0, + nr_pages * page_size * 2)) + err("fallocate"); } printf("nr_pages: %lu, nr_pages_per_cpu: %lu\n", nr_pages, nr_pages_per_cpu); From patchwork Thu Jul 1 01:49:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353151 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83999C11F6B for ; Thu, 1 Jul 2021 01:49:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 346DE61260 for ; Thu, 1 Jul 2021 01:49:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 346DE61260 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B240B8D0201; Wed, 30 Jun 2021 21:49:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AFB138D01D0; Wed, 30 Jun 2021 21:49:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 99C808D0201; Wed, 30 Jun 2021 21:49:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 768148D01D0 for ; Wed, 30 Jun 2021 21:49:40 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 4387A8163EEF for ; Thu, 1 Jul 2021 01:49:40 +0000 (UTC) X-FDA: 78312337320.15.CFB0A70 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id DCE45900032B for ; Thu, 1 Jul 2021 01:49:39 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 920F56105A; Thu, 1 Jul 2021 01:49:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104179; bh=8KNx8WWwsmy+fp9z4NZNRRslQL5pls+oaMECB9+Zhck=; h=Date:From:To:Subject:In-Reply-To:From; b=JxmEYO5JkJ+cIjVXjKqqA3wk011M9JxTemfdg7Gsg2YLoANskoZYzc748HEDLBpIm MPAp0649Ea/hbcYplM8UoHGMScQhNkFK4tXugpXSMRRYphxAZRw1dRWA0pY8/hvTVu Bdf/5pxjn7q+veK3uJexUt7VlqEWWJ0IymkYYztY= Date: Wed, 30 Jun 2021 18:49:38 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 046/192] userfaultfd/selftests: create alias mappings in the shmem test Message-ID: <20210701014938.PK7FRdk21%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DCE45900032B Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=JxmEYO5J; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: ory7xyjp9inu1mm1uh9tug1ud98fsdcf X-HE-Tag: 1625104179-825851 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/selftests: create alias mappings in the shmem test Previously, we just allocated two shm areas: area_src and area_dst. With this commit, change this so we also allocate area_src_alias, and area_dst_alias. area_*_alias and area_* (respectively) point to the same underlying physical pages, but are different VMAs. In a future commit in this series, we'll leverage this setup to exercise minor fault handling support for shmem, just like we do in the hugetlb_shared test. Link: https://lkml.kernel.org/r/20210503180737.2487560-9-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 22 ++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-create-alias-mappings-in-the-shmem-test +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -278,13 +278,29 @@ static void shmem_release_pages(char *re static void shmem_allocate_area(void **alloc_area) { - unsigned long offset = - alloc_area == (void **)&area_src ? 0 : nr_pages * page_size; + void *area_alias = NULL; + bool is_src = alloc_area == (void **)&area_src; + unsigned long offset = is_src ? 0 : nr_pages * page_size; *alloc_area = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE, MAP_SHARED, shm_fd, offset); if (*alloc_area == MAP_FAILED) err("mmap of memfd failed"); + + area_alias = mmap(NULL, nr_pages * page_size, PROT_READ | PROT_WRITE, + MAP_SHARED, shm_fd, offset); + if (area_alias == MAP_FAILED) + err("mmap of memfd alias failed"); + + if (is_src) + area_src_alias = area_alias; + else + area_dst_alias = area_alias; +} + +static void shmem_alias_mapping(__u64 *start, size_t len, unsigned long offset) +{ + *start = (unsigned long)area_dst_alias + offset; } struct uffd_test_ops { @@ -314,7 +330,7 @@ static struct uffd_test_ops shmem_uffd_t .expected_ioctls = SHMEM_EXPECTED_IOCTLS, .allocate_area = shmem_allocate_area, .release_pages = shmem_release_pages, - .alias_mapping = noop_alias_mapping, + .alias_mapping = shmem_alias_mapping, }; static struct uffd_test_ops hugetlb_uffd_test_ops = { From patchwork Thu Jul 1 01:49:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353153 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49985C11F69 for ; Thu, 1 Jul 2021 01:49:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D823B6105A for ; Thu, 1 Jul 2021 01:49:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D823B6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F3B88D01D0; Wed, 30 Jun 2021 21:49:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5CB298D0202; Wed, 30 Jun 2021 21:49:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 450AC8D01D0; Wed, 30 Jun 2021 21:49:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0072.hostedemail.com [216.40.44.72]) by kanga.kvack.org (Postfix) with ESMTP id 1BB7C8D01D0 for ; Wed, 30 Jun 2021 21:49:44 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E3F6D25F5A for ; Thu, 1 Jul 2021 01:49:43 +0000 (UTC) X-FDA: 78312337446.21.65DD3AF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 8C1F31985 for ; Thu, 1 Jul 2021 01:49:43 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 295E661467; Thu, 1 Jul 2021 01:49:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104182; bh=2Sh/9B+VY5XHKYgd3d2dOaCg5KcIL4ZGrYGh9vQQVB4=; h=Date:From:To:Subject:In-Reply-To:From; b=Hotcykx0yN/NULl1rUm5hXRyf1+udwxvsVd/QFVJP1ImzuzvrAiHy4c2Agqxs78RN SGNJ+0LGFhvjyaDrN3uzEBcut2/6F0ZQXaeKFqRwS47DGFO+ENPDZ50t95Ed7/1Zip CIVFreXQ+0qxuFqzJppPjuvkrA3cjDwWWcs0GpT0= Date: Wed, 30 Jun 2021 18:49:41 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 047/192] userfaultfd/selftests: reinitialize test context in each test Message-ID: <20210701014941.wOltpZ3gz%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Hotcykx0; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8C1F31985 X-Stat-Signature: us71axgf9dhyg1kp64x51nb6jbqkzk9c X-HE-Tag: 1625104183-998665 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/selftests: reinitialize test context in each test Currently, the context (fds, mmap-ed areas, etc.) are global. Each test mutates this state in some way, in some cases really "clobbering it" (e.g., the events test mremap-ing area_dst over the top of area_src, or the minor faults tests overwriting the count_verify values in the test areas). We run the tests in a particular order, each test is careful to make the right assumptions about its starting state, etc. But, this is fragile. It's better for a test's success or failure to not depend on what some other prior test case did to the global state. To that end, clear and reinitialize the test context at the start of each test case, so whatever prior test cases did doesn't affect future tests. This is particularly relevant to this series because the events test's mremap of area_dst screws up assumptions the minor fault test was relying on. This wasn't a problem for hugetlb, as we don't mremap in that case. [peterx@redhat.com: fix conflict between this patch and the uffd pagemap series] Link: https://lkml.kernel.org/r/YKQqKrl+/cQ1utrb@t490s Link: https://lkml.kernel.org/r/20210503180737.2487560-10-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Signed-off-by: Peter Xu Reviewed-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 222 +++++++++++---------- 1 file changed, 117 insertions(+), 105 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-reinitialize-test-context-in-each-test +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -89,7 +89,8 @@ static int shm_fd; static int huge_fd; static char *huge_fd_off0; static unsigned long long *count_verify; -static int uffd, uffd_flags, finished, *pipefd; +static int uffd = -1; +static int uffd_flags, finished, *pipefd; static char *area_src, *area_src_alias, *area_dst, *area_dst_alias; static char *zeropage; pthread_attr_t attr; @@ -342,6 +343,111 @@ static struct uffd_test_ops hugetlb_uffd static struct uffd_test_ops *uffd_test_ops; +static void userfaultfd_open(uint64_t *features) +{ + struct uffdio_api uffdio_api; + + uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY); + if (uffd < 0) + err("userfaultfd syscall not available in this kernel"); + uffd_flags = fcntl(uffd, F_GETFD, NULL); + + uffdio_api.api = UFFD_API; + uffdio_api.features = *features; + if (ioctl(uffd, UFFDIO_API, &uffdio_api)) + err("UFFDIO_API failed.\nPlease make sure to " + "run with either root or ptrace capability."); + if (uffdio_api.api != UFFD_API) + err("UFFDIO_API error: %" PRIu64, (uint64_t)uffdio_api.api); + + *features = uffdio_api.features; +} + +static inline void munmap_area(void **area) +{ + if (*area) + if (munmap(*area, nr_pages * page_size)) + err("munmap"); + + *area = NULL; +} + +static void uffd_test_ctx_clear(void) +{ + size_t i; + + if (pipefd) { + for (i = 0; i < nr_cpus * 2; ++i) { + if (close(pipefd[i])) + err("close pipefd"); + } + free(pipefd); + pipefd = NULL; + } + + if (count_verify) { + free(count_verify); + count_verify = NULL; + } + + if (uffd != -1) { + if (close(uffd)) + err("close uffd"); + uffd = -1; + } + + huge_fd_off0 = NULL; + munmap_area((void **)&area_src); + munmap_area((void **)&area_src_alias); + munmap_area((void **)&area_dst); + munmap_area((void **)&area_dst_alias); +} + +static void uffd_test_ctx_init_ext(uint64_t *features) +{ + unsigned long nr, cpu; + + uffd_test_ctx_clear(); + + uffd_test_ops->allocate_area((void **)&area_src); + uffd_test_ops->allocate_area((void **)&area_dst); + + uffd_test_ops->release_pages(area_src); + uffd_test_ops->release_pages(area_dst); + + userfaultfd_open(features); + + count_verify = malloc(nr_pages * sizeof(unsigned long long)); + if (!count_verify) + err("count_verify"); + + for (nr = 0; nr < nr_pages; nr++) { + *area_mutex(area_src, nr) = + (pthread_mutex_t)PTHREAD_MUTEX_INITIALIZER; + count_verify[nr] = *area_count(area_src, nr) = 1; + /* + * In the transition between 255 to 256, powerpc will + * read out of order in my_bcmp and see both bytes as + * zero, so leave a placeholder below always non-zero + * after the count, to avoid my_bcmp to trigger false + * positives. + */ + *(area_count(area_src, nr) + 1) = 1; + } + + pipefd = malloc(sizeof(int) * nr_cpus * 2); + if (!pipefd) + err("pipefd"); + for (cpu = 0; cpu < nr_cpus; cpu++) + if (pipe2(&pipefd[cpu * 2], O_CLOEXEC | O_NONBLOCK)) + err("pipe"); +} + +static inline void uffd_test_ctx_init(uint64_t features) +{ + uffd_test_ctx_init_ext(&features); +} + static int my_bcmp(char *str1, char *str2, size_t n) { unsigned long i; @@ -726,40 +832,6 @@ static int stress(struct uffd_stats *uff return 0; } -static int userfaultfd_open_ext(uint64_t *features) -{ - struct uffdio_api uffdio_api; - - uffd = syscall(__NR_userfaultfd, O_CLOEXEC | O_NONBLOCK | UFFD_USER_MODE_ONLY); - if (uffd < 0) { - fprintf(stderr, - "userfaultfd syscall not available in this kernel\n"); - return 1; - } - uffd_flags = fcntl(uffd, F_GETFD, NULL); - - uffdio_api.api = UFFD_API; - uffdio_api.features = *features; - if (ioctl(uffd, UFFDIO_API, &uffdio_api)) { - fprintf(stderr, "UFFDIO_API failed.\nPlease make sure to " - "run with either root or ptrace capability.\n"); - return 1; - } - if (uffdio_api.api != UFFD_API) { - fprintf(stderr, "UFFDIO_API error: %" PRIu64 "\n", - (uint64_t)uffdio_api.api); - return 1; - } - - *features = uffdio_api.features; - return 0; -} - -static int userfaultfd_open(uint64_t features) -{ - return userfaultfd_open_ext(&features); -} - sigjmp_buf jbuf, *sigbuf; static void sighndl(int sig, siginfo_t *siginfo, void *ptr) @@ -868,6 +940,8 @@ static int faulting_process(int signal_t MREMAP_MAYMOVE | MREMAP_FIXED, area_src); if (area_dst == MAP_FAILED) err("mremap"); + /* Reset area_src since we just clobbered it */ + area_src = NULL; for (; nr < nr_pages; nr++) { count = *area_count(area_dst, nr); @@ -961,10 +1035,8 @@ static int userfaultfd_zeropage_test(voi printf("testing UFFDIO_ZEROPAGE: "); fflush(stdout); - uffd_test_ops->release_pages(area_dst); + uffd_test_ctx_init(0); - if (userfaultfd_open(0)) - return 1; uffdio_register.range.start = (unsigned long) area_dst; uffdio_register.range.len = nr_pages * page_size; uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING; @@ -981,7 +1053,6 @@ static int userfaultfd_zeropage_test(voi if (my_bcmp(area_dst, zeropage, page_size)) err("zeropage is not zero"); - close(uffd); printf("done.\n"); return 0; } @@ -999,12 +1070,10 @@ static int userfaultfd_events_test(void) printf("testing events (fork, remap, remove): "); fflush(stdout); - uffd_test_ops->release_pages(area_dst); - features = UFFD_FEATURE_EVENT_FORK | UFFD_FEATURE_EVENT_REMAP | UFFD_FEATURE_EVENT_REMOVE; - if (userfaultfd_open(features)) - return 1; + uffd_test_ctx_init(features); + fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK); uffdio_register.range.start = (unsigned long) area_dst; @@ -1037,8 +1106,6 @@ static int userfaultfd_events_test(void) if (pthread_join(uffd_mon, NULL)) return 1; - close(uffd); - uffd_stats_report(&stats, 1); return stats.missing_faults != nr_pages; @@ -1058,11 +1125,9 @@ static int userfaultfd_sig_test(void) printf("testing signal delivery: "); fflush(stdout); - uffd_test_ops->release_pages(area_dst); - features = UFFD_FEATURE_EVENT_FORK|UFFD_FEATURE_SIGBUS; - if (userfaultfd_open(features)) - return 1; + uffd_test_ctx_init(features); + fcntl(uffd, F_SETFL, uffd_flags | O_NONBLOCK); uffdio_register.range.start = (unsigned long) area_dst; @@ -1103,7 +1168,6 @@ static int userfaultfd_sig_test(void) printf("done.\n"); if (userfaults) err("Signal test failed, userfaults: %ld", userfaults); - close(uffd); return userfaults != 0; } @@ -1126,10 +1190,7 @@ static int userfaultfd_minor_test(void) printf("testing minor faults: "); fflush(stdout); - uffd_test_ops->release_pages(area_dst); - - if (userfaultfd_open_ext(&features)) - return 1; + uffd_test_ctx_init_ext(&features); /* If kernel reports the feature isn't supported, skip the test. */ if (!(features & UFFD_FEATURE_MINOR_HUGETLBFS)) { printf("skipping test due to lack of feature support\n"); @@ -1183,8 +1244,6 @@ static int userfaultfd_minor_test(void) if (pthread_join(uffd_mon, NULL)) return 1; - close(uffd); - uffd_stats_report(&stats, 1); return stats.missing_faults != 0 || stats.minor_faults != nr_pages; @@ -1267,7 +1326,7 @@ static void userfaultfd_pagemap_test(uns /* Flush so it doesn't flush twice in parent/child later */ fflush(stdout); - uffd_test_ops->release_pages(area_dst); + uffd_test_ctx_init(0); if (test_pgsize > page_size) { /* This is a thp test */ @@ -1279,9 +1338,6 @@ static void userfaultfd_pagemap_test(uns err("madvise(MADV_NOHUGEPAGE) failed"); } - if (userfaultfd_open(0)) - err("userfaultfd_open"); - uffdio_register.range.start = (unsigned long) area_dst; uffdio_register.range.len = nr_pages * page_size; uffdio_register.mode = UFFDIO_REGISTER_MODE_WP; @@ -1324,7 +1380,6 @@ static void userfaultfd_pagemap_test(uns pagemap_check_wp(value, false); close(pagemap_fd); - close(uffd); printf("done\n"); } @@ -1334,50 +1389,9 @@ static int userfaultfd_stress(void) char *tmp_area; unsigned long nr; struct uffdio_register uffdio_register; - unsigned long cpu; struct uffd_stats uffd_stats[nr_cpus]; - uffd_test_ops->allocate_area((void **)&area_src); - if (!area_src) - return 1; - uffd_test_ops->allocate_area((void **)&area_dst); - if (!area_dst) - return 1; - - if (userfaultfd_open(0)) - return 1; - - count_verify = malloc(nr_pages * sizeof(unsigned long long)); - if (!count_verify) { - perror("count_verify"); - return 1; - } - - for (nr = 0; nr < nr_pages; nr++) { - *area_mutex(area_src, nr) = (pthread_mutex_t) - PTHREAD_MUTEX_INITIALIZER; - count_verify[nr] = *area_count(area_src, nr) = 1; - /* - * In the transition between 255 to 256, powerpc will - * read out of order in my_bcmp and see both bytes as - * zero, so leave a placeholder below always non-zero - * after the count, to avoid my_bcmp to trigger false - * positives. - */ - *(area_count(area_src, nr) + 1) = 1; - } - - pipefd = malloc(sizeof(int) * nr_cpus * 2); - if (!pipefd) { - perror("pipefd"); - return 1; - } - for (cpu = 0; cpu < nr_cpus; cpu++) { - if (pipe2(&pipefd[cpu*2], O_CLOEXEC | O_NONBLOCK)) { - perror("pipe"); - return 1; - } - } + uffd_test_ctx_init(0); if (posix_memalign(&area, page_size, page_size)) err("out of memory"); @@ -1498,8 +1512,6 @@ static int userfaultfd_stress(void) uffd_stats_report(uffd_stats, nr_cpus); } - close(uffd); - if (test_type == TEST_ANON) { /* * shmem/hugetlb won't be able to run since they have different From patchwork Thu Jul 1 01:49:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353155 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92E85C11F68 for ; Thu, 1 Jul 2021 01:49:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4589E6135C for ; Thu, 1 Jul 2021 01:49:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4589E6135C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC02D8D0203; Wed, 30 Jun 2021 21:49:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B97248D0202; Wed, 30 Jun 2021 21:49:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A392D8D0203; Wed, 30 Jun 2021 21:49:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id 8101D8D0202 for ; Wed, 30 Jun 2021 21:49:47 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5A44E18030090 for ; Thu, 1 Jul 2021 01:49:47 +0000 (UTC) X-FDA: 78312337614.08.61A511E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 078F430000A0 for ; Thu, 1 Jul 2021 01:49:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B2A9A61260; Thu, 1 Jul 2021 01:49:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104186; bh=wUf9Ta9PDoGCvEfqnRV7KLRjUO9QmL155mmeq3yioL8=; h=Date:From:To:Subject:In-Reply-To:From; b=zxDNvTl39/o6vFd8lO/lP9KyK4VHxVh3HdDwfTHclqM5g/fgl817s0+kCygXRh1+W Nh7ob97jK63oGEEKL5qt8dlp2VXsPq8W+MFjDvi+x5CsoRuSB+O/a4L6Sks5YtqXjv yNvbtjKqNVXjmmV/fLnkQoomPVog5dmfWeFKge4A= Date: Wed, 30 Jun 2021 18:49:45 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, almasrymina@google.com, axelrasmussen@google.com, bgeffon@google.com, dgilbert@redhat.com, hughd@google.com, jglisse@redhat.com, joe@perches.com, kirill@shutemov.name, linux-mm@kvack.org, lokeshgidra@google.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, oupton@google.com, peterx@redhat.com, rppt@linux.vnet.ibm.com, sfr@canb.auug.org.au, shli@fb.com, shuah@kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, wangqing@vivo.com Subject: [patch 048/192] userfaultfd/selftests: exercise minor fault handling shmem support Message-ID: <20210701014945.rz3pIQwZx%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=zxDNvTl3; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: eq843kjs9nf6we7445iz9zpcsea76yew X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 078F430000A0 X-HE-Tag: 1625104186-83778 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Axel Rasmussen Subject: userfaultfd/selftests: exercise minor fault handling shmem support Enable test_uffdio_minor for test_type == TEST_SHMEM, and modify the test slightly to pass in / check for the right feature flags. Link: https://lkml.kernel.org/r/20210503180737.2487560-11-axelrasmussen@google.com Signed-off-by: Axel Rasmussen Reviewed-by: Peter Xu Cc: Alexander Viro Cc: Andrea Arcangeli Cc: Brian Geffon Cc: "Dr . David Alan Gilbert" Cc: Hugh Dickins Cc: Jerome Glisse Cc: Joe Perches Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Mike Kravetz Cc: Mike Rapoport Cc: Mina Almasry Cc: Oliver Upton Cc: Shaohua Li Cc: Shuah Khan Cc: Stephen Rothwell Cc: Wang Qing Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/userfaultfd.c | 29 ++++++++++++++++++--- 1 file changed, 25 insertions(+), 4 deletions(-) --- a/tools/testing/selftests/vm/userfaultfd.c~userfaultfd-selftests-exercise-minor-fault-handling-shmem-support +++ a/tools/testing/selftests/vm/userfaultfd.c @@ -474,6 +474,7 @@ static void wp_range(int ufd, __u64 star static void continue_range(int ufd, __u64 start, __u64 len) { struct uffdio_continue req; + int ret; req.range.start = start; req.range.len = len; @@ -482,6 +483,17 @@ static void continue_range(int ufd, __u6 if (ioctl(ufd, UFFDIO_CONTINUE, &req)) err("UFFDIO_CONTINUE failed for address 0x%" PRIx64, (uint64_t)start); + + /* + * Error handling within the kernel for continue is subtly different + * from copy or zeropage, so it may be a source of bugs. Trigger an + * error (-EEXIST) on purpose, to verify doing so doesn't cause a BUG. + */ + req.mapped = 0; + ret = ioctl(ufd, UFFDIO_CONTINUE, &req); + if (ret >= 0 || req.mapped != -EEXIST) + err("failed to exercise UFFDIO_CONTINUE error handling, ret=%d, mapped=%" PRId64, + ret, (int64_t) req.mapped); } static void *locking_thread(void *arg) @@ -1182,7 +1194,7 @@ static int userfaultfd_minor_test(void) void *expected_page; char c; struct uffd_stats stats = { 0 }; - uint64_t features = UFFD_FEATURE_MINOR_HUGETLBFS; + uint64_t req_features, features_out; if (!test_uffdio_minor) return 0; @@ -1190,9 +1202,17 @@ static int userfaultfd_minor_test(void) printf("testing minor faults: "); fflush(stdout); - uffd_test_ctx_init_ext(&features); - /* If kernel reports the feature isn't supported, skip the test. */ - if (!(features & UFFD_FEATURE_MINOR_HUGETLBFS)) { + if (test_type == TEST_HUGETLB) + req_features = UFFD_FEATURE_MINOR_HUGETLBFS; + else if (test_type == TEST_SHMEM) + req_features = UFFD_FEATURE_MINOR_SHMEM; + else + return 1; + + features_out = req_features; + uffd_test_ctx_init_ext(&features_out); + /* If kernel reports required features aren't supported, skip test. */ + if ((features_out & req_features) != req_features) { printf("skipping test due to lack of feature support\n"); fflush(stdout); return 0; @@ -1575,6 +1595,7 @@ static void set_test_type(const char *ty map_shared = true; test_type = TEST_SHMEM; uffd_test_ops = &shmem_uffd_test_ops; + test_uffdio_minor = true; } else { err("Unknown test type: %s", type); } From patchwork Thu Jul 1 01:49:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353159 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5AF07C11F69 for ; Thu, 1 Jul 2021 01:49:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 11F8261260 for ; Thu, 1 Jul 2021 01:49:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11F8261260 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8D9C18D0204; Wed, 30 Jun 2021 21:49:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 862E68D0202; Wed, 30 Jun 2021 21:49:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72AE98D0204; Wed, 30 Jun 2021 21:49:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 4E1C58D0202 for ; Wed, 30 Jun 2021 21:49:50 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 27A6F1802BC66 for ; Thu, 1 Jul 2021 01:49:50 +0000 (UTC) X-FDA: 78312337740.25.E61E6AA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id DAE3DF00008C for ; Thu, 1 Jul 2021 01:49:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 044446135C; Thu, 1 Jul 2021 01:49:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104189; bh=NZWefgvsCW72KL/kGw2AC3WzJaXpkbIKPTeRLoczNm0=; h=Date:From:To:Subject:In-Reply-To:From; b=SBtwUcUTcpr6U24E6CVCSNPw65jH3QSeLn2m4/y69cq/okh718/7UDUIzyOKF/d6i cMS1OmPDVERwqcL8RmfcoVEkOvCUk++AxV1uSTNBX0WkQ0d2EtSaODdlrZG40y7d5W Y3JsYiDXF7ZtcvCxMzrVHh2Dco3QHrD/6eZPP8E4= Date: Wed, 30 Jun 2021 18:49:48 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yuzhao@google.com Subject: [patch 049/192] mm/vmscan.c: fix potential deadlock in reclaim_pages() Message-ID: <20210701014948.8QuhSTdO4%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=SBtwUcUT; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: DAE3DF00008C X-Stat-Signature: 97jmo359zy31pj84ziyngs1dashgkota X-HE-Tag: 1625104189-431332 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/vmscan.c: fix potential deadlock in reclaim_pages() Theoretically without the protect from memalloc_noreclaim_save() and memalloc_noreclaim_restore(), reclaim_pages() can go into the block I/O layer recursively and deadlock. Querying 'reclaim_pages' in our kernel crash databases didn't yield any results. So the deadlock seems unlikely to happen. A possible explanation is that the only user of reclaim_pages(), i.e., MADV_PAGEOUT, is usually called before memory pressure builds up, e.g., on Android and Chrome OS. Under such a condition, allocations in the block I/O layer can be fulfilled without diverting to direct reclaim and therefore the recursion is avoided. Link: https://lkml.kernel.org/r/20210622074642.785473-1-yuzhao@google.com Link: https://lkml.kernel.org/r/20210614194727.2684053-1-yuzhao@google.com Signed-off-by: Yu Zhao Cc: Minchan Kim Signed-off-by: Andrew Morton --- mm/vmscan.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) --- a/mm/vmscan.c~mm-vmscanc-fix-potential-deadlock-in-reclaim_pages +++ a/mm/vmscan.c @@ -1701,6 +1701,7 @@ unsigned int reclaim_clean_pages_from_li unsigned int nr_reclaimed; struct page *page, *next; LIST_HEAD(clean_pages); + unsigned int noreclaim_flag; list_for_each_entry_safe(page, next, page_list, lru) { if (!PageHuge(page) && page_is_file_lru(page) && @@ -1711,8 +1712,17 @@ unsigned int reclaim_clean_pages_from_li } } + /* + * We should be safe here since we are only dealing with file pages and + * we are not kswapd and therefore cannot write dirty file pages. But + * call memalloc_noreclaim_save() anyway, just in case these conditions + * change in the future. + */ + noreclaim_flag = memalloc_noreclaim_save(); nr_reclaimed = shrink_page_list(&clean_pages, zone->zone_pgdat, &sc, &stat, true); + memalloc_noreclaim_restore(noreclaim_flag); + list_splice(&clean_pages, page_list); mod_node_page_state(zone->zone_pgdat, NR_ISOLATED_FILE, -(long)nr_reclaimed); @@ -2306,6 +2316,7 @@ unsigned long reclaim_pages(struct list_ LIST_HEAD(node_page_list); struct reclaim_stat dummy_stat; struct page *page; + unsigned int noreclaim_flag; struct scan_control sc = { .gfp_mask = GFP_KERNEL, .priority = DEF_PRIORITY, @@ -2314,6 +2325,8 @@ unsigned long reclaim_pages(struct list_ .may_swap = 1, }; + noreclaim_flag = memalloc_noreclaim_save(); + while (!list_empty(page_list)) { page = lru_to_page(page_list); if (nid == NUMA_NO_NODE) { @@ -2350,6 +2363,8 @@ unsigned long reclaim_pages(struct list_ } } + memalloc_noreclaim_restore(noreclaim_flag); + return nr_reclaimed; } From patchwork Thu Jul 1 01:49:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4352EC11F68 for ; Thu, 1 Jul 2021 01:49:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0455A61468 for ; Thu, 1 Jul 2021 01:49:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0455A61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 77A408D0205; Wed, 30 Jun 2021 21:49:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 72A448D0202; Wed, 30 Jun 2021 21:49:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F1788D0205; Wed, 30 Jun 2021 21:49:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id 326EA8D0202 for ; Wed, 30 Jun 2021 21:49:53 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 020B625F5A for ; Thu, 1 Jul 2021 01:49:53 +0000 (UTC) X-FDA: 78312337866.08.5FDDE50 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id ACA11B0000A7 for ; Thu, 1 Jul 2021 01:49:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CEB7561466; Thu, 1 Jul 2021 01:49:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104192; bh=UaiPca7gs407bkmCVK4bU9Y93+uF2oRhgy/5oNT0hcI=; h=Date:From:To:Subject:In-Reply-To:From; b=Rn3JLFPAmIikthGgjrQQpdkqhrsICZYskSpyop+a97rJ2ygqFeFsFNBKYVQsXvadB vpy1qknnNy2PnE6ipBKLuskEeLLJ73SJxl5rAXrBRybnBciFEjQ5DbxBfrFv8iG5IE 49eA+bLKOSRqNpMoP6pVnMtRR+asXeCMzeW94mwQ= Date: Wed, 30 Jun 2021 18:49:51 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yuzhao@google.com Subject: [patch 050/192] include/trace/events/vmscan.h: remove mm_vmscan_inactive_list_is_low Message-ID: <20210701014951.I2J96pCkx%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: ACA11B0000A7 X-Stat-Signature: y9twcnzqzzqgix96o77wjxgysj4haqii Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Rn3JLFPA; dmarc=none; spf=pass (imf19.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104192-570168 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: include/trace/events/vmscan.h: remove mm_vmscan_inactive_list_is_low mm_vmscan_inactive_list_is_low has no users after commit b91ac374346b ("mm: vmscan: enforce inactive:active ratio at the reclaim root"). Remove it. Link: https://lkml.kernel.org/r/20210614194554.2683395-1-yuzhao@google.com Signed-off-by: Yu Zhao Acked-by: Johannes Weiner Signed-off-by: Andrew Morton --- include/trace/events/vmscan.h | 41 -------------------------------- 1 file changed, 41 deletions(-) --- a/include/trace/events/vmscan.h~include-trace-events-vmscanh-remove-mm_vmscan_inactive_list_is_low +++ a/include/trace/events/vmscan.h @@ -423,47 +423,6 @@ TRACE_EVENT(mm_vmscan_lru_shrink_active, show_reclaim_flags(__entry->reclaim_flags)) ); -TRACE_EVENT(mm_vmscan_inactive_list_is_low, - - TP_PROTO(int nid, int reclaim_idx, - unsigned long total_inactive, unsigned long inactive, - unsigned long total_active, unsigned long active, - unsigned long ratio, int file), - - TP_ARGS(nid, reclaim_idx, total_inactive, inactive, total_active, active, ratio, file), - - TP_STRUCT__entry( - __field(int, nid) - __field(int, reclaim_idx) - __field(unsigned long, total_inactive) - __field(unsigned long, inactive) - __field(unsigned long, total_active) - __field(unsigned long, active) - __field(unsigned long, ratio) - __field(int, reclaim_flags) - ), - - TP_fast_assign( - __entry->nid = nid; - __entry->reclaim_idx = reclaim_idx; - __entry->total_inactive = total_inactive; - __entry->inactive = inactive; - __entry->total_active = total_active; - __entry->active = active; - __entry->ratio = ratio; - __entry->reclaim_flags = trace_reclaim_flags(file) & - RECLAIM_WB_LRU; - ), - - TP_printk("nid=%d reclaim_idx=%d total_inactive=%ld inactive=%ld total_active=%ld active=%ld ratio=%ld flags=%s", - __entry->nid, - __entry->reclaim_idx, - __entry->total_inactive, __entry->inactive, - __entry->total_active, __entry->active, - __entry->ratio, - show_reclaim_flags(__entry->reclaim_flags)) -); - TRACE_EVENT(mm_vmscan_node_reclaim_begin, TP_PROTO(int nid, int order, gfp_t gfp_flags), From patchwork Thu Jul 1 01:49:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353161 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29DBEC11F68 for ; Thu, 1 Jul 2021 01:49:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CBAEC6135C for ; Thu, 1 Jul 2021 01:49:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CBAEC6135C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 52FE48D0206; Wed, 30 Jun 2021 21:49:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E0658D0202; Wed, 30 Jun 2021 21:49:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CF408D0206; Wed, 30 Jun 2021 21:49:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 150918D0202 for ; Wed, 30 Jun 2021 21:49:56 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id E18F21802E6EA for ; Thu, 1 Jul 2021 01:49:55 +0000 (UTC) X-FDA: 78312337950.13.8BB34B2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 9D6C1801AB08 for ; Thu, 1 Jul 2021 01:49:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A54746105A; Thu, 1 Jul 2021 01:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104194; bh=jfervTFIl6D7+YaQowUqIZ/SImxUQ6eoqGhYYf6uadE=; h=Date:From:To:Subject:In-Reply-To:From; b=ujVOIHFcF5fV2pP4oLZJLAhw5aP8idk+reEUeJ6p5bxX6Jd62ZAL3IUdYGIN2mt6V f8V431U0kPGYP08mLogXNV2LhRt9orumfV3ULuS0euvILr+NFJhD3hpthjOX8yPf+3 YdsWLUK2q65vQdv1Pw5895sJE4mEfyHbOmCGfavw= Date: Wed, 30 Jun 2021 18:49:54 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 051/192] mm: workingset: define macro WORKINGSET_SHIFT Message-ID: <20210701014954.a8xL3Jys0%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ujVOIHFc; dmarc=none; spf=pass (imf06.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: ij8mmtp96384s5b8ounguyef3m6eocqf X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9D6C1801AB08 X-HE-Tag: 1625104195-988254 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm: workingset: define macro WORKINGSET_SHIFT The magic number 1 is used in several places in workingset.c. Define a macro WORKINGSET_SHIFT for it to improve code readability. Link: https://lkml.kernel.org/r/20210624122307.1759342-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Johannes Weiner Signed-off-by: Andrew Morton --- mm/workingset.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) --- a/mm/workingset.c~mm-workingset-define-macro-workingset_shift +++ a/mm/workingset.c @@ -168,8 +168,10 @@ * refault distance will immediately activate the refaulting page. */ +#define WORKINGSET_SHIFT 1 #define EVICTION_SHIFT ((BITS_PER_LONG - BITS_PER_XA_VALUE) + \ - 1 + NODES_SHIFT + MEM_CGROUP_ID_SHIFT) + WORKINGSET_SHIFT + NODES_SHIFT + \ + MEM_CGROUP_ID_SHIFT) #define EVICTION_MASK (~0UL >> EVICTION_SHIFT) /* @@ -189,7 +191,7 @@ static void *pack_shadow(int memcgid, pg eviction &= EVICTION_MASK; eviction = (eviction << MEM_CGROUP_ID_SHIFT) | memcgid; eviction = (eviction << NODES_SHIFT) | pgdat->node_id; - eviction = (eviction << 1) | workingset; + eviction = (eviction << WORKINGSET_SHIFT) | workingset; return xa_mk_value(eviction); } @@ -201,8 +203,8 @@ static void unpack_shadow(void *shadow, int memcgid, nid; bool workingset; - workingset = entry & 1; - entry >>= 1; + workingset = entry & ((1UL << WORKINGSET_SHIFT) - 1); + entry >>= WORKINGSET_SHIFT; nid = entry & ((1UL << NODES_SHIFT) - 1); entry >>= NODES_SHIFT; memcgid = entry & ((1UL << MEM_CGROUP_ID_SHIFT) - 1); From patchwork Thu Jul 1 01:49:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353163 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0FE6C11F66 for ; Thu, 1 Jul 2021 01:49:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A92606105A for ; Thu, 1 Jul 2021 01:49:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A92606105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 30D278D0207; Wed, 30 Jun 2021 21:49:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BDA28D0202; Wed, 30 Jun 2021 21:49:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 185488D0207; Wed, 30 Jun 2021 21:49:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id E74E38D0202 for ; Wed, 30 Jun 2021 21:49:58 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B7411181AEF3F for ; Thu, 1 Jul 2021 01:49:58 +0000 (UTC) X-FDA: 78312338076.15.BE25FC4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 76CC0F0000AC for ; Thu, 1 Jul 2021 01:49:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 898F361468; Thu, 1 Jul 2021 01:49:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104197; bh=PrsCtYpteRybHkrDOsVQ1CWUHS+LHfr/gXVJF8uRmgE=; h=Date:From:To:Subject:In-Reply-To:From; b=nkfHjxM9HqF76mxH2BlPxOuUksGYZnjpkB3RhSBSJHA5C0iBLfJTo74b54z/xahIP 7rc0YGqkAgpAsRYBe0KRUsdergoLgPCPmRka2Hiwi+M5No7UDlGO9Ips3WWOa5sWql GTgykIkG0qp1nS0ZkM64+cxc6CPNH3WNnNBPOnuo= Date: Wed, 30 Jun 2021 18:49:57 -0700 From: Andrew Morton To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, wangkefeng.wang@huawei.com, will@kernel.org Subject: [patch 052/192] mm/kconfig: move HOLES_IN_ZONE into mm Message-ID: <20210701014957.DhuivUJRL%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=nkfHjxM9; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: b6udh87diax6ewpwy95x7qjrre6dey9h X-Rspamd-Queue-Id: 76CC0F0000AC X-Rspamd-Server: rspam06 X-HE-Tag: 1625104198-505973 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Kefeng Wang Subject: mm/kconfig: move HOLES_IN_ZONE into mm commit a55749639dc1 ("ia64: drop marked broken DISCONTIGMEM and VIRTUAL_MEM_MAP") drop VIRTUAL_MEM_MAP, so there is no need HOLES_IN_ZONE on ia64. Also move HOLES_IN_ZONE into mm/Kconfig, select it if architecture needs this feature. Link: https://lkml.kernel.org/r/20210417075946.181402-1-wangkefeng.wang@huawei.com Signed-off-by: Kefeng Wang Acked-by: Catalin Marinas [arm64] Cc: Will Deacon Cc: Thomas Bogendoerfer Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 4 +--- arch/ia64/Kconfig | 3 --- arch/mips/Kconfig | 3 --- mm/Kconfig | 3 +++ 4 files changed, 4 insertions(+), 9 deletions(-) --- a/arch/arm64/Kconfig~mm-move-holes_in_zone-into-mm +++ a/arch/arm64/Kconfig @@ -201,6 +201,7 @@ config ARM64 select HAVE_KPROBES select HAVE_KRETPROBES select HAVE_GENERIC_VDSO + select HOLES_IN_ZONE select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING @@ -1052,9 +1053,6 @@ config NEED_PER_CPU_EMBED_FIRST_CHUNK def_bool y depends on NUMA -config HOLES_IN_ZONE - def_bool y - source "kernel/Kconfig.hz" config ARCH_SPARSEMEM_ENABLE --- a/arch/ia64/Kconfig~mm-move-holes_in_zone-into-mm +++ a/arch/ia64/Kconfig @@ -308,9 +308,6 @@ config NODES_SHIFT MAX_NUMNODES will be 2^(This value). If in doubt, use the default. -config HOLES_IN_ZONE - bool - config HAVE_ARCH_NODEDATA_EXTENSION def_bool y depends on NUMA --- a/arch/mips/Kconfig~mm-move-holes_in_zone-into-mm +++ a/arch/mips/Kconfig @@ -1233,9 +1233,6 @@ config HAVE_PLAT_MEMCPY config ISA_DMA_API bool -config HOLES_IN_ZONE - bool - config SYS_SUPPORTS_RELOCATABLE bool help --- a/mm/Kconfig~mm-move-holes_in_zone-into-mm +++ a/mm/Kconfig @@ -96,6 +96,9 @@ config HAVE_FAST_GUP depends on MMU bool +config HOLES_IN_ZONE + bool + # Don't discard allocated memory used to track "memory" and "reserved" memblocks # after early boot, so it can still be used to test for validity of memory. # Also, memblocks are updated with memory hot(un)plug. From patchwork Thu Jul 1 01:50:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2451AC11F66 for ; Thu, 1 Jul 2021 01:50:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D0DED61468 for ; Thu, 1 Jul 2021 01:50:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D0DED61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 57EB88D0208; Wed, 30 Jun 2021 21:50:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 52ECE8D0202; Wed, 30 Jun 2021 21:50:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F6D48D0208; Wed, 30 Jun 2021 21:50:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id 1C0258D0202 for ; Wed, 30 Jun 2021 21:50:02 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DFD551804654B for ; Thu, 1 Jul 2021 01:50:01 +0000 (UTC) X-FDA: 78312338202.20.67C420E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id 9742D6001A8C for ; Thu, 1 Jul 2021 01:50:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8633C6105A; Thu, 1 Jul 2021 01:50:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104200; bh=DN83jSjQlIGmXuJ3pDrx5I5PaeOcOqI2mGZ1P2n1rKs=; h=Date:From:To:Subject:In-Reply-To:From; b=jtua0N9PEjI/q4NDC0xozoZJJBNdhhRnavlf2tjom5XSOc3w2ryEa8weYWUqUAFq7 XODQnDsMF1iDu/4dzwTWCW9s2xN+DXAVLkyRpxmOv6jbWqJDNmteF+erAZj+zezG8R Qz2CcoKtYWhWHL2gBEmJqJ8dF280SxydyGVwKnOY= Date: Wed, 30 Jun 2021 18:50:00 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, corbet@lwn.net, eric.dumazet@gmail.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org Subject: [patch 053/192] docs: proc.rst: meminfo: briefly describe gaps in memory accounting Message-ID: <20210701015000.a7W7wztCi%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=jtua0N9P; dmarc=none; spf=pass (imf10.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: kzqgsiiwg5bgurc79tmiat3bg7ynj5gd X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9742D6001A8C X-HE-Tag: 1625104201-923535 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport Subject: docs: proc.rst: meminfo: briefly describe gaps in memory accounting Add a paragraph that explains that it may happen that the counters in /proc/meminfo do not add up to the overall memory usage. Link: https://lkml.kernel.org/r/20210421061127.1182723-1-rppt@kernel.org Signed-off-by: Mike Rapoport Acked-by: Michal Hocko Reviewed-by: Matthew Wilcox (Oracle) Acked-by: Vlastimil Babka Cc: Jonathan Corbet Cc: Alexey Dobriyan Cc: Eric Dumazet Signed-off-by: Andrew Morton --- Documentation/filesystems/proc.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) --- a/Documentation/filesystems/proc.rst~docs-procrst-meminfo-briefly-describe-gaps-in-memory-accounting +++ a/Documentation/filesystems/proc.rst @@ -933,8 +933,15 @@ meminfo ~~~~~~~ Provides information about distribution and utilization of memory. This -varies by architecture and compile options. The following is from a -16GB PIII, which has highmem enabled. You may not have all of these fields. +varies by architecture and compile options. Some of the counters reported +here overlap. The memory reported by the non overlapping counters may not +add up to the overall memory usage and the difference for some workloads +can be substantial. In many cases there are other means to find out +additional memory using subsystem specific interfaces, for instance +/proc/net/sockstat for TCP memory allocations. + +The following is from a 16GB PIII, which has highmem enabled. +You may not have all of these fields. :: From patchwork Thu Jul 1 01:50:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D7A97C11F6A for ; Thu, 1 Jul 2021 01:50:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9049E61241 for ; Thu, 1 Jul 2021 01:50:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9049E61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 15F488D0209; Wed, 30 Jun 2021 21:50:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 10FD98D0202; Wed, 30 Jun 2021 21:50:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F19FE8D0209; Wed, 30 Jun 2021 21:50:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0145.hostedemail.com [216.40.44.145]) by kanga.kvack.org (Postfix) with ESMTP id CA5248D0202 for ; Wed, 30 Jun 2021 21:50:05 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A0000181AEF3F for ; Thu, 1 Jul 2021 01:50:05 +0000 (UTC) X-FDA: 78312338370.26.BF2EBFA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 4762B6001AA5 for ; Thu, 1 Jul 2021 01:50:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D626C61468; Thu, 1 Jul 2021 01:50:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104204; bh=l0/S+LLQunTS3Zar5n1r2cgGpT8Ax0M5iwOAkUwL3XY=; h=Date:From:To:Subject:In-Reply-To:From; b=sqv76ERwmL46uFLB/6ugHWrV0ac+cG6uzVl5V5N66qFQG0yVXftKIkEmZhiuYWv6v bdDyuxQdBuNpZMLH9tuNO1ob3agu2TmtleNhGtwKhuzU4CxojtrAYs64fx4+Nxhd32 YD2CmtZ5iRCjJfE4Y/JQ8adLXefAcXPSU06GJt3E= Date: Wed, 30 Jun 2021 18:50:03 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, alex.shi@linux.alibaba.com, david@redhat.com, guro@fb.com, haiyangz@microsoft.com, jasowang@redhat.com, jbohac@suse.cz, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mst@redhat.com, naoya.horiguchi@nec.com, osalvador@suse.de, rppt@linux.ibm.com, steven.price@arm.com, sthemmin@microsoft.com, torvalds@linux-foundation.org, wei.liu@kernel.org, willy@infradead.org, yaoaili@kingsoft.com Subject: [patch 054/192] fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER Message-ID: <20210701015003.2E6L6dEh3%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=sqv76ERw; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: z7dg3zpic8fgyzsgw4i1wjxkt11t9zir X-Rspamd-Queue-Id: 4762B6001AA5 X-HE-Tag: 1625104205-435049 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: fs/proc/kcore: drop KCORE_REMAP and KCORE_OTHER Patch series "fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages", v3. Looking for places where the kernel might unconditionally read PageOffline() pages, I stumbled over /proc/kcore; turns out /proc/kcore needs some more love to not touch some other pages we really don't want to read -- i.e., hwpoisoned ones. Examples for PageOffline() pages are pages inflated in a balloon, memory unplugged via virtio-mem, and partially-present sections in memory added by the Hyper-V balloon. When reading pages inflated in a balloon, we essentially produce unnecessary load in the hypervisor; holes in partially present sections in case of Hyper-V are not accessible and already were a problem for /proc/vmcore, fixed in makedumpfile by detecting PageOffline() pages. In the future, virtio-mem might disallow reading unplugged memory -- marked as PageOffline() -- in some environments, resulting in undefined behavior when accessed; therefore, I'm trying to identify and rework all these (corner) cases. With this series, there is really only access via /dev/mem, /proc/vmcore and kdb left after I ripped out /dev/kmem. kdb is an advanced corner-case use case -- we won't care for now if someone explicitly tries to do nasty things by reading from/writing to physical addresses we better not touch. /dev/mem is a use case we won't support for virtio-mem, at least for now, so we'll simply disallow mapping any virtio-mem memory via /dev/mem next. /proc/vmcore is really only a problem when dumping the old kernel via something that's not makedumpfile (read: basically never), however, we'll try sanitizing that as well in the second kernel in the future. Tested via kcore_dump: https://github.com/schlafwandler/kcore_dump This patch (of 6): Commit db779ef67ffe ("proc/kcore: Remove unused kclist_add_remap()") removed the last user of KCORE_REMAP. Commit 595dd46ebfc1 ("vfs/proc/kcore, x86/mm/kcore: Fix SMAP fault when dumping vsyscall user page") removed the last user of KCORE_OTHER. Let's drop both types. While at it, also drop vaddr in "struct kcore_list", used by KCORE_REMAP only. Link: https://lkml.kernel.org/r/20210526093041.8800-1-david@redhat.com Link: https://lkml.kernel.org/r/20210526093041.8800-2-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Alexey Dobriyan Cc: "Matthew Wilcox (Oracle)" Cc: Oscar Salvador Cc: Michal Hocko Cc: Roman Gushchin Cc: Alex Shi Cc: Steven Price Cc: Mike Kravetz Cc: Aili Yao Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Naoya Horiguchi Signed-off-by: Andrew Morton --- fs/proc/kcore.c | 7 ++----- include/linux/kcore.h | 3 --- 2 files changed, 2 insertions(+), 8 deletions(-) --- a/fs/proc/kcore.c~fs-proc-kcore-drop-kcore_remap-and-kcore_other +++ a/fs/proc/kcore.c @@ -380,11 +380,8 @@ read_kcore(struct file *file, char __use phdr->p_type = PT_LOAD; phdr->p_flags = PF_R | PF_W | PF_X; phdr->p_offset = kc_vaddr_to_offset(m->addr) + data_offset; - if (m->type == KCORE_REMAP) - phdr->p_vaddr = (size_t)m->vaddr; - else - phdr->p_vaddr = (size_t)m->addr; - if (m->type == KCORE_RAM || m->type == KCORE_REMAP) + phdr->p_vaddr = (size_t)m->addr; + if (m->type == KCORE_RAM) phdr->p_paddr = __pa(m->addr); else if (m->type == KCORE_TEXT) phdr->p_paddr = __pa_symbol(m->addr); --- a/include/linux/kcore.h~fs-proc-kcore-drop-kcore_remap-and-kcore_other +++ a/include/linux/kcore.h @@ -11,14 +11,11 @@ enum kcore_type { KCORE_RAM, KCORE_VMEMMAP, KCORE_USER, - KCORE_OTHER, - KCORE_REMAP, }; struct kcore_list { struct list_head list; unsigned long addr; - unsigned long vaddr; size_t size; int type; }; From patchwork Thu Jul 1 01:50:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353169 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82BA8C11F6A for ; Thu, 1 Jul 2021 01:50:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3728361474 for ; Thu, 1 Jul 2021 01:50:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3728361474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD31D8D020B; Wed, 30 Jun 2021 21:50:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE48F8D0202; Wed, 30 Jun 2021 21:50:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A87528D020B; Wed, 30 Jun 2021 21:50:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0075.hostedemail.com [216.40.44.75]) by kanga.kvack.org (Postfix) with ESMTP id 800608D0202 for ; Wed, 30 Jun 2021 21:50:09 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 58E60181AEF3F for ; Thu, 1 Jul 2021 01:50:09 +0000 (UTC) X-FDA: 78312338538.21.76B962C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id 04909B0000A5 for ; Thu, 1 Jul 2021 01:50:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DB7F6135C; Thu, 1 Jul 2021 01:50:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104208; bh=X0y+MmNBinFDW/Wm+TmdoRMm+GG9SaimXMSOqHWddiA=; h=Date:From:To:Subject:In-Reply-To:From; b=sBgZBLqBT9CLTHNqZyZn0ZgnoWqT19cSOB0+Ii7cOx+x+AMsMFZDCztUVokscM+cu N43ujBBqYFhBY6e9RJxOdIBzsK/jHfchuMd0viP41RTf2ExUufxa0h8B4rQQMHtFFg Qu/o+uqrOckvnrZrQ46txg7wFT2kfvPChJwCG8IA= Date: Wed, 30 Jun 2021 18:50:07 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, alex.shi@linux.alibaba.com, david@redhat.com, guro@fb.com, haiyangz@microsoft.com, jasowang@redhat.com, jbohac@suse.cz, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mst@redhat.com, naoya.horiguchi@nec.com, osalvador@suse.de, rppt@linux.ibm.com, steven.price@arm.com, sthemmin@microsoft.com, torvalds@linux-foundation.org, wei.liu@kernel.org, willy@infradead.org, yaoaili@kingsoft.com Subject: [patch 055/192] fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM Message-ID: <20210701015007.IRGAO9pLj%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=sBgZBLqB; spf=pass (imf19.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 5ufohse77nwkwt45ns95rrc6oxhku8px X-Rspamd-Queue-Id: 04909B0000A5 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104208-290936 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: fs/proc/kcore: pfn_is_ram check only applies to KCORE_RAM Let's resturcture the code, using switch-case, and checking pfn_is_ram() only when we are dealing with KCORE_RAM. Link: https://lkml.kernel.org/r/20210526093041.8800-3-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport Cc: Aili Yao Cc: Alexey Dobriyan Cc: Alex Shi Cc: Haiyang Zhang Cc: Jason Wang Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: "Matthew Wilcox (Oracle)" Cc: "Michael S. Tsirkin" Cc: Michal Hocko Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Oscar Salvador Cc: Roman Gushchin Cc: Stephen Hemminger Cc: Steven Price Cc: Wei Liu Signed-off-by: Andrew Morton --- fs/proc/kcore.c | 35 +++++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-) --- a/fs/proc/kcore.c~fs-proc-kcore-pfn_is_ram-check-only-applies-to-kcore_ram +++ a/fs/proc/kcore.c @@ -483,25 +483,36 @@ read_kcore(struct file *file, char __use goto out; } m = NULL; /* skip the list anchor */ - } else if (!pfn_is_ram(__pa(start) >> PAGE_SHIFT)) { - if (clear_user(buffer, tsz)) { - ret = -EFAULT; - goto out; - } - } else if (m->type == KCORE_VMALLOC) { + goto skip; + } + + switch (m->type) { + case KCORE_VMALLOC: vread(buf, (char *)start, tsz); /* we have to zero-fill user buffer even if no read */ if (copy_to_user(buffer, buf, tsz)) { ret = -EFAULT; goto out; } - } else if (m->type == KCORE_USER) { + break; + case KCORE_USER: /* User page is handled prior to normal kernel page: */ if (copy_to_user(buffer, (char *)start, tsz)) { ret = -EFAULT; goto out; } - } else { + break; + case KCORE_RAM: + if (!pfn_is_ram(__pa(start) >> PAGE_SHIFT)) { + if (clear_user(buffer, tsz)) { + ret = -EFAULT; + goto out; + } + break; + } + fallthrough; + case KCORE_VMEMMAP: + case KCORE_TEXT: if (kern_addr_valid(start)) { /* * Using bounce buffer to bypass the @@ -525,7 +536,15 @@ read_kcore(struct file *file, char __use goto out; } } + break; + default: + pr_warn_once("Unhandled KCORE type: %d\n", m->type); + if (clear_user(buffer, tsz)) { + ret = -EFAULT; + goto out; + } } +skip: buflen -= tsz; *fpos += tsz; buffer += tsz; From patchwork Thu Jul 1 01:50:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353171 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 414EAC11F6A for ; Thu, 1 Jul 2021 01:50:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E6C0761469 for ; Thu, 1 Jul 2021 01:50:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6C0761469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 68FD58D020C; Wed, 30 Jun 2021 21:50:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6406A8D0202; Wed, 30 Jun 2021 21:50:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E1B08D020C; Wed, 30 Jun 2021 21:50:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id 2A89F8D0202 for ; Wed, 30 Jun 2021 21:50:13 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id F25A0252D7 for ; Thu, 1 Jul 2021 01:50:12 +0000 (UTC) X-FDA: 78312338664.05.07E4FCC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 8A8A0900032B for ; Thu, 1 Jul 2021 01:50:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 367606105A; Thu, 1 Jul 2021 01:50:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104211; bh=U7q2SNv2XL2VMJUK6O9N7koxRTuh+HEVxidHJUWSZ/0=; h=Date:From:To:Subject:In-Reply-To:From; b=07c5W6ajiBmdBcwW/gtK3IZ3rvlvX20A+5CQKPXOY7J8E7HvfAZzvkvoJpd6BeLIL 68zF5v3jMSjDjqlCG3g3za3FpRVrggyKB5QYlPzMa4QAy+xdjcBoiKv5bDf1F6mNrN e83bgBs5gJ2BVYuLXyN/kbX5dhBTvABNgKUBNHIo= Date: Wed, 30 Jun 2021 18:50:10 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, alex.shi@linux.alibaba.com, david@redhat.com, guro@fb.com, haiyangz@microsoft.com, jasowang@redhat.com, jbohac@suse.cz, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mst@redhat.com, naoya.horiguchi@nec.com, osalvador@suse.de, rppt@linux.ibm.com, steven.price@arm.com, sthemmin@microsoft.com, torvalds@linux-foundation.org, wei.liu@kernel.org, willy@infradead.org, yaoaili@kingsoft.com Subject: [patch 056/192] fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages Message-ID: <20210701015010.PMff3kM77%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8A8A0900032B Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=07c5W6aj; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: f3isoj96nx8km1mm5x9jkssh8fegk4eh X-HE-Tag: 1625104212-697753 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: fs/proc/kcore: don't read offline sections, logically offline pages and hwpoisoned pages Let's avoid reading: 1) Offline memory sections: the content of offline memory sections is stale as the memory is effectively unused by the kernel. On s390x with standby memory, offline memory sections (belonging to offline storage increments) are not accessible. With virtio-mem and the hyper-v balloon, we can have unavailable memory chunks that should not be accessed inside offline memory sections. Last but not least, offline memory sections might contain hwpoisoned pages which we can no longer identify because the memmap is stale. 2) PG_offline pages: logically offline pages that are documented as "The content of these pages is effectively stale. Such pages should not be touched (read/write/dump/save) except by their owner.". Examples include pages inflated in a balloon or unavailble memory ranges inside hotplugged memory sections with virtio-mem or the hyper-v balloon. 3) PG_hwpoison pages: Reading pages marked as hwpoisoned can be fatal. As documented: "Accessing is not safe since it may cause another machine check. Don't touch!" Introduce is_page_hwpoison(), adding a comment that it is inherently racy but best we can really do. Reading /proc/kcore now performs similar checks as when reading /proc/vmcore for kdump via makedumpfile: problematic pages are exclude. It's also similar to hibernation code, however, we don't skip hwpoisoned pages when processing pages in kernel/power/snapshot.c:saveable_page() yet. Note 1: we can race against memory offlining code, especially memory going offline and getting unplugged: however, we will properly tear down the identity mapping and handle faults gracefully when accessing this memory from kcore code. Note 2: we can race against drivers setting PageOffline() and turning memory inaccessible in the hypervisor. We'll handle this in a follow-up patch. Link: https://lkml.kernel.org/r/20210526093041.8800-4-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport Reviewed-by: Oscar Salvador Cc: Aili Yao Cc: Alexey Dobriyan Cc: Alex Shi Cc: Haiyang Zhang Cc: Jason Wang Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: "Matthew Wilcox (Oracle)" Cc: "Michael S. Tsirkin" Cc: Michal Hocko Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Roman Gushchin Cc: Stephen Hemminger Cc: Steven Price Cc: Wei Liu Signed-off-by: Andrew Morton --- fs/proc/kcore.c | 14 +++++++++++++- include/linux/page-flags.h | 12 ++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) --- a/fs/proc/kcore.c~fs-proc-kcore-dont-read-offline-sections-logically-offline-pages-and-hwpoisoned-pages +++ a/fs/proc/kcore.c @@ -465,6 +465,9 @@ read_kcore(struct file *file, char __use m = NULL; while (buflen) { + struct page *page; + unsigned long pfn; + /* * If this is the first iteration or the address is not within * the previous entry, search for a matching entry. @@ -503,7 +506,16 @@ read_kcore(struct file *file, char __use } break; case KCORE_RAM: - if (!pfn_is_ram(__pa(start) >> PAGE_SHIFT)) { + pfn = __pa(start) >> PAGE_SHIFT; + page = pfn_to_online_page(pfn); + + /* + * Don't read offline sections, logically offline pages + * (e.g., inflated in a balloon), hwpoisoned pages, + * and explicitly excluded physical ranges. + */ + if (!page || PageOffline(page) || + is_page_hwpoison(page) || !pfn_is_ram(pfn)) { if (clear_user(buffer, tsz)) { ret = -EFAULT; goto out; --- a/include/linux/page-flags.h~fs-proc-kcore-dont-read-offline-sections-logically-offline-pages-and-hwpoisoned-pages +++ a/include/linux/page-flags.h @@ -695,6 +695,18 @@ PAGEFLAG_FALSE(DoubleMap) #endif /* + * Check if a page is currently marked HWPoisoned. Note that this check is + * best effort only and inherently racy: there is no way to synchronize with + * failing hardware. + */ +static inline bool is_page_hwpoison(struct page *page) +{ + if (PageHWPoison(page)) + return true; + return PageHuge(page) && PageHWPoison(compound_head(page)); +} + +/* * For pages that are never mapped to userspace (and aren't PageSlab), * page_type may be used. Because it is initialised to -1, we invert the * sense of the bit, so __SetPageFoo *clears* the bit used for PageFoo, and From patchwork Thu Jul 1 01:50:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6561C11F69 for ; Thu, 1 Jul 2021 01:50:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8E88561474 for ; Thu, 1 Jul 2021 01:50:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8E88561474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 15F9F8D020D; Wed, 30 Jun 2021 21:50:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 137218D0202; Wed, 30 Jun 2021 21:50:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F19988D020D; Wed, 30 Jun 2021 21:50:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id CF6C38D0202 for ; Wed, 30 Jun 2021 21:50:16 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A08CF8249980 for ; Thu, 1 Jul 2021 01:50:16 +0000 (UTC) X-FDA: 78312338832.17.13789DF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 3A004801AB06 for ; Thu, 1 Jul 2021 01:50:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CBD4361469; Thu, 1 Jul 2021 01:50:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104215; bh=kXAnhwZPo/GQsexU8Xv3wlLS01pcQsizPtnTqNthHxk=; h=Date:From:To:Subject:In-Reply-To:From; b=ERWrvNB85R3RrAZnRKzYbgKnXr1Ql92N/00QiUJMqJG1G3nRELtdRx4NnP5WO6H66 2SqTn32FL4EmYIt7JX4CcoRfqJU93KzkdCBkxj5Ora++1d7YdAeAjiGQXalrfHX+o/ /Z2lG8w01sWHo+KSDD2OMOnw1NL1/xOBSuXUAVCk= Date: Wed, 30 Jun 2021 18:50:14 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, alex.shi@linux.alibaba.com, david@redhat.com, guro@fb.com, haiyangz@microsoft.com, jasowang@redhat.com, jbohac@suse.cz, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mst@redhat.com, naoya.horiguchi@nec.com, osalvador@suse.de, rppt@linux.ibm.com, steven.price@arm.com, sthemmin@microsoft.com, torvalds@linux-foundation.org, wei.liu@kernel.org, willy@infradead.org, yaoaili@kingsoft.com Subject: [patch 057/192] mm: introduce page_offline_(begin|end|freeze|thaw) to synchronize setting PageOffline() Message-ID: <20210701015014.kKUZ9uEOa%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ERWrvNB8; dmarc=none; spf=pass (imf06.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: gitw4mgwpcpnmco8u6dk4qjzm9inp15q X-Rspamd-Queue-Id: 3A004801AB06 X-HE-Tag: 1625104216-844737 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm: introduce page_offline_(begin|end|freeze|thaw) to synchronize setting PageOffline() A driver might set a page logically offline -- PageOffline() -- and turn the page inaccessible in the hypervisor; after that, access to page content can be fatal. One example is virtio-mem; while unplugged memory -- marked as PageOffline() can currently be read in the hypervisor, this will no longer be the case in the future; for example, when having a virtio-mem device backed by huge pages in the hypervisor. Some special PFN walkers -- i.e., /proc/kcore -- read content of random pages after checking PageOffline(); however, these PFN walkers can race with drivers that set PageOffline(). Let's introduce page_offline_(begin|end|freeze|thaw) for synchronizing. page_offline_freeze()/page_offline_thaw() allows for a subsystem to synchronize with such drivers, achieving that a page cannot be set PageOffline() while frozen. page_offline_begin()/page_offline_end() is used by drivers that care about such races when setting a page PageOffline(). For simplicity, use a rwsem for now; neither drivers nor users are performance sensitive. Link: https://lkml.kernel.org/r/20210526093041.8800-5-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Michal Hocko Reviewed-by: Mike Rapoport Reviewed-by: Oscar Salvador Cc: Aili Yao Cc: Alexey Dobriyan Cc: Alex Shi Cc: Haiyang Zhang Cc: Jason Wang Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: "Matthew Wilcox (Oracle)" Cc: "Michael S. Tsirkin" Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Roman Gushchin Cc: Stephen Hemminger Cc: Steven Price Cc: Wei Liu Signed-off-by: Andrew Morton --- include/linux/page-flags.h | 10 ++++++++ mm/util.c | 40 +++++++++++++++++++++++++++++++++++ 2 files changed, 50 insertions(+) --- a/include/linux/page-flags.h~mm-introduce-page_offline_beginendfreezethaw-to-synchronize-setting-pageoffline +++ a/include/linux/page-flags.h @@ -769,9 +769,19 @@ PAGE_TYPE_OPS(Buddy, buddy) * relies on this feature is aware that re-onlining the memory block will * require to re-set the pages PageOffline() and not giving them to the * buddy via online_page_callback_t. + * + * There are drivers that mark a page PageOffline() and expect there won't be + * any further access to page content. PFN walkers that read content of random + * pages should check PageOffline() and synchronize with such drivers using + * page_offline_freeze()/page_offline_thaw(). */ PAGE_TYPE_OPS(Offline, offline) +extern void page_offline_freeze(void); +extern void page_offline_thaw(void); +extern void page_offline_begin(void); +extern void page_offline_end(void); + /* * Marks pages in use as page tables. */ --- a/mm/util.c~mm-introduce-page_offline_beginendfreezethaw-to-synchronize-setting-pageoffline +++ a/mm/util.c @@ -1010,3 +1010,43 @@ void mem_dump_obj(void *object) } EXPORT_SYMBOL_GPL(mem_dump_obj); #endif + +/* + * A driver might set a page logically offline -- PageOffline() -- and + * turn the page inaccessible in the hypervisor; after that, access to page + * content can be fatal. + * + * Some special PFN walkers -- i.e., /proc/kcore -- read content of random + * pages after checking PageOffline(); however, these PFN walkers can race + * with drivers that set PageOffline(). + * + * page_offline_freeze()/page_offline_thaw() allows for a subsystem to + * synchronize with such drivers, achieving that a page cannot be set + * PageOffline() while frozen. + * + * page_offline_begin()/page_offline_end() is used by drivers that care about + * such races when setting a page PageOffline(). + */ +static DECLARE_RWSEM(page_offline_rwsem); + +void page_offline_freeze(void) +{ + down_read(&page_offline_rwsem); +} + +void page_offline_thaw(void) +{ + up_read(&page_offline_rwsem); +} + +void page_offline_begin(void) +{ + down_write(&page_offline_rwsem); +} +EXPORT_SYMBOL(page_offline_begin); + +void page_offline_end(void) +{ + up_write(&page_offline_rwsem); +} +EXPORT_SYMBOL(page_offline_end); From patchwork Thu Jul 1 01:50:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 619DAC11F69 for ; Thu, 1 Jul 2021 01:50:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 157FC61241 for ; Thu, 1 Jul 2021 01:50:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 157FC61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 900598D020E; Wed, 30 Jun 2021 21:50:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8620D8D0202; Wed, 30 Jun 2021 21:50:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 72A618D020E; Wed, 30 Jun 2021 21:50:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0076.hostedemail.com [216.40.44.76]) by kanga.kvack.org (Postfix) with ESMTP id 4F0758D0202 for ; Wed, 30 Jun 2021 21:50:20 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1ED45252D7 for ; Thu, 1 Jul 2021 01:50:20 +0000 (UTC) X-FDA: 78312339000.23.4C1FF54 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id D0ACAF0000B0 for ; Thu, 1 Jul 2021 01:50:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 63C366147D; Thu, 1 Jul 2021 01:50:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104219; bh=QeVHaLFTs9qKXUbknKXET0E5BhAvOeThhD4tDDWPTUU=; h=Date:From:To:Subject:In-Reply-To:From; b=PHz5T58UpENwTsa9qdnQgp0T/SHro782+7pp6u8Be9Q+Bs+T0QFro43Y7ysikKJR1 yqv/Nb5sPv/9MHQsKloJTYW3JcMttk5I0sUuLZsygtQfIxJdZ7Gj7NitQe+1LuWAzC DhbwzqjXi1eDAhtVRBbUgyKxgjAqMECGlPf6rnLs= Date: Wed, 30 Jun 2021 18:50:18 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, alex.shi@linux.alibaba.com, david@redhat.com, guro@fb.com, haiyangz@microsoft.com, jasowang@redhat.com, jbohac@suse.cz, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mst@redhat.com, naoya.horiguchi@nec.com, osalvador@suse.de, rppt@linux.ibm.com, steven.price@arm.com, sthemmin@microsoft.com, torvalds@linux-foundation.org, wei.liu@kernel.org, willy@infradead.org, yaoaili@kingsoft.com Subject: [patch 058/192] virtio-mem: use page_offline_(start|end) when setting PageOffline() Message-ID: <20210701015018.qXPlnv3P5%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=PHz5T58U; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: ah6b8uesng76tcak6a9kwg6a5a9c5ez9 X-Rspamd-Queue-Id: D0ACAF0000B0 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104219-339265 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: virtio-mem: use page_offline_(start|end) when setting PageOffline() Let's properly use page_offline_(start|end) to synchronize setting PageOffline(), so we won't have valid page access to unplugged memory regions from /proc/kcore. Existing balloon implementations usually allow reading inflated memory; doing so might result in unnecessary overhead in the hypervisor, which is currently the case with virtio-mem. For future virtio-mem use cases, it will be different when using shmem, huge pages, !anonymous private mappings, ... as backing storage for a VM. virtio-mem unplugged memory must no longer be accessed and access might result in undefined behavior. There will be a virtio spec extension to document this change, including a new feature flag indicating the changed behavior. We really don't want to race against PFN walkers reading random page content. Link: https://lkml.kernel.org/r/20210526093041.8800-6-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Michael S. Tsirkin Acked-by: Mike Rapoport Reviewed-by: Oscar Salvador Cc: Aili Yao Cc: Alexey Dobriyan Cc: Alex Shi Cc: Haiyang Zhang Cc: Jason Wang Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: "Matthew Wilcox (Oracle)" Cc: Michal Hocko Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Roman Gushchin Cc: Stephen Hemminger Cc: Steven Price Cc: Wei Liu Signed-off-by: Andrew Morton --- drivers/virtio/virtio_mem.c | 2 ++ 1 file changed, 2 insertions(+) --- a/drivers/virtio/virtio_mem.c~virtio-mem-use-page_offline_startend-when-setting-pageoffline +++ a/drivers/virtio/virtio_mem.c @@ -1065,6 +1065,7 @@ static int virtio_mem_memory_notifier_cb static void virtio_mem_set_fake_offline(unsigned long pfn, unsigned long nr_pages, bool onlined) { + page_offline_begin(); for (; nr_pages--; pfn++) { struct page *page = pfn_to_page(pfn); @@ -1075,6 +1076,7 @@ static void virtio_mem_set_fake_offline( ClearPageReserved(page); } } + page_offline_end(); } /* From patchwork Thu Jul 1 01:50:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCCDFC11F69 for ; Thu, 1 Jul 2021 01:50:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7534C6105A for ; Thu, 1 Jul 2021 01:50:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7534C6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F1B338D020F; Wed, 30 Jun 2021 21:50:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ECABC8D0202; Wed, 30 Jun 2021 21:50:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D92B98D020F; Wed, 30 Jun 2021 21:50:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id B745E8D0202 for ; Wed, 30 Jun 2021 21:50:23 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 969A4252D7 for ; Thu, 1 Jul 2021 01:50:23 +0000 (UTC) X-FDA: 78312339126.37.B5E3AD3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 5FB6610004CD for ; Thu, 1 Jul 2021 01:50:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EA0AE61468; Thu, 1 Jul 2021 01:50:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104222; bh=1at29HZqa4xcbE0QkL8RimpRBMUAwriLfuBiCNL1g4M=; h=Date:From:To:Subject:In-Reply-To:From; b=Ny7HHTjs78yCRkIbWdb56tiilMMclpPAsiw9Pz3tLqdf95ZiLU/3dhYmJp5txR0lm yXu0aoF5RlNGL1Lrinzk4vQ8ngnAmmHzgXFP1F1H2nI3tBVZb2zFMmD1fLl4YZzxSu GL0xsrId7IhUnAaTFLVWrmEevTHIZSTGcd3/pIDY= Date: Wed, 30 Jun 2021 18:50:21 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, alex.shi@linux.alibaba.com, david@redhat.com, guro@fb.com, haiyangz@microsoft.com, jasowang@redhat.com, jbohac@suse.cz, kys@microsoft.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, mst@redhat.com, naoya.horiguchi@nec.com, osalvador@suse.de, rppt@linux.ibm.com, steven.price@arm.com, sthemmin@microsoft.com, torvalds@linux-foundation.org, wei.liu@kernel.org, willy@infradead.org, yaoaili@kingsoft.com Subject: [patch 059/192] fs/proc/kcore: use page_offline_(freeze|thaw) Message-ID: <20210701015021.y4sXmyoGE%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5FB6610004CD X-Stat-Signature: 6p5yk3o6qmk8p6u9btdmewbah59n74z5 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Ny7HHTjs; dmarc=none; spf=pass (imf07.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104223-723301 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: fs/proc/kcore: use page_offline_(freeze|thaw) Let's properly synchronize with drivers that set PageOffline(). Unfreeze/thaw every now and then, so drivers that want to set PageOffline() can make progress. Link: https://lkml.kernel.org/r/20210526093041.8800-7-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Mike Rapoport Reviewed-by: Oscar Salvador Cc: Aili Yao Cc: Alexey Dobriyan Cc: Alex Shi Cc: Haiyang Zhang Cc: Jason Wang Cc: Jiri Bohac Cc: "K. Y. Srinivasan" Cc: "Matthew Wilcox (Oracle)" Cc: "Michael S. Tsirkin" Cc: Michal Hocko Cc: Mike Kravetz Cc: Naoya Horiguchi Cc: Roman Gushchin Cc: Stephen Hemminger Cc: Steven Price Cc: Wei Liu Signed-off-by: Andrew Morton --- fs/proc/kcore.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/fs/proc/kcore.c~fs-proc-kcore-use-page_offline_freezethaw +++ a/fs/proc/kcore.c @@ -313,6 +313,7 @@ read_kcore(struct file *file, char __use { char *buf = file->private_data; size_t phdrs_offset, notes_offset, data_offset; + size_t page_offline_frozen = 1; size_t phdrs_len, notes_len; struct kcore_list *m; size_t tsz; @@ -322,6 +323,11 @@ read_kcore(struct file *file, char __use int ret = 0; down_read(&kclist_lock); + /* + * Don't race against drivers that set PageOffline() and expect no + * further page access. + */ + page_offline_freeze(); get_kcore_size(&nphdr, &phdrs_len, ¬es_len, &data_offset); phdrs_offset = sizeof(struct elfhdr); @@ -480,6 +486,12 @@ read_kcore(struct file *file, char __use } } + if (page_offline_frozen++ % MAX_ORDER_NR_PAGES == 0) { + page_offline_thaw(); + cond_resched(); + page_offline_freeze(); + } + if (&m->list == &kclist_head) { if (clear_user(buffer, tsz)) { ret = -EFAULT; @@ -565,6 +577,7 @@ skip: } out: + page_offline_thaw(); up_read(&kclist_lock); if (ret) return ret; From patchwork Thu Jul 1 01:50:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB2E8C11F68 for ; Thu, 1 Jul 2021 01:50:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6D9886135C for ; Thu, 1 Jul 2021 01:50:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D9886135C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EA6AE8D0210; Wed, 30 Jun 2021 21:50:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E558F8D0202; Wed, 30 Jun 2021 21:50:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1D798D0210; Wed, 30 Jun 2021 21:50:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id AEC638D0202 for ; Wed, 30 Jun 2021 21:50:26 -0400 (EDT) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 820B182E514D for ; Thu, 1 Jul 2021 01:50:26 +0000 (UTC) X-FDA: 78312339252.04.56F9703 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 348F010000A2 for ; Thu, 1 Jul 2021 01:50:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 539BE61241; Thu, 1 Jul 2021 01:50:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104225; bh=f+6Qn2B22xd5M7NPCeDNlxnSj2n14tkoCR1HN9n0za8=; h=Date:From:To:Subject:In-Reply-To:From; b=e13PMWw4TLg7xS/nuJ6/O1nMtRt2MItASwTkLS1S4LwQxMqQJX3/fO/wpyW3tYgCT xBThqp+mzD7Mu1ecRxHqerbEZBKeksYH85LWqgCUOJ4w3uYV6dKRXqcbH6Y6jZcEx6 uaxq2yw3AU17D3EQgtIQawHTJeAOiY83/ont+roo= Date: Wed, 30 Jun 2021 18:50:24 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hdanton@sina.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 060/192] mm/z3fold: define macro NCHUNKS as TOTAL_CHUNKS - ZHDR_CHUNKS Message-ID: <20210701015024.4W8zL1V8o%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 348F010000A2 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=e13PMWw4; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: rg3tcr1o78nk65h7etipgubbpfk5ou4b X-HE-Tag: 1625104226-742393 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/z3fold: define macro NCHUNKS as TOTAL_CHUNKS - ZHDR_CHUNKS Patch series "Cleanup and fixup for z3fold". This series contains cleanups to remove unused function, redefine macro to improve readability and so on. Also this fixes several bugs in z3fold, such as memory leak in z3fold_destroy_pool(). More details can be found in the respective changelogs. This patch (of 6): To improve code readability, we could define macro NCHUNKS as TOTAL_CHUNKS - ZHDR_CHUNKS. No functional change intended. Link: https://lkml.kernel.org/r/20210619093151.1492174-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210619093151.1492174-2-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool Cc: Hillf Danton Signed-off-by: Andrew Morton --- mm/z3fold.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/z3fold.c~mm-z3fold-define-macro-nchunks-as-total_chunks-zhdr_chunks +++ a/mm/z3fold.c @@ -62,7 +62,7 @@ #define ZHDR_SIZE_ALIGNED round_up(sizeof(struct z3fold_header), CHUNK_SIZE) #define ZHDR_CHUNKS (ZHDR_SIZE_ALIGNED >> CHUNK_SHIFT) #define TOTAL_CHUNKS (PAGE_SIZE >> CHUNK_SHIFT) -#define NCHUNKS ((PAGE_SIZE - ZHDR_SIZE_ALIGNED) >> CHUNK_SHIFT) +#define NCHUNKS (TOTAL_CHUNKS - ZHDR_CHUNKS) #define BUDDY_MASK (0x3) #define BUDDY_SHIFT 2 From patchwork Thu Jul 1 01:50:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DEC9C11F69 for ; Thu, 1 Jul 2021 01:50:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4FC2661241 for ; Thu, 1 Jul 2021 01:50:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4FC2661241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD1238D0211; Wed, 30 Jun 2021 21:50:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C807D8D0202; Wed, 30 Jun 2021 21:50:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B22298D0211; Wed, 30 Jun 2021 21:50:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 8D0E28D0202 for ; Wed, 30 Jun 2021 21:50:29 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 61F8882E514D for ; Thu, 1 Jul 2021 01:50:29 +0000 (UTC) X-FDA: 78312339378.33.F10C6DD Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 2FB621984 for ; Thu, 1 Jul 2021 01:50:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 36C3F6105A; Thu, 1 Jul 2021 01:50:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104228; bh=pUNghmS3UghhRIbrzL3KTLY8WvTwvWERnFSE/dvBWbY=; h=Date:From:To:Subject:In-Reply-To:From; b=lvTlrjZlu83aCpxLHWiVTgvMRILS6sdL4YojRZ86xdZjZuFyMtlB3fuDRWAafLjfW BiseYcKapK3HCC8i6jGRILaPBz2JuX1jeXFDqqy3XxTDB5/npS7yaQEu4GnuzDqwV5 dJhmh2u1qhGVoiXTAP2WrneBAcLmoXvAA5OIfa9U= Date: Wed, 30 Jun 2021 18:50:27 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hdanton@sina.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 061/192] mm/z3fold: avoid possible underflow in z3fold_alloc() Message-ID: <20210701015027.60S0CzrHj%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lvTlrjZl; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2FB621984 X-Stat-Signature: i8w7q69ywkxfsg9axuyw35pd9sisomw8 X-HE-Tag: 1625104229-344747 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/z3fold: avoid possible underflow in z3fold_alloc() It is not enough to just make sure the z3fold header is not larger than the page size. When z3fold header is equal to PAGE_SIZE, we would underflow when check alloc size against PAGE_SIZE - ZHDR_SIZE_ALIGNED - CHUNK_SIZE in z3fold_alloc(). Make sure there has remaining spaces for its buddy to fix this theoretical issue. Link: https://lkml.kernel.org/r/20210619093151.1492174-3-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool Cc: Hillf Danton Signed-off-by: Andrew Morton --- mm/z3fold.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) --- a/mm/z3fold.c~mm-z3fold-avoid-possible-underflow-in-z3fold_alloc +++ a/mm/z3fold.c @@ -1803,8 +1803,11 @@ static int __init init_z3fold(void) { int ret; - /* Make sure the z3fold header is not larger than the page size */ - BUILD_BUG_ON(ZHDR_SIZE_ALIGNED > PAGE_SIZE); + /* + * Make sure the z3fold header is not larger than the page size and + * there has remaining spaces for its buddy. + */ + BUILD_BUG_ON(ZHDR_SIZE_ALIGNED > PAGE_SIZE - CHUNK_SIZE); ret = z3fold_mount(); if (ret) return ret; From patchwork Thu Jul 1 01:50:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73F71C11F66 for ; Thu, 1 Jul 2021 01:50:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2B0B36146D for ; Thu, 1 Jul 2021 01:50:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2B0B36146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A388A8D0212; Wed, 30 Jun 2021 21:50:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E89E8D0202; Wed, 30 Jun 2021 21:50:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B1088D0212; Wed, 30 Jun 2021 21:50:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id 650758D0202 for ; Wed, 30 Jun 2021 21:50:32 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3848E25F5A for ; Thu, 1 Jul 2021 01:50:32 +0000 (UTC) X-FDA: 78312339504.31.382C11C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id E55411983 for ; Thu, 1 Jul 2021 01:50:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 140BA61468; Thu, 1 Jul 2021 01:50:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104231; bh=bMZghlfwKOQnCH7xwNfRdzA6t6/oFT5kgSFzQsMvzLU=; h=Date:From:To:Subject:In-Reply-To:From; b=h70G1lb7JeJ6TBPZyfMTFkVEsiQ5fC8NEni3BBtOjpm+A7KMHmmPnPWSDJqXEg7wf IpFWWcDSqUD+NdSq5+bW2HyztgBYAKisFbfTk8LCq4OPar7lV7wEm9TRlIX66APfu2 Xhyfmb7EBkctktQyyplQblvNp/Sg6IJVQ0zpVrRo= Date: Wed, 30 Jun 2021 18:50:30 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hdanton@sina.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 062/192] mm/z3fold: remove magic number in z3fold_create_pool() Message-ID: <20210701015030.igMLYc0-G%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E55411983 X-Stat-Signature: wy9tjgc8ys9nihtk9bnxrojfzh7tsqks Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=h70G1lb7; dmarc=none; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104231-136727 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/z3fold: remove magic number in z3fold_create_pool() It's meaningless to pass a magic number 2 to __alloc_percpu() as there is a minimum alignment size of PCPU_MIN_ALLOC_SIZE (> 2) in it. Also there is no special alignment requirement for unbuddied. So we could replace this magic number with nature alignment, i.e. __alignof__(struct list_head), to improve readability. Link: https://lkml.kernel.org/r/20210619093151.1492174-4-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool Cc: Hillf Danton Signed-off-by: Andrew Morton --- mm/z3fold.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/mm/z3fold.c~mm-z3fold-remove-magic-number-in-z3fold_create_pool +++ a/mm/z3fold.c @@ -998,7 +998,8 @@ static struct z3fold_pool *z3fold_create goto out_c; spin_lock_init(&pool->lock); spin_lock_init(&pool->stale_lock); - pool->unbuddied = __alloc_percpu(sizeof(struct list_head)*NCHUNKS, 2); + pool->unbuddied = __alloc_percpu(sizeof(struct list_head) * NCHUNKS, + __alignof__(struct list_head)); if (!pool->unbuddied) goto out_pool; for_each_possible_cpu(cpu) { From patchwork Thu Jul 1 01:50:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353185 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8400AC11F66 for ; Thu, 1 Jul 2021 01:50:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 37E0F61468 for ; Thu, 1 Jul 2021 01:50:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 37E0F61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AFCBD8D0213; Wed, 30 Jun 2021 21:50:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AACA88D0202; Wed, 30 Jun 2021 21:50:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 973EA8D0213; Wed, 30 Jun 2021 21:50:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id 7563A8D0202 for ; Wed, 30 Jun 2021 21:50:35 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 471CA16950 for ; Thu, 1 Jul 2021 01:50:35 +0000 (UTC) X-FDA: 78312339630.39.70B4CBA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 03B0A30000A5 for ; Thu, 1 Jul 2021 01:50:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id F23B86105A; Thu, 1 Jul 2021 01:50:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104234; bh=J4cFi94OqIwcIG1fLpadUmpda5w2L840R4tDI0HI/z4=; h=Date:From:To:Subject:In-Reply-To:From; b=K16hYQdLNhx72fLwE0EOAjGQxV83DZDpQASaLVA8aY0mFC4dq4MK4A0tg5MDWW6Sy Nz3o9V+DaLqrsZe5QLtERqQVBkn1gJUSlb3CT0Z6/DFelPY2vg8fRSdu8iO+rSxvRO usw348gLwyqCUAXjBxg82YR0fqE6KAgwvbh0BWwI= Date: Wed, 30 Jun 2021 18:50:33 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hdanton@sina.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 063/192] mm/z3fold: remove unused function handle_to_z3fold_header() Message-ID: <20210701015033.bZHcaN-yR%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=K16hYQdL; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 03B0A30000A5 X-Stat-Signature: t94tubf33wpsdw8gi5x5qo783jx1uuqf X-HE-Tag: 1625104234-881598 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/z3fold: remove unused function handle_to_z3fold_header() handle_to_z3fold_header() is unused now. So we can remove it. As a result, get_z3fold_header() becomes the only caller of __get_z3fold_header() and the argument lock is always true. Therefore we could further fold the __get_z3fold_header() into get_z3fold_header() with lock = true. Link: https://lkml.kernel.org/r/20210619093151.1492174-5-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool Cc: Hillf Danton Signed-off-by: Andrew Morton --- mm/z3fold.c | 22 ++++------------------ 1 file changed, 4 insertions(+), 18 deletions(-) --- a/mm/z3fold.c~mm-z3fold-remove-unused-function-handle_to_z3fold_header +++ a/mm/z3fold.c @@ -253,9 +253,8 @@ static inline void z3fold_page_unlock(st spin_unlock(&zhdr->page_lock); } - -static inline struct z3fold_header *__get_z3fold_header(unsigned long handle, - bool lock) +/* return locked z3fold page if it's not headless */ +static inline struct z3fold_header *get_z3fold_header(unsigned long handle) { struct z3fold_buddy_slots *slots; struct z3fold_header *zhdr; @@ -269,13 +268,12 @@ static inline struct z3fold_header *__ge read_lock(&slots->lock); addr = *(unsigned long *)handle; zhdr = (struct z3fold_header *)(addr & PAGE_MASK); - if (lock) - locked = z3fold_page_trylock(zhdr); + locked = z3fold_page_trylock(zhdr); read_unlock(&slots->lock); if (locked) break; cpu_relax(); - } while (lock); + } while (true); } else { zhdr = (struct z3fold_header *)(handle & PAGE_MASK); } @@ -283,18 +281,6 @@ static inline struct z3fold_header *__ge return zhdr; } -/* Returns the z3fold page where a given handle is stored */ -static inline struct z3fold_header *handle_to_z3fold_header(unsigned long h) -{ - return __get_z3fold_header(h, false); -} - -/* return locked z3fold page if it's not headless */ -static inline struct z3fold_header *get_z3fold_header(unsigned long h) -{ - return __get_z3fold_header(h, true); -} - static inline void put_z3fold_header(struct z3fold_header *zhdr) { struct page *page = virt_to_page(zhdr); From patchwork Thu Jul 1 01:50:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353187 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F9F0C11F68 for ; Thu, 1 Jul 2021 01:50:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 01C1261468 for ; Thu, 1 Jul 2021 01:50:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01C1261468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C8368D0214; Wed, 30 Jun 2021 21:50:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7773D8D0202; Wed, 30 Jun 2021 21:50:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CB268D0214; Wed, 30 Jun 2021 21:50:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 3797F8D0202 for ; Wed, 30 Jun 2021 21:50:38 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0BD58180883DD for ; Thu, 1 Jul 2021 01:50:38 +0000 (UTC) X-FDA: 78312339756.23.BA1715E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id C8F499000326 for ; Thu, 1 Jul 2021 01:50:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E346D61241; Thu, 1 Jul 2021 01:50:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104237; bh=jjK0vP4z5S8Qw8Nrw75eoUT+YTFBaPnX8il2WKqqGh4=; h=Date:From:To:Subject:In-Reply-To:From; b=iVzsDawq87TsysRHGA+lzCNFxIKtz0ZJWKzN4ig0vR1G44TlgCRAui2JD0ULWjR4f UiGrz6Hh4+CSmTYYug9M9bi+SC8i5qCM2evaWnYVaCkLnfBAGGklS7euKCeY//mPWL Ob4spCDwaW5GgMu5r81yXWlusKzL9pU5YBdtGtV0= Date: Wed, 30 Jun 2021 18:50:36 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hdanton@sina.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 064/192] mm/z3fold: fix potential memory leak in z3fold_destroy_pool() Message-ID: <20210701015036.QdGlpLe6T%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=iVzsDawq; dmarc=none; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: co4wrh5hc39cfr5tpszjh6g3auzf798q X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C8F499000326 X-HE-Tag: 1625104237-197771 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/z3fold: fix potential memory leak in z3fold_destroy_pool() There is a memory leak in z3fold_destroy_pool() as it forgets to free_percpu pool->unbuddied. Call free_percpu for pool->unbuddied to fix this issue. Link: https://lkml.kernel.org/r/20210619093151.1492174-6-linmiaohe@huawei.com Fixes: d30561c56f41 ("z3fold: use per-cpu unbuddied lists") Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool Cc: Hillf Danton Signed-off-by: Andrew Morton --- mm/z3fold.c | 1 + 1 file changed, 1 insertion(+) --- a/mm/z3fold.c~mm-z3fold-fix-potential-memory-leak-in-z3fold_destroy_pool +++ a/mm/z3fold.c @@ -1046,6 +1046,7 @@ static void z3fold_destroy_pool(struct z destroy_workqueue(pool->compact_wq); destroy_workqueue(pool->release_wq); z3fold_unregister_migration(pool); + free_percpu(pool->unbuddied); kfree(pool); } From patchwork Thu Jul 1 01:50:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F730C11F69 for ; Thu, 1 Jul 2021 01:50:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F360561468 for ; Thu, 1 Jul 2021 01:50:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F360561468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7A0738D0215; Wed, 30 Jun 2021 21:50:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 776EA8D0202; Wed, 30 Jun 2021 21:50:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63F348D0215; Wed, 30 Jun 2021 21:50:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0110.hostedemail.com [216.40.44.110]) by kanga.kvack.org (Postfix) with ESMTP id 41B838D0202 for ; Wed, 30 Jun 2021 21:50:41 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 14ED02488D for ; Thu, 1 Jul 2021 01:50:41 +0000 (UTC) X-FDA: 78312339882.10.B8C607F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id C4EDA30000A0 for ; Thu, 1 Jul 2021 01:50:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D3A556105A; Thu, 1 Jul 2021 01:50:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104240; bh=4t4tLlA0YIs4aLmPXUV2HxdlmBdxAJZgCR1Najhl0Q4=; h=Date:From:To:Subject:In-Reply-To:From; b=LtMWwKvCsvOhthzBZfyFGS9fbvnLhC600kXrB4CgjJlL6QsrK5TdyEvXq8aY8Rvm6 MvPkhaG5/9VNrTd9SQvVfQEpeOeinlSx9yMTRgfSu5eSjW4aUtWY0w30BCc/yfppS1 YkvGx9/vNJsJ3paR5sf+A5POpG/NShrPRJRlUEMA= Date: Wed, 30 Jun 2021 18:50:39 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hdanton@sina.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 065/192] mm/z3fold: use release_z3fold_page_locked() to release locked z3fold page Message-ID: <20210701015039.95EwGK4nc%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C4EDA30000A0 X-Stat-Signature: cgsbuamdt4qxt3wifdyx89tpjxty94p1 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=LtMWwKvC; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104240-42285 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000058, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/z3fold: use release_z3fold_page_locked() to release locked z3fold page We should use release_z3fold_page_locked() to release z3fold page when it's locked, although it looks harmless to use release_z3fold_page() now. Link: https://lkml.kernel.org/r/20210619093151.1492174-7-linmiaohe@huawei.com Fixes: dcf5aedb24f8 ("z3fold: stricter locking and more careful reclaim") Signed-off-by: Miaohe Lin Reviewed-by: Vitaly Wool Cc: Hillf Danton Signed-off-by: Andrew Morton --- mm/z3fold.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/z3fold.c~mm-z3fold-use-release_z3fold_page_locked-to-release-locked-z3fold-page +++ a/mm/z3fold.c @@ -1370,7 +1370,7 @@ static int z3fold_reclaim_page(struct z3 if (zhdr->foreign_handles || test_and_set_bit(PAGE_CLAIMED, &page->private)) { if (kref_put(&zhdr->refcount, - release_z3fold_page)) + release_z3fold_page_locked)) atomic64_dec(&pool->pages_nr); else z3fold_page_unlock(zhdr); From patchwork Thu Jul 1 01:50:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353191 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31094C11F68 for ; Thu, 1 Jul 2021 01:50:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DFAAC61241 for ; Thu, 1 Jul 2021 01:50:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFAAC61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B6298D0216; Wed, 30 Jun 2021 21:50:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 68C988D0202; Wed, 30 Jun 2021 21:50:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 555EE8D0216; Wed, 30 Jun 2021 21:50:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id 306448D0202 for ; Wed, 30 Jun 2021 21:50:44 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F231B18030090 for ; Thu, 1 Jul 2021 01:50:43 +0000 (UTC) X-FDA: 78312339966.13.2272F02 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id A39A5F0000AE for ; Thu, 1 Jul 2021 01:50:43 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BA0696146D; Thu, 1 Jul 2021 01:50:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104243; bh=ENApBjEaO1KjfEOhm60Hc0/aVXPQIXc8yzvZc6IY0Ck=; h=Date:From:To:Subject:In-Reply-To:From; b=SQTBmV+oX8L/dPdkw5XiOSypuyI14pAdWiJpXIt8EX5awzM0j69OajqXnZnbyWo3r yiEK9kbguXf9w5dq7uYJnclTEemY7uGijy5ajiVWrm6vErt1Eqz3jiYb4mIqovblF1 u13jFxcJ9W04GQEeKwDQR9pvL5LElafndlR9Ogck= Date: Wed, 30 Jun 2021 18:50:42 -0700 From: Andrew Morton To: akpm@linux-foundation.org, ddstreet@ieee.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, sjenning@redhat.com, torvalds@linux-foundation.org Subject: [patch 066/192] mm/zbud: reuse unbuddied[0] as buddied in zbud_pool Message-ID: <20210701015042.NdLnvShOK%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=SQTBmV+o; dmarc=none; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: ja6wgr7wow8qi8m6gkryb3qjj51kee96 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A39A5F0000AE X-HE-Tag: 1625104243-477681 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zbud: reuse unbuddied[0] as buddied in zbud_pool Patch series "Cleanups for zbud", v2. This series contains just cleanups to save some possible memory in zbud_pool and avoid exporting any unneeded zbud API. More details can be found in the respective changelogs This patch (of 2): Since commit 9d8c5b5284e4 ("mm: zbud: fix condition check on allocation size"), zbud_pool.unbuddied[0] is always unused. We can reuse it as buddied field to save some possible memory. Link: https://lkml.kernel.org/r/20210608114515.206992-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210608114515.206992-2-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Seth Jennings Cc: Dan Streetman Signed-off-by: Andrew Morton --- mm/zbud.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) --- a/mm/zbud.c~mm-zbud-reuse-unbuddied-as-buddied-in-zbud_pool +++ a/mm/zbud.c @@ -93,8 +93,14 @@ */ struct zbud_pool { spinlock_t lock; - struct list_head unbuddied[NCHUNKS]; - struct list_head buddied; + union { + /* + * Reuse unbuddied[0] as buddied on the ground that + * unbuddied[0] is unused. + */ + struct list_head buddied; + struct list_head unbuddied[NCHUNKS]; + }; struct list_head lru; u64 pages_nr; const struct zbud_ops *ops; From patchwork Thu Jul 1 01:50:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353193 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D872C11F68 for ; Thu, 1 Jul 2021 01:50:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E467361468 for ; Thu, 1 Jul 2021 01:50:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E467361468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B5AA8D0217; Wed, 30 Jun 2021 21:50:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 665A78D0202; Wed, 30 Jun 2021 21:50:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DFAA8D0217; Wed, 30 Jun 2021 21:50:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id 2B4888D0202 for ; Wed, 30 Jun 2021 21:50:47 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 02E5D8020E4A for ; Thu, 1 Jul 2021 01:50:47 +0000 (UTC) X-FDA: 78312340134.35.9DBCF19 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 9F37A10000A0 for ; Thu, 1 Jul 2021 01:50:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A458B6105A; Thu, 1 Jul 2021 01:50:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104245; bh=OhISEyIemJ93vEofxgyx+8ydHy9/ssXQuoYk4J1wLdc=; h=Date:From:To:Subject:In-Reply-To:From; b=BvvEtgachBdqrbkQtX2XMjHz9JTR7QLAmRFJgSTufFsOm7HHHhYu9FkZtxdoBcXtV vnYrmQTWSSb3B8LOshQSwgToD6/O6UW/R4OYpJTrNOUiUKUPtW9Ao+hStcQzV1nIZe SlObIMwBKUVYg1ROxUh3iLFGdQn2wbB6j3UQ7sp8= Date: Wed, 30 Jun 2021 18:50:45 -0700 From: Andrew Morton To: akpm@linux-foundation.org, ddstreet@ieee.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, nathan@kernel.org, sjenning@redhat.com, torvalds@linux-foundation.org Subject: [patch 067/192] mm/zbud: don't export any zbud API Message-ID: <20210701015045.Gy10gvNw3%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=BvvEtgac; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 9F37A10000A0 X-Stat-Signature: uf3uohafn86e5s8jx8zm5cth6ptguctr X-HE-Tag: 1625104246-481155 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zbud: don't export any zbud API The zbud doesn't need to export any API and it is meant to be used via zpool API since the commit 12d79d64bfd3 ("mm/zpool: update zswap to use zpool"). So we can remove the unneeded zbud.h and move down zpool API to avoid any forward declaration. [linmiaohe@huawei.com: fix unused function warnings when CONFIG_ZPOOL is disabled] Link: https://lkml.kernel.org/r/20210619025508.1239386-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210608114515.206992-3-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Dan Streetman Cc: Seth Jennings Cc: Nathan Chancellor Signed-off-by: Andrew Morton --- MAINTAINERS | 1 include/linux/zbud.h | 23 ---- mm/Kconfig | 1 mm/zbud.c | 223 ++++++++++++++++++++--------------------- 4 files changed, 110 insertions(+), 138 deletions(-) --- a/include/linux/zbud.h +++ /dev/null @@ -1,23 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -#ifndef _ZBUD_H_ -#define _ZBUD_H_ - -#include - -struct zbud_pool; - -struct zbud_ops { - int (*evict)(struct zbud_pool *pool, unsigned long handle); -}; - -struct zbud_pool *zbud_create_pool(gfp_t gfp, const struct zbud_ops *ops); -void zbud_destroy_pool(struct zbud_pool *pool); -int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, - unsigned long *handle); -void zbud_free(struct zbud_pool *pool, unsigned long handle); -int zbud_reclaim_page(struct zbud_pool *pool, unsigned int retries); -void *zbud_map(struct zbud_pool *pool, unsigned long handle); -void zbud_unmap(struct zbud_pool *pool, unsigned long handle); -u64 zbud_get_pool_size(struct zbud_pool *pool); - -#endif /* _ZBUD_H_ */ --- a/MAINTAINERS~mm-zbud-dont-export-any-zbud-api +++ a/MAINTAINERS @@ -20172,7 +20172,6 @@ M: Seth Jennings M: Dan Streetman L: linux-mm@kvack.org S: Maintained -F: include/linux/zbud.h F: mm/zbud.c ZD1211RW WIRELESS DRIVER --- a/mm/Kconfig~mm-zbud-dont-export-any-zbud-api +++ a/mm/Kconfig @@ -674,6 +674,7 @@ config ZPOOL config ZBUD tristate "Low (Up to 2x) density storage for compressed pages" + depends on ZPOOL help A special purpose allocator for storing compressed pages. It is designed to store up to two compressed pages per physical --- a/mm/zbud.c~mm-zbud-dont-export-any-zbud-api +++ a/mm/zbud.c @@ -51,7 +51,6 @@ #include #include #include -#include #include /***************** @@ -73,6 +72,12 @@ #define ZHDR_SIZE_ALIGNED CHUNK_SIZE #define NCHUNKS ((PAGE_SIZE - ZHDR_SIZE_ALIGNED) >> CHUNK_SHIFT) +struct zbud_pool; + +struct zbud_ops { + int (*evict)(struct zbud_pool *pool, unsigned long handle); +}; + /** * struct zbud_pool - stores metadata for each zbud pool * @lock: protects all pool fields and first|last_chunk fields of any @@ -104,10 +109,8 @@ struct zbud_pool { struct list_head lru; u64 pages_nr; const struct zbud_ops *ops; -#ifdef CONFIG_ZPOOL struct zpool *zpool; const struct zpool_ops *zpool_ops; -#endif }; /* @@ -127,104 +130,6 @@ struct zbud_header { }; /***************** - * zpool - ****************/ - -#ifdef CONFIG_ZPOOL - -static int zbud_zpool_evict(struct zbud_pool *pool, unsigned long handle) -{ - if (pool->zpool && pool->zpool_ops && pool->zpool_ops->evict) - return pool->zpool_ops->evict(pool->zpool, handle); - else - return -ENOENT; -} - -static const struct zbud_ops zbud_zpool_ops = { - .evict = zbud_zpool_evict -}; - -static void *zbud_zpool_create(const char *name, gfp_t gfp, - const struct zpool_ops *zpool_ops, - struct zpool *zpool) -{ - struct zbud_pool *pool; - - pool = zbud_create_pool(gfp, zpool_ops ? &zbud_zpool_ops : NULL); - if (pool) { - pool->zpool = zpool; - pool->zpool_ops = zpool_ops; - } - return pool; -} - -static void zbud_zpool_destroy(void *pool) -{ - zbud_destroy_pool(pool); -} - -static int zbud_zpool_malloc(void *pool, size_t size, gfp_t gfp, - unsigned long *handle) -{ - return zbud_alloc(pool, size, gfp, handle); -} -static void zbud_zpool_free(void *pool, unsigned long handle) -{ - zbud_free(pool, handle); -} - -static int zbud_zpool_shrink(void *pool, unsigned int pages, - unsigned int *reclaimed) -{ - unsigned int total = 0; - int ret = -EINVAL; - - while (total < pages) { - ret = zbud_reclaim_page(pool, 8); - if (ret < 0) - break; - total++; - } - - if (reclaimed) - *reclaimed = total; - - return ret; -} - -static void *zbud_zpool_map(void *pool, unsigned long handle, - enum zpool_mapmode mm) -{ - return zbud_map(pool, handle); -} -static void zbud_zpool_unmap(void *pool, unsigned long handle) -{ - zbud_unmap(pool, handle); -} - -static u64 zbud_zpool_total_size(void *pool) -{ - return zbud_get_pool_size(pool) * PAGE_SIZE; -} - -static struct zpool_driver zbud_zpool_driver = { - .type = "zbud", - .sleep_mapped = true, - .owner = THIS_MODULE, - .create = zbud_zpool_create, - .destroy = zbud_zpool_destroy, - .malloc = zbud_zpool_malloc, - .free = zbud_zpool_free, - .shrink = zbud_zpool_shrink, - .map = zbud_zpool_map, - .unmap = zbud_zpool_unmap, - .total_size = zbud_zpool_total_size, -}; - -MODULE_ALIAS("zpool-zbud"); -#endif /* CONFIG_ZPOOL */ - -/***************** * Helpers *****************/ /* Just to make the code easier to read */ @@ -310,7 +215,7 @@ static int num_free_chunks(struct zbud_h * Return: pointer to the new zbud pool or NULL if the metadata allocation * failed. */ -struct zbud_pool *zbud_create_pool(gfp_t gfp, const struct zbud_ops *ops) +static struct zbud_pool *zbud_create_pool(gfp_t gfp, const struct zbud_ops *ops) { struct zbud_pool *pool; int i; @@ -334,7 +239,7 @@ struct zbud_pool *zbud_create_pool(gfp_t * * The pool should be emptied before this function is called. */ -void zbud_destroy_pool(struct zbud_pool *pool) +static void zbud_destroy_pool(struct zbud_pool *pool) { kfree(pool); } @@ -358,7 +263,7 @@ void zbud_destroy_pool(struct zbud_pool * gfp arguments are invalid or -ENOMEM if the pool was unable to allocate * a new page. */ -int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, +static int zbud_alloc(struct zbud_pool *pool, size_t size, gfp_t gfp, unsigned long *handle) { int chunks, i, freechunks; @@ -433,7 +338,7 @@ found: * only sets the first|last_chunks to 0. The page is actually freed * once both buddies are evicted (see zbud_reclaim_page() below). */ -void zbud_free(struct zbud_pool *pool, unsigned long handle) +static void zbud_free(struct zbud_pool *pool, unsigned long handle) { struct zbud_header *zhdr; int freechunks; @@ -505,7 +410,7 @@ void zbud_free(struct zbud_pool *pool, u * no pages to evict or an eviction handler is not registered, -EAGAIN if * the retry limit was hit. */ -int zbud_reclaim_page(struct zbud_pool *pool, unsigned int retries) +static int zbud_reclaim_page(struct zbud_pool *pool, unsigned int retries) { int i, ret, freechunks; struct zbud_header *zhdr; @@ -587,7 +492,7 @@ next: * * Returns: a pointer to the mapped allocation */ -void *zbud_map(struct zbud_pool *pool, unsigned long handle) +static void *zbud_map(struct zbud_pool *pool, unsigned long handle) { return (void *)(handle); } @@ -597,7 +502,7 @@ void *zbud_map(struct zbud_pool *pool, u * @pool: pool in which the allocation resides * @handle: handle associated with the allocation to be unmapped */ -void zbud_unmap(struct zbud_pool *pool, unsigned long handle) +static void zbud_unmap(struct zbud_pool *pool, unsigned long handle) { } @@ -608,30 +513,120 @@ void zbud_unmap(struct zbud_pool *pool, * Returns: size in pages of the given pool. The pool lock need not be * taken to access pages_nr. */ -u64 zbud_get_pool_size(struct zbud_pool *pool) +static u64 zbud_get_pool_size(struct zbud_pool *pool) { return pool->pages_nr; } +/***************** + * zpool + ****************/ + +static int zbud_zpool_evict(struct zbud_pool *pool, unsigned long handle) +{ + if (pool->zpool && pool->zpool_ops && pool->zpool_ops->evict) + return pool->zpool_ops->evict(pool->zpool, handle); + else + return -ENOENT; +} + +static const struct zbud_ops zbud_zpool_ops = { + .evict = zbud_zpool_evict +}; + +static void *zbud_zpool_create(const char *name, gfp_t gfp, + const struct zpool_ops *zpool_ops, + struct zpool *zpool) +{ + struct zbud_pool *pool; + + pool = zbud_create_pool(gfp, zpool_ops ? &zbud_zpool_ops : NULL); + if (pool) { + pool->zpool = zpool; + pool->zpool_ops = zpool_ops; + } + return pool; +} + +static void zbud_zpool_destroy(void *pool) +{ + zbud_destroy_pool(pool); +} + +static int zbud_zpool_malloc(void *pool, size_t size, gfp_t gfp, + unsigned long *handle) +{ + return zbud_alloc(pool, size, gfp, handle); +} +static void zbud_zpool_free(void *pool, unsigned long handle) +{ + zbud_free(pool, handle); +} + +static int zbud_zpool_shrink(void *pool, unsigned int pages, + unsigned int *reclaimed) +{ + unsigned int total = 0; + int ret = -EINVAL; + + while (total < pages) { + ret = zbud_reclaim_page(pool, 8); + if (ret < 0) + break; + total++; + } + + if (reclaimed) + *reclaimed = total; + + return ret; +} + +static void *zbud_zpool_map(void *pool, unsigned long handle, + enum zpool_mapmode mm) +{ + return zbud_map(pool, handle); +} +static void zbud_zpool_unmap(void *pool, unsigned long handle) +{ + zbud_unmap(pool, handle); +} + +static u64 zbud_zpool_total_size(void *pool) +{ + return zbud_get_pool_size(pool) * PAGE_SIZE; +} + +static struct zpool_driver zbud_zpool_driver = { + .type = "zbud", + .sleep_mapped = true, + .owner = THIS_MODULE, + .create = zbud_zpool_create, + .destroy = zbud_zpool_destroy, + .malloc = zbud_zpool_malloc, + .free = zbud_zpool_free, + .shrink = zbud_zpool_shrink, + .map = zbud_zpool_map, + .unmap = zbud_zpool_unmap, + .total_size = zbud_zpool_total_size, +}; + +MODULE_ALIAS("zpool-zbud"); + static int __init init_zbud(void) { /* Make sure the zbud header will fit in one chunk */ BUILD_BUG_ON(sizeof(struct zbud_header) > ZHDR_SIZE_ALIGNED); pr_info("loaded\n"); -#ifdef CONFIG_ZPOOL zpool_register_driver(&zbud_zpool_driver); -#endif return 0; } static void __exit exit_zbud(void) { -#ifdef CONFIG_ZPOOL zpool_unregister_driver(&zbud_zpool_driver); -#endif - pr_info("unloaded\n"); } From patchwork Thu Jul 1 01:50:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353195 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE8FFC11F68 for ; Thu, 1 Jul 2021 01:50:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8F82861241 for ; Thu, 1 Jul 2021 01:50:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F82861241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 185AF8D0218; Wed, 30 Jun 2021 21:50:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 135CA8D0202; Wed, 30 Jun 2021 21:50:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F18658D0218; Wed, 30 Jun 2021 21:50:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0235.hostedemail.com [216.40.44.235]) by kanga.kvack.org (Postfix) with ESMTP id CF0638D0202 for ; Wed, 30 Jun 2021 21:50:49 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A95151802B4D3 for ; Thu, 1 Jul 2021 01:50:49 +0000 (UTC) X-FDA: 78312340218.21.CC20310 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 67D4330000A6 for ; Thu, 1 Jul 2021 01:50:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8852E6105A; Thu, 1 Jul 2021 01:50:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104248; bh=z3Z39tCLifEa5QUv67IYFRRwi8ywQ7lTDjexKpXSuhM=; h=Date:From:To:Subject:In-Reply-To:From; b=Spri/IJKJju+xA3f+ZNv/LzXi3EMBrOeyaelOw+ZV4s7+APWri7VkZt63e/o44faJ v0DSB04/IF0XgZkZi/JcaA8dnXZ/+U0P2Z91+ctGA3xz1yQxBKW5xc5OTB8K1k0fxC 3KnIxdpRPf7YbM7VNV7dtr/h0AgwL4P3mZSD2EfQ= Date: Wed, 30 Jun 2021 18:50:48 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yuehaibing@huawei.com Subject: [patch 068/192] mm/compaction: use DEVICE_ATTR_WO macro Message-ID: <20210701015048.s99_O6WH0%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 67D4330000A6 X-Stat-Signature: 1jtbnrk3arc1zf9uxcrnm5jysdtwkyhd Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="Spri/IJK"; dmarc=none; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104249-626239 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: YueHaibing Subject: mm/compaction: use DEVICE_ATTR_WO macro Use DEVICE_ATTR_WO helper instead of plain DEVICE_ATTR, which makes the code a bit shorter and easier to read. Link: https://lkml.kernel.org/r/20210523064521.32912-1-yuehaibing@huawei.com Signed-off-by: YueHaibing Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/compaction.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/compaction.c~mm-compaction-use-device_attr_wo-macro +++ a/mm/compaction.c @@ -2722,9 +2722,9 @@ int sysctl_compaction_handler(struct ctl } #if defined(CONFIG_SYSFS) && defined(CONFIG_NUMA) -static ssize_t sysfs_compact_node(struct device *dev, - struct device_attribute *attr, - const char *buf, size_t count) +static ssize_t compact_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) { int nid = dev->id; @@ -2737,7 +2737,7 @@ static ssize_t sysfs_compact_node(struct return count; } -static DEVICE_ATTR(compact, 0200, NULL, sysfs_compact_node); +static DEVICE_ATTR_WO(compact); int compaction_register_node(struct node *node) { From patchwork Thu Jul 1 01:50:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCDFDC11F69 for ; Thu, 1 Jul 2021 01:50:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 803E861241 for ; Thu, 1 Jul 2021 01:50:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 803E861241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 04D1E8D0219; Wed, 30 Jun 2021 21:50:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3EE48D0202; Wed, 30 Jun 2021 21:50:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E07218D0219; Wed, 30 Jun 2021 21:50:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0030.hostedemail.com [216.40.44.30]) by kanga.kvack.org (Postfix) with ESMTP id B8E2C8D0202 for ; Wed, 30 Jun 2021 21:50:52 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 860F3181AF5CA for ; Thu, 1 Jul 2021 01:50:52 +0000 (UTC) X-FDA: 78312340344.07.C2CB379 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 45F2630002A4 for ; Thu, 1 Jul 2021 01:50:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5AD246105A; Thu, 1 Jul 2021 01:50:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104251; bh=sW2YVX7JTuv9Ix1neyEH1wc4ioUwYFcLg/KnAEK56IU=; h=Date:From:To:Subject:In-Reply-To:From; b=R0ZKT8tfejr1U1nVw8VVLIoGqnl24//nDqcbxAZx1i/pRcjCADRXy5shOap6GCoR6 /6hD70M/Hc1CdonIjnrbWGXfIHGCdQ0zZMyOMEQvTDcUHpit0nIjqrRszxcXQZXoEd yglUG2zjulto5GFvHHwvE6eFb7KYShC0NvDUEQMs= Date: Wed, 30 Jun 2021 18:50:51 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, liu.xiang@zlingsmart.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 069/192] mm: compaction: remove duplicate !list_empty(&sublist) check Message-ID: <20210701015051.qyVr1V2h4%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 45F2630002A4 X-Stat-Signature: tmiuceoexpq3s7np939onm3dwckz1jpi Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=R0ZKT8tf; dmarc=none; spf=pass (imf09.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104252-545136 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Liu Xiang Subject: mm: compaction: remove duplicate !list_empty(&sublist) check The list_splice_tail(&sublist, freelist) also do !list_empty(&sublist) check, so remove the duplicate call. Link: https://lkml.kernel.org/r/20210609095409.19920-1-liu.xiang@zlingsmart.com Signed-off-by: Liu Xiang Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/compaction.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/mm/compaction.c~mm-compaction-remove-duplicate-list_emptysublist-check +++ a/mm/compaction.c @@ -1297,8 +1297,7 @@ move_freelist_head(struct list_head *fre if (!list_is_last(freelist, &freepage->lru)) { list_cut_before(&sublist, freelist, &freepage->lru); - if (!list_empty(&sublist)) - list_splice_tail(&sublist, freelist); + list_splice_tail(&sublist, freelist); } } @@ -1315,8 +1314,7 @@ move_freelist_tail(struct list_head *fre if (!list_is_first(freelist, &freepage->lru)) { list_cut_position(&sublist, freelist, &freepage->lru); - if (!list_empty(&sublist)) - list_splice_tail(&sublist, freelist); + list_splice_tail(&sublist, freelist); } } From patchwork Thu Jul 1 01:50:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AADA5C11F68 for ; Thu, 1 Jul 2021 01:50:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6359E6105A for ; Thu, 1 Jul 2021 01:50:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6359E6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DE21A8D021A; Wed, 30 Jun 2021 21:50:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DB8918D0202; Wed, 30 Jun 2021 21:50:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C81BE8D021A; Wed, 30 Jun 2021 21:50:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id A607B8D0202 for ; Wed, 30 Jun 2021 21:50:55 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7A5CB181AEF3F for ; Thu, 1 Jul 2021 01:50:55 +0000 (UTC) X-FDA: 78312340470.12.9F67496 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id 3B937900009A for ; Thu, 1 Jul 2021 01:50:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5023561469; Thu, 1 Jul 2021 01:50:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104254; bh=UJtTJYrCRsLD4FfW1JymU0J77+YJVyR9MC3Xyr/ThOU=; h=Date:From:To:Subject:In-Reply-To:From; b=zyGW2MG7Gxv8myiLus7P1ylSM8SY8pB2aSaQRrZ3qBYI5lor+/ufGPGDYafbg3Bnm yP9ae6/gzkqSUTVWAAd1EFRTke+RJ9EMUm4i5cUG0BODpdqhOKnw1RlXGS6rnptKGf ntbw+eWnRuIQ+yHRhIM/1nFHUWr2qPIHZqEXFC3U= Date: Wed, 30 Jun 2021 18:50:53 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vvghjk1234@gmail.com Subject: [patch 070/192] mm/compaction: fix 'limit' in fast_isolate_freepages Message-ID: <20210701015053.dU5ZOGwUJ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=zyGW2MG7; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 3tzjo1rnfj7jxpriikdwy1kfxpm7cwby X-Rspamd-Queue-Id: 3B937900009A X-Rspamd-Server: rspam06 X-HE-Tag: 1625104255-670620 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wonhyuk Yang Subject: mm/compaction: fix 'limit' in fast_isolate_freepages Because of 'min(1, ...)', fast_isolate_freepages set 'limit' to 0 or 1. This takes away the opportunities of find candinate pages. So, by making enough scans available, increases the probability of finding the appropriate freepage. Tested it on the thpscale and the results are as follows. 5.12.0 5.12.0 valnilla patched Amean fault-both-1 598.15 ( 0.00%) 592.56 ( 0.93%) Amean fault-both-3 1494.47 ( 0.00%) 1514.35 ( -1.33%) Amean fault-both-5 2519.48 ( 0.00%) 2471.76 ( 1.89%) Amean fault-both-7 3173.85 ( 0.00%) 3079.19 ( 2.98%) Amean fault-both-12 8063.83 ( 0.00%) 7858.29 ( 2.55%) Amean fault-both-18 8781.20 ( 0.00%) 7827.70 * 10.86%* Amean fault-both-24 12576.44 ( 0.00%) 12250.20 ( 2.59%) Amean fault-both-30 18503.27 ( 0.00%) 17528.11 * 5.27%* Amean fault-both-32 16133.69 ( 0.00%) 13874.24 * 14.00%* 5.12.0 5.12.0 vanilla patched Ops Compaction migrate scanned 6547133.00 5963901.00 Ops Compaction free scanned 32452453.00 26609101.00 5.12 5.12 vanilla patched Duration User 27.99 28.84 Duration System 244.08 236.76 Duration Elapsed 78.27 78.38 Link: https://lkml.kernel.org/r/20210626082443.22547-1-vvghjk1234@gmail.com Fixes: 5a811889de10f ("mm, compaction: use free lists to quickly locate a migration target") Signed-off-by: Wonhyuk Yang Acked-by: Mel Gorman Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/compaction.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/mm/compaction.c~mm-compaction-fix-limit-in-fast_isolate_freepages +++ a/mm/compaction.c @@ -1378,7 +1378,7 @@ static int next_search_order(struct comp static unsigned long fast_isolate_freepages(struct compact_control *cc) { - unsigned int limit = min(1U, freelist_scan_limit(cc) >> 1); + unsigned int limit = max(1U, freelist_scan_limit(cc) >> 1); unsigned int nr_scanned = 0; unsigned long low_pfn, min_pfn, highest = 0; unsigned long nr_isolated = 0; @@ -1490,11 +1490,11 @@ fast_isolate_freepages(struct compact_co spin_unlock_irqrestore(&cc->zone->lock, flags); /* - * Smaller scan on next order so the total scan ig related + * Smaller scan on next order so the total scan is related * to freelist_scan_limit. */ if (order_scanned >= limit) - limit = min(1U, limit >> 1); + limit = max(1U, limit >> 1); } if (!page) { From patchwork Thu Jul 1 01:50:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28436C11F69 for ; Thu, 1 Jul 2021 01:51:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D676E6105A for ; Thu, 1 Jul 2021 01:50:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D676E6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F2E08D021B; Wed, 30 Jun 2021 21:50:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C9558D0202; Wed, 30 Jun 2021 21:50:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 491668D021B; Wed, 30 Jun 2021 21:50:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0109.hostedemail.com [216.40.44.109]) by kanga.kvack.org (Postfix) with ESMTP id 2479B8D0202 for ; Wed, 30 Jun 2021 21:50:59 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id ECD0F1802ED99 for ; Thu, 1 Jul 2021 01:50:58 +0000 (UTC) X-FDA: 78312340596.14.26C4D77 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 9B0D630002A7 for ; Thu, 1 Jul 2021 01:50:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4AF2061468; Thu, 1 Jul 2021 01:50:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104257; bh=F+BU3Z9FbYyWHj/3lx0puQHv3vZJMTIU/TOiZqchxU4=; h=Date:From:To:Subject:In-Reply-To:From; b=lXAjz3yebpmFto3qXOzLVeiiRrqO1Owjyaoj7m9Dq+cfQV2kUnUVwBU03qWzyLls5 5bH9FaWeM6F64xJ75rZkzW1ofOKWqPCEy3CcJPJja/MypsKNex5VzWxOo3c1QKZKGI ddTMYRcELwLgrMthOFk/DXzU2HhUtGv895yURz00= Date: Wed, 30 Jun 2021 18:50:56 -0700 From: Andrew Morton To: aarcange@redhat.com, ak@linux.intel.com, akpm@linux-foundation.org, ben.widawsky@intel.com, dan.j.williams@intel.com, dave.hansen@intel.com, feng.tang@intel.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, rdunlap@infradead.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, ying.huang@intel.com Subject: [patch 071/192] mm/mempolicy: cleanup nodemask intersection check for oom Message-ID: <20210701015056.QHBu0th6x%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lXAjz3ye; dmarc=none; spf=pass (imf09.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: nrcfosjwknhzie9szwydj3cb5ahfj17k X-Rspamd-Queue-Id: 9B0D630002A7 X-HE-Tag: 1625104258-7888 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Feng Tang Subject: mm/mempolicy: cleanup nodemask intersection check for oom Patch series "mm/mempolicy: some fix and semantics cleanup", v4. Current memory policy code has some confusing and ambiguous part about MPOL_LOCAL policy, as it is handled as a faked MPOL_PREFERRED one, and there are many places having to distinguish them. Also the nodemask intersection check needs cleanup to be more explicit for OOM use, and handle MPOL_INTERLEAVE correctly. This patchset cleans up these and unifies the parameter sanity check for mbind() and set_mempolicy(). This patch (of 3): mempolicy_nodemask_intersects seem to be a general purpose mempolicy function. In fact it is partially tailored for the OOM purpose instead. The oom proper is the only existing user so rename the function to make that purpose explicit. While at it drop the MPOL_INTERLEAVE as those allocations never has a nodemask defined (see alloc_page_interleave) so this is a dead code and a confusing one because MPOL_INTERLEAVE is a hint rather than a hard requirement so it shouldn't be considered during the OOM. The final code can be reduced to a check for MPOL_BIND which is the only memory policy that is a hard requirement and thus relevant to a constrained OOM logic. [mhocko@suse.com: changelog edits] Link: https://lkml.kernel.org/r/1622560492-1294-1-git-send-email-feng.tang@intel.com Link: https://lkml.kernel.org/r/1622560492-1294-2-git-send-email-feng.tang@intel.com Link: https://lkml.kernel.org/r/1622469956-82897-1-git-send-email-feng.tang@intel.com Link: https://lkml.kernel.org/r/1622469956-82897-2-git-send-email-feng.tang@intel.com Signed-off-by: Feng Tang Suggested-by: Michal Hocko Acked-by: Michal Hocko Cc: Andi Kleen Cc: Andrea Arcangeli Cc: Ben Widawsky Cc: Dan Williams Cc: Dave Hansen Cc: David Rientjes Cc: Huang Ying Cc: Mel Gorman Cc: Mike Kravetz Cc: Randy Dunlap Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mempolicy.h | 2 +- mm/mempolicy.c | 34 +++++++++------------------------- mm/oom_kill.c | 2 +- 3 files changed, 11 insertions(+), 27 deletions(-) --- a/include/linux/mempolicy.h~mm-mempolicy-cleanup-nodemask-intersection-check-for-oom +++ a/include/linux/mempolicy.h @@ -150,7 +150,7 @@ extern int huge_node(struct vm_area_stru unsigned long addr, gfp_t gfp_flags, struct mempolicy **mpol, nodemask_t **nodemask); extern bool init_nodemask_of_mempolicy(nodemask_t *mask); -extern bool mempolicy_nodemask_intersects(struct task_struct *tsk, +extern bool mempolicy_in_oom_domain(struct task_struct *tsk, const nodemask_t *mask); extern nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy); --- a/mm/mempolicy.c~mm-mempolicy-cleanup-nodemask-intersection-check-for-oom +++ a/mm/mempolicy.c @@ -2094,16 +2094,16 @@ bool init_nodemask_of_mempolicy(nodemask #endif /* - * mempolicy_nodemask_intersects + * mempolicy_in_oom_domain * - * If tsk's mempolicy is "default" [NULL], return 'true' to indicate default - * policy. Otherwise, check for intersection between mask and the policy - * nodemask for 'bind' or 'interleave' policy. For 'preferred' or 'local' - * policy, always return true since it may allocate elsewhere on fallback. + * If tsk's mempolicy is "bind", check for intersection between mask and + * the policy nodemask. Otherwise, return true for all other policies + * including "interleave", as a tsk with "interleave" policy may have + * memory allocated from all nodes in system. * * Takes task_lock(tsk) to prevent freeing of its mempolicy. */ -bool mempolicy_nodemask_intersects(struct task_struct *tsk, +bool mempolicy_in_oom_domain(struct task_struct *tsk, const nodemask_t *mask) { struct mempolicy *mempolicy; @@ -2111,29 +2111,13 @@ bool mempolicy_nodemask_intersects(struc if (!mask) return ret; + task_lock(tsk); mempolicy = tsk->mempolicy; - if (!mempolicy) - goto out; - - switch (mempolicy->mode) { - case MPOL_PREFERRED: - /* - * MPOL_PREFERRED and MPOL_F_LOCAL are only preferred nodes to - * allocate from, they may fallback to other nodes when oom. - * Thus, it's possible for tsk to have allocated memory from - * nodes in mask. - */ - break; - case MPOL_BIND: - case MPOL_INTERLEAVE: + if (mempolicy && mempolicy->mode == MPOL_BIND) ret = nodes_intersects(mempolicy->v.nodes, *mask); - break; - default: - BUG(); - } -out: task_unlock(tsk); + return ret; } --- a/mm/oom_kill.c~mm-mempolicy-cleanup-nodemask-intersection-check-for-oom +++ a/mm/oom_kill.c @@ -104,7 +104,7 @@ static bool oom_cpuset_eligible(struct t * mempolicy intersects current, otherwise it may be * needlessly killed. */ - ret = mempolicy_nodemask_intersects(tsk, mask); + ret = mempolicy_in_oom_domain(tsk, mask); } else { /* * This is not a mempolicy constrained oom, so only From patchwork Thu Jul 1 01:51:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 922AAC11F6B for ; Thu, 1 Jul 2021 01:51:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3E1056105A for ; Thu, 1 Jul 2021 01:51:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E1056105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE5618D021C; Wed, 30 Jun 2021 21:51:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BBC848D0202; Wed, 30 Jun 2021 21:51:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A376A8D021C; Wed, 30 Jun 2021 21:51:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id 7CDDC8D0202 for ; Wed, 30 Jun 2021 21:51:02 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 50F7F24886 for ; Thu, 1 Jul 2021 01:51:02 +0000 (UTC) X-FDA: 78312340764.28.24EF67A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id EDB0A1000160 for ; Thu, 1 Jul 2021 01:51:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C156661469; Thu, 1 Jul 2021 01:51:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104261; bh=mCUiWRIs3VetdrhlFTl6whUUUyDFWLdcZRT2Rk8pe54=; h=Date:From:To:Subject:In-Reply-To:From; b=uodQUiHxmHAQX0yEAHf1uEM/rgLE96M5a0z+ev+glGnl1zXGyJqWOG7o9fyLyp5SR KzhiZdb6q5VmJw2atRXboen2By/jrlQ9I0h7Jy5CRA+tBoufdZ1cmyYpCBbiBQE2Yn kAyw85NRUVhmuHppFaTyl992W/kXwn4e4dGlMjbI= Date: Wed, 30 Jun 2021 18:51:00 -0700 From: Andrew Morton To: aarcange@redhat.com, ak@linux.intel.com, akpm@linux-foundation.org, ben.widawsky@intel.com, dan.j.williams@intel.com, dave.hansen@intel.com, feng.tang@intel.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, rdunlap@infradead.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, ying.huang@intel.com Subject: [patch 072/192] mm/mempolicy: don't handle MPOL_LOCAL like a fake MPOL_PREFERRED policy Message-ID: <20210701015100.66QymrF0r%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=uodQUiHx; dmarc=none; spf=pass (imf13.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: zdwiyxd8q78nez19qexyrifk7yz79cgf X-Rspamd-Queue-Id: EDB0A1000160 X-HE-Tag: 1625104261-130573 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Feng Tang Subject: mm/mempolicy: don't handle MPOL_LOCAL like a fake MPOL_PREFERRED policy MPOL_LOCAL policy has been setup as a real policy, but it is still handled like a faked POL_PREFERRED policy with one internal MPOL_F_LOCAL flag bit set, and there are many places having to judge the real 'prefer' or the 'local' policy, which are quite confusing. In current code, there are 4 cases that MPOL_LOCAL are used: 1. user specifies 'local' policy 2. user specifies 'prefer' policy, but with empty nodemask 3. system 'default' policy is used 4. 'prefer' policy + valid 'preferred' node with MPOL_F_STATIC_NODES flag set, and when it is 'rebind' to a nodemask which doesn't contains the 'preferred' node, it will perform as 'local' policy So make 'local' a real policy instead of a fake 'prefer' one, and kill MPOL_F_LOCAL bit, which can greatly reduce the confusion for code reading. For case 4, the logic of mpol_rebind_preferred() is confusing, as Michal Hocko pointed out: : I do believe that rebinding preferred policy is just bogus and it should : be dropped altogether on the ground that a preference is a mere hint from : userspace where to start the allocation. Unless I am missing something : cpusets will be always authoritative for the final placement. The : preferred node just acts as a starting point and it should be really : preserved when cpusets changes. Otherwise we have a very subtle behavior : corner cases. So dump all the tricky transformation between 'prefer' and 'local', and just record the new nodemask of rebinding. [feng.tang@intel.com: fix a problem in mpol_set_nodemask(), per Michal Hocko] Link: https://lkml.kernel.org/r/1622560492-1294-3-git-send-email-feng.tang@intel.com [feng.tang@intel.com: refine code and comments of mpol_set_nodemask(), per Michal] Link: https://lkml.kernel.org/r/20210603081807.GE56979@shbuild999.sh.intel.com Link: https://lkml.kernel.org/r/1622469956-82897-3-git-send-email-feng.tang@intel.com Signed-off-by: Feng Tang Suggested-by: Michal Hocko Acked-by: Michal Hocko Cc: Andi Kleen Cc: Andrea Arcangeli Cc: Ben Widawsky Cc: Dan Williams Cc: Dave Hansen Cc: David Rientjes Cc: Huang Ying Cc: Mel Gorman Cc: Michal Hocko Cc: Mike Kravetz Cc: Randy Dunlap Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/uapi/linux/mempolicy.h | 1 mm/mempolicy.c | 136 ++++++++++++------------------- 2 files changed, 56 insertions(+), 81 deletions(-) --- a/include/uapi/linux/mempolicy.h~mm-mempolicy-dont-handle-mpol_local-like-a-fake-mpol_preferred-policy +++ a/include/uapi/linux/mempolicy.h @@ -60,7 +60,6 @@ enum { * are never OR'ed into the mode in mempolicy API arguments. */ #define MPOL_F_SHARED (1 << 0) /* identify shared policies */ -#define MPOL_F_LOCAL (1 << 1) /* preferred local allocation */ #define MPOL_F_MOF (1 << 3) /* this policy wants migrate on fault */ #define MPOL_F_MORON (1 << 4) /* Migrate On protnone Reference On Node */ --- a/mm/mempolicy.c~mm-mempolicy-dont-handle-mpol_local-like-a-fake-mpol_preferred-policy +++ a/mm/mempolicy.c @@ -121,8 +121,7 @@ enum zone_type policy_zone = 0; */ static struct mempolicy default_policy = { .refcnt = ATOMIC_INIT(1), /* never free it */ - .mode = MPOL_PREFERRED, - .flags = MPOL_F_LOCAL, + .mode = MPOL_LOCAL, }; static struct mempolicy preferred_node_policy[MAX_NUMNODES]; @@ -200,12 +199,9 @@ static int mpol_new_interleave(struct me static int mpol_new_preferred(struct mempolicy *pol, const nodemask_t *nodes) { - if (!nodes) - pol->flags |= MPOL_F_LOCAL; /* local allocation */ - else if (nodes_empty(*nodes)) - return -EINVAL; /* no allowed nodes */ - else - pol->v.preferred_node = first_node(*nodes); + if (nodes_empty(*nodes)) + return -EINVAL; + pol->v.preferred_node = first_node(*nodes); return 0; } @@ -220,8 +216,7 @@ static int mpol_new_bind(struct mempolic /* * mpol_set_nodemask is called after mpol_new() to set up the nodemask, if * any, for the new policy. mpol_new() has already validated the nodes - * parameter with respect to the policy mode and flags. But, we need to - * handle an empty nodemask with MPOL_PREFERRED here. + * parameter with respect to the policy mode and flags. * * Must be called holding task's alloc_lock to protect task's mems_allowed * and mempolicy. May also be called holding the mmap_lock for write. @@ -231,33 +226,31 @@ static int mpol_set_nodemask(struct memp { int ret; - /* if mode is MPOL_DEFAULT, pol is NULL. This is right. */ - if (pol == NULL) + /* + * Default (pol==NULL) resp. local memory policies are not a + * subject of any remapping. They also do not need any special + * constructor. + */ + if (!pol || pol->mode == MPOL_LOCAL) return 0; + /* Check N_MEMORY */ nodes_and(nsc->mask1, cpuset_current_mems_allowed, node_states[N_MEMORY]); VM_BUG_ON(!nodes); - if (pol->mode == MPOL_PREFERRED && nodes_empty(*nodes)) - nodes = NULL; /* explicit local allocation */ - else { - if (pol->flags & MPOL_F_RELATIVE_NODES) - mpol_relative_nodemask(&nsc->mask2, nodes, &nsc->mask1); - else - nodes_and(nsc->mask2, *nodes, nsc->mask1); - if (mpol_store_user_nodemask(pol)) - pol->w.user_nodemask = *nodes; - else - pol->w.cpuset_mems_allowed = - cpuset_current_mems_allowed; - } + if (pol->flags & MPOL_F_RELATIVE_NODES) + mpol_relative_nodemask(&nsc->mask2, nodes, &nsc->mask1); + else + nodes_and(nsc->mask2, *nodes, nsc->mask1); - if (nodes) - ret = mpol_ops[pol->mode].create(pol, &nsc->mask2); + if (mpol_store_user_nodemask(pol)) + pol->w.user_nodemask = *nodes; else - ret = mpol_ops[pol->mode].create(pol, NULL); + pol->w.cpuset_mems_allowed = cpuset_current_mems_allowed; + + ret = mpol_ops[pol->mode].create(pol, &nsc->mask2); return ret; } @@ -290,13 +283,14 @@ static struct mempolicy *mpol_new(unsign if (((flags & MPOL_F_STATIC_NODES) || (flags & MPOL_F_RELATIVE_NODES))) return ERR_PTR(-EINVAL); + + mode = MPOL_LOCAL; } } else if (mode == MPOL_LOCAL) { if (!nodes_empty(*nodes) || (flags & MPOL_F_STATIC_NODES) || (flags & MPOL_F_RELATIVE_NODES)) return ERR_PTR(-EINVAL); - mode = MPOL_PREFERRED; } else if (nodes_empty(*nodes)) return ERR_PTR(-EINVAL); policy = kmem_cache_alloc(policy_cache, GFP_KERNEL); @@ -344,25 +338,7 @@ static void mpol_rebind_nodemask(struct static void mpol_rebind_preferred(struct mempolicy *pol, const nodemask_t *nodes) { - nodemask_t tmp; - - if (pol->flags & MPOL_F_STATIC_NODES) { - int node = first_node(pol->w.user_nodemask); - - if (node_isset(node, *nodes)) { - pol->v.preferred_node = node; - pol->flags &= ~MPOL_F_LOCAL; - } else - pol->flags |= MPOL_F_LOCAL; - } else if (pol->flags & MPOL_F_RELATIVE_NODES) { - mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes); - pol->v.preferred_node = first_node(tmp); - } else if (!(pol->flags & MPOL_F_LOCAL)) { - pol->v.preferred_node = node_remap(pol->v.preferred_node, - pol->w.cpuset_mems_allowed, - *nodes); - pol->w.cpuset_mems_allowed = *nodes; - } + pol->w.cpuset_mems_allowed = *nodes; } /* @@ -376,7 +352,7 @@ static void mpol_rebind_policy(struct me { if (!pol) return; - if (!mpol_store_user_nodemask(pol) && !(pol->flags & MPOL_F_LOCAL) && + if (!mpol_store_user_nodemask(pol) && nodes_equal(pol->w.cpuset_mems_allowed, *newmask)) return; @@ -427,6 +403,9 @@ static const struct mempolicy_operations .create = mpol_new_bind, .rebind = mpol_rebind_nodemask, }, + [MPOL_LOCAL] = { + .rebind = mpol_rebind_default, + }, }; static int migrate_page_add(struct page *page, struct list_head *pagelist, @@ -919,10 +898,12 @@ static void get_policy_nodemask(struct m case MPOL_INTERLEAVE: *nodes = p->v.nodes; break; + case MPOL_LOCAL: + /* return empty node mask for local allocation */ + break; + case MPOL_PREFERRED: - if (!(p->flags & MPOL_F_LOCAL)) - node_set(p->v.preferred_node, *nodes); - /* else return empty node mask for local allocation */ + node_set(p->v.preferred_node, *nodes); break; default: BUG(); @@ -1894,9 +1875,9 @@ nodemask_t *policy_nodemask(gfp_t gfp, s /* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node(gfp_t gfp, struct mempolicy *policy, int nd) { - if (policy->mode == MPOL_PREFERRED && !(policy->flags & MPOL_F_LOCAL)) + if (policy->mode == MPOL_PREFERRED) { nd = policy->v.preferred_node; - else { + } else { /* * __GFP_THISNODE shouldn't even be used with the bind policy * because we might easily break the expectation to stay on the @@ -1933,14 +1914,11 @@ unsigned int mempolicy_slab_node(void) return node; policy = current->mempolicy; - if (!policy || policy->flags & MPOL_F_LOCAL) + if (!policy) return node; switch (policy->mode) { case MPOL_PREFERRED: - /* - * handled MPOL_F_LOCAL above - */ return policy->v.preferred_node; case MPOL_INTERLEAVE: @@ -1960,6 +1938,8 @@ unsigned int mempolicy_slab_node(void) &policy->v.nodes); return z->zone ? zone_to_nid(z->zone) : node; } + case MPOL_LOCAL: + return node; default: BUG(); @@ -2072,16 +2052,18 @@ bool init_nodemask_of_mempolicy(nodemask mempolicy = current->mempolicy; switch (mempolicy->mode) { case MPOL_PREFERRED: - if (mempolicy->flags & MPOL_F_LOCAL) - nid = numa_node_id(); - else - nid = mempolicy->v.preferred_node; + nid = mempolicy->v.preferred_node; init_nodemask_of_node(mask, nid); break; case MPOL_BIND: case MPOL_INTERLEAVE: - *mask = mempolicy->v.nodes; + *mask = mempolicy->v.nodes; + break; + + case MPOL_LOCAL: + nid = numa_node_id(); + init_nodemask_of_node(mask, nid); break; default: @@ -2188,7 +2170,7 @@ struct page *alloc_pages_vma(gfp_t gfp, * If the policy is interleave, or does not allow the current * node in its nodemask, we allocate the standard way. */ - if (pol->mode == MPOL_PREFERRED && !(pol->flags & MPOL_F_LOCAL)) + if (pol->mode == MPOL_PREFERRED) hpage_node = pol->v.preferred_node; nmask = policy_nodemask(gfp, pol); @@ -2324,10 +2306,9 @@ bool __mpol_equal(struct mempolicy *a, s case MPOL_INTERLEAVE: return !!nodes_equal(a->v.nodes, b->v.nodes); case MPOL_PREFERRED: - /* a's ->flags is the same as b's */ - if (a->flags & MPOL_F_LOCAL) - return true; return a->v.preferred_node == b->v.preferred_node; + case MPOL_LOCAL: + return true; default: BUG(); return false; @@ -2465,10 +2446,11 @@ int mpol_misplaced(struct page *page, st break; case MPOL_PREFERRED: - if (pol->flags & MPOL_F_LOCAL) - polnid = numa_node_id(); - else - polnid = pol->v.preferred_node; + polnid = pol->v.preferred_node; + break; + + case MPOL_LOCAL: + polnid = numa_node_id(); break; case MPOL_BIND: @@ -2835,9 +2817,6 @@ void numa_default_policy(void) * Parse and format mempolicy from/to strings */ -/* - * "local" is implemented internally by MPOL_PREFERRED with MPOL_F_LOCAL flag. - */ static const char * const policy_modes[] = { [MPOL_DEFAULT] = "default", @@ -2915,7 +2894,6 @@ int mpol_parse_str(char *str, struct mem */ if (nodelist) goto out; - mode = MPOL_PREFERRED; break; case MPOL_DEFAULT: /* @@ -2959,7 +2937,7 @@ int mpol_parse_str(char *str, struct mem else if (nodelist) new->v.preferred_node = first_node(nodes); else - new->flags |= MPOL_F_LOCAL; + new->mode = MPOL_LOCAL; /* * Save nodes for contextualization: this will be used to "clone" @@ -3005,12 +2983,10 @@ void mpol_to_str(char *buffer, int maxle switch (mode) { case MPOL_DEFAULT: + case MPOL_LOCAL: break; case MPOL_PREFERRED: - if (flags & MPOL_F_LOCAL) - mode = MPOL_LOCAL; - else - node_set(pol->v.preferred_node, nodes); + node_set(pol->v.preferred_node, nodes); break; case MPOL_BIND: case MPOL_INTERLEAVE: From patchwork Thu Jul 1 01:51:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353205 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34840C11F68 for ; Thu, 1 Jul 2021 01:51:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E08BF61469 for ; Thu, 1 Jul 2021 01:51:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E08BF61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 667548D021D; Wed, 30 Jun 2021 21:51:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63E798D0202; Wed, 30 Jun 2021 21:51:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52DA88D021D; Wed, 30 Jun 2021 21:51:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 2B8788D0202 for ; Wed, 30 Jun 2021 21:51:06 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 01EA32287D for ; Thu, 1 Jul 2021 01:51:06 +0000 (UTC) X-FDA: 78312340932.24.6493A54 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 6E64030000A5 for ; Thu, 1 Jul 2021 01:51:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 218E26105A; Thu, 1 Jul 2021 01:51:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104264; bh=f7LPnw86Ap+ikHRqhja7nyyAa3xRirXQNc40PynI9/A=; h=Date:From:To:Subject:In-Reply-To:From; b=Ceft8B61AbVjYWPqIyPSZePIbmEfDqGN29EQ4dhoOgipSHZL+NdByjfmbPKhD5pZ0 q5miDD9XTQBJHRjneGuOo7+vyvWJrVKXvT9s+oaS1DXo9Apz2Fhu9FOarVBGbncKc3 +22fPHTKWeEHDKbpJoqz+athUVVp9mpy2DASwLSE= Date: Wed, 30 Jun 2021 18:51:03 -0700 From: Andrew Morton To: aarcange@redhat.com, ak@linux.intel.com, akpm@linux-foundation.org, ben.widawsky@intel.com, dan.j.williams@intel.com, dave.hansen@intel.com, feng.tang@intel.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, rdunlap@infradead.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, ying.huang@intel.com Subject: [patch 073/192] mm/mempolicy: unify the parameter sanity check for mbind and set_mempolicy Message-ID: <20210701015103.yXMghYrpe%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Ceft8B61; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: gtr8duprb5youd1kamj1grfgp41yyc88 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6E64030000A5 X-HE-Tag: 1625104265-570016 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Feng Tang Subject: mm/mempolicy: unify the parameter sanity check for mbind and set_mempolicy Currently the kernel_mbind() and kernel_set_mempolicy() do almost the same operation for parameter sanity check. Add a helper function to unify the code to reduce the redundancy, and make it easier for changing the sanity check code in future. [thanks to David Rientjes for suggesting using helper function instead of macro]. [feng.tang@intel.com: add comment] Link: https://lkml.kernel.org/r/1622560492-1294-4-git-send-email-feng.tang@intel.com Link: https://lkml.kernel.org/r/1622469956-82897-4-git-send-email-feng.tang@intel.com Signed-off-by: Feng Tang Acked-by: Michal Hocko Cc: David Rientjes Cc: Dave Hansen Cc: Ben Widawsky Cc: Andrea Arcangeli Cc: Mel Gorman Cc: Mike Kravetz Cc: Randy Dunlap Cc: Vlastimil Babka Cc: Andi Kleen Cc: Dan Williams Cc: Huang Ying Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/mempolicy.c | 48 +++++++++++++++++++++++++++++------------------ 1 file changed, 30 insertions(+), 18 deletions(-) --- a/mm/mempolicy.c~mm-mempolicy-unify-the-parameter-sanity-check-for-mbind-and-set_mempolicy +++ a/mm/mempolicy.c @@ -1441,26 +1441,38 @@ static int copy_nodes_to_user(unsigned l return copy_to_user(mask, nodes_addr(*nodes), copy) ? -EFAULT : 0; } +/* Basic parameter sanity check used by both mbind() and set_mempolicy() */ +static inline int sanitize_mpol_flags(int *mode, unsigned short *flags) +{ + *flags = *mode & MPOL_MODE_FLAGS; + *mode &= ~MPOL_MODE_FLAGS; + if ((unsigned int)(*mode) >= MPOL_MAX) + return -EINVAL; + if ((*flags & MPOL_F_STATIC_NODES) && (*flags & MPOL_F_RELATIVE_NODES)) + return -EINVAL; + + return 0; +} + static long kernel_mbind(unsigned long start, unsigned long len, unsigned long mode, const unsigned long __user *nmask, unsigned long maxnode, unsigned int flags) { + unsigned short mode_flags; nodemask_t nodes; + int lmode = mode; int err; - unsigned short mode_flags; start = untagged_addr(start); - mode_flags = mode & MPOL_MODE_FLAGS; - mode &= ~MPOL_MODE_FLAGS; - if (mode >= MPOL_MAX) - return -EINVAL; - if ((mode_flags & MPOL_F_STATIC_NODES) && - (mode_flags & MPOL_F_RELATIVE_NODES)) - return -EINVAL; + err = sanitize_mpol_flags(&lmode, &mode_flags); + if (err) + return err; + err = get_nodes(&nodes, nmask, maxnode); if (err) return err; - return do_mbind(start, len, mode, mode_flags, &nodes, flags); + + return do_mbind(start, len, lmode, mode_flags, &nodes, flags); } SYSCALL_DEFINE6(mbind, unsigned long, start, unsigned long, len, @@ -1474,20 +1486,20 @@ SYSCALL_DEFINE6(mbind, unsigned long, st static long kernel_set_mempolicy(int mode, const unsigned long __user *nmask, unsigned long maxnode) { - int err; + unsigned short mode_flags; nodemask_t nodes; - unsigned short flags; + int lmode = mode; + int err; + + err = sanitize_mpol_flags(&lmode, &mode_flags); + if (err) + return err; - flags = mode & MPOL_MODE_FLAGS; - mode &= ~MPOL_MODE_FLAGS; - if ((unsigned int)mode >= MPOL_MAX) - return -EINVAL; - if ((flags & MPOL_F_STATIC_NODES) && (flags & MPOL_F_RELATIVE_NODES)) - return -EINVAL; err = get_nodes(&nodes, nmask, maxnode); if (err) return err; - return do_set_mempolicy(mode, flags, &nodes); + + return do_set_mempolicy(lmode, mode_flags, &nodes); } SYSCALL_DEFINE3(set_mempolicy, int, mode, const unsigned long __user *, nmask, From patchwork Thu Jul 1 01:51:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353207 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E15CC11F69 for ; Thu, 1 Jul 2021 01:51:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B49C561241 for ; Thu, 1 Jul 2021 01:51:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B49C561241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3CF158D021E; Wed, 30 Jun 2021 21:51:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 37F1E8D0202; Wed, 30 Jun 2021 21:51:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26DE88D021E; Wed, 30 Jun 2021 21:51:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id F086F8D0202 for ; Wed, 30 Jun 2021 21:51:08 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A41B5827DC63 for ; Thu, 1 Jul 2021 01:51:08 +0000 (UTC) X-FDA: 78312341016.10.0DEC2E7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id 63D73F000129 for ; Thu, 1 Jul 2021 01:51:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6D61161469; Thu, 1 Jul 2021 01:51:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104267; bh=wRLv2R4QkVRlU9cMNpyvXuwwWaXnK1gjWRMys2tx4m4=; h=Date:From:To:Subject:In-Reply-To:From; b=YHJTdK7r+SzJpFm3Nh46cN7R4Na9264fbXr8eE3g+D80zpp7+4Gffu3e5tDDA+E4G 9dePyASu7qcMkf/QshB6gNhpZ7lv0cbQwUpUWgq9/h5EqIrRjkm3E5kK5ootxYoS0s AU0KQinEnniGljgw5Y4OuRZfOe3vHCLiLHyFQ0wY= Date: Wed, 30 Jun 2021 18:51:07 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, nao.horiguchi@gmail.com, shy828301@gmail.com, torvalds@linux-foundation.org, ziy@nvidia.com Subject: [patch 074/192] mm: mempolicy: don't have to split pmd for huge zero page Message-ID: <20210701015107.mLNtftQiK%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=YHJTdK7r; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 4hso7xqozjbmoenste57iz9e9buz4zao X-Rspamd-Queue-Id: 63D73F000129 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104268-952101 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: mempolicy: don't have to split pmd for huge zero page When trying to migrate pages to obey mempolicy, the huge zero page is split by inserting base zero pfn to all PTEs, then the page table walk fallback to PTE level and just skips zero page. Skipping zero page for mempolicy has been the behavior of kernel since v2.6.16 due to commit f4598c8b3678 ("[PATCH] migration: make sure there is no attempt to migrate reserved pages."). So it seems pointless to split huge zero page, it could be just skipped like base zero page. Set ACTION_CONTINUE to prevent the walk_page_range() split the pmd for this case. Link: https://lkml.kernel.org/r/20210609172146.3594-1-shy828301@gmail.com Link: https://lkml.kernel.org/r/20210604203513.240709-1-shy828301@gmail.com Signed-off-by: Yang Shi Reviewed-by: Zi Yan Acked-by: Michal Hocko Cc: Naoya Horiguchi Cc: Kirill A. Shutemov Cc: Hugh Dickins Signed-off-by: Andrew Morton --- mm/mempolicy.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) --- a/mm/mempolicy.c~mm-mempolicy-dont-have-to-split-pmd-for-huge-zero-page +++ a/mm/mempolicy.c @@ -437,7 +437,8 @@ static inline bool queue_pages_required( /* * queue_pages_pmd() has four possible return values: - * 0 - pages are placed on the right node or queued successfully. + * 0 - pages are placed on the right node or queued successfully, or + * special page is met, i.e. huge zero page. * 1 - there is unmovable page, and MPOL_MF_MOVE* & MPOL_MF_STRICT were * specified. * 2 - THP was split. @@ -461,8 +462,7 @@ static int queue_pages_pmd(pmd_t *pmd, s page = pmd_page(*pmd); if (is_huge_zero_page(page)) { spin_unlock(ptl); - __split_huge_pmd(walk->vma, pmd, addr, false, NULL); - ret = 2; + walk->action = ACTION_CONTINUE; goto out; } if (!queue_pages_required(page, qp)) @@ -489,7 +489,8 @@ out: * and move them to the pagelist if they do. * * queue_pages_pte_range() has three possible return values: - * 0 - pages are placed on the right node or queued successfully. + * 0 - pages are placed on the right node or queued successfully, or + * special page is met, i.e. zero page. * 1 - there is unmovable page, and MPOL_MF_MOVE* & MPOL_MF_STRICT were * specified. * -EIO - only MPOL_MF_STRICT was specified and an existing page was already From patchwork Thu Jul 1 01:51:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D21FC11F68 for ; Thu, 1 Jul 2021 01:51:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CF8C86105A for ; Thu, 1 Jul 2021 01:51:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CF8C86105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 57D368D021F; Wed, 30 Jun 2021 21:51:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 553898D0202; Wed, 30 Jun 2021 21:51:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F52B8D021F; Wed, 30 Jun 2021 21:51:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 176B98D0202 for ; Wed, 30 Jun 2021 21:51:12 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DBDAE24886 for ; Thu, 1 Jul 2021 01:51:11 +0000 (UTC) X-FDA: 78312341142.19.398FCE1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id 84600900009A for ; Thu, 1 Jul 2021 01:51:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E5D161468; Thu, 1 Jul 2021 01:51:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104270; bh=JnVUbSLfZsZmoOyg28767H4B8Pe64nRGdlXby+6qmPY=; h=Date:From:To:Subject:In-Reply-To:From; b=zvykVeDhZZ97bfxv8Kicgwr0bX6Ch1bsxTMmIUv1HOg61G4ljIRz9kz0Y52u8TflQ lrAimpfj3c6SVyv+cVjCHu4GjXhufyA9oZooU2Mp2CMfW+zitsLU6fIFLFDxwwcuoN KZHj/nsoHKQ3S5NnXAYwycELQ46fTjHdWljQwGDQ= Date: Wed, 30 Jun 2021 18:51:10 -0700 From: Andrew Morton To: aarcange@redhat.com, ak@linux.intel.com, akpm@linux-foundation.org, ben.widawsky@intel.com, dave.hansen@intel.com, feng.tang@intel.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 075/192] mm/mempolicy: use unified 'nodes' for bind/interleave/prefer policies Message-ID: <20210701015110.uKksWhAPf%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 84600900009A X-Stat-Signature: bsiwjo3am3twy6fj78ur4rprfofd3pg7 Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=zvykVeDh; dmarc=none; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104271-755971 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ben Widawsky Subject: mm/mempolicy: use unified 'nodes' for bind/interleave/prefer policies Current structure 'mempolicy' uses a union to store the node info for bind/interleave/perfer policies. union { short preferred_node; /* preferred */ nodemask_t nodes; /* interleave/bind */ /* undefined for default */ } v; Since preferred node can also be represented by a nodemask_t with only ont bit set, unify these policies with using one nodemask_t 'nodes', which can remove a union, simplify the code and make it easier to support future's new policy's node info. Link: https://lore.kernel.org/r/20200630212517.308045-7-ben.widawsky@intel.com Link: https://lkml.kernel.org/r/1623399825-75651-1-git-send-email-feng.tang@intel.com Co-developed-by: Feng Tang Signed-off-by: Ben Widawsky Signed-off-by: Feng Tang Cc: Michal Hocko Cc: David Rientjes Cc: Dave Hansen Cc: Andrea Arcangeli Cc: Mel Gorman Cc: Mike Kravetz Cc: Vlastimil Babka Cc: Andi Kleen Signed-off-by: Andrew Morton --- include/linux/mempolicy.h | 7 -- mm/mempolicy.c | 96 ++++++++++++++++-------------------- 2 files changed, 46 insertions(+), 57 deletions(-) --- a/include/linux/mempolicy.h~mm-mempolicy-use-unified-nodes-for-bind-interleave-prefer-policies +++ a/include/linux/mempolicy.h @@ -46,11 +46,8 @@ struct mempolicy { atomic_t refcnt; unsigned short mode; /* See MPOL_* above */ unsigned short flags; /* See set_mempolicy() MPOL_F_* above */ - union { - short preferred_node; /* preferred */ - nodemask_t nodes; /* interleave/bind */ - /* undefined for default */ - } v; + nodemask_t nodes; /* interleave/bind/perfer */ + union { nodemask_t cpuset_mems_allowed; /* relative to these nodes */ nodemask_t user_nodemask; /* nodemask passed by user */ --- a/mm/mempolicy.c~mm-mempolicy-use-unified-nodes-for-bind-interleave-prefer-policies +++ a/mm/mempolicy.c @@ -193,7 +193,7 @@ static int mpol_new_interleave(struct me { if (nodes_empty(*nodes)) return -EINVAL; - pol->v.nodes = *nodes; + pol->nodes = *nodes; return 0; } @@ -201,7 +201,9 @@ static int mpol_new_preferred(struct mem { if (nodes_empty(*nodes)) return -EINVAL; - pol->v.preferred_node = first_node(*nodes); + + nodes_clear(pol->nodes); + node_set(first_node(*nodes), pol->nodes); return 0; } @@ -209,7 +211,7 @@ static int mpol_new_bind(struct mempolic { if (nodes_empty(*nodes)) return -EINVAL; - pol->v.nodes = *nodes; + pol->nodes = *nodes; return 0; } @@ -324,7 +326,7 @@ static void mpol_rebind_nodemask(struct else if (pol->flags & MPOL_F_RELATIVE_NODES) mpol_relative_nodemask(&tmp, &pol->w.user_nodemask, nodes); else { - nodes_remap(tmp, pol->v.nodes, pol->w.cpuset_mems_allowed, + nodes_remap(tmp, pol->nodes, pol->w.cpuset_mems_allowed, *nodes); pol->w.cpuset_mems_allowed = *nodes; } @@ -332,7 +334,7 @@ static void mpol_rebind_nodemask(struct if (nodes_empty(tmp)) tmp = *nodes; - pol->v.nodes = tmp; + pol->nodes = tmp; } static void mpol_rebind_preferred(struct mempolicy *pol, @@ -897,15 +899,12 @@ static void get_policy_nodemask(struct m switch (p->mode) { case MPOL_BIND: case MPOL_INTERLEAVE: - *nodes = p->v.nodes; + case MPOL_PREFERRED: + *nodes = p->nodes; break; case MPOL_LOCAL: /* return empty node mask for local allocation */ break; - - case MPOL_PREFERRED: - node_set(p->v.preferred_node, *nodes); - break; default: BUG(); } @@ -989,7 +988,7 @@ static long do_get_mempolicy(int *policy *policy = err; } else if (pol == current->mempolicy && pol->mode == MPOL_INTERLEAVE) { - *policy = next_node_in(current->il_prev, pol->v.nodes); + *policy = next_node_in(current->il_prev, pol->nodes); } else { err = -EINVAL; goto out; @@ -1857,14 +1856,14 @@ static int apply_policy_zone(struct memp BUG_ON(dynamic_policy_zone == ZONE_MOVABLE); /* - * if policy->v.nodes has movable memory only, + * if policy->nodes has movable memory only, * we apply policy when gfp_zone(gfp) = ZONE_MOVABLE only. * - * policy->v.nodes is intersect with node_states[N_MEMORY]. + * policy->nodes is intersect with node_states[N_MEMORY]. * so if the following test fails, it implies - * policy->v.nodes has movable memory only. + * policy->nodes has movable memory only. */ - if (!nodes_intersects(policy->v.nodes, node_states[N_HIGH_MEMORY])) + if (!nodes_intersects(policy->nodes, node_states[N_HIGH_MEMORY])) dynamic_policy_zone = ZONE_MOVABLE; return zone >= dynamic_policy_zone; @@ -1879,8 +1878,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, s /* Lower zones don't get a nodemask applied for MPOL_BIND */ if (unlikely(policy->mode == MPOL_BIND) && apply_policy_zone(policy, gfp_zone(gfp)) && - cpuset_nodemask_valid_mems_allowed(&policy->v.nodes)) - return &policy->v.nodes; + cpuset_nodemask_valid_mems_allowed(&policy->nodes)) + return &policy->nodes; return NULL; } @@ -1889,7 +1888,7 @@ nodemask_t *policy_nodemask(gfp_t gfp, s static int policy_node(gfp_t gfp, struct mempolicy *policy, int nd) { if (policy->mode == MPOL_PREFERRED) { - nd = policy->v.preferred_node; + nd = first_node(policy->nodes); } else { /* * __GFP_THISNODE shouldn't even be used with the bind policy @@ -1908,7 +1907,7 @@ static unsigned interleave_nodes(struct unsigned next; struct task_struct *me = current; - next = next_node_in(me->il_prev, policy->v.nodes); + next = next_node_in(me->il_prev, policy->nodes); if (next < MAX_NUMNODES) me->il_prev = next; return next; @@ -1932,7 +1931,7 @@ unsigned int mempolicy_slab_node(void) switch (policy->mode) { case MPOL_PREFERRED: - return policy->v.preferred_node; + return first_node(policy->nodes); case MPOL_INTERLEAVE: return interleave_nodes(policy); @@ -1948,7 +1947,7 @@ unsigned int mempolicy_slab_node(void) enum zone_type highest_zoneidx = gfp_zone(GFP_KERNEL); zonelist = &NODE_DATA(node)->node_zonelists[ZONELIST_FALLBACK]; z = first_zones_zonelist(zonelist, highest_zoneidx, - &policy->v.nodes); + &policy->nodes); return z->zone ? zone_to_nid(z->zone) : node; } case MPOL_LOCAL: @@ -1961,12 +1960,12 @@ unsigned int mempolicy_slab_node(void) /* * Do static interleaving for a VMA with known offset @n. Returns the n'th - * node in pol->v.nodes (starting from n=0), wrapping around if n exceeds the + * node in pol->nodes (starting from n=0), wrapping around if n exceeds the * number of present nodes. */ static unsigned offset_il_node(struct mempolicy *pol, unsigned long n) { - unsigned nnodes = nodes_weight(pol->v.nodes); + unsigned nnodes = nodes_weight(pol->nodes); unsigned target; int i; int nid; @@ -1974,9 +1973,9 @@ static unsigned offset_il_node(struct me if (!nnodes) return numa_node_id(); target = (unsigned int)n % nnodes; - nid = first_node(pol->v.nodes); + nid = first_node(pol->nodes); for (i = 0; i < target; i++) - nid = next_node(nid, pol->v.nodes); + nid = next_node(nid, pol->nodes); return nid; } @@ -2032,7 +2031,7 @@ int huge_node(struct vm_area_struct *vma } else { nid = policy_node(gfp_flags, *mpol, numa_node_id()); if ((*mpol)->mode == MPOL_BIND) - *nodemask = &(*mpol)->v.nodes; + *nodemask = &(*mpol)->nodes; } return nid; } @@ -2056,7 +2055,6 @@ int huge_node(struct vm_area_struct *vma bool init_nodemask_of_mempolicy(nodemask_t *mask) { struct mempolicy *mempolicy; - int nid; if (!(mask && current->mempolicy)) return false; @@ -2065,18 +2063,13 @@ bool init_nodemask_of_mempolicy(nodemask mempolicy = current->mempolicy; switch (mempolicy->mode) { case MPOL_PREFERRED: - nid = mempolicy->v.preferred_node; - init_nodemask_of_node(mask, nid); - break; - case MPOL_BIND: case MPOL_INTERLEAVE: - *mask = mempolicy->v.nodes; + *mask = mempolicy->nodes; break; case MPOL_LOCAL: - nid = numa_node_id(); - init_nodemask_of_node(mask, nid); + init_nodemask_of_node(mask, numa_node_id()); break; default: @@ -2110,7 +2103,7 @@ bool mempolicy_in_oom_domain(struct task task_lock(tsk); mempolicy = tsk->mempolicy; if (mempolicy && mempolicy->mode == MPOL_BIND) - ret = nodes_intersects(mempolicy->v.nodes, *mask); + ret = nodes_intersects(mempolicy->nodes, *mask); task_unlock(tsk); return ret; @@ -2184,7 +2177,7 @@ struct page *alloc_pages_vma(gfp_t gfp, * node in its nodemask, we allocate the standard way. */ if (pol->mode == MPOL_PREFERRED) - hpage_node = pol->v.preferred_node; + hpage_node = first_node(pol->nodes); nmask = policy_nodemask(gfp, pol); if (!nmask || node_isset(hpage_node, *nmask)) { @@ -2317,9 +2310,8 @@ bool __mpol_equal(struct mempolicy *a, s switch (a->mode) { case MPOL_BIND: case MPOL_INTERLEAVE: - return !!nodes_equal(a->v.nodes, b->v.nodes); case MPOL_PREFERRED: - return a->v.preferred_node == b->v.preferred_node; + return !!nodes_equal(a->nodes, b->nodes); case MPOL_LOCAL: return true; default: @@ -2459,7 +2451,7 @@ int mpol_misplaced(struct page *page, st break; case MPOL_PREFERRED: - polnid = pol->v.preferred_node; + polnid = first_node(pol->nodes); break; case MPOL_LOCAL: @@ -2469,7 +2461,7 @@ int mpol_misplaced(struct page *page, st case MPOL_BIND: /* Optimize placement among multiple nodes via NUMA balancing */ if (pol->flags & MPOL_F_MORON) { - if (node_isset(thisnid, pol->v.nodes)) + if (node_isset(thisnid, pol->nodes)) break; goto out; } @@ -2480,12 +2472,12 @@ int mpol_misplaced(struct page *page, st * else select nearest allowed node, if any. * If no allowed nodes, use current [!misplaced]. */ - if (node_isset(curnid, pol->v.nodes)) + if (node_isset(curnid, pol->nodes)) goto out; z = first_zones_zonelist( node_zonelist(numa_node_id(), GFP_HIGHUSER), gfp_zone(GFP_HIGHUSER), - &pol->v.nodes); + &pol->nodes); polnid = zone_to_nid(z->zone); break; @@ -2688,7 +2680,7 @@ int mpol_set_shared_policy(struct shared vma->vm_pgoff, sz, npol ? npol->mode : -1, npol ? npol->flags : -1, - npol ? nodes_addr(npol->v.nodes)[0] : NUMA_NO_NODE); + npol ? nodes_addr(npol->nodes)[0] : NUMA_NO_NODE); if (npol) { new = sp_alloc(vma->vm_pgoff, vma->vm_pgoff + sz, npol); @@ -2786,7 +2778,7 @@ void __init numa_policy_init(void) .refcnt = ATOMIC_INIT(1), .mode = MPOL_PREFERRED, .flags = MPOL_F_MOF | MPOL_F_MORON, - .v = { .preferred_node = nid, }, + .nodes = nodemask_of_node(nid), }; } @@ -2945,12 +2937,14 @@ int mpol_parse_str(char *str, struct mem * Save nodes for mpol_to_str() to show the tmpfs mount options * for /proc/mounts, /proc/pid/mounts and /proc/pid/mountinfo. */ - if (mode != MPOL_PREFERRED) - new->v.nodes = nodes; - else if (nodelist) - new->v.preferred_node = first_node(nodes); - else + if (mode != MPOL_PREFERRED) { + new->nodes = nodes; + } else if (nodelist) { + nodes_clear(new->nodes); + node_set(first_node(nodes), new->nodes); + } else { new->mode = MPOL_LOCAL; + } /* * Save nodes for contextualization: this will be used to "clone" @@ -2999,11 +2993,9 @@ void mpol_to_str(char *buffer, int maxle case MPOL_LOCAL: break; case MPOL_PREFERRED: - node_set(pol->v.preferred_node, nodes); - break; case MPOL_BIND: case MPOL_INTERLEAVE: - nodes = pol->v.nodes; + nodes = pol->nodes; break; default: WARN_ON_ONCE(1); From patchwork Thu Jul 1 01:51:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62A43C11F68 for ; Thu, 1 Jul 2021 01:51:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 16C6961241 for ; Thu, 1 Jul 2021 01:51:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16C6961241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8FFDA8D0220; Wed, 30 Jun 2021 21:51:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D6C38D0202; Wed, 30 Jun 2021 21:51:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DAF48D0220; Wed, 30 Jun 2021 21:51:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0164.hostedemail.com [216.40.44.164]) by kanga.kvack.org (Postfix) with ESMTP id 48E078D0202 for ; Wed, 30 Jun 2021 21:51:15 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1D1571802DA45 for ; Thu, 1 Jul 2021 01:51:15 +0000 (UTC) X-FDA: 78312341310.30.897B2FE Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id C159E90000A0 for ; Thu, 1 Jul 2021 01:51:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A48736105A; Thu, 1 Jul 2021 01:51:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104274; bh=gs9zDii9CyRCJNPF36isXF8ekseigKt6pZr9N5sv9JA=; h=Date:From:To:Subject:In-Reply-To:From; b=D/IijvlnzdDYRCsMkqTSltTapBkC3UpuNZZiW3CxdGnHiel17nnQcDn62VN5PQ7HH YDcsUCTFTD7jIU6Y8CY0VkurMbGaVlwI1Dd39D/lOs1mUAdJHxHQFaKpVWf6Hm4HOu 1jdduNlLAIkdkz6YISHB0THhMZIFIva8RbeKTW3Q= Date: Wed, 30 Jun 2021 18:51:13 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, wangkefeng.wang@huawei.com, will@kernel.org Subject: [patch 076/192] include/linux/mmzone.h: add documentation for pfn_valid() Message-ID: <20210701015113.if3pngvZe%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C159E90000A0 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="D/Iijvln"; spf=pass (imf23.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: rwtfwfqpenuqr6uhaqxpfqsbkzopzqaw X-HE-Tag: 1625104274-805169 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport Subject: include/linux/mmzone.h: add documentation for pfn_valid() Patch series "arm64: drop pfn_valid_within() and simplify pfn_valid()", v4. These patches aim to remove CONFIG_HOLES_IN_ZONE and essentially hardwire pfn_valid_within() to 1. The idea is to mark NOMAP pages as reserved in the memory map and restore the intended semantics of pfn_valid() to designate availability of struct page for a pfn. With this the core mm will be able to cope with the fact that it cannot use NOMAP pages and the holes created by NOMAP ranges within MAX_ORDER blocks will be treated correctly even without the need for pfn_valid_within. This patch (of 4): Add comment describing the semantics of pfn_valid() that clarifies that pfn_valid() only checks for availability of a memory map entry (i.e. struct page) for a PFN rather than availability of usable memory backing that PFN. The most "generic" version of pfn_valid() used by the configurations with SPARSEMEM enabled resides in include/linux/mmzone.h so this is the most suitable place for documentation about semantics of pfn_valid(). Link: https://lkml.kernel.org/r/20210511100550.28178-1-rppt@kernel.org Link: https://lkml.kernel.org/r/20210511100550.28178-2-rppt@kernel.org Signed-off-by: Mike Rapoport Suggested-by: Anshuman Khandual Reviewed-by: Anshuman Khandual Acked-by: Ard Biesheuvel Reviewed-by: Kefeng Wang Cc: Catalin Marinas Cc: David Hildenbrand Cc: Marc Zyngier Cc: Mark Rutland Cc: Mike Rapoport Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 11 +++++++++++ 1 file changed, 11 insertions(+) --- a/include/linux/mmzone.h~include-linux-mmzoneh-add-documentation-for-pfn_valid +++ a/include/linux/mmzone.h @@ -1445,6 +1445,17 @@ static inline int pfn_section_valid(stru #endif #ifndef CONFIG_HAVE_ARCH_PFN_VALID +/** + * pfn_valid - check if there is a valid memory map entry for a PFN + * @pfn: the page frame number to check + * + * Check if there is a valid memory map entry aka struct page for the @pfn. + * Note, that availability of the memory map entry does not imply that + * there is actual usable memory at that @pfn. The struct page may + * represent a hole or an unusable page frame. + * + * Return: 1 for PFNs that have memory map entries and 0 otherwise + */ static inline int pfn_valid(unsigned long pfn) { struct mem_section *ms; From patchwork Thu Jul 1 01:51:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353213 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8ED75C11F69 for ; Thu, 1 Jul 2021 01:51:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4735F61469 for ; Thu, 1 Jul 2021 01:51:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4735F61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C0CD98D0221; Wed, 30 Jun 2021 21:51:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BE3448D0202; Wed, 30 Jun 2021 21:51:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AABBC8D0221; Wed, 30 Jun 2021 21:51:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 89C018D0202 for ; Wed, 30 Jun 2021 21:51:18 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6740D24881 for ; Thu, 1 Jul 2021 01:51:18 +0000 (UTC) X-FDA: 78312341436.31.A31CB4C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 01584D000096 for ; Thu, 1 Jul 2021 01:51:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CD00261468; Thu, 1 Jul 2021 01:51:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104277; bh=Ow4VczhFIOphbSnW2FjqESy98R/wHerGjgUTcfHXkJc=; h=Date:From:To:Subject:In-Reply-To:From; b=SrgG26g2y9gXYv/XLZ55f013b1h4fHVnUdoe2amPqlJ+yYUtzHhKuMIzeJbTEe9JF o6heRXiAQMGxoJAFc7qJcdC/4kNT3SmMKeFOH+64BMZjLhLpE4MSHMKnQkulKqopHI cGgpLqqvWJxTQAojgu4xKt5pSXKjQn46OpgEk7cs= Date: Wed, 30 Jun 2021 18:51:16 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, wangkefeng.wang@huawei.com, will@kernel.org Subject: [patch 077/192] memblock: update initialization of reserved pages Message-ID: <20210701015116.pqqz6odd4%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 01584D000096 X-Stat-Signature: 7o1t1k5fw4ht98gia16n554uasa7m9so Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=SrgG26g2; dmarc=none; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104277-843133 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport Subject: memblock: update initialization of reserved pages The struct pages representing a reserved memory region are initialized using reserve_bootmem_range() function. This function is called for each reserved region just before the memory is freed from memblock to the buddy page allocator. The struct pages for MEMBLOCK_NOMAP regions are kept with the default values set by the memory map initialization which makes it necessary to have a special treatment for such pages in pfn_valid() and pfn_valid_within(). Split out initialization of the reserved pages to a function with a meaningful name and treat the MEMBLOCK_NOMAP regions the same way as the reserved regions and mark struct pages for the NOMAP regions as PageReserved. Link: https://lkml.kernel.org/r/20210511100550.28178-3-rppt@kernel.org Signed-off-by: Mike Rapoport Reviewed-by: David Hildenbrand Reviewed-by: Anshuman Khandual Acked-by: Ard Biesheuvel Reviewed-by: Kefeng Wang Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Rutland Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/memblock.h | 4 +++- mm/memblock.c | 28 ++++++++++++++++++++++++++-- 2 files changed, 29 insertions(+), 3 deletions(-) --- a/include/linux/memblock.h~memblock-update-initialization-of-reserved-pages +++ a/include/linux/memblock.h @@ -30,7 +30,9 @@ extern unsigned long long max_possible_p * @MEMBLOCK_NONE: no special request * @MEMBLOCK_HOTPLUG: hotpluggable region * @MEMBLOCK_MIRROR: mirrored region - * @MEMBLOCK_NOMAP: don't add to kernel direct mapping + * @MEMBLOCK_NOMAP: don't add to kernel direct mapping and treat as + * reserved in the memory map; refer to memblock_mark_nomap() description + * for further details */ enum memblock_flags { MEMBLOCK_NONE = 0x0, /* No special request */ --- a/mm/memblock.c~memblock-update-initialization-of-reserved-pages +++ a/mm/memblock.c @@ -906,6 +906,11 @@ int __init_memblock memblock_mark_mirror * @base: the base phys addr of the region * @size: the size of the region * + * The memory regions marked with %MEMBLOCK_NOMAP will not be added to the + * direct mapping of the physical memory. These regions will still be + * covered by the memory map. The struct page representing NOMAP memory + * frames in the memory map will be PageReserved() + * * Return: 0 on success, -errno on failure. */ int __init_memblock memblock_mark_nomap(phys_addr_t base, phys_addr_t size) @@ -2002,6 +2007,26 @@ static unsigned long __init __free_memor return end_pfn - start_pfn; } +static void __init memmap_init_reserved_pages(void) +{ + struct memblock_region *region; + phys_addr_t start, end; + u64 i; + + /* initialize struct pages for the reserved regions */ + for_each_reserved_mem_range(i, &start, &end) + reserve_bootmem_region(start, end); + + /* and also treat struct pages for the NOMAP regions as PageReserved */ + for_each_mem_region(region) { + if (memblock_is_nomap(region)) { + start = region->base; + end = start + region->size; + reserve_bootmem_region(start, end); + } + } +} + static unsigned long __init free_low_memory_core_early(void) { unsigned long count = 0; @@ -2010,8 +2035,7 @@ static unsigned long __init free_low_mem memblock_clear_hotplug(0, -1); - for_each_reserved_mem_range(i, &start, &end) - reserve_bootmem_region(start, end); + memmap_init_reserved_pages(); /* * We need to use NUMA_NO_NODE instead of NODE_DATA(0)->node_id From patchwork Thu Jul 1 01:51:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353215 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE91EC11F68 for ; Thu, 1 Jul 2021 01:51:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8968361469 for ; Thu, 1 Jul 2021 01:51:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8968361469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 110788D0222; Wed, 30 Jun 2021 21:51:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E7928D0202; Wed, 30 Jun 2021 21:51:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF1EE8D0222; Wed, 30 Jun 2021 21:51:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0189.hostedemail.com [216.40.44.189]) by kanga.kvack.org (Postfix) with ESMTP id CC7168D0202 for ; Wed, 30 Jun 2021 21:51:21 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 95E8D1C9A1 for ; Thu, 1 Jul 2021 01:51:21 +0000 (UTC) X-FDA: 78312341562.29.9542A91 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id 22BC5900009A for ; Thu, 1 Jul 2021 01:51:21 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 133346105A; Thu, 1 Jul 2021 01:51:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104280; bh=YYTDxqTLO8oAwMJ92YvzyVaLGdGA6vqLkJ2NZzliz7U=; h=Date:From:To:Subject:In-Reply-To:From; b=xIyumq6JDdZC3dSAHPq+NdsaXjY+RE1BjQWbgDi0eI1mlOYAar3MtcHYSvOFtNrzy BE+Q4r6/50UeWNGRXPgmNjz9rhUbAYcDYCFmfmBNJiUMPoyMx2MOE9I3ShJU/KDi6F IXYwkgpuV4dS6qpoyG4ueNX7wY3bvSKzkMzhAICk= Date: Wed, 30 Jun 2021 18:51:19 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, wangkefeng.wang@huawei.com, will@kernel.org Subject: [patch 078/192] arm64: decouple check whether pfn is in linear map from pfn_valid() Message-ID: <20210701015119.-TOEhD-PZ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 22BC5900009A Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=xIyumq6J; spf=pass (imf28.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 5di58tganigeqfcupukajqmdzmexxyx1 X-HE-Tag: 1625104281-2688 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport Subject: arm64: decouple check whether pfn is in linear map from pfn_valid() The intended semantics of pfn_valid() is to verify whether there is a struct page for the pfn in question and nothing else. Yet, on arm64 it is used to distinguish memory areas that are mapped in the linear map vs those that require ioremap() to access them. Introduce a dedicated pfn_is_map_memory() wrapper for memblock_is_map_memory() to perform such check and use it where appropriate. Using a wrapper allows to avoid cyclic include dependencies. While here also update style of pfn_valid() so that both pfn_valid() and pfn_is_map_memory() declarations will be consistent. Link: https://lkml.kernel.org/r/20210511100550.28178-4-rppt@kernel.org Signed-off-by: Mike Rapoport Acked-by: David Hildenbrand Acked-by: Ard Biesheuvel Reviewed-by: Kefeng Wang Cc: Anshuman Khandual Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Rutland Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/memory.h | 2 +- arch/arm64/include/asm/page.h | 3 ++- arch/arm64/kvm/mmu.c | 2 +- arch/arm64/mm/init.c | 12 ++++++++++++ arch/arm64/mm/ioremap.c | 4 ++-- arch/arm64/mm/mmu.c | 2 +- 6 files changed, 19 insertions(+), 6 deletions(-) --- a/arch/arm64/include/asm/memory.h~arm64-decouple-check-whether-pfn-is-in-linear-map-from-pfn_valid +++ a/arch/arm64/include/asm/memory.h @@ -369,7 +369,7 @@ static inline void *phys_to_virt(phys_ad #define virt_addr_valid(addr) ({ \ __typeof__(addr) __addr = __tag_reset(addr); \ - __is_lm_address(__addr) && pfn_valid(virt_to_pfn(__addr)); \ + __is_lm_address(__addr) && pfn_is_map_memory(virt_to_pfn(__addr)); \ }) void dump_mem_limit(void); --- a/arch/arm64/include/asm/page.h~arm64-decouple-check-whether-pfn-is-in-linear-map-from-pfn_valid +++ a/arch/arm64/include/asm/page.h @@ -37,7 +37,8 @@ void copy_highpage(struct page *to, stru typedef struct page *pgtable_t; -extern int pfn_valid(unsigned long); +int pfn_valid(unsigned long pfn); +int pfn_is_map_memory(unsigned long pfn); #include --- a/arch/arm64/kvm/mmu.c~arm64-decouple-check-whether-pfn-is-in-linear-map-from-pfn_valid +++ a/arch/arm64/kvm/mmu.c @@ -85,7 +85,7 @@ void kvm_flush_remote_tlbs(struct kvm *k static bool kvm_is_device_pfn(unsigned long pfn) { - return !pfn_valid(pfn); + return !pfn_is_map_memory(pfn); } static void *stage2_memcache_zalloc_page(void *arg) --- a/arch/arm64/mm/init.c~arm64-decouple-check-whether-pfn-is-in-linear-map-from-pfn_valid +++ a/arch/arm64/mm/init.c @@ -256,6 +256,18 @@ int pfn_valid(unsigned long pfn) } EXPORT_SYMBOL(pfn_valid); +int pfn_is_map_memory(unsigned long pfn) +{ + phys_addr_t addr = PFN_PHYS(pfn); + + /* avoid false positives for bogus PFNs, see comment in pfn_valid() */ + if (PHYS_PFN(addr) != pfn) + return 0; + + return memblock_is_map_memory(addr); +} +EXPORT_SYMBOL(pfn_is_map_memory); + static phys_addr_t memory_limit = PHYS_ADDR_MAX; /* --- a/arch/arm64/mm/ioremap.c~arm64-decouple-check-whether-pfn-is-in-linear-map-from-pfn_valid +++ a/arch/arm64/mm/ioremap.c @@ -43,7 +43,7 @@ static void __iomem *__ioremap_caller(ph /* * Don't allow RAM to be mapped. */ - if (WARN_ON(pfn_valid(__phys_to_pfn(phys_addr)))) + if (WARN_ON(pfn_is_map_memory(__phys_to_pfn(phys_addr)))) return NULL; area = get_vm_area_caller(size, VM_IOREMAP, caller); @@ -84,7 +84,7 @@ EXPORT_SYMBOL(iounmap); void __iomem *ioremap_cache(phys_addr_t phys_addr, size_t size) { /* For normal memory we already have a cacheable mapping. */ - if (pfn_valid(__phys_to_pfn(phys_addr))) + if (pfn_is_map_memory(__phys_to_pfn(phys_addr))) return (void __iomem *)__phys_to_virt(phys_addr); return __ioremap_caller(phys_addr, size, __pgprot(PROT_NORMAL), --- a/arch/arm64/mm/mmu.c~arm64-decouple-check-whether-pfn-is-in-linear-map-from-pfn_valid +++ a/arch/arm64/mm/mmu.c @@ -82,7 +82,7 @@ void set_swapper_pgd(pgd_t *pgdp, pgd_t pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, unsigned long size, pgprot_t vma_prot) { - if (!pfn_valid(pfn)) + if (!pfn_is_map_memory(pfn)) return pgprot_noncached(vma_prot); else if (file->f_flags & O_SYNC) return pgprot_writecombine(vma_prot); From patchwork Thu Jul 1 01:51:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1BB0C11F69 for ; Thu, 1 Jul 2021 01:51:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A128661468 for ; Thu, 1 Jul 2021 01:51:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A128661468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 26F758D0223; Wed, 30 Jun 2021 21:51:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 245E28D0202; Wed, 30 Jun 2021 21:51:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 136808D0223; Wed, 30 Jun 2021 21:51:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0125.hostedemail.com [216.40.44.125]) by kanga.kvack.org (Postfix) with ESMTP id DFA038D0202 for ; Wed, 30 Jun 2021 21:51:24 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id B6AE0180300DD for ; Thu, 1 Jul 2021 01:51:24 +0000 (UTC) X-FDA: 78312341688.32.96BAEF9 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 78C345000098 for ; Thu, 1 Jul 2021 01:51:24 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4605161469; Thu, 1 Jul 2021 01:51:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104283; bh=Ty1K6Dbhu2efMQRHCLoL8TCxA2XV9UNWYXZg0QHl6nY=; h=Date:From:To:Subject:In-Reply-To:From; b=CPYDBy9gATsYp4QOPqS660Lz05pmXr03jzr1ZW6WIx5X49V89naUtZALDnFBe4K2S G9TJ8HxrBDL0UyIZCNwE80+uQkreWGzsKaFNih0NrfVZxfcowSTdJ6J+1DxOaPB+ns H7tE2zUx88WntvB1deucjbmfajaxhoDq4KB9S2kw= Date: Wed, 30 Jun 2021 18:51:22 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, ardb@kernel.org, catalin.marinas@arm.com, david@redhat.com, linux-mm@kvack.org, mark.rutland@arm.com, maz@kernel.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, wangkefeng.wang@huawei.com, will@kernel.org Subject: [patch 079/192] arm64: drop pfn_valid_within() and simplify pfn_valid() Message-ID: <20210701015122.VdSqIdHx4%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 78C345000098 X-Stat-Signature: 6jgti9hk8psnjerosjuhwuac1bb7poew Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=CPYDBy9g; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104284-97817 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Rapoport Subject: arm64: drop pfn_valid_within() and simplify pfn_valid() The arm64's version of pfn_valid() differs from the generic because of two reasons: * Parts of the memory map are freed during boot. This makes it necessary to verify that there is actual physical memory that corresponds to a pfn which is done by querying memblock. * There are NOMAP memory regions. These regions are not mapped in the linear map and until the previous commit the struct pages representing these areas had default values. As the consequence of absence of the special treatment of NOMAP regions in the memory map it was necessary to use memblock_is_map_memory() in pfn_valid() and to have pfn_valid_within() aliased to pfn_valid() so that generic mm functionality would not treat a NOMAP page as a normal page. Since the NOMAP regions are now marked as PageReserved(), pfn walkers and the rest of core mm will treat them as unusable memory and thus pfn_valid_within() is no longer required at all and can be disabled on arm64. pfn_valid() can be slightly simplified by replacing memblock_is_map_memory() with memblock_is_memory(). [rppt@kernel.org: fix merge fix] Link: https://lkml.kernel.org/r/YJtoQhidtIJOhYsV@kernel.org Link: https://lkml.kernel.org/r/20210511100550.28178-5-rppt@kernel.org Signed-off-by: Mike Rapoport Acked-by: David Hildenbrand Acked-by: Ard Biesheuvel Reviewed-by: Kefeng Wang Cc: Anshuman Khandual Cc: Catalin Marinas Cc: Marc Zyngier Cc: Mark Rutland Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 1 - arch/arm64/mm/init.c | 2 +- 2 files changed, 1 insertion(+), 2 deletions(-) --- a/arch/arm64/Kconfig~arm64-drop-pfn_valid_within-and-simplify-pfn_valid +++ a/arch/arm64/Kconfig @@ -201,7 +201,6 @@ config ARM64 select HAVE_KPROBES select HAVE_KRETPROBES select HAVE_GENERIC_VDSO - select HOLES_IN_ZONE select IOMMU_DMA if IOMMU_SUPPORT select IRQ_DOMAIN select IRQ_FORCED_THREADING --- a/arch/arm64/mm/init.c~arm64-drop-pfn_valid_within-and-simplify-pfn_valid +++ a/arch/arm64/mm/init.c @@ -252,7 +252,7 @@ int pfn_valid(unsigned long pfn) if (!early_section(ms)) return pfn_section_valid(ms, pfn); - return memblock_is_map_memory(addr); + return memblock_is_memory(addr); } EXPORT_SYMBOL(pfn_valid); From patchwork Thu Jul 1 01:51:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353219 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDFB5C11F66 for ; Thu, 1 Jul 2021 01:51:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A54796105A for ; Thu, 1 Jul 2021 01:51:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A54796105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2E4C28D0224; Wed, 30 Jun 2021 21:51:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BADE8D0202; Wed, 30 Jun 2021 21:51:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 183438D0224; Wed, 30 Jun 2021 21:51:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id EA26B8D0202 for ; Wed, 30 Jun 2021 21:51:27 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C6B4E82E4C0A for ; Thu, 1 Jul 2021 01:51:27 +0000 (UTC) X-FDA: 78312341814.14.FC1713A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 720E7F0000AE for ; Thu, 1 Jul 2021 01:51:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6FF4661468; Thu, 1 Jul 2021 01:51:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104286; bh=+jBkxEESAKb+5hPJN3twpYZizEk5SUOLczGPvYeg5+0=; h=Date:From:To:Subject:In-Reply-To:From; b=iccqal9Y2etDGTsMTvcJdQ3fadDIQYio2zzpcxj7/kfDHgpjl1qCc0UUCerNxAHHu QUSayEqAS5V1CDLn5TZd47T+Y82bJfN0/FREV03i9ZKiq0EZBlzsgv3OSNauhXUTv5 yacWTP7eZ4aJKC6LecJNhPN8AtzVBlWFchb+uZ+o= Date: Wed, 30 Jun 2021 18:51:26 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, catalin.marinas@arm.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, will@kernel.org Subject: [patch 080/192] arm64/mm: drop HAVE_ARCH_PFN_VALID Message-ID: <20210701015126.ZgBRuFydL%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 720E7F0000AE Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=iccqal9Y; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 1j99wb6ifb1r67giud3dbngqcjxfeero X-HE-Tag: 1625104287-918287 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: arm64/mm: drop HAVE_ARCH_PFN_VALID CONFIG_SPARSEMEM_VMEMMAP is now the only available memory model on arm64 platforms and free_unused_memmap() would just return without creating any holes in the memmap mapping. There is no need for any special handling in pfn_valid() and HAVE_ARCH_PFN_VALID can just be dropped. This also moves the pfn upper bits sanity check into generic pfn_valid(). Link: https://lkml.kernel.org/r/1621947349-25421-1-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Acked-by: David Hildenbrand Acked-by: Mike Rapoport Cc: Catalin Marinas Cc: Will Deacon Cc: David Hildenbrand Cc: Mike Rapoport Signed-off-by: Andrew Morton --- arch/arm64/Kconfig | 1 arch/arm64/include/asm/page.h | 1 arch/arm64/mm/init.c | 37 -------------------------------- include/linux/mmzone.h | 9 +++++++ 4 files changed, 9 insertions(+), 39 deletions(-) --- a/arch/arm64/include/asm/page.h~arm64-mm-drop-have_arch_pfn_valid +++ a/arch/arm64/include/asm/page.h @@ -37,7 +37,6 @@ void copy_highpage(struct page *to, stru typedef struct page *pgtable_t; -int pfn_valid(unsigned long pfn); int pfn_is_map_memory(unsigned long pfn); #include --- a/arch/arm64/Kconfig~arm64-mm-drop-have_arch_pfn_valid +++ a/arch/arm64/Kconfig @@ -154,7 +154,6 @@ config ARM64 select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT - select HAVE_ARCH_PFN_VALID select HAVE_ARCH_PREL32_RELOCATIONS select HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET select HAVE_ARCH_SECCOMP_FILTER --- a/arch/arm64/mm/init.c~arm64-mm-drop-have_arch_pfn_valid +++ a/arch/arm64/mm/init.c @@ -219,43 +219,6 @@ static void __init zone_sizes_init(unsig free_area_init(max_zone_pfns); } -int pfn_valid(unsigned long pfn) -{ - phys_addr_t addr = PFN_PHYS(pfn); - struct mem_section *ms; - - /* - * Ensure the upper PAGE_SHIFT bits are clear in the - * pfn. Else it might lead to false positives when - * some of the upper bits are set, but the lower bits - * match a valid pfn. - */ - if (PHYS_PFN(addr) != pfn) - return 0; - - if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) - return 0; - - ms = __pfn_to_section(pfn); - if (!valid_section(ms)) - return 0; - - /* - * ZONE_DEVICE memory does not have the memblock entries. - * memblock_is_map_memory() check for ZONE_DEVICE based - * addresses will always fail. Even the normal hotplugged - * memory will never have MEMBLOCK_NOMAP flag set in their - * memblock entries. Skip memblock search for all non early - * memory sections covering all of hotplug memory including - * both normal and ZONE_DEVICE based. - */ - if (!early_section(ms)) - return pfn_section_valid(ms, pfn); - - return memblock_is_memory(addr); -} -EXPORT_SYMBOL(pfn_valid); - int pfn_is_map_memory(unsigned long pfn) { phys_addr_t addr = PFN_PHYS(pfn); --- a/include/linux/mmzone.h~arm64-mm-drop-have_arch_pfn_valid +++ a/include/linux/mmzone.h @@ -1460,6 +1460,15 @@ static inline int pfn_valid(unsigned lon { struct mem_section *ms; + /* + * Ensure the upper PAGE_SHIFT bits are clear in the + * pfn. Else it might lead to false positives when + * some of the upper bits are set, but the lower bits + * match a valid pfn. + */ + if (PHYS_PFN(PFN_PHYS(pfn)) != pfn) + return 0; + if (pfn_to_section_nr(pfn) >= NR_MEM_SECTIONS) return 0; ms = __nr_to_section(pfn_to_section_nr(pfn)); From patchwork Thu Jul 1 01:51:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F042C11F69 for ; Thu, 1 Jul 2021 01:51:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE77661468 for ; Thu, 1 Jul 2021 01:51:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE77661468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 778D88D0225; Wed, 30 Jun 2021 21:51:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74ED08D0202; Wed, 30 Jun 2021 21:51:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 617478D0225; Wed, 30 Jun 2021 21:51:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 3E77A8D0202 for ; Wed, 30 Jun 2021 21:51:31 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 04B44181AEF3F for ; Thu, 1 Jul 2021 01:51:31 +0000 (UTC) X-FDA: 78312341982.27.3EFD4B2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id AE611400208B for ; Thu, 1 Jul 2021 01:51:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8C4F26105A; Thu, 1 Jul 2021 01:51:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104289; bh=xW4yljaUS6tfIAND3LK85LpGFwp48IwJfWOcN1kgIJs=; h=Date:From:To:Subject:In-Reply-To:From; b=SzHLSisM8eefH7WEcJJw4orqutWkkfwtGTXsu1YpRi5kka7iEkKpdT7hbxdz3R7Y2 TruA2z2hgcgiZoyy5pvgkWLBxV9uGqD1MVHMdcaOuEoUEoxFhWxuBxCafUDsrKCZUT ITZcFxjtun2UYAnB+ZldBuLvsSo2iuk2wb/efkhY= Date: Wed, 30 Jun 2021 18:51:29 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, david@redhat.com, duanxiongchun@bytedance.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 081/192] mm: migrate: fix missing update page_private to hugetlb_page_subpool Message-ID: <20210701015129.UHD_0dddK%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=SzHLSisM; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: AE611400208B X-Stat-Signature: iw9tz331wo4ia5cgnpnr53gf96namc9r X-HE-Tag: 1625104290-687647 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: migrate: fix missing update page_private to hugetlb_page_subpool Since commit d6995da31122 ("hugetlb: use page.private for hugetlb specific page flags") converts page.private for hugetlb specific page flags. We should use hugetlb_page_subpool() to get the subpool pointer instead of page_private(). This 'could' prevent the migration of hugetlb pages. page_private(hpage) is now used for hugetlb page specific flags. At migration time, the only flag which could be set is HPageVmemmapOptimized. This flag will only be set if the new vmemmap reduction feature is enabled. In addition, !page_mapping() implies an anonymous mapping. So, this will prevent migration of hugetb pages in anonymous mappings if the vmemmap reduction feature is enabled. In addition, that if statement checked for the rare race condition of a page being migrated while in the process of being freed. Since that check is now wrong, we could leak hugetlb subpool usage counts. The commit forgot to update it in the page migration routine. So fix it. [songmuchun@bytedance.com: fix compiler error when !CONFIG_HUGETLB_PAGE reported by Randy] Link: https://lkml.kernel.org/r/20210521022747.35736-1-songmuchun@bytedance.com Link: https://lkml.kernel.org/r/20210520025949.1866-1-songmuchun@bytedance.com Fixes: d6995da31122 ("hugetlb: use page.private for hugetlb specific page flags") Signed-off-by: Muchun Song Reported-by: Anshuman Khandual Reviewed-by: Mike Kravetz Acked-by: Michal Hocko Tested-by: Anshuman Khandual [arm64] Cc: Oscar Salvador Cc: David Hildenbrand Cc: Matthew Wilcox Cc: Xiongchun Duan Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 5 +++++ mm/migrate.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) --- a/include/linux/hugetlb.h~mm-migrate-fix-missing-update-page_private-to-hugetlb_page_subpool +++ a/include/linux/hugetlb.h @@ -898,6 +898,11 @@ static inline void huge_ptep_modify_prot #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; +static inline struct hugepage_subpool *hugetlb_page_subpool(struct page *hpage) +{ + return NULL; +} + static inline int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list) { --- a/mm/migrate.c~mm-migrate-fix-missing-update-page_private-to-hugetlb_page_subpool +++ a/mm/migrate.c @@ -1293,7 +1293,7 @@ static int unmap_and_move_huge_page(new_ * page_mapping() set, hugetlbfs specific move page routine will not * be called and we could leak usage counts for subpools. */ - if (page_private(hpage) && !page_mapping(hpage)) { + if (hugetlb_page_subpool(hpage) && !page_mapping(hpage)) { rc = -EBUSY; goto out_unlock; } From patchwork Thu Jul 1 01:51:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C3DEC11F68 for ; Thu, 1 Jul 2021 01:51:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 262286146D for ; Thu, 1 Jul 2021 01:51:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 262286146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A10E48D0226; Wed, 30 Jun 2021 21:51:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E6968D0202; Wed, 30 Jun 2021 21:51:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AFDA8D0226; Wed, 30 Jun 2021 21:51:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 672B38D0202 for ; Wed, 30 Jun 2021 21:51:34 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3977424886 for ; Thu, 1 Jul 2021 01:51:34 +0000 (UTC) X-FDA: 78312342108.29.7116E33 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id D0F42100009D for ; Thu, 1 Jul 2021 01:51:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C291461468; Thu, 1 Jul 2021 01:51:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104293; bh=8YUHM8dYnPIYa+Nhy8nNmU9CFXOEVM0BzMzTrZBtiNI=; h=Date:From:To:Subject:In-Reply-To:From; b=SMqC7uwcVHXzOhWO5b/8wBuYcVb3cABrPc6fWf14F1fVwyEJCbDupwgKmKEZu1kV5 MdzZhOQFQbFLiHAsks7aCxAmgoAGOpsqiR9lhVDrZ7OA40iNjhXUj4KALIJee8IFwr CSn05pIPmnWDcxaM/OSWC6ThN24I0MCZFD1iSDgQ= Date: Wed, 30 Jun 2021 18:51:32 -0700 From: Andrew Morton To: akpm@linux-foundation.org, cfijalkovich@google.com, hridya@google.com, hughd@google.com, kaleshsingh@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, song@kernel.org, surenb@google.com, timmurray@google.com, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, william.kucharski@oracle.com, willy@infradead.org Subject: [patch 082/192] mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs Message-ID: <20210701015132.RLAMMMWAa%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=SMqC7uwc; dmarc=none; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: 5sbp64sh1qgthe7phsddhhih5hbocppn X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D0F42100009D X-HE-Tag: 1625104293-866802 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Collin Fijalkovich Subject: mm, thp: relax the VM_DENYWRITE constraint on file-backed THPs Transparent huge pages are supported for read-only non-shmem files, but are only used for vmas with VM_DENYWRITE. This condition ensures that file THPs are protected from writes while an application is running (ETXTBSY). Any existing file THPs are then dropped from the page cache when a file is opened for write in do_dentry_open(). Since sys_mmap ignores MAP_DENYWRITE, this constrains the use of file THPs to vmas produced by execve(). Systems that make heavy use of shared libraries (e.g. Android) are unable to apply VM_DENYWRITE through the dynamic linker, preventing them from benefiting from the resultant reduced contention on the TLB. This patch reduces the constraint on file THPs allowing use with any executable mapping from a file not opened for write (see inode_is_open_for_write()). It also introduces additional conditions to ensure that files opened for write will never be backed by file THPs. Restricting the use of THPs to executable mappings eliminates the risk that a read-only file later opened for write would encounter significant latencies due to page cache truncation. The ld linker flag '-z max-page-size=(hugepage size)' can be used to produce executables with the necessary layout. The dynamic linker must map these file's segments at a hugepage size aligned vma for the mapping to be backed with THPs. Comparison of the performance characteristics of 4KB and 2MB-backed libraries follows; the Android dex2oat tool was used to AOT compile an example application on a single ARM core. 4KB Pages: ========== count event_name # count / runtime 598,995,035,942 cpu-cycles # 1.800861 GHz 81,195,620,851 raw-stall-frontend # 244.112 M/sec 347,754,466,597 iTLB-loads # 1.046 G/sec 2,970,248,900 iTLB-load-misses # 0.854122% miss rate Total test time: 332.854998 seconds. 2MB Pages: ========== count event_name # count / runtime 592,872,663,047 cpu-cycles # 1.800358 GHz 76,485,624,143 raw-stall-frontend # 232.261 M/sec 350,478,413,710 iTLB-loads # 1.064 G/sec 803,233,322 iTLB-load-misses # 0.229182% miss rate Total test time: 329.826087 seconds A check of /proc/$(pidof dex2oat64)/smaps shows THPs in use: /apex/com.android.art/lib64/libart.so FilePmdMapped: 4096 kB /apex/com.android.art/lib64/libart-compiler.so FilePmdMapped: 2048 kB Link: https://lkml.kernel.org/r/20210406000930.3455850-1-cfijalkovich@google.com Signed-off-by: Collin Fijalkovich Acked-by: Hugh Dickins Reviewed-by: William Kucharski Acked-by: Song Liu Cc: Suren Baghdasaryan Cc: Hridya Valsaraju Cc: Kalesh Singh Cc: Tim Murray Cc: Matthew Wilcox Cc: Alexander Viro Signed-off-by: Andrew Morton --- fs/open.c | 13 +++++++++++-- mm/khugepaged.c | 16 +++++++++++++++- 2 files changed, 26 insertions(+), 3 deletions(-) --- a/fs/open.c~mm-thp-relax-the-vm_denywrite-constraint-on-file-backed-thps +++ a/fs/open.c @@ -852,8 +852,17 @@ static int do_dentry_open(struct file *f * XXX: Huge page cache doesn't support writing yet. Drop all page * cache for this file before processing writes. */ - if ((f->f_mode & FMODE_WRITE) && filemap_nr_thps(inode->i_mapping)) - truncate_pagecache(inode, 0); + if (f->f_mode & FMODE_WRITE) { + /* + * Paired with smp_mb() in collapse_file() to ensure nr_thps + * is up to date and the update to i_writecount by + * get_write_access() is visible. Ensures subsequent insertion + * of THPs into the page cache will fail. + */ + smp_mb(); + if (filemap_nr_thps(inode->i_mapping)) + truncate_pagecache(inode, 0); + } return 0; --- a/mm/khugepaged.c~mm-thp-relax-the-vm_denywrite-constraint-on-file-backed-thps +++ a/mm/khugepaged.c @@ -457,7 +457,8 @@ static bool hugepage_vma_check(struct vm /* Read-only file mappings need to be aligned for THP to work. */ if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file && - (vm_flags & VM_DENYWRITE)) { + !inode_is_open_for_write(vma->vm_file->f_inode) && + (vm_flags & VM_EXEC)) { return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR); } @@ -1862,6 +1863,19 @@ out_unlock: else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, nr); filemap_nr_thps_inc(mapping); + /* + * Paired with smp_mb() in do_dentry_open() to ensure + * i_writecount is up to date and the update to nr_thps is + * visible. Ensures the page cache will be truncated if the + * file is opened writable. + */ + smp_mb(); + if (inode_is_open_for_write(mapping->host)) { + result = SCAN_FAIL; + __mod_lruvec_page_state(new_page, NR_FILE_THPS, -nr); + filemap_nr_thps_dec(mapping); + goto xa_locked; + } } if (nr_none) { From patchwork Thu Jul 1 01:51:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F19B7C11F69 for ; Thu, 1 Jul 2021 01:51:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A3A7C61241 for ; Thu, 1 Jul 2021 01:51:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3A7C61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2BCEB8D0227; Wed, 30 Jun 2021 21:51:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 293758D0202; Wed, 30 Jun 2021 21:51:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 135D88D0227; Wed, 30 Jun 2021 21:51:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id E5D4A8D0202 for ; Wed, 30 Jun 2021 21:51:37 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B3742181AEF3F for ; Thu, 1 Jul 2021 01:51:37 +0000 (UTC) X-FDA: 78312342234.31.8EA1C1B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 601E95000098 for ; Thu, 1 Jul 2021 01:51:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 381C261469; Thu, 1 Jul 2021 01:51:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104296; bh=Rt/oHMa1L1El/BUpRprYlUKkViEDfGzSGjXG32ptjhI=; h=Date:From:To:Subject:In-Reply-To:From; b=pTSpvGPFgNwrm1JULidu+oDI1efQVCCuqs7QAplNzOgY2+f++fPOGDCUb1mV4cbj1 ZXhMxD5XBqh8DuZCQh6l5sJ4G7dBKRjbF9mDHAGBtNr3fGgRjVlsSnuh9GdVkv5/a3 mzTXzS8NQc8h3SGCy+qrkwfjEWemgTIPNoHxCLqA= Date: Wed, 30 Jun 2021 18:51:35 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 083/192] mm: memory: add orig_pmd to struct vm_fault Message-ID: <20210701015135.V8qrg8rwW%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=pTSpvGPF; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: zb14r9q9mcsbmf64ci5rm5i1rb4xz4qm X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 601E95000098 X-HE-Tag: 1625104297-690726 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: memory: add orig_pmd to struct vm_fault Pach series "mm: thp: use generic THP migration for NUMA hinting fault", v3. When the THP NUMA fault support was added THP migration was not supported yet. So the ad hoc THP migration was implemented in NUMA fault handling. Since v4.14 THP migration has been supported so it doesn't make too much sense to still keep another THP migration implementation rather than using the generic migration code. It is definitely a maintenance burden to keep two THP migration implementation for different code paths and it is more error prone. Using the generic THP migration implementation allows us remove the duplicate code and some hacks needed by the old ad hoc implementation. A quick grep shows x86_64, PowerPC (book3s), ARM64 ans S390 support both THP and NUMA balancing. The most of them support THP migration except for S390. Zi Yan tried to add THP migration support for S390 before but it was not accepted due to the design of S390 PMD. For the discussion, please see: https://lkml.org/lkml/2018/4/27/953. Per the discussion with Gerald Schaefer in v1 it is acceptible to skip huge PMD for S390 for now. I saw there were some hacks about gup from git history, but I didn't figure out if they have been removed or not since I just found FOLL_NUMA code in the current gup implementation and they seems useful. Patch #1 ~ #2 are preparation patches. Patch #3 is the real meat. Patch #4 ~ #6 keep consistent counters and behaviors with before. Patch #7 skips change huge PMD to prot_none if thp migration is not supported. Test ---- Did some tests to measure the latency of do_huge_pmd_numa_page. The test VM has 80 vcpus and 64G memory. The test would create 2 processes to consume 128G memory together which would incur memory pressure to cause THP splits. And it also creates 80 processes to hog cpu, and the memory consumer processes are bound to different nodes periodically in order to increase NUMA faults. The below test script is used: echo 3 > /proc/sys/vm/drop_caches # Run stress-ng for 24 hours ./stress-ng/stress-ng --vm 2 --vm-bytes 64G --timeout 24h & PID=$! ./stress-ng/stress-ng --cpu $NR_CPUS --timeout 24h & # Wait for vm stressors forked sleep 5 PID_1=`pgrep -P $PID | awk 'NR == 1'` PID_2=`pgrep -P $PID | awk 'NR == 2'` JOB1=`pgrep -P $PID_1` JOB2=`pgrep -P $PID_2` # Bind load jobs to different nodes periodically to force generate # cross node memory access while [ -d "/proc/$PID" ] do taskset -apc 8 $JOB1 taskset -apc 8 $JOB2 sleep 300 taskset -apc 58 $JOB1 taskset -apc 58 $JOB2 sleep 300 done With the above test the histogram of latency of do_huge_pmd_numa_page is as shown below. Since the number of do_huge_pmd_numa_page varies drastically for each run (should be due to scheduler), so I converted the raw number to percentage. patched base @us[stress-ng]: [0] 3.57% 0.16% [1] 55.68% 18.36% [2, 4) 10.46% 40.44% [4, 8) 7.26% 17.82% [8, 16) 21.12% 13.41% [16, 32) 1.06% 4.27% [32, 64) 0.56% 4.07% [64, 128) 0.16% 0.35% [128, 256) < 0.1% < 0.1% [256, 512) < 0.1% < 0.1% [512, 1K) < 0.1% < 0.1% [1K, 2K) < 0.1% < 0.1% [2K, 4K) < 0.1% < 0.1% [4K, 8K) < 0.1% < 0.1% [8K, 16K) < 0.1% < 0.1% [16K, 32K) < 0.1% < 0.1% [32K, 64K) < 0.1% < 0.1% Per the result, patched kernel is even slightly better than the base kernel. I think this is because the lock contention against THP split is less than base kernel due to the refactor. To exclude the affect from THP split, I also did test w/o memory pressure. No obvious regression is spotted. The below is the test result *w/o* memory pressure. patched base @us[stress-ng]: [0] 7.97% 18.4% [1] 69.63% 58.24% [2, 4) 4.18% 2.63% [4, 8) 0.22% 0.17% [8, 16) 1.03% 0.92% [16, 32) 0.14% < 0.1% [32, 64) < 0.1% < 0.1% [64, 128) < 0.1% < 0.1% [128, 256) < 0.1% < 0.1% [256, 512) 0.45% 1.19% [512, 1K) 15.45% 17.27% [1K, 2K) < 0.1% < 0.1% [2K, 4K) < 0.1% < 0.1% [4K, 8K) < 0.1% < 0.1% [8K, 16K) 0.86% 0.88% [16K, 32K) < 0.1% 0.15% [32K, 64K) < 0.1% < 0.1% [64K, 128K) < 0.1% < 0.1% [128K, 256K) < 0.1% < 0.1% The series also survived a series of tests that exercise NUMA balancing migrations by Mel. This patch (of 7): Add orig_pmd to struct vm_fault so the "orig_pmd" parameter used by huge page fault could be removed, just like its PTE counterpart does. Link: https://lkml.kernel.org/r/20210518200801.7413-1-shy828301@gmail.com Link: https://lkml.kernel.org/r/20210518200801.7413-2-shy828301@gmail.com Signed-off-by: Yang Shi Acked-by: Mel Gorman Cc: Kirill A. Shutemov Cc: Zi Yan Cc: Huang Ying Cc: Michal Hocko Cc: Hugh Dickins Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Christian Borntraeger Signed-off-by: Andrew Morton --- include/linux/huge_mm.h | 9 ++++----- include/linux/mm.h | 7 ++++++- mm/huge_memory.c | 9 ++++++--- mm/memory.c | 26 +++++++++++++------------- 4 files changed, 29 insertions(+), 22 deletions(-) --- a/include/linux/huge_mm.h~mm-memory-add-orig_pmd-to-struct-vm_fault +++ a/include/linux/huge_mm.h @@ -11,7 +11,7 @@ vm_fault_t do_huge_pmd_anonymous_page(st int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *dst_vma, struct vm_area_struct *src_vma); -void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd); +void huge_pmd_set_accessed(struct vm_fault *vmf); int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, pud_t *dst_pud, pud_t *src_pud, unsigned long addr, struct vm_area_struct *vma); @@ -24,7 +24,7 @@ static inline void huge_pud_set_accessed } #endif -vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd); +vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf); struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd, unsigned int flags); @@ -288,7 +288,7 @@ struct page *follow_devmap_pmd(struct vm struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr, pud_t *pud, int flags, struct dev_pagemap **pgmap); -vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t orig_pmd); +vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf); extern struct page *huge_zero_page; extern unsigned long huge_zero_pfn; @@ -441,8 +441,7 @@ static inline spinlock_t *pud_trans_huge return NULL; } -static inline vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, - pmd_t orig_pmd) +static inline vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) { return 0; } --- a/include/linux/mm.h~mm-memory-add-orig_pmd-to-struct-vm_fault +++ a/include/linux/mm.h @@ -550,7 +550,12 @@ struct vm_fault { pud_t *pud; /* Pointer to pud entry matching * the 'address' */ - pte_t orig_pte; /* Value of PTE at the time of fault */ + union { + pte_t orig_pte; /* Value of PTE at the time of fault */ + pmd_t orig_pmd; /* Value of PMD at the time of fault, + * used by PMD fault only. + */ + }; struct page *cow_page; /* Page handler may use for COW fault */ struct page *page; /* ->fault handlers should return a --- a/mm/huge_memory.c~mm-memory-add-orig_pmd-to-struct-vm_fault +++ a/mm/huge_memory.c @@ -1257,11 +1257,12 @@ unlock: } #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ -void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd) +void huge_pmd_set_accessed(struct vm_fault *vmf) { pmd_t entry; unsigned long haddr; bool write = vmf->flags & FAULT_FLAG_WRITE; + pmd_t orig_pmd = vmf->orig_pmd; vmf->ptl = pmd_lock(vmf->vma->vm_mm, vmf->pmd); if (unlikely(!pmd_same(*vmf->pmd, orig_pmd))) @@ -1278,11 +1279,12 @@ unlock: spin_unlock(vmf->ptl); } -vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) +vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; struct page *page; unsigned long haddr = vmf->address & HPAGE_PMD_MASK; + pmd_t orig_pmd = vmf->orig_pmd; vmf->ptl = pmd_lockptr(vma->vm_mm, vmf->pmd); VM_BUG_ON_VMA(!vma->anon_vma, vma); @@ -1418,9 +1420,10 @@ out: } /* NUMA hinting page fault entry point for trans huge pmds */ -vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) +vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; + pmd_t pmd = vmf->orig_pmd; struct anon_vma *anon_vma = NULL; struct page *page; unsigned long haddr = vmf->address & HPAGE_PMD_MASK; --- a/mm/memory.c~mm-memory-add-orig_pmd-to-struct-vm_fault +++ a/mm/memory.c @@ -4298,12 +4298,12 @@ static inline vm_fault_t create_huge_pmd } /* `inline' is required to avoid gcc 4.1.2 build error */ -static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf, pmd_t orig_pmd) +static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf) { if (vma_is_anonymous(vmf->vma)) { - if (userfaultfd_huge_pmd_wp(vmf->vma, orig_pmd)) + if (userfaultfd_huge_pmd_wp(vmf->vma, vmf->orig_pmd)) return handle_userfault(vmf, VM_UFFD_WP); - return do_huge_pmd_wp_page(vmf, orig_pmd); + return do_huge_pmd_wp_page(vmf); } if (vmf->vma->vm_ops->huge_fault) { vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD); @@ -4530,26 +4530,26 @@ retry_pud: if (!(ret & VM_FAULT_FALLBACK)) return ret; } else { - pmd_t orig_pmd = *vmf.pmd; + vmf.orig_pmd = *vmf.pmd; barrier(); - if (unlikely(is_swap_pmd(orig_pmd))) { + if (unlikely(is_swap_pmd(vmf.orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - if (is_pmd_migration_entry(orig_pmd)) + !is_pmd_migration_entry(vmf.orig_pmd)); + if (is_pmd_migration_entry(vmf.orig_pmd)) pmd_migration_entry_wait(mm, vmf.pmd); return 0; } - if (pmd_trans_huge(orig_pmd) || pmd_devmap(orig_pmd)) { - if (pmd_protnone(orig_pmd) && vma_is_accessible(vma)) - return do_huge_pmd_numa_page(&vmf, orig_pmd); + if (pmd_trans_huge(vmf.orig_pmd) || pmd_devmap(vmf.orig_pmd)) { + if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) + return do_huge_pmd_numa_page(&vmf); - if (dirty && !pmd_write(orig_pmd)) { - ret = wp_huge_pmd(&vmf, orig_pmd); + if (dirty && !pmd_write(vmf.orig_pmd)) { + ret = wp_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; } else { - huge_pmd_set_accessed(&vmf, orig_pmd); + huge_pmd_set_accessed(&vmf); return 0; } } From patchwork Thu Jul 1 01:51:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07072C11F6A for ; Thu, 1 Jul 2021 01:51:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B15226105A for ; Thu, 1 Jul 2021 01:51:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B15226105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3A77D8D0228; Wed, 30 Jun 2021 21:51:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 37E1E8D0202; Wed, 30 Jun 2021 21:51:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2472B8D0228; Wed, 30 Jun 2021 21:51:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0193.hostedemail.com [216.40.44.193]) by kanga.kvack.org (Postfix) with ESMTP id 02ADF8D0202 for ; Wed, 30 Jun 2021 21:51:40 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BECA2181AEF3F for ; Thu, 1 Jul 2021 01:51:40 +0000 (UTC) X-FDA: 78312342360.34.61C8D83 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 7DF4CD0000A0 for ; Thu, 1 Jul 2021 01:51:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6011561241; Thu, 1 Jul 2021 01:51:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104299; bh=ncayqTaMm8i8J06ewDQI1J4TfCvG093pumynsfT/Ys8=; h=Date:From:To:Subject:In-Reply-To:From; b=zBASmNfCsjnXP1EjM/7Ua2tELl3i+tZYO/x9RC38PRtzG4mOPsTkF1FOxST/lvg0c 6PJ8eIPsrzZjJmkhWYX5VnTFvoTMmwkso3692ZcHrHRczUNCMPbqIuiBZVuNcBswWv s7qoBElcZfPMWVJKhyuASk7bx/TeDxxoyEzKTrag= Date: Wed, 30 Jun 2021 18:51:39 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 084/192] mm: memory: make numa_migrate_prep() non-static Message-ID: <20210701015139.o4cDNVwiX%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=zBASmNfC; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: e5aten5rqob91xps5x6b9axkw9mmh94r X-Rspamd-Queue-Id: 7DF4CD0000A0 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104300-406272 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: memory: make numa_migrate_prep() non-static The numa_migrate_prep() will be used by huge NUMA fault as well in the following patch, make it non-static. Link: https://lkml.kernel.org/r/20210518200801.7413-3-shy828301@gmail.com Signed-off-by: Yang Shi Acked-by: Mel Gorman Cc: Christian Borntraeger Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Huang Ying Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Michal Hocko Cc: Vasily Gorbik Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/internal.h | 3 +++ mm/memory.c | 5 ++--- 2 files changed, 5 insertions(+), 3 deletions(-) --- a/mm/internal.h~mm-memory-make-numa_migrate_prep-non-static +++ a/mm/internal.h @@ -672,4 +672,7 @@ int vmap_pages_range_noflush(unsigned lo void vunmap_range_noflush(unsigned long start, unsigned long end); +int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, + unsigned long addr, int page_nid, int *flags); + #endif /* __MM_INTERNAL_H */ --- a/mm/memory.c~mm-memory-make-numa_migrate_prep-non-static +++ a/mm/memory.c @@ -4175,9 +4175,8 @@ static vm_fault_t do_fault(struct vm_fau return ret; } -static int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, - unsigned long addr, int page_nid, - int *flags) +int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, + unsigned long addr, int page_nid, int *flags) { get_page(page); From patchwork Thu Jul 1 01:51:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8B71EC11F6B for ; Thu, 1 Jul 2021 01:51:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2E05C61468 for ; Thu, 1 Jul 2021 01:51:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E05C61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A5E958D0229; Wed, 30 Jun 2021 21:51:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A34748D0202; Wed, 30 Jun 2021 21:51:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8875C8D0229; Wed, 30 Jun 2021 21:51:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0183.hostedemail.com [216.40.44.183]) by kanga.kvack.org (Postfix) with ESMTP id 65F478D0202 for ; Wed, 30 Jun 2021 21:51:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 31FF0253D3 for ; Thu, 1 Jul 2021 01:51:44 +0000 (UTC) X-FDA: 78312342528.01.F08955B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id B291610000B5 for ; Thu, 1 Jul 2021 01:51:43 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DFA56105A; Thu, 1 Jul 2021 01:51:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104303; bh=0vgGKBCwL0tx/7ZDcKJOrrqA/KfB4jBrOV0l0EdsBeU=; h=Date:From:To:Subject:In-Reply-To:From; b=A8XSgd3j3HK51Ti9snsM6zX6Zk9lcBYl320U016SS2QMPCSDg9Qgo9tY4Vr2BYfet Nj5NefYiNSq/HAoxQgf6Viyp5N+bD11gK+f+3iBP0ZlX8jGXHFw9/YV+BSdYd82D/J aURbloiEHT6ZVPz5MjnAo0v2Ob/2mdYLNjvT9hbg= Date: Wed, 30 Jun 2021 18:51:42 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, dan.carpenter@oracle.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 085/192] mm: thp: refactor NUMA fault handling Message-ID: <20210701015142._nsjOVpxs%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=A8XSgd3j; spf=pass (imf07.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 6kbdbfrqerom9pqgdae5aa5xem5c1fsj X-Rspamd-Queue-Id: B291610000B5 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104303-646238 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: thp: refactor NUMA fault handling When the THP NUMA fault support was added THP migration was not supported yet. So the ad hoc THP migration was implemented in NUMA fault handling. Since v4.14 THP migration has been supported so it doesn't make too much sense to still keep another THP migration implementation rather than using the generic migration code. This patch reworks the NUMA fault handling to use generic migration implementation to migrate misplaced page. There is no functional change. After the refactor the flow of NUMA fault handling looks just like its PTE counterpart: Acquire ptl Prepare for migration (elevate page refcount) Release ptl Isolate page from lru and elevate page refcount Migrate the misplaced THP If migration fails just restore the old normal PMD. In the old code anon_vma lock was needed to serialize THP migration against THP split, but since then the THP code has been reworked a lot, it seems anon_vma lock is not required anymore to avoid the race. The page refcount elevation when holding ptl should prevent from THP split. Use migrate_misplaced_page() for both base page and THP NUMA hinting fault and remove all the dead and duplicate code. [dan.carpenter@oracle.com: fix a double unlock bug] Link: https://lkml.kernel.org/r/YLX8uYN01JmfLnlK@mwanda Link: https://lkml.kernel.org/r/20210518200801.7413-4-shy828301@gmail.com Signed-off-by: Yang Shi Signed-off-by: Dan Carpenter Acked-by: Mel Gorman Cc: Christian Borntraeger Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Huang Ying Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Michal Hocko Cc: Vasily Gorbik Cc: Zi Yan Signed-off-by: Andrew Morton --- include/linux/migrate.h | 23 ---- mm/huge_memory.c | 145 +++++++++---------------------- mm/internal.h | 18 --- mm/migrate.c | 177 +++++++------------------------------- 4 files changed, 77 insertions(+), 286 deletions(-) --- a/include/linux/migrate.h~mm-thp-refactor-numa-fault-handling +++ a/include/linux/migrate.h @@ -99,14 +99,9 @@ static inline void __ClearPageMovable(st #endif #ifdef CONFIG_NUMA_BALANCING -extern bool pmd_trans_migrating(pmd_t pmd); extern int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, int node); #else -static inline bool pmd_trans_migrating(pmd_t pmd) -{ - return false; -} static inline int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, int node) { @@ -114,24 +109,6 @@ static inline int migrate_misplaced_page } #endif /* CONFIG_NUMA_BALANCING */ -#if defined(CONFIG_NUMA_BALANCING) && defined(CONFIG_TRANSPARENT_HUGEPAGE) -extern int migrate_misplaced_transhuge_page(struct mm_struct *mm, - struct vm_area_struct *vma, - pmd_t *pmd, pmd_t entry, - unsigned long address, - struct page *page, int node); -#else -static inline int migrate_misplaced_transhuge_page(struct mm_struct *mm, - struct vm_area_struct *vma, - pmd_t *pmd, pmd_t entry, - unsigned long address, - struct page *page, int node) -{ - return -EAGAIN; -} -#endif /* CONFIG_NUMA_BALANCING && CONFIG_TRANSPARENT_HUGEPAGE*/ - - #ifdef CONFIG_MIGRATION /* --- a/mm/huge_memory.c~mm-thp-refactor-numa-fault-handling +++ a/mm/huge_memory.c @@ -1423,94 +1423,22 @@ out: vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - pmd_t pmd = vmf->orig_pmd; - struct anon_vma *anon_vma = NULL; + pmd_t oldpmd = vmf->orig_pmd; + pmd_t pmd; struct page *page; unsigned long haddr = vmf->address & HPAGE_PMD_MASK; - int page_nid = NUMA_NO_NODE, this_nid = numa_node_id(); + int page_nid = NUMA_NO_NODE; int target_nid, last_cpupid = -1; - bool page_locked; bool migrated = false; - bool was_writable; + bool was_writable = pmd_savedwrite(oldpmd); int flags = 0; vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); - if (unlikely(!pmd_same(pmd, *vmf->pmd))) - goto out_unlock; - - /* - * If there are potential migrations, wait for completion and retry - * without disrupting NUMA hinting information. Do not relock and - * check_same as the page may no longer be mapped. - */ - if (unlikely(pmd_trans_migrating(*vmf->pmd))) { - page = pmd_page(*vmf->pmd); - if (!get_page_unless_zero(page)) - goto out_unlock; + if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) { spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } - page = pmd_page(pmd); - BUG_ON(is_huge_zero_page(page)); - page_nid = page_to_nid(page); - last_cpupid = page_cpupid_last(page); - count_vm_numa_event(NUMA_HINT_FAULTS); - if (page_nid == this_nid) { - count_vm_numa_event(NUMA_HINT_FAULTS_LOCAL); - flags |= TNF_FAULT_LOCAL; - } - - /* See similar comment in do_numa_page for explanation */ - if (!pmd_savedwrite(pmd)) - flags |= TNF_NO_GROUP; - - /* - * Acquire the page lock to serialise THP migrations but avoid dropping - * page_table_lock if at all possible - */ - page_locked = trylock_page(page); - target_nid = mpol_misplaced(page, vma, haddr); - /* Migration could have started since the pmd_trans_migrating check */ - if (!page_locked) { - page_nid = NUMA_NO_NODE; - if (!get_page_unless_zero(page)) - goto out_unlock; - spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); - goto out; - } else if (target_nid == NUMA_NO_NODE) { - /* There are no parallel migrations and page is in the right - * node. Clear the numa hinting info in this pmd. - */ - goto clear_pmdnuma; - } - - /* - * Page is misplaced. Page lock serialises migrations. Acquire anon_vma - * to serialises splits - */ - get_page(page); - spin_unlock(vmf->ptl); - anon_vma = page_lock_anon_vma_read(page); - - /* Confirm the PMD did not change while page_table_lock was released */ - spin_lock(vmf->ptl); - if (unlikely(!pmd_same(pmd, *vmf->pmd))) { - unlock_page(page); - put_page(page); - page_nid = NUMA_NO_NODE; - goto out_unlock; - } - - /* Bail if we fail to protect against THP splits for any reason */ - if (unlikely(!anon_vma)) { - put_page(page); - page_nid = NUMA_NO_NODE; - goto clear_pmdnuma; - } - /* * Since we took the NUMA fault, we must have observed the !accessible * bit. Make sure all other CPUs agree with that, to avoid them @@ -1537,43 +1465,58 @@ vm_fault_t do_huge_pmd_numa_page(struct haddr + HPAGE_PMD_SIZE); } - /* - * Migrate the THP to the requested node, returns with page unlocked - * and access rights restored. - */ + pmd = pmd_modify(oldpmd, vma->vm_page_prot); + page = vm_normal_page_pmd(vma, haddr, pmd); + if (!page) + goto out_map; + + /* See similar comment in do_numa_page for explanation */ + if (!was_writable) + flags |= TNF_NO_GROUP; + + page_nid = page_to_nid(page); + last_cpupid = page_cpupid_last(page); + target_nid = numa_migrate_prep(page, vma, haddr, page_nid, + &flags); + + if (target_nid == NUMA_NO_NODE) { + put_page(page); + goto out_map; + } + spin_unlock(vmf->ptl); - migrated = migrate_misplaced_transhuge_page(vma->vm_mm, vma, - vmf->pmd, pmd, vmf->address, page, target_nid); + migrated = migrate_misplaced_page(page, vma, target_nid); if (migrated) { flags |= TNF_MIGRATED; page_nid = target_nid; - } else + } else { flags |= TNF_MIGRATE_FAIL; - - goto out; -clear_pmdnuma: - BUG_ON(!PageLocked(page)); - was_writable = pmd_savedwrite(pmd); - pmd = pmd_modify(pmd, vma->vm_page_prot); - pmd = pmd_mkyoung(pmd); - if (was_writable) - pmd = pmd_mkwrite(pmd); - set_pmd_at(vma->vm_mm, haddr, vmf->pmd, pmd); - update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); - unlock_page(page); -out_unlock: - spin_unlock(vmf->ptl); + vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); + if (unlikely(!pmd_same(oldpmd, *vmf->pmd))) { + spin_unlock(vmf->ptl); + goto out; + } + goto out_map; + } out: - if (anon_vma) - page_unlock_anon_vma_read(anon_vma); - if (page_nid != NUMA_NO_NODE) task_numa_fault(last_cpupid, page_nid, HPAGE_PMD_NR, flags); return 0; + +out_map: + /* Restore the PMD */ + pmd = pmd_modify(oldpmd, vma->vm_page_prot); + pmd = pmd_mkyoung(pmd); + if (was_writable) + pmd = pmd_mkwrite(pmd); + set_pmd_at(vma->vm_mm, haddr, vmf->pmd, pmd); + update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); + spin_unlock(vmf->ptl); + goto out; } /* --- a/mm/internal.h~mm-thp-refactor-numa-fault-handling +++ a/mm/internal.h @@ -369,23 +369,6 @@ extern unsigned int munlock_vma_page(str */ extern void clear_page_mlock(struct page *page); -/* - * mlock_migrate_page - called only from migrate_misplaced_transhuge_page() - * (because that does not go through the full procedure of migration ptes): - * to migrate the Mlocked page flag; update statistics. - */ -static inline void mlock_migrate_page(struct page *newpage, struct page *page) -{ - if (TestClearPageMlocked(page)) { - int nr_pages = thp_nr_pages(page); - - /* Holding pmd lock, no change in irq context: __mod is safe */ - __mod_zone_page_state(page_zone(page), NR_MLOCK, -nr_pages); - SetPageMlocked(newpage); - __mod_zone_page_state(page_zone(newpage), NR_MLOCK, nr_pages); - } -} - extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma); /* @@ -461,7 +444,6 @@ static inline struct file *maybe_unlock_ #else /* !CONFIG_MMU */ static inline void clear_page_mlock(struct page *page) { } static inline void mlock_vma_page(struct page *page) { } -static inline void mlock_migrate_page(struct page *new, struct page *old) { } static inline void vunmap_range_noflush(unsigned long start, unsigned long end) { } --- a/mm/migrate.c~mm-thp-refactor-numa-fault-handling +++ a/mm/migrate.c @@ -2048,6 +2048,23 @@ static struct page *alloc_misplaced_dst_ return newpage; } +static struct page *alloc_misplaced_dst_page_thp(struct page *page, + unsigned long data) +{ + int nid = (int) data; + struct page *newpage; + + newpage = alloc_pages_node(nid, (GFP_TRANSHUGE_LIGHT | __GFP_THISNODE), + HPAGE_PMD_ORDER); + if (!newpage) + goto out; + + prep_transhuge_page(newpage); + +out: + return newpage; +} + static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) { int page_lru; @@ -2086,12 +2103,6 @@ static int numamigrate_isolate_page(pg_d return 1; } -bool pmd_trans_migrating(pmd_t pmd) -{ - struct page *page = pmd_page(pmd); - return PageLocked(page); -} - /* * Attempt to migrate a misplaced page to the specified destination * node. Caller is expected to have an elevated reference count on @@ -2104,6 +2115,20 @@ int migrate_misplaced_page(struct page * int isolated; int nr_remaining; LIST_HEAD(migratepages); + new_page_t *new; + bool compound; + + /* + * PTE mapped THP or HugeTLB page can't reach here so the page could + * be either base page or THP. And it must be head page if it is + * THP. + */ + compound = PageTransHuge(page); + + if (compound) + new = alloc_misplaced_dst_page_thp; + else + new = alloc_misplaced_dst_page; /* * Don't migrate file pages that are mapped in multiple processes @@ -2125,9 +2150,8 @@ int migrate_misplaced_page(struct page * goto out; list_add(&page->lru, &migratepages); - nr_remaining = migrate_pages(&migratepages, alloc_misplaced_dst_page, - NULL, node, MIGRATE_ASYNC, - MR_NUMA_MISPLACED); + nr_remaining = migrate_pages(&migratepages, *new, NULL, node, + MIGRATE_ASYNC, MR_NUMA_MISPLACED); if (nr_remaining) { if (!list_empty(&migratepages)) { list_del(&page->lru); @@ -2146,141 +2170,6 @@ out: return 0; } #endif /* CONFIG_NUMA_BALANCING */ - -#if defined(CONFIG_NUMA_BALANCING) && defined(CONFIG_TRANSPARENT_HUGEPAGE) -/* - * Migrates a THP to a given target node. page must be locked and is unlocked - * before returning. - */ -int migrate_misplaced_transhuge_page(struct mm_struct *mm, - struct vm_area_struct *vma, - pmd_t *pmd, pmd_t entry, - unsigned long address, - struct page *page, int node) -{ - spinlock_t *ptl; - pg_data_t *pgdat = NODE_DATA(node); - int isolated = 0; - struct page *new_page = NULL; - int page_lru = page_is_file_lru(page); - unsigned long start = address & HPAGE_PMD_MASK; - - new_page = alloc_pages_node(node, - (GFP_TRANSHUGE_LIGHT | __GFP_THISNODE), - HPAGE_PMD_ORDER); - if (!new_page) - goto out_fail; - prep_transhuge_page(new_page); - - isolated = numamigrate_isolate_page(pgdat, page); - if (!isolated) { - put_page(new_page); - goto out_fail; - } - - /* Prepare a page as a migration target */ - __SetPageLocked(new_page); - if (PageSwapBacked(page)) - __SetPageSwapBacked(new_page); - - /* anon mapping, we can simply copy page->mapping to the new page: */ - new_page->mapping = page->mapping; - new_page->index = page->index; - /* flush the cache before copying using the kernel virtual address */ - flush_cache_range(vma, start, start + HPAGE_PMD_SIZE); - migrate_page_copy(new_page, page); - WARN_ON(PageLRU(new_page)); - - /* Recheck the target PMD */ - ptl = pmd_lock(mm, pmd); - if (unlikely(!pmd_same(*pmd, entry) || !page_ref_freeze(page, 2))) { - spin_unlock(ptl); - - /* Reverse changes made by migrate_page_copy() */ - if (TestClearPageActive(new_page)) - SetPageActive(page); - if (TestClearPageUnevictable(new_page)) - SetPageUnevictable(page); - - unlock_page(new_page); - put_page(new_page); /* Free it */ - - /* Retake the callers reference and putback on LRU */ - get_page(page); - putback_lru_page(page); - mod_node_page_state(page_pgdat(page), - NR_ISOLATED_ANON + page_lru, -HPAGE_PMD_NR); - - goto out_unlock; - } - - entry = mk_huge_pmd(new_page, vma->vm_page_prot); - entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); - - /* - * Overwrite the old entry under pagetable lock and establish - * the new PTE. Any parallel GUP will either observe the old - * page blocking on the page lock, block on the page table - * lock or observe the new page. The SetPageUptodate on the - * new page and page_add_new_anon_rmap guarantee the copy is - * visible before the pagetable update. - */ - page_add_anon_rmap(new_page, vma, start, true); - /* - * At this point the pmd is numa/protnone (i.e. non present) and the TLB - * has already been flushed globally. So no TLB can be currently - * caching this non present pmd mapping. There's no need to clear the - * pmd before doing set_pmd_at(), nor to flush the TLB after - * set_pmd_at(). Clearing the pmd here would introduce a race - * condition against MADV_DONTNEED, because MADV_DONTNEED only holds the - * mmap_lock for reading. If the pmd is set to NULL at any given time, - * MADV_DONTNEED won't wait on the pmd lock and it'll skip clearing this - * pmd. - */ - set_pmd_at(mm, start, pmd, entry); - update_mmu_cache_pmd(vma, address, &entry); - - page_ref_unfreeze(page, 2); - mlock_migrate_page(new_page, page); - page_remove_rmap(page, true); - set_page_owner_migrate_reason(new_page, MR_NUMA_MISPLACED); - - spin_unlock(ptl); - - /* Take an "isolate" reference and put new page on the LRU. */ - get_page(new_page); - putback_lru_page(new_page); - - unlock_page(new_page); - unlock_page(page); - put_page(page); /* Drop the rmap reference */ - put_page(page); /* Drop the LRU isolation reference */ - - count_vm_events(PGMIGRATE_SUCCESS, HPAGE_PMD_NR); - count_vm_numa_events(NUMA_PAGE_MIGRATE, HPAGE_PMD_NR); - - mod_node_page_state(page_pgdat(page), - NR_ISOLATED_ANON + page_lru, - -HPAGE_PMD_NR); - return isolated; - -out_fail: - count_vm_events(PGMIGRATE_FAIL, HPAGE_PMD_NR); - ptl = pmd_lock(mm, pmd); - if (pmd_same(*pmd, entry)) { - entry = pmd_modify(entry, vma->vm_page_prot); - set_pmd_at(mm, start, pmd, entry); - update_mmu_cache_pmd(vma, address, &entry); - } - spin_unlock(ptl); - -out_unlock: - unlock_page(page); - put_page(page); - return 0; -} -#endif /* CONFIG_NUMA_BALANCING */ - #endif /* CONFIG_NUMA */ #ifdef CONFIG_DEVICE_PRIVATE From patchwork Thu Jul 1 01:51:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353231 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 510BEC11F66 for ; Thu, 1 Jul 2021 01:51:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 05E1D6105A for ; Thu, 1 Jul 2021 01:51:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05E1D6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 83B728D022A; Wed, 30 Jun 2021 21:51:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 811CB8D0202; Wed, 30 Jun 2021 21:51:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6DA518D022A; Wed, 30 Jun 2021 21:51:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id 4C9628D0202 for ; Wed, 30 Jun 2021 21:51:47 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 24F9982E4C0A for ; Thu, 1 Jul 2021 01:51:47 +0000 (UTC) X-FDA: 78312342654.25.F4C8702 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id D788830000A8 for ; Thu, 1 Jul 2021 01:51:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BCF0C6146D; Thu, 1 Jul 2021 01:51:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104306; bh=/6mnSpYzINtTUTiv5Z/54zQUx/epEwshhh6aQvPtpas=; h=Date:From:To:Subject:In-Reply-To:From; b=LAQ04JoGVc8qP0vDgViTGXL8TZF2NhNSfaSlce7MZ2e3vWh60tU7kh+Nj3UNu998c FynBRC3c9dlVaToXRd0T92Lsu7jckW/rmV5kClwXXqG06wy39fP56U3yZD/aim+B9H q6A5Xgri4Q+kler4QrAMQpXI3gI7xgIZFOXqpy2g= Date: Wed, 30 Jun 2021 18:51:45 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 086/192] mm: migrate: account THP NUMA migration counters correctly Message-ID: <20210701015145.SwXVL5Q0x%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=LAQ04JoG; dmarc=none; spf=pass (imf08.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: fcm9akkcfkou1wsonhuo84b1n6pfg4ni X-Rspamd-Queue-Id: D788830000A8 X-HE-Tag: 1625104306-678800 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: migrate: account THP NUMA migration counters correctly Now both base page and THP NUMA migration is done via migrate_misplaced_page(), keep the counters correctly for THP. Link: https://lkml.kernel.org/r/20210518200801.7413-5-shy828301@gmail.com Signed-off-by: Yang Shi Acked-by: Mel Gorman Cc: Christian Borntraeger Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Huang Ying Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Michal Hocko Cc: Vasily Gorbik Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/migrate.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) --- a/mm/migrate.c~mm-migrate-account-thp-numa-migration-counters-correctly +++ a/mm/migrate.c @@ -2117,6 +2117,7 @@ int migrate_misplaced_page(struct page * LIST_HEAD(migratepages); new_page_t *new; bool compound; + unsigned int nr_pages = thp_nr_pages(page); /* * PTE mapped THP or HugeTLB page can't reach here so the page could @@ -2155,13 +2156,13 @@ int migrate_misplaced_page(struct page * if (nr_remaining) { if (!list_empty(&migratepages)) { list_del(&page->lru); - dec_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_lru(page)); + mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON + + page_is_file_lru(page), -nr_pages); putback_lru_page(page); } isolated = 0; } else - count_vm_numa_event(NUMA_PAGE_MIGRATE); + count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_pages); BUG_ON(!list_empty(&migratepages)); return isolated; From patchwork Thu Jul 1 01:51:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353233 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A174AC11F6C for ; Thu, 1 Jul 2021 01:51:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5661661241 for ; Thu, 1 Jul 2021 01:51:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5661661241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D1DA48D022B; Wed, 30 Jun 2021 21:51:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF3C08D0202; Wed, 30 Jun 2021 21:51:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BBBD28D022B; Wed, 30 Jun 2021 21:51:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0133.hostedemail.com [216.40.44.133]) by kanga.kvack.org (Postfix) with ESMTP id 987C88D0202 for ; Wed, 30 Jun 2021 21:51:50 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 68A04250AF for ; Thu, 1 Jul 2021 01:51:50 +0000 (UTC) X-FDA: 78312342780.02.265E0CE Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 33B6FD000155 for ; Thu, 1 Jul 2021 01:51:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 033C86105A; Thu, 1 Jul 2021 01:51:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104309; bh=kfFHHb408x5/Xe3br+j7sBP8MtK3lQSIzDoD9RrJpQs=; h=Date:From:To:Subject:In-Reply-To:From; b=qGULMZS6MsaS+8i9UyQ/skBZuWJiyDsG9eCQe5bKCJOLUCvlZu1ZHnXOgDzPwJ6WO LdV5hop2vfWc5R30oZZOgbossXyuwMIVvK222wmIiDYBNpw9rhWHpblpTyo15CqmBu cjL/R3NBbrcz89n8ji6zbfrouEhVgMdMEWjOyd/4= Date: Wed, 30 Jun 2021 18:51:48 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 087/192] mm: migrate: don't split THP for misplaced NUMA page Message-ID: <20210701015148.gUkYuLq9J%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 33B6FD000155 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=qGULMZS6; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: ztmujpwsjy9n5tsouroca7dt8cmnpxu3 X-HE-Tag: 1625104310-477677 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: migrate: don't split THP for misplaced NUMA page The old behavior didn't split THP if migration is failed due to lack of memory on the target node. But the THP migration does split THP, so keep the old behavior for misplaced NUMA page migration. Link: https://lkml.kernel.org/r/20210518200801.7413-6-shy828301@gmail.com Signed-off-by: Yang Shi Acked-by: Mel Gorman Cc: Christian Borntraeger Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Huang Ying Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Michal Hocko Cc: Vasily Gorbik Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/migrate.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) --- a/mm/migrate.c~mm-migrate-dont-split-thp-for-misplaced-numa-page +++ a/mm/migrate.c @@ -1423,6 +1423,7 @@ int migrate_pages(struct list_head *from int swapwrite = current->flags & PF_SWAPWRITE; int rc, nr_subpages; LIST_HEAD(ret_pages); + bool nosplit = (reason == MR_NUMA_MISPLACED); trace_mm_migrate_pages_start(mode, reason); @@ -1494,8 +1495,9 @@ retry: /* * When memory is low, don't bother to try to migrate * other pages, just exit. + * THP NUMA faulting doesn't split THP to retry. */ - if (is_thp) { + if (is_thp && !nosplit) { if (!try_split_thp(page, &page2, from)) { nr_thp_split++; goto retry; From patchwork Thu Jul 1 01:51:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353235 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C05DDC11F6A for ; Thu, 1 Jul 2021 01:51:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7900061474 for ; Thu, 1 Jul 2021 01:51:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7900061474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F407B8D022C; Wed, 30 Jun 2021 21:51:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F16AB8D0202; Wed, 30 Jun 2021 21:51:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDFB98D022C; Wed, 30 Jun 2021 21:51:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0008.hostedemail.com [216.40.44.8]) by kanga.kvack.org (Postfix) with ESMTP id BCB828D0202 for ; Wed, 30 Jun 2021 21:51:53 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 90AA8250AF for ; Thu, 1 Jul 2021 01:51:53 +0000 (UTC) X-FDA: 78312342906.09.EA97FB3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id 4D650B0000A7 for ; Thu, 1 Jul 2021 01:51:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 30FFF61468; Thu, 1 Jul 2021 01:51:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104312; bh=u54k/CjEHa1zzENVGfXNHYSuUkrbAJJvnxYkwPqsebs=; h=Date:From:To:Subject:In-Reply-To:From; b=ub2pMqhHAxREIOFOcve9x2TlS0+tRN58UC3O24rzz9HivMmEEr1TySGzCCvBfB4Lk 9+kbbiKDIuJrtXRdVE4zt+fAe48gYSw4R7hHee5APQzfrA7L3c43XtzfEEbKab6dMV t6AOnXSzGxRL424UqahDoWb/xC3RQ4joSFoVk9vQ= Date: Wed, 30 Jun 2021 18:51:51 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 088/192] mm: migrate: check mapcount for THP instead of refcount Message-ID: <20210701015151.3eYyc1NfJ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ub2pMqhH; dmarc=none; spf=pass (imf19.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: pr7fmhj3aa3psoha9kd11ncf5bb55g6i X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4D650B0000A7 X-HE-Tag: 1625104313-294663 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: migrate: check mapcount for THP instead of refcount The generic migration path will check refcount, so no need check refcount here. But the old code actually prevents from migrating shared THP (mapped by multiple processes), so bail out early if mapcount is > 1 to keep the behavior. Link: https://lkml.kernel.org/r/20210518200801.7413-7-shy828301@gmail.com Signed-off-by: Yang Shi Cc: Christian Borntraeger Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Huang Ying Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Mel Gorman Cc: Michal Hocko Cc: Vasily Gorbik Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/migrate.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) --- a/mm/migrate.c~mm-migrate-check-mapcount-for-thp-instead-of-refcount +++ a/mm/migrate.c @@ -2073,6 +2073,10 @@ static int numamigrate_isolate_page(pg_d VM_BUG_ON_PAGE(compound_order(page) && !PageTransHuge(page), page); + /* Do not migrate THP mapped by multiple processes */ + if (PageTransHuge(page) && total_mapcount(page) > 1) + return 0; + /* Avoid migrating to a node that is nearly full */ if (!migrate_balanced_pgdat(pgdat, compound_nr(page))) return 0; @@ -2080,18 +2084,6 @@ static int numamigrate_isolate_page(pg_d if (isolate_lru_page(page)) return 0; - /* - * migrate_misplaced_transhuge_page() skips page migration's usual - * check on page_count(), so we must do it here, now that the page - * has been isolated: a GUP pin, or any other pin, prevents migration. - * The expected page count is 3: 1 for page's mapcount and 1 for the - * caller's pin and 1 for the reference taken by isolate_lru_page(). - */ - if (PageTransHuge(page) && page_count(page) != 3) { - putback_lru_page(page); - return 0; - } - page_lru = page_is_file_lru(page); mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON + page_lru, thp_nr_pages(page)); From patchwork Thu Jul 1 01:51:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16052C11F68 for ; Thu, 1 Jul 2021 01:51:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C3FA561474 for ; Thu, 1 Jul 2021 01:51:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C3FA561474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4E00E8D022D; Wed, 30 Jun 2021 21:51:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B5CC8D0202; Wed, 30 Jun 2021 21:51:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 357038D022D; Wed, 30 Jun 2021 21:51:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0020.hostedemail.com [216.40.44.20]) by kanga.kvack.org (Postfix) with ESMTP id 10E6C8D0202 for ; Wed, 30 Jun 2021 21:51:57 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DC19382E4C0A for ; Thu, 1 Jul 2021 01:51:56 +0000 (UTC) X-FDA: 78312343032.35.882CE66 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 94AD3D0000A0 for ; Thu, 1 Jul 2021 01:51:56 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6802C6146D; Thu, 1 Jul 2021 01:51:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104315; bh=uANmw+e78nTpk2QCb7SCehF0iEDyB5ErkP6kLyFncNc=; h=Date:From:To:Subject:In-Reply-To:From; b=CIx1tKBzGtNTzULqGpeMh/fc9oAN6u8a9mlSlvjux/QODdy+exnsPo30KCeQ7PzIy MJxVwx542wawMebhkHYU/MjDJCdD892Oua2HQn+G7g+mb8VswMU354uXJBcjtRkgMY sWyz3vVlQZ09SIxHNrF+KT92lffKPHi8uxEJpkYw= Date: Wed, 30 Jun 2021 18:51:55 -0700 From: Andrew Morton To: akpm@linux-foundation.org, borntraeger@de.ibm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, hughd@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com, ziy@nvidia.com Subject: [patch 089/192] mm: thp: skip make PMD PROT_NONE if THP migration is not supported Message-ID: <20210701015155.l-dNiqU7C%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 94AD3D0000A0 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=CIx1tKBz; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: j3t1iq8a9r8rmukx7gn9s6un71k888qh X-HE-Tag: 1625104316-262834 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: thp: skip make PMD PROT_NONE if THP migration is not supported A quick grep shows x86_64, PowerPC (book3s), ARM64 and S390 support both NUMA balancing and THP. But S390 doesn't support THP migration so NUMA balancing actually can't migrate any misplaced pages. Skip make PMD PROT_NONE for such case otherwise CPU cycles may be wasted by pointless NUMA hinting faults on S390. Link: https://lkml.kernel.org/r/20210518200801.7413-8-shy828301@gmail.com Signed-off-by: Yang Shi Acked-by: Mel Gorman Cc: Christian Borntraeger Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Huang Ying Cc: Hugh Dickins Cc: Kirill A. Shutemov Cc: Michal Hocko Cc: Vasily Gorbik Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+) --- a/mm/huge_memory.c~mm-thp-skip-make-pmd-prot_none-if-thp-migration-is-not-supported +++ a/mm/huge_memory.c @@ -1742,6 +1742,7 @@ bool move_huge_pmd(struct vm_area_struct * Returns * - 0 if PMD could not be locked * - 1 if PMD was locked but protections unchanged and TLB flush unnecessary + * or if prot_numa but THP migration is not supported * - HPAGE_PMD_NR if protections changed and TLB flush necessary */ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, @@ -1756,6 +1757,9 @@ int change_huge_pmd(struct vm_area_struc bool uffd_wp = cp_flags & MM_CP_UFFD_WP; bool uffd_wp_resolve = cp_flags & MM_CP_UFFD_WP_RESOLVE; + if (prot_numa && !thp_migration_supported()) + return 1; + ptl = __pmd_trans_huge_lock(pmd, vma); if (!ptl) return 0; From patchwork Thu Jul 1 01:51:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0552FC11F6A for ; Thu, 1 Jul 2021 01:52:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A7C8761241 for ; Thu, 1 Jul 2021 01:52:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7C8761241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3321D8D022E; Wed, 30 Jun 2021 21:52:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 308298D0202; Wed, 30 Jun 2021 21:52:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F7E78D022E; Wed, 30 Jun 2021 21:52:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id ED9DD8D0202 for ; Wed, 30 Jun 2021 21:51:59 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C71121802ED99 for ; Thu, 1 Jul 2021 01:51:59 +0000 (UTC) X-FDA: 78312343158.10.D84CA1D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 8A702400208A for ; Thu, 1 Jul 2021 01:51:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8B4BF61474; Thu, 1 Jul 2021 01:51:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104318; bh=XFuB63iS5k2hf2niZ9v7nEmLoo5ci92rgzRtof4P49s=; h=Date:From:To:Subject:In-Reply-To:From; b=yOisVyqCISVNYNSaG3GxOxYSMeUEZ8eQLd9KGGtdT+pXe2SOlL0VynqF4o1YTpp3y QY5WatAd7WuLx2iZvfQfjuW9VccOqC5HO8yQ2rq2P0qh8POYLnIjLP8EHU4RMMDMEm uk1Ek+31SV4i+Sow9IOk+7DfYcPwSzFDluMN+O5Y= Date: Wed, 30 Jun 2021 18:51:58 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, gerald.schaefer@linux.ibm.com, gor@linux.ibm.com, hca@linux.ibm.com, linux-mm@kvack.org, mingo@redhat.com, mm-commits@vger.kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 090/192] mm/thp: make ARCH_ENABLE_SPLIT_PMD_PTLOCK dependent on PGTABLE_LEVELS > 2 Message-ID: <20210701015158.X1cghgNmq%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8A702400208A Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=yOisVyqC; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: unatnqzrkrsudqidxggcx83qqa6drr19 X-HE-Tag: 1625104319-131548 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm/thp: make ARCH_ENABLE_SPLIT_PMD_PTLOCK dependent on PGTABLE_LEVELS > 2 ARCH_ENABLE_SPLIT_PMD_PTLOCK is irrelevant unless there are more than two page table levels including PMD (also per Documentation/vm/split_page_table_lock.rst). Make this dependency explicit on remaining platforms i.e x86 and s390 where ARCH_ENABLE_SPLIT_PMD_PTLOCK is subscribed. Link: https://lkml.kernel.org/r/1622013501-20409-1-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Acked-by: Gerald Schaefer # s390 Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Thomas Gleixner Cc: Ingo Molnar Signed-off-by: Andrew Morton --- arch/s390/Kconfig | 2 +- arch/x86/Kconfig | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) --- a/arch/s390/Kconfig~mm-thp-make-arch_enable_split_pmd_ptlock-dependent-on-pgtable_levels-2 +++ a/arch/s390/Kconfig @@ -62,7 +62,7 @@ config S390 select ARCH_BINFMT_ELF_STATE select ARCH_ENABLE_MEMORY_HOTPLUG if SPARSEMEM select ARCH_ENABLE_MEMORY_HOTREMOVE - select ARCH_ENABLE_SPLIT_PMD_PTLOCK + select ARCH_ENABLE_SPLIT_PMD_PTLOCK if PGTABLE_LEVELS > 2 select ARCH_HAS_DEBUG_VM_PGTABLE select ARCH_HAS_DEBUG_WX select ARCH_HAS_DEVMEM_IS_ALLOWED --- a/arch/x86/Kconfig~mm-thp-make-arch_enable_split_pmd_ptlock-dependent-on-pgtable_levels-2 +++ a/arch/x86/Kconfig @@ -63,7 +63,7 @@ config X86 select ARCH_ENABLE_HUGEPAGE_MIGRATION if X86_64 && HUGETLB_PAGE && MIGRATION select ARCH_ENABLE_MEMORY_HOTPLUG if X86_64 || (X86_32 && HIGHMEM) select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG - select ARCH_ENABLE_SPLIT_PMD_PTLOCK if X86_64 || X86_PAE + select ARCH_ENABLE_SPLIT_PMD_PTLOCK if (PGTABLE_LEVELS > 2) && (X86_64 || X86_PAE) select ARCH_ENABLE_THP_MIGRATION if X86_64 && TRANSPARENT_HUGEPAGE select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI select ARCH_HAS_CACHE_LINE_SIZE From patchwork Thu Jul 1 01:52:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96735C11F6B for ; Thu, 1 Jul 2021 01:52:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4D06F61468 for ; Thu, 1 Jul 2021 01:52:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D06F61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C59E88D022F; Wed, 30 Jun 2021 21:52:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C30628D0202; Wed, 30 Jun 2021 21:52:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF87F8D022F; Wed, 30 Jun 2021 21:52:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id 8DEFA8D0202 for ; Wed, 30 Jun 2021 21:52:03 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 62254253D3 for ; Thu, 1 Jul 2021 01:52:03 +0000 (UTC) X-FDA: 78312343326.03.6CD9546 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id F2D836001AA8 for ; Thu, 1 Jul 2021 01:52:02 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9E4FB6146D; Thu, 1 Jul 2021 01:52:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104322; bh=sSdelPm01ave48Y2ecTahcYVqBfBy+Uk6DbljjcyDcg=; h=Date:From:To:Subject:In-Reply-To:From; b=QMfJ/jrhIbIGgcwM6Jt5ykMi7HeLBOyFxXMwWjlDWn5NgrfMENHji5/Zd8/kWHyWn vmkca7UsJVbr+axUI75Y0yialawADmoC1oOcjzI1v621UQBPxck7NmKKJ3UbBNHVRC sUkb+q7aOc5DKea8o8r8ILMzJhkkfiwsHhSla3KQ= Date: Wed, 30 Jun 2021 18:52:01 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, hughd@google.com, jack@suse.cz, juew@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, osalvador@suse.de, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, shy828301@gmail.com, torvalds@linux-foundation.org, wangyugui@e16-tech.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 091/192] mm: rmap: make try_to_unmap() void function Message-ID: <20210701015201.N4tR2xm1a%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="QMfJ/jrh"; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: o5ddsa5kosw15da48c6xstuxkkqbgce4 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F2D836001AA8 X-HE-Tag: 1625104322-696878 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Shi Subject: mm: rmap: make try_to_unmap() void function Currently try_to_unmap() return bool value by checking page_mapcount(), however this may return false positive since page_mapcount() doesn't check all subpages of compound page. The total_mapcount() could be used instead, but its cost is higher since it traverses all subpages. Actually the most callers of try_to_unmap() don't care about the return value at all. So just need check if page is still mapped by page_mapped() when necessary. And page_mapped() does bail out early when it finds mapped subpage. Link: https://lkml.kernel.org/r/bb27e3fe-6036-b637-5086-272befbfe3da@google.com Suggested-by: Hugh Dickins Signed-off-by: Yang Shi Acked-by: Minchan Kim Reviewed-by: Shakeel Butt Acked-by: Kirill A. Shutemov Signed-off-by: Hugh Dickins Acked-by: Naoya Horiguchi Cc: Alistair Popple Cc: Jan Kara Cc: Jue Wang Cc: "Matthew Wilcox (Oracle)" Cc: Miaohe Lin Cc: Oscar Salvador Cc: Peter Xu Cc: Ralph Campbell Cc: Wang Yugui Cc: Zi Yan Signed-off-by: Andrew Morton --- include/linux/rmap.h | 2 +- mm/memory-failure.c | 15 +++++++-------- mm/rmap.c | 15 ++++----------- mm/vmscan.c | 3 ++- 4 files changed, 14 insertions(+), 21 deletions(-) --- a/include/linux/rmap.h~mm-rmap-make-try_to_unmap-void-function +++ a/include/linux/rmap.h @@ -195,7 +195,7 @@ static inline void page_dup_rmap(struct int page_referenced(struct page *, int is_locked, struct mem_cgroup *memcg, unsigned long *vm_flags); -bool try_to_unmap(struct page *, enum ttu_flags flags); +void try_to_unmap(struct page *, enum ttu_flags flags); /* Avoid racy checks */ #define PVMW_SYNC (1 << 0) --- a/mm/memory-failure.c~mm-rmap-make-try_to_unmap-void-function +++ a/mm/memory-failure.c @@ -1269,7 +1269,7 @@ static bool hwpoison_user_mappings(struc enum ttu_flags ttu = TTU_IGNORE_MLOCK; struct address_space *mapping; LIST_HEAD(tokill); - bool unmap_success = true; + bool unmap_success; int kill = 1, forcekill; struct page *hpage = *hpagep; bool mlocked = PageMlocked(hpage); @@ -1332,7 +1332,7 @@ static bool hwpoison_user_mappings(struc collect_procs(hpage, &tokill, flags & MF_ACTION_REQUIRED); if (!PageHuge(hpage)) { - unmap_success = try_to_unmap(hpage, ttu); + try_to_unmap(hpage, ttu); } else { if (!PageAnon(hpage)) { /* @@ -1344,17 +1344,16 @@ static bool hwpoison_user_mappings(struc */ mapping = hugetlb_page_mapping_lock_write(hpage); if (mapping) { - unmap_success = try_to_unmap(hpage, - ttu|TTU_RMAP_LOCKED); + try_to_unmap(hpage, ttu|TTU_RMAP_LOCKED); i_mmap_unlock_write(mapping); - } else { + } else pr_info("Memory failure: %#lx: could not lock mapping for mapped huge page\n", pfn); - unmap_success = false; - } } else { - unmap_success = try_to_unmap(hpage, ttu); + try_to_unmap(hpage, ttu); } } + + unmap_success = !page_mapped(hpage); if (!unmap_success) pr_err("Memory failure: %#lx: failed to unmap page (mapcount=%d)\n", pfn, page_mapcount(hpage)); --- a/mm/rmap.c~mm-rmap-make-try_to_unmap-void-function +++ a/mm/rmap.c @@ -1405,7 +1405,7 @@ static bool try_to_unmap_one(struct page /* * When racing against e.g. zap_pte_range() on another cpu, * in between its ptep_get_and_clear_full() and page_remove_rmap(), - * try_to_unmap() may return false when it is about to become true, + * try_to_unmap() may return before page_mapped() has become false, * if page table locking is skipped: use TTU_SYNC to wait for that. */ if (flags & TTU_SYNC) @@ -1756,9 +1756,10 @@ static int page_not_mapped(struct page * * Tries to remove all the page table entries which are mapping this * page, used in the pageout path. Caller must hold the page lock. * - * If unmap is successful, return true. Otherwise, false. + * It is the caller's responsibility to check if the page is still + * mapped when needed (use TTU_SYNC to prevent accounting races). */ -bool try_to_unmap(struct page *page, enum ttu_flags flags) +void try_to_unmap(struct page *page, enum ttu_flags flags) { struct rmap_walk_control rwc = { .rmap_one = try_to_unmap_one, @@ -1783,14 +1784,6 @@ bool try_to_unmap(struct page *page, enu rmap_walk_locked(page, &rwc); else rmap_walk(page, &rwc); - - /* - * When racing against e.g. zap_pte_range() on another cpu, - * in between its ptep_get_and_clear_full() and page_remove_rmap(), - * try_to_unmap() may return false when it is about to become true, - * if page table locking is skipped: use TTU_SYNC to wait for that. - */ - return !page_mapcount(page); } /** --- a/mm/vmscan.c~mm-rmap-make-try_to_unmap-void-function +++ a/mm/vmscan.c @@ -1499,7 +1499,8 @@ static unsigned int shrink_page_list(str if (unlikely(PageTransHuge(page))) flags |= TTU_SPLIT_HUGE_PMD; - if (!try_to_unmap(page, flags)) { + try_to_unmap(page, flags); + if (page_mapped(page)) { stat->nr_unmap_fail += nr_pages; if (!was_swapbacked && PageSwapBacked(page)) stat->nr_lazyfree_fail += nr_pages; From patchwork Thu Jul 1 01:52:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD8F1C11F69 for ; Thu, 1 Jul 2021 01:52:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 84E1B61468 for ; Thu, 1 Jul 2021 01:52:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 84E1B61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0C1808D0230; Wed, 30 Jun 2021 21:52:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0992D8D0202; Wed, 30 Jun 2021 21:52:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7C2D8D0230; Wed, 30 Jun 2021 21:52:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id C22A98D0202 for ; Wed, 30 Jun 2021 21:52:06 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9E883181AEF3F for ; Thu, 1 Jul 2021 01:52:06 +0000 (UTC) X-FDA: 78312343452.16.B070838 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 5B68FE0016BE for ; Thu, 1 Jul 2021 01:52:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1F6AC61474; Thu, 1 Jul 2021 01:52:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104325; bh=YB2daD+Wi9X9GUd02UzY33OL+VvqiKaf8fIYoCEIhWI=; h=Date:From:To:Subject:In-Reply-To:From; b=XQjRMY9NR1mTaY75pRRee5NtCerDPlMRm5CeNigaZsL6V6o7/3Nw6gtgqAHtaB/Gy BBxpPXQW6Ymk0Q4MMBo09JbOx+5J8T/5hgT+Vf5wTnS9vg1DellVu6cqMaqVfmnib6 Mi8+9Z7Mb55Suwh5zPgs3pa2scO8kHWkWmUS4ANw= Date: Wed, 30 Jun 2021 18:52:04 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, hughd@google.com, jack@suse.cz, juew@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, osalvador@suse.de, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, shy828301@gmail.com, torvalds@linux-foundation.org, wangyugui@e16-tech.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 092/192] mm/thp: remap_page() is only needed on anonymous THP Message-ID: <20210701015204.PUckIgqJ6%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5B68FE0016BE X-Stat-Signature: nwdk4dgouaom46k9wsdr37touwpg1ujo Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=XQjRMY9N; dmarc=none; spf=pass (imf30.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104326-622867 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hugh Dickins Subject: mm/thp: remap_page() is only needed on anonymous THP THP splitting's unmap_page() only sets TTU_SPLIT_FREEZE when PageAnon, and migration entries are only inserted when TTU_MIGRATION (unused here) or TTU_SPLIT_FREEZE is set: so it's just a waste of time for remap_page() to search for migration entries to remove when !PageAnon. Link: https://lkml.kernel.org/r/f987bc44-f28e-688d-2424-b4722153ed8@google.com Fixes: baa355fd3314 ("thp: file pages support for split_huge_page()") Signed-off-by: Hugh Dickins Reviewed-by: Yang Shi Acked-by: Kirill A. Shutemov Cc: Alistair Popple Cc: Jan Kara Cc: Jue Wang Cc: "Matthew Wilcox (Oracle)" Cc: Miaohe Lin Cc: Minchan Kim Cc: Naoya Horiguchi Cc: Oscar Salvador Cc: Peter Xu Cc: Ralph Campbell Cc: Shakeel Butt Cc: Wang Yugui Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/huge_memory.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/mm/huge_memory.c~mm-thp-remap_page-is-only-needed-on-anonymous-thp +++ a/mm/huge_memory.c @@ -2307,6 +2307,7 @@ static void unmap_page(struct page *page VM_BUG_ON_PAGE(!PageHead(page), page); + /* If TTU_SPLIT_FREEZE is ever extended to file, update remap_page() */ if (PageAnon(page)) ttu_flags |= TTU_SPLIT_FREEZE; @@ -2318,6 +2319,10 @@ static void unmap_page(struct page *page static void remap_page(struct page *page, unsigned int nr) { int i; + + /* If TTU_SPLIT_FREEZE is ever extended to file, remove this check */ + if (!PageAnon(page)) + return; if (PageTransHuge(page)) { remove_migration_ptes(page, page, true); } else { From patchwork Thu Jul 1 01:52:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55FE3C11F6A for ; Thu, 1 Jul 2021 01:52:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0CF6261241 for ; Thu, 1 Jul 2021 01:52:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0CF6261241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7ECB88D0231; Wed, 30 Jun 2021 21:52:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7C3388D0202; Wed, 30 Jun 2021 21:52:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 68C068D0231; Wed, 30 Jun 2021 21:52:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id 42DF18D0202 for ; Wed, 30 Jun 2021 21:52:10 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0E523181AEF3F for ; Thu, 1 Jul 2021 01:52:10 +0000 (UTC) X-FDA: 78312343620.12.0123E97 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id C5108F0000AD for ; Thu, 1 Jul 2021 01:52:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 78AE661468; Thu, 1 Jul 2021 01:52:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104329; bh=2dZWw63BUDFoIX03mW6b/itp6jdWGNjUtM9R8vs/XTg=; h=Date:From:To:Subject:In-Reply-To:From; b=s+LYQvHSQlqg+3J09xpPnX0w9v/EVNA1+l0Xl+VO/JhOptpQHG3qsptan03fTlhiY 1rKczMnSSBYNwH/LdHOSjdJvm1kpiGLF54vdCMJdqsKueVSJku1U3qVfr6Mcy3vNkE ILOi29TwUYfQjo/zIO2QARrVoJGznmctyJf8WpNM= Date: Wed, 30 Jun 2021 18:52:08 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, hughd@google.com, jack@suse.cz, juew@google.com, kirill.shutemov@linux.intel.com, linmiaohe@huawei.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, osalvador@suse.de, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, shy828301@gmail.com, torvalds@linux-foundation.org, wangyugui@e16-tech.com, willy@infradead.org, ziy@nvidia.com Subject: [patch 093/192] mm: hwpoison_user_mappings() try_to_unmap() with TTU_SYNC Message-ID: <20210701015208.xKgFl8Tfb%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=s+LYQvHS; dmarc=none; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: yeb54pq7hdmqg87acn6rdq56e6ezg6ix X-Rspamd-Queue-Id: C5108F0000AD X-HE-Tag: 1625104329-58198 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hugh Dickins Subject: mm: hwpoison_user_mappings() try_to_unmap() with TTU_SYNC TTU_SYNC prevents an unlikely race, when try_to_unmap() returns shortly before the page is accounted as unmapped. It is unlikely to coincide with hwpoisoning, but now that we have the flag, hwpoison_user_mappings() would do well to use it. Link: https://lkml.kernel.org/r/329c28ed-95df-9a2c-8893-b444d8a6d340@google.com Signed-off-by: Hugh Dickins Acked-by: Kirill A. Shutemov Acked-by: Naoya Horiguchi Cc: Alistair Popple Cc: Jan Kara Cc: Jue Wang Cc: "Matthew Wilcox (Oracle)" Cc: Miaohe Lin Cc: Minchan Kim Cc: Oscar Salvador Cc: Peter Xu Cc: Ralph Campbell Cc: Shakeel Butt Cc: Wang Yugui Cc: Yang Shi Cc: Zi Yan Signed-off-by: Andrew Morton --- mm/memory-failure.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/memory-failure.c~mm-hwpoison_user_mappings-try_to_unmap-with-ttu_sync +++ a/mm/memory-failure.c @@ -1266,7 +1266,7 @@ static int get_hwpoison_page(struct page static bool hwpoison_user_mappings(struct page *p, unsigned long pfn, int flags, struct page **hpagep) { - enum ttu_flags ttu = TTU_IGNORE_MLOCK; + enum ttu_flags ttu = TTU_IGNORE_MLOCK | TTU_SYNC; struct address_space *mapping; LIST_HEAD(tokill); bool unmap_success; From patchwork Thu Jul 1 01:52:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC4A3C11F69 for ; Thu, 1 Jul 2021 01:52:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BBADD61468 for ; Thu, 1 Jul 2021 01:52:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBADD61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 41C358D0232; Wed, 30 Jun 2021 21:52:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FF9A8D0202; Wed, 30 Jun 2021 21:52:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BB098D0232; Wed, 30 Jun 2021 21:52:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id 0711D8D0202 for ; Wed, 30 Jun 2021 21:52:12 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D2D3224886 for ; Thu, 1 Jul 2021 01:52:12 +0000 (UTC) X-FDA: 78312343704.34.FADF9C8 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 94B6BF00008C for ; Thu, 1 Jul 2021 01:52:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B312C61474; Thu, 1 Jul 2021 01:52:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104331; bh=2fBKr6yCuCxyiEB0OEX48/16v+DiafEG5O7H/lh1Oc0=; h=Date:From:To:Subject:In-Reply-To:From; b=btgyzVSsqDPNioKFPDFd98ItYfZLBGRdDRdtXQSjZfTLOrRKP+zPzD/qJ1RvbyMCU telp4WfkudzXCkErImeJVCrnO5XdIyZlz15xD/63uC44JVZ0FK2Uig1iUm4eqgIYug yuirgMOYsl9fe3x0Wy02DrPkSitldIv17MFOF66I= Date: Wed, 30 Jun 2021 18:52:11 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 094/192] mm/thp: fix strncpy warning Message-ID: <20210701015211.j_262-htJ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 94B6BF00008C X-Stat-Signature: d1s7eh94zxqitb83w9qhjumjr398tqkq Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=btgyzVSs; dmarc=none; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104332-407661 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/thp: fix strncpy warning Using MAX_INPUT_BUF_SZ as the maximum length of the string makes fortify complain as it thinks the string might be longer than the buffer, and if it is, we will end up with a "string" that is missing a NUL terminator. It's trivial to show that 'tok' points to a NUL-terminated string which is less than MAX_INPUT_BUF_SZ in length, so we may as well just use strcpy() and avoid the warning. Link: https://lkml.kernel.org/r/20210615200242.1716568-4-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/huge_memory.c~mm-thp-fix-strncpy-warning +++ a/mm/huge_memory.c @@ -3101,7 +3101,7 @@ static ssize_t split_huge_pages_write(st tok = strsep(&buf, ","); if (tok) { - strncpy(file_path, tok, MAX_INPUT_BUF_SZ); + strcpy(file_path, tok); } else { ret = -EINVAL; goto out; From patchwork Thu Jul 1 01:52:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25417C11F66 for ; Thu, 1 Jul 2021 01:52:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CD44C6146D for ; Thu, 1 Jul 2021 01:52:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD44C6146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 491D28D0233; Wed, 30 Jun 2021 21:52:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 467CA8D0202; Wed, 30 Jun 2021 21:52:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3575C8D0233; Wed, 30 Jun 2021 21:52:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 0C7288D0202 for ; Wed, 30 Jun 2021 21:52:16 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D30B682E8741 for ; Thu, 1 Jul 2021 01:52:15 +0000 (UTC) X-FDA: 78312343830.17.E494F9A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 85BB710004CC for ; Thu, 1 Jul 2021 01:52:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 80F0A61469; Thu, 1 Jul 2021 01:52:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104334; bh=TZ02/5BRV3MVGz6H7yhrD7JgNllspAGhg7ulbsDIIws=; h=Date:From:To:Subject:In-Reply-To:From; b=PchLKqP2i/PEF2rI96lR7wqUsu8+5uR7IJpmpfbGKZkZB1j51N8WANRroi+Yq4v3p 9B8pCP7QxtZ3ay+XpPEkLki+n8+mdgkn3nf3z0RVNukGpoJBgU5J0oEG5/s7BQhDYU 85ybDdPgJwQwRKqoR9VzTmZVxN2oBW7f4CwgQJSc= Date: Wed, 30 Jun 2021 18:52:14 -0700 From: Andrew Morton To: akpm@linux-foundation.org, chenli@uniontech.com, david@redhat.com, geert@linux-m68k.org, gerg@linux-m68k.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 095/192] nommu: remove __GFP_HIGHMEM in vmalloc/vzalloc Message-ID: <20210701015214.X6biaOQaV%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=PchLKqP2; dmarc=none; spf=pass (imf07.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: i8ty5c7665yn89c8we5ssu1ph9idmy4y X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 85BB710004CC X-HE-Tag: 1625104335-342999 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chen Li Subject: nommu: remove __GFP_HIGHMEM in vmalloc/vzalloc mm/nommu.c: void *__vmalloc(unsigned long size, gfp_t gfp_mask) { /* * You can't specify __GFP_HIGHMEM with kmalloc() since kmalloc() * returns only a logical address. */ return kmalloc(size, (gfp_mask | __GFP_COMP) & ~__GFP_HIGHMEM); } nommu's __vmalloc just uses kmalloc internally and elimitates __GFP_HIGHMEM, so it makes no sense to add __GFP_HIGHMEM for nommu's vmalloc/vzalloc. [akpm@linux-foundation.org: coding style fixes] Link: https://lkml.kernel.org/r/875z00rnp8.wl-chenli@uniontech.com Signed-off-by: Chen Li Reviewed-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Cc: Greg Ungerer Cc: Geert Uytterhoeven Signed-off-by: Andrew Morton --- mm/nommu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/nommu.c~nommu-remove-__gfp_highmem-in-vmalloc-vzalloc +++ a/mm/nommu.c @@ -223,7 +223,7 @@ long vread(char *buf, char *addr, unsign */ void *vmalloc(unsigned long size) { - return __vmalloc(size, GFP_KERNEL | __GFP_HIGHMEM); + return __vmalloc(size, GFP_KERNEL); } EXPORT_SYMBOL(vmalloc); @@ -241,7 +241,7 @@ EXPORT_SYMBOL(vmalloc); */ void *vzalloc(unsigned long size) { - return __vmalloc(size, GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); + return __vmalloc(size, GFP_KERNEL | __GFP_ZERO); } EXPORT_SYMBOL(vzalloc); From patchwork Thu Jul 1 01:52:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFFEEC11F68 for ; Thu, 1 Jul 2021 01:52:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A0F0E6146D for ; Thu, 1 Jul 2021 01:52:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A0F0E6146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2214B8D0234; Wed, 30 Jun 2021 21:52:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F7268D0202; Wed, 30 Jun 2021 21:52:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E6BC8D0234; Wed, 30 Jun 2021 21:52:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0240.hostedemail.com [216.40.44.240]) by kanga.kvack.org (Postfix) with ESMTP id DC87F8D0202 for ; Wed, 30 Jun 2021 21:52:18 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B124B82F4874 for ; Thu, 1 Jul 2021 01:52:18 +0000 (UTC) X-FDA: 78312343956.16.425DFE1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 59CB46001AA5 for ; Thu, 1 Jul 2021 01:52:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6D49261241; Thu, 1 Jul 2021 01:52:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104337; bh=bIYmOeWvPd5sLcxcaDg8HxtQCgr2Nz9dnoQh5TQY8A4=; h=Date:From:To:Subject:In-Reply-To:From; b=NsO0qq7+uob/APMCgAknMeDeOyPOO/pO7GLWDB/Ku5qyXNGVAGFf5GhtkNt5aPLDe fjn+g4EHbc2xKyfsUA9ACPRkciYWDX0h7q3wtpFC/pJ6aqklzclwbKFMpEpZAqqZgm dsk3jWGbtBPFs+D5sqXw6ZzlEtrJjBo+KhcUUhRc= Date: Wed, 30 Jun 2021 18:52:17 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, Liam.Howlett@Oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 096/192] mm/nommu: unexport do_munmap() Message-ID: <20210701015217.qU-VNmZsg%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=NsO0qq7+; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: tgc6gjgja64rh8rxj9ii4zprtyfr6798 X-Rspamd-Queue-Id: 59CB46001AA5 X-HE-Tag: 1625104338-942867 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Liam Howlett Subject: mm/nommu: unexport do_munmap() do_munmap() does not take the mmap_write_lock(). vm_munmap() should be used instead. Link: https://lkml.kernel.org/r/20210604194002.648037-1-Liam.Howlett@Oracle.com Signed-off-by: Liam R. Howlett Reviewed-by: Andrew Morton Reviewed-by: Matthew Wilcox (Oracle) Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/nommu.c | 1 - 1 file changed, 1 deletion(-) --- a/mm/nommu.c~mm-nommu-unexport-do_munmap +++ a/mm/nommu.c @@ -1501,7 +1501,6 @@ erase_whole_vma: delete_vma(mm, vma); return 0; } -EXPORT_SYMBOL(do_munmap); int vm_munmap(unsigned long addr, size_t len) { From patchwork Thu Jul 1 01:52:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12356431 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH,RDNS_NONE, SPF_HELO_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org ([198.145.29.99]:44358 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238447AbhGAByu (ORCPT ); Wed, 30 Jun 2021 21:54:50 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8B30261241; Thu, 1 Jul 2021 01:52:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104341; bh=MAN3R6hwaXK6OVgDLodQ1kFVjzdBfbmW8KvAXATB+Sk=; h=Date:From:To:Subject:In-Reply-To:From; b=ywB66YlJE05EayGRHUHiUyejgyxuectuumnfpW84OPo77nKxavg/5J0t+URuMWz5F CPE4sL0ph0ZJaK5od+tYyUdRRq3vIjBrKOhC4MU6AaGTfsYZQRDUUaQBkHhVXHMY0i UVbIfCMTcn6Nv2bIKvAqke4QGHVrmgkKc6ATbqJg= Date: Wed, 30 Jun 2021 18:52:20 -0700 From: Andrew Morton To: akpm@linux-foundation.org, bp@alien8.de, catalin.marinas@arm.com, davem@davemloft.net, geert@linux-m68k.org, linux-mm@kvack.org, linux@armlinux.org.uk, michal.simek@xilinx.com, mingo@redhat.com, mm-commits@vger.kernel.org, mpe@ellerman.id.au, palmerdabbelt@google.com, rppt@linux.ibm.com, rth@twiddle.net, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, wangkefeng.wang@huawei.com, will@kernel.org Subject: [patch 097/192] mm: generalize ZONE_[DMA|DMA32] Message-ID: <20210701015220.X8rakqyg8%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Return-Path: X-Envelope-To: <"|/opt/Majordomo/wrapper resend -l mm-commits -f x-mm-commits -h vger.kernel.org -L /opt/mail/bin/ta/listexpand -O mm-commits-owner /opt/Majordomo/lists/mm-commits"> (uid 400) X-Orcpt: rfc822;mm-commits@vger.kernel.org Original-Recipient: rfc822;mm-commits@vger.kernel.org List-Id: From: Kefeng Wang Subject: mm: generalize ZONE_[DMA|DMA32] ZONE_[DMA|DMA32] configs have duplicate definitions on platforms that subscribe to them. Instead, just make them generic options which can be selected on applicable platforms. Also only x86/arm64 architectures could enable both ZONE_DMA and ZONE_DMA32 if EXPERT, add ARCH_HAS_ZONE_DMA_SET to make dma zone configurable and visible on the two architectures. Link: https://lkml.kernel.org/r/20210528074557.17768-1-wangkefeng.wang@huawei.com Signed-off-by: Kefeng Wang Acked-by: Catalin Marinas [arm64] Acked-by: Geert Uytterhoeven [m68k] Acked-by: Mike Rapoport Acked-by: Palmer Dabbelt [RISC-V] Acked-by: Michal Simek [microblaze] Acked-by: Michael Ellerman [powerpc] Cc: Catalin Marinas Cc: Will Deacon Cc: Geert Uytterhoeven Cc: Thomas Bogendoerfer Cc: "David S. Miller" Cc: Ingo Molnar Cc: Borislav Petkov Cc: Richard Henderson Cc: Russell King Signed-off-by: Andrew Morton --- arch/alpha/Kconfig | 5 +---- arch/arm/Kconfig | 3 --- arch/arm64/Kconfig | 9 +-------- arch/ia64/Kconfig | 4 +--- arch/m68k/Kconfig | 5 +---- arch/microblaze/Kconfig | 4 +--- arch/mips/Kconfig | 7 ------- arch/powerpc/Kconfig | 4 ---- arch/powerpc/platforms/Kconfig.cputype | 1 + arch/riscv/Kconfig | 5 +---- arch/s390/Kconfig | 4 +--- arch/sparc/Kconfig | 5 +---- arch/x86/Kconfig | 15 ++------------- mm/Kconfig | 12 ++++++++++++ 14 files changed, 23 insertions(+), 60 deletions(-) --- a/arch/alpha/Kconfig~mm-generalize-zone_ +++ a/arch/alpha/Kconfig @@ -40,6 +40,7 @@ config ALPHA select MMU_GATHER_NO_RANGE select SET_FS select SPARSEMEM_EXTREME if SPARSEMEM + select ZONE_DMA help The Alpha is a 64-bit general-purpose processor designed and marketed by the Digital Equipment Corporation of blessed memory, @@ -65,10 +66,6 @@ config GENERIC_CALIBRATE_DELAY bool default y -config ZONE_DMA - bool - default y - config GENERIC_ISA_DMA bool default y --- a/arch/arm64/Kconfig~mm-generalize-zone_ +++ a/arch/arm64/Kconfig @@ -42,6 +42,7 @@ config ARM64 select ARCH_HAS_SYSCALL_WRAPPER select ARCH_HAS_TEARDOWN_DMA_OPS if IOMMU_SUPPORT select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST + select ARCH_HAS_ZONE_DMA_SET if EXPERT select ARCH_HAVE_ELF_PROT select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_INLINE_READ_LOCK if !PREEMPTION @@ -306,14 +307,6 @@ config GENERIC_CSUM config GENERIC_CALIBRATE_DELAY def_bool y -config ZONE_DMA - bool "Support DMA zone" if EXPERT - default y - -config ZONE_DMA32 - bool "Support DMA32 zone" if EXPERT - default y - config ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE def_bool y --- a/arch/arm/Kconfig~mm-generalize-zone_ +++ a/arch/arm/Kconfig @@ -218,9 +218,6 @@ config GENERIC_CALIBRATE_DELAY config ARCH_MAY_HAVE_PC_FDC bool -config ZONE_DMA - bool - config ARCH_SUPPORTS_UPROBES def_bool y --- a/arch/ia64/Kconfig~mm-generalize-zone_ +++ a/arch/ia64/Kconfig @@ -60,6 +60,7 @@ config IA64 select NUMA if !FLATMEM select PCI_MSI_ARCH_FALLBACKS if PCI_MSI select SET_FS + select ZONE_DMA32 default y help The Itanium Processor Family is Intel's 64-bit successor to @@ -72,9 +73,6 @@ config 64BIT select ATA_NONSTANDARD if ATA default y -config ZONE_DMA32 - def_bool y - config MMU bool default y --- a/arch/m68k/Kconfig~mm-generalize-zone_ +++ a/arch/m68k/Kconfig @@ -34,6 +34,7 @@ config M68K select SET_FS select UACCESS_MEMCPY if !MMU select VIRT_TO_BUS + select ZONE_DMA config CPU_BIG_ENDIAN def_bool y @@ -62,10 +63,6 @@ config TIME_LOW_RES config NO_IOPORT_MAP def_bool y -config ZONE_DMA - bool - default y - config HZ int default 1000 if CLEOPATRA --- a/arch/microblaze/Kconfig~mm-generalize-zone_ +++ a/arch/microblaze/Kconfig @@ -43,6 +43,7 @@ config MICROBLAZE select MMU_GATHER_NO_RANGE select SPARSE_IRQ select SET_FS + select ZONE_DMA # Endianness selection choice @@ -60,9 +61,6 @@ config CPU_LITTLE_ENDIAN endchoice -config ZONE_DMA - def_bool y - config ARCH_HAS_ILOG2_U32 def_bool n --- a/arch/mips/Kconfig~mm-generalize-zone_ +++ a/arch/mips/Kconfig @@ -3274,13 +3274,6 @@ config I8253 select CLKSRC_I8253 select CLKEVT_I8253 select MIPS_EXTERNAL_TIMER - -config ZONE_DMA - bool - -config ZONE_DMA32 - bool - endmenu config TRAD_SIGNALS --- a/arch/powerpc/Kconfig~mm-generalize-zone_ +++ a/arch/powerpc/Kconfig @@ -403,10 +403,6 @@ config PPC_ADV_DEBUG_DAC_RANGE config PPC_DAWR bool -config ZONE_DMA - bool - default y if PPC_BOOK3E_64 - config PGTABLE_LEVELS int default 2 if !PPC64 --- a/arch/powerpc/platforms/Kconfig.cputype~mm-generalize-zone_ +++ a/arch/powerpc/platforms/Kconfig.cputype @@ -111,6 +111,7 @@ config PPC_BOOK3E_64 select PPC_FPU # Make it a choice ? select PPC_SMP_MUXED_IPI select PPC_DOORBELL + select ZONE_DMA endchoice --- a/arch/riscv/Kconfig~mm-generalize-zone_ +++ a/arch/riscv/Kconfig @@ -104,6 +104,7 @@ config RISCV select SYSCTL_EXCEPTION_TRACE select THREAD_INFO_IN_TASK select UACCESS_MEMCPY if !MMU + select ZONE_DMA32 if 64BIT config ARCH_MMAP_RND_BITS_MIN default 18 if 64BIT @@ -133,10 +134,6 @@ config MMU Select if you want MMU-based virtualised addressing space support by paged memory management. If unsure, say 'Y'. -config ZONE_DMA32 - bool - default y if 64BIT - config VA_BITS int default 32 if 32BIT --- a/arch/s390/Kconfig~mm-generalize-zone_ +++ a/arch/s390/Kconfig @@ -2,9 +2,6 @@ config MMU def_bool y -config ZONE_DMA - def_bool y - config CPU_BIG_ENDIAN def_bool y @@ -210,6 +207,7 @@ config S390 select THREAD_INFO_IN_TASK select TTY select VIRT_CPU_ACCOUNTING + select ZONE_DMA # Note: keep the above list sorted alphabetically config SCHED_OMIT_FRAME_POINTER --- a/arch/sparc/Kconfig~mm-generalize-zone_ +++ a/arch/sparc/Kconfig @@ -59,6 +59,7 @@ config SPARC32 select CLZ_TAB select HAVE_UID16 select OLD_SIGACTION + select ZONE_DMA config SPARC64 def_bool 64BIT @@ -141,10 +142,6 @@ config HIGHMEM default y if SPARC32 select KMAP_LOCAL -config ZONE_DMA - bool - default y if SPARC32 - config GENERIC_ISA_DMA bool default y if SPARC32 --- a/arch/x86/Kconfig~mm-generalize-zone_ +++ a/arch/x86/Kconfig @@ -33,6 +33,7 @@ config X86_64 select NEED_DMA_MAP_STATE select SWIOTLB select ARCH_HAS_ELFCORE_COMPAT + select ZONE_DMA32 config FORCE_DYNAMIC_FTRACE def_bool y @@ -93,6 +94,7 @@ config X86 select ARCH_HAS_SYSCALL_WRAPPER select ARCH_HAS_UBSAN_SANITIZE_ALL select ARCH_HAS_DEBUG_WX + select ARCH_HAS_ZONE_DMA_SET if EXPERT select ARCH_HAVE_NMI_SAFE_CMPXCHG select ARCH_MIGHT_HAVE_ACPI_PDC if ACPI select ARCH_MIGHT_HAVE_PC_PARPORT @@ -343,9 +345,6 @@ config ARCH_SUSPEND_POSSIBLE config ARCH_WANT_GENERAL_HUGETLB def_bool y -config ZONE_DMA32 - def_bool y if X86_64 - config AUDIT_ARCH def_bool y if X86_64 @@ -393,16 +392,6 @@ config CC_HAS_SANE_STACKPROTECTOR menu "Processor type and features" -config ZONE_DMA - bool "DMA memory allocation support" if EXPERT - default y - help - DMA memory allocation support allows devices with less than 32-bit - addressing to allocate within the first 16MB of address space. - Disable if no such devices will be used. - - If unsure, say Y. - config SMP bool "Symmetric multi-processing support" help --- a/mm/Kconfig~mm-generalize-zone_ +++ a/mm/Kconfig @@ -761,6 +761,18 @@ config ARCH_HAS_CACHE_LINE_SIZE config ARCH_HAS_PTE_DEVMAP bool +config ARCH_HAS_ZONE_DMA_SET + bool + +config ZONE_DMA + bool "Support DMA zone" if ARCH_HAS_ZONE_DMA_SET + default y if ARM64 || X86 + +config ZONE_DMA32 + bool "Support DMA32 zone" if ARCH_HAS_ZONE_DMA_SET + depends on !X86_32 + default y if ARM64 + config ZONE_DEVICE bool "Device memory (pmem, HMM, etc...) hotplug support" depends on MEMORY_HOTPLUG From patchwork Thu Jul 1 01:52:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6671DC11F69 for ; Thu, 1 Jul 2021 01:52:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 13B9861241 for ; Thu, 1 Jul 2021 01:52:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 13B9861241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 888908D0235; Wed, 30 Jun 2021 21:52:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 85F908D0202; Wed, 30 Jun 2021 21:52:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7274A8D0235; Wed, 30 Jun 2021 21:52:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id 52C298D0202 for ; Wed, 30 Jun 2021 21:52:26 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2529724886 for ; Thu, 1 Jul 2021 01:52:26 +0000 (UTC) X-FDA: 78312344292.21.84E65B2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id D3C3F4002087 for ; Thu, 1 Jul 2021 01:52:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2B6C86105A; Thu, 1 Jul 2021 01:52:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104345; bh=srZOmjHweQG1tAHO8xMW9Tn8yCWrDqidrbxEW2al5Hc=; h=Date:From:To:Subject:In-Reply-To:From; b=YT05pTQyIPQC7TRfrkLLJjD9qn9ZLWo6UiG3fyDYgF0xCmltWBn5KilJW6P4RO2Y6 46tiX3DWUvmadt3EBQ3w81VuXvx971AnVHsRHQRjW0zX41g/TVFlJYhKLQULjmQHbk wmSTfduU6y7qXNvtVI3KgRABi0n4nb8FdkfuwOZE= Date: Wed, 30 Jun 2021 18:52:23 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, chris@zankel.net, dave.hansen@intel.com, david@redhat.com, deller@gmx.de, eike-kernel@sf-tec.de, hughd@google.com, ink@jurassic.park.msu.ru, James.Bottomley@HansenPartnership.com, jannh@google.com, jcmvbkbc@gmail.com, jgg@ziepe.ca, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, linuxram@us.ibm.com, mattst88@gmail.com, mhocko@suse.com, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, mst@redhat.com, osalvador@suse.de, peterx@redhat.com, riel@surriel.com, rth@twiddle.net, shuah@kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vbabka@suse.cz, willy@infradead.org Subject: [patch 098/192] mm: make variable names for populate_vma_page_range() consistent Message-ID: <20210701015223.QTgRbro8j%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D3C3F4002087 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=YT05pTQy; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 3momx5xrzu54phhtr1edwuoge7e37hho X-HE-Tag: 1625104345-253398 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm: make variable names for populate_vma_page_range() consistent Patch series "mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables", v2. Excessive details on MADV_POPULATE_(READ|WRITE) can be found in patch #2. This patch (of 5): Let's make the variable names in the function declaration match the variable names used in the definition. Link: https://lkml.kernel.org/r/20210419135443.12822-1-david@redhat.com Link: https://lkml.kernel.org/r/20210419135443.12822-2-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador Cc: Michal Hocko Cc: Jason Gunthorpe Cc: Peter Xu Cc: Andrea Arcangeli Cc: Arnd Bergmann Cc: Chris Zankel Cc: Dave Hansen Cc: Helge Deller Cc: Hugh Dickins Cc: Ivan Kokshaysky Cc: "James E.J. Bottomley" Cc: Jann Horn Cc: Kirill A. Shutemov Cc: Matthew Wilcox (Oracle) Cc: Matt Turner Cc: Max Filippov Cc: Michael S. Tsirkin Cc: Mike Kravetz Cc: Minchan Kim Cc: Ram Pai Cc: Richard Henderson Cc: Rik van Riel Cc: Rolf Eike Beer Cc: Shuah Khan Cc: Thomas Bogendoerfer Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/internal.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/internal.h~mm-make-variable-names-for-populate_vma_page_range-consistent +++ a/mm/internal.h @@ -344,7 +344,7 @@ void __vma_unlink_list(struct mm_struct #ifdef CONFIG_MMU extern long populate_vma_page_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end, int *nonblocking); + unsigned long start, unsigned long end, int *locked); extern void munlock_vma_pages_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); static inline void munlock_vma_pages_all(struct vm_area_struct *vma) From patchwork Thu Jul 1 01:52:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2409AC11F68 for ; Thu, 1 Jul 2021 01:52:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B304261469 for ; Thu, 1 Jul 2021 01:52:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B304261469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3CDAD8D0202; Wed, 30 Jun 2021 21:52:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A4758D0236; Wed, 30 Jun 2021 21:52:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 254428D0202; Wed, 30 Jun 2021 21:52:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id E9E498D0202 for ; Wed, 30 Jun 2021 21:52:30 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BF151253D3 for ; Thu, 1 Jul 2021 01:52:30 +0000 (UTC) X-FDA: 78312344460.16.E656D50 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 542BF6001AA9 for ; Thu, 1 Jul 2021 01:52:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B1C0861468; Thu, 1 Jul 2021 01:52:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104349; bh=nAZaYso07xxJt9fyRje9fi45hhdPx6s78HxSrE8uJL8=; h=Date:From:To:Subject:In-Reply-To:From; b=xjcez30Lf9aKMSeZQrCsm3tqEwNJhovvIOkJS+9QViG2tUDu3Gejf5dYXExh8Jd2P NiJeab7EyqkndnY1r+og2G28HFfe9O47RAEkn12hxiCdXORlhf/7Zta6f6u0X0DoF5 ErRRg6pW3PfoiyQOhnmL5KZiSW2LZwQvirJzmUUU= Date: Wed, 30 Jun 2021 18:52:28 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, chris@zankel.net, dave.hansen@intel.com, david@redhat.com, deller@gmx.de, eike-kernel@sf-tec.de, hughd@google.com, ink@jurassic.park.msu.ru, James.Bottomley@HansenPartnership.com, jannh@google.com, jcmvbkbc@gmail.com, jgg@ziepe.ca, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, linuxram@us.ibm.com, mattst88@gmail.com, mhocko@suse.com, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, mst@redhat.com, osalvador@suse.de, peterx@redhat.com, riel@surriel.com, rth@twiddle.net, shuah@kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vbabka@suse.cz, willy@infradead.org Subject: [patch 099/192] mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables Message-ID: <20210701015228.QXA77Jpli%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=xjcez30L; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: giwzfybiwx4j6pd8r1fjnncu5837ttzp X-Rspamd-Queue-Id: 542BF6001AA9 X-HE-Tag: 1625104350-597030 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm/madvise: introduce MADV_POPULATE_(READ|WRITE) to prefault page tables I. Background: Sparse Memory Mappings When we manage sparse memory mappings dynamically in user space - also sometimes involving MAP_NORESERVE - we want to dynamically populate/ discard memory inside such a sparse memory region. Example users are hypervisors (especially implementing memory ballooning or similar technologies like virtio-mem) and memory allocators. In addition, we want to fail in a nice way (instead of generating SIGBUS) if populating does not succeed because we are out of backend memory (which can happen easily with file-based mappings, especially tmpfs and hugetlbfs). While MADV_DONTNEED, MADV_REMOVE and FALLOC_FL_PUNCH_HOLE allow for reliably discarding memory for most mapping types, there is no generic approach to populate page tables and preallocate memory. Although mmap() supports MAP_POPULATE, it is not applicable to the concept of sparse memory mappings, where we want to populate/discard dynamically and avoid expensive/problematic remappings. In addition, we never actually report errors during the final populate phase - it is best-effort only. fallocate() can be used to preallocate file-based memory and fail in a safe way. However, it cannot really be used for any private mappings on anonymous files via memfd due to COW semantics. In addition, fallocate() does not actually populate page tables, so we still always get pagefaults on first access - which is sometimes undesired (i.e., real-time workloads) and requires real prefaulting of page tables, not just a preallocation of backend storage. There might be interesting use cases for sparse memory regions along with mlockall(MCL_ONFAULT) which fallocate() cannot satisfy as it does not prefault page tables. II. On preallcoation/prefaulting from user space Because we don't have a proper interface, what applications (like QEMU and databases) end up doing is touching (i.e., reading+writing one byte to not overwrite existing data) all individual pages. However, that approach 1) Can result in wear on storage backing, because we end up reading/writing each page; this is especially a problem for dax/pmem. 2) Can result in mmap_sem contention when prefaulting via multiple threads. 3) Requires expensive signal handling, especially to catch SIGBUS in case of hugetlbfs/shmem/file-backed memory. For example, this is problematic in hypervisors like QEMU where SIGBUS handlers might already be used by other subsystems concurrently to e.g, handle hardware errors. "Simply" doing preallocation concurrently from other thread is not that easy. III. On MADV_WILLNEED Extending MADV_WILLNEED is not an option because 1. It would change the semantics: "Expect access in the near future." and "might be a good idea to read some pages" vs. "Definitely populate/ preallocate all memory and definitely fail on errors.". 2. Existing users (like virtio-balloon in QEMU when deflating the balloon) don't want populate/prealloc semantics. They treat this rather as a hint to give a little performance boost without too much overhead - and don't expect that a lot of memory might get consumed or a lot of time might be spent. IV. MADV_POPULATE_READ and MADV_POPULATE_WRITE Let's introduce MADV_POPULATE_READ and MADV_POPULATE_WRITE, inspired by MAP_POPULATE, with the following semantics: 1. MADV_POPULATE_READ can be used to prefault page tables just like manually reading each individual page. This will not break any COW mappings. The shared zero page might get mapped and no backend storage might get preallocated -- allocation might be deferred to write-fault time. Especially shared file mappings require an explicit fallocate() upfront to actually preallocate backend memory (blocks in the file system) in case the file might have holes. 2. If MADV_POPULATE_READ succeeds, all page tables have been populated (prefaulted) readable once. 3. MADV_POPULATE_WRITE can be used to preallocate backend memory and prefault page tables just like manually writing (or reading+writing) each individual page. This will break any COW mappings -- e.g., the shared zeropage is never populated. 4. If MADV_POPULATE_WRITE succeeds, all page tables have been populated (prefaulted) writable once. 5. MADV_POPULATE_READ and MADV_POPULATE_WRITE cannot be applied to special mappings marked with VM_PFNMAP and VM_IO. Also, proper access permissions (e.g., PROT_READ, PROT_WRITE) are required. If any such mapping is encountered, madvise() fails with -EINVAL. 6. If MADV_POPULATE_READ or MADV_POPULATE_WRITE fails, some page tables might have been populated. 7. MADV_POPULATE_READ and MADV_POPULATE_WRITE will return -EHWPOISON when encountering a HW poisoned page in the range. 8. Similar to MAP_POPULATE, MADV_POPULATE_READ and MADV_POPULATE_WRITE cannot protect from the OOM (Out Of Memory) handler killing the process. While the use case for MADV_POPULATE_WRITE is fairly obvious (i.e., preallocate memory and prefault page tables for VMs), one issue is that whenever we prefault pages writable, the pages have to be marked dirty, because the CPU could dirty them any time. while not a real problem for hugetlbfs or dax/pmem, it can be a problem for shared file mappings: each page will be marked dirty and has to be written back later when evicting. MADV_POPULATE_READ allows for optimizing this scenario: Pre-read a whole mapping from backend storage without marking it dirty, such that eviction won't have to write it back. As discussed above, shared file mappings might require an explciit fallocate() upfront to achieve preallcoation+prepopulation. Although sparse memory mappings are the primary use case, this will also be useful for other preallocate/prefault use cases where MAP_POPULATE is not desired or the semantics of MAP_POPULATE are not sufficient: as one example, QEMU users can trigger preallocation/prefaulting of guest RAM after the mapping was created -- and don't want errors to be silently suppressed. Looking at the history, MADV_POPULATE was already proposed in 2013 [1], however, the main motivation back than was performance improvements -- which should also still be the case. V. Single-threaded performance comparison I did a short experiment, prefaulting page tables on completely *empty mappings/files* and repeated the experiment 10 times. The results correspond to the shortest execution time. In general, the performance benefit for huge pages is negligible with small mappings. V.1: Private mappings POPULATE_READ and POPULATE_WRITE is fastest. Note that Reading/POPULATE_READ will populate the shared zeropage where applicable -- which result in short population times. The fastest way to allocate backend storage (here: swap or huge pages) and prefault page tables is POPULATE_WRITE. V.2: Shared mappings fallocate() is fastest, however, doesn't prefault page tables. POPULATE_WRITE is faster than simple writes and read/writes. POPULATE_READ is faster than simple reads. Without a fd, the fastest way to allocate backend storage and prefault page tables is POPULATE_WRITE. With an fd, the fastest way is usually FALLOCATE+POPULATE_READ or FALLOCATE+POPULATE_WRITE respectively; one exception are actual files: FALLOCATE+Read is slightly faster than FALLOCATE+POPULATE_READ. The fastest way to allocate backend storage prefault page tables is FALLOCATE+POPULATE_WRITE -- except when dealing with actual files; then, FALLOCATE+POPULATE_READ is fastest and won't directly mark all pages as dirty. v.3: Detailed results ================================================== 2 MiB MAP_PRIVATE: ************************************************** Anon 4 KiB : Read : 0.119 ms Anon 4 KiB : Write : 0.222 ms Anon 4 KiB : Read/Write : 0.380 ms Anon 4 KiB : POPULATE_READ : 0.060 ms Anon 4 KiB : POPULATE_WRITE : 0.158 ms Memfd 4 KiB : Read : 0.034 ms Memfd 4 KiB : Write : 0.310 ms Memfd 4 KiB : Read/Write : 0.362 ms Memfd 4 KiB : POPULATE_READ : 0.039 ms Memfd 4 KiB : POPULATE_WRITE : 0.229 ms Memfd 2 MiB : Read : 0.030 ms Memfd 2 MiB : Write : 0.030 ms Memfd 2 MiB : Read/Write : 0.030 ms Memfd 2 MiB : POPULATE_READ : 0.030 ms Memfd 2 MiB : POPULATE_WRITE : 0.030 ms tmpfs : Read : 0.033 ms tmpfs : Write : 0.313 ms tmpfs : Read/Write : 0.406 ms tmpfs : POPULATE_READ : 0.039 ms tmpfs : POPULATE_WRITE : 0.285 ms file : Read : 0.033 ms file : Write : 0.351 ms file : Read/Write : 0.408 ms file : POPULATE_READ : 0.039 ms file : POPULATE_WRITE : 0.290 ms hugetlbfs : Read : 0.030 ms hugetlbfs : Write : 0.030 ms hugetlbfs : Read/Write : 0.030 ms hugetlbfs : POPULATE_READ : 0.030 ms hugetlbfs : POPULATE_WRITE : 0.030 ms ************************************************** 4096 MiB MAP_PRIVATE: ************************************************** Anon 4 KiB : Read : 237.940 ms Anon 4 KiB : Write : 708.409 ms Anon 4 KiB : Read/Write : 1054.041 ms Anon 4 KiB : POPULATE_READ : 124.310 ms Anon 4 KiB : POPULATE_WRITE : 572.582 ms Memfd 4 KiB : Read : 136.928 ms Memfd 4 KiB : Write : 963.898 ms Memfd 4 KiB : Read/Write : 1106.561 ms Memfd 4 KiB : POPULATE_READ : 78.450 ms Memfd 4 KiB : POPULATE_WRITE : 805.881 ms Memfd 2 MiB : Read : 357.116 ms Memfd 2 MiB : Write : 357.210 ms Memfd 2 MiB : Read/Write : 357.606 ms Memfd 2 MiB : POPULATE_READ : 356.094 ms Memfd 2 MiB : POPULATE_WRITE : 356.937 ms tmpfs : Read : 137.536 ms tmpfs : Write : 954.362 ms tmpfs : Read/Write : 1105.954 ms tmpfs : POPULATE_READ : 80.289 ms tmpfs : POPULATE_WRITE : 822.826 ms file : Read : 137.874 ms file : Write : 987.025 ms file : Read/Write : 1107.439 ms file : POPULATE_READ : 80.413 ms file : POPULATE_WRITE : 857.622 ms hugetlbfs : Read : 355.607 ms hugetlbfs : Write : 355.729 ms hugetlbfs : Read/Write : 356.127 ms hugetlbfs : POPULATE_READ : 354.585 ms hugetlbfs : POPULATE_WRITE : 355.138 ms ************************************************** 2 MiB MAP_SHARED: ************************************************** Anon 4 KiB : Read : 0.394 ms Anon 4 KiB : Write : 0.348 ms Anon 4 KiB : Read/Write : 0.400 ms Anon 4 KiB : POPULATE_READ : 0.326 ms Anon 4 KiB : POPULATE_WRITE : 0.273 ms Anon 2 MiB : Read : 0.030 ms Anon 2 MiB : Write : 0.030 ms Anon 2 MiB : Read/Write : 0.030 ms Anon 2 MiB : POPULATE_READ : 0.030 ms Anon 2 MiB : POPULATE_WRITE : 0.030 ms Memfd 4 KiB : Read : 0.412 ms Memfd 4 KiB : Write : 0.372 ms Memfd 4 KiB : Read/Write : 0.419 ms Memfd 4 KiB : POPULATE_READ : 0.343 ms Memfd 4 KiB : POPULATE_WRITE : 0.288 ms Memfd 4 KiB : FALLOCATE : 0.137 ms Memfd 4 KiB : FALLOCATE+Read : 0.446 ms Memfd 4 KiB : FALLOCATE+Write : 0.330 ms Memfd 4 KiB : FALLOCATE+Read/Write : 0.454 ms Memfd 4 KiB : FALLOCATE+POPULATE_READ : 0.379 ms Memfd 4 KiB : FALLOCATE+POPULATE_WRITE : 0.268 ms Memfd 2 MiB : Read : 0.030 ms Memfd 2 MiB : Write : 0.030 ms Memfd 2 MiB : Read/Write : 0.030 ms Memfd 2 MiB : POPULATE_READ : 0.030 ms Memfd 2 MiB : POPULATE_WRITE : 0.030 ms Memfd 2 MiB : FALLOCATE : 0.030 ms Memfd 2 MiB : FALLOCATE+Read : 0.031 ms Memfd 2 MiB : FALLOCATE+Write : 0.031 ms Memfd 2 MiB : FALLOCATE+Read/Write : 0.031 ms Memfd 2 MiB : FALLOCATE+POPULATE_READ : 0.030 ms Memfd 2 MiB : FALLOCATE+POPULATE_WRITE : 0.030 ms tmpfs : Read : 0.416 ms tmpfs : Write : 0.369 ms tmpfs : Read/Write : 0.425 ms tmpfs : POPULATE_READ : 0.346 ms tmpfs : POPULATE_WRITE : 0.295 ms tmpfs : FALLOCATE : 0.139 ms tmpfs : FALLOCATE+Read : 0.447 ms tmpfs : FALLOCATE+Write : 0.333 ms tmpfs : FALLOCATE+Read/Write : 0.454 ms tmpfs : FALLOCATE+POPULATE_READ : 0.380 ms tmpfs : FALLOCATE+POPULATE_WRITE : 0.272 ms file : Read : 0.191 ms file : Write : 0.511 ms file : Read/Write : 0.524 ms file : POPULATE_READ : 0.196 ms file : POPULATE_WRITE : 0.434 ms file : FALLOCATE : 0.004 ms file : FALLOCATE+Read : 0.197 ms file : FALLOCATE+Write : 0.554 ms file : FALLOCATE+Read/Write : 0.480 ms file : FALLOCATE+POPULATE_READ : 0.201 ms file : FALLOCATE+POPULATE_WRITE : 0.381 ms hugetlbfs : Read : 0.030 ms hugetlbfs : Write : 0.030 ms hugetlbfs : Read/Write : 0.030 ms hugetlbfs : POPULATE_READ : 0.030 ms hugetlbfs : POPULATE_WRITE : 0.030 ms hugetlbfs : FALLOCATE : 0.030 ms hugetlbfs : FALLOCATE+Read : 0.031 ms hugetlbfs : FALLOCATE+Write : 0.031 ms hugetlbfs : FALLOCATE+Read/Write : 0.030 ms hugetlbfs : FALLOCATE+POPULATE_READ : 0.030 ms hugetlbfs : FALLOCATE+POPULATE_WRITE : 0.030 ms ************************************************** 4096 MiB MAP_SHARED: ************************************************** Anon 4 KiB : Read : 1053.090 ms Anon 4 KiB : Write : 913.642 ms Anon 4 KiB : Read/Write : 1060.350 ms Anon 4 KiB : POPULATE_READ : 893.691 ms Anon 4 KiB : POPULATE_WRITE : 782.885 ms Anon 2 MiB : Read : 358.553 ms Anon 2 MiB : Write : 358.419 ms Anon 2 MiB : Read/Write : 357.992 ms Anon 2 MiB : POPULATE_READ : 357.533 ms Anon 2 MiB : POPULATE_WRITE : 357.808 ms Memfd 4 KiB : Read : 1078.144 ms Memfd 4 KiB : Write : 942.036 ms Memfd 4 KiB : Read/Write : 1100.391 ms Memfd 4 KiB : POPULATE_READ : 925.829 ms Memfd 4 KiB : POPULATE_WRITE : 804.394 ms Memfd 4 KiB : FALLOCATE : 304.632 ms Memfd 4 KiB : FALLOCATE+Read : 1163.359 ms Memfd 4 KiB : FALLOCATE+Write : 933.186 ms Memfd 4 KiB : FALLOCATE+Read/Write : 1187.304 ms Memfd 4 KiB : FALLOCATE+POPULATE_READ : 1013.660 ms Memfd 4 KiB : FALLOCATE+POPULATE_WRITE : 794.560 ms Memfd 2 MiB : Read : 358.131 ms Memfd 2 MiB : Write : 358.099 ms Memfd 2 MiB : Read/Write : 358.250 ms Memfd 2 MiB : POPULATE_READ : 357.563 ms Memfd 2 MiB : POPULATE_WRITE : 357.334 ms Memfd 2 MiB : FALLOCATE : 356.735 ms Memfd 2 MiB : FALLOCATE+Read : 358.152 ms Memfd 2 MiB : FALLOCATE+Write : 358.331 ms Memfd 2 MiB : FALLOCATE+Read/Write : 358.018 ms Memfd 2 MiB : FALLOCATE+POPULATE_READ : 357.286 ms Memfd 2 MiB : FALLOCATE+POPULATE_WRITE : 357.523 ms tmpfs : Read : 1087.265 ms tmpfs : Write : 950.840 ms tmpfs : Read/Write : 1107.567 ms tmpfs : POPULATE_READ : 922.605 ms tmpfs : POPULATE_WRITE : 810.094 ms tmpfs : FALLOCATE : 306.320 ms tmpfs : FALLOCATE+Read : 1169.796 ms tmpfs : FALLOCATE+Write : 933.730 ms tmpfs : FALLOCATE+Read/Write : 1191.610 ms tmpfs : FALLOCATE+POPULATE_READ : 1020.474 ms tmpfs : FALLOCATE+POPULATE_WRITE : 798.945 ms file : Read : 654.101 ms file : Write : 1259.142 ms file : Read/Write : 1289.509 ms file : POPULATE_READ : 661.642 ms file : POPULATE_WRITE : 1106.816 ms file : FALLOCATE : 1.864 ms file : FALLOCATE+Read : 656.328 ms file : FALLOCATE+Write : 1153.300 ms file : FALLOCATE+Read/Write : 1180.613 ms file : FALLOCATE+POPULATE_READ : 668.347 ms file : FALLOCATE+POPULATE_WRITE : 996.143 ms hugetlbfs : Read : 357.245 ms hugetlbfs : Write : 357.413 ms hugetlbfs : Read/Write : 357.120 ms hugetlbfs : POPULATE_READ : 356.321 ms hugetlbfs : POPULATE_WRITE : 356.693 ms hugetlbfs : FALLOCATE : 355.927 ms hugetlbfs : FALLOCATE+Read : 357.074 ms hugetlbfs : FALLOCATE+Write : 357.120 ms hugetlbfs : FALLOCATE+Read/Write : 356.983 ms hugetlbfs : FALLOCATE+POPULATE_READ : 356.413 ms hugetlbfs : FALLOCATE+POPULATE_WRITE : 356.266 ms ************************************************** [1] https://lkml.org/lkml/2013/6/27/698 [akpm@linux-foundation.org: coding style fixes] Link: https://lkml.kernel.org/r/20210419135443.12822-3-david@redhat.com Signed-off-by: David Hildenbrand Cc: Arnd Bergmann Cc: Michal Hocko Cc: Oscar Salvador Cc: Matthew Wilcox (Oracle) Cc: Andrea Arcangeli Cc: Minchan Kim Cc: Jann Horn Cc: Jason Gunthorpe Cc: Dave Hansen Cc: Hugh Dickins Cc: Rik van Riel Cc: Michael S. Tsirkin Cc: Kirill A. Shutemov Cc: Vlastimil Babka Cc: Richard Henderson Cc: Ivan Kokshaysky Cc: Matt Turner Cc: Thomas Bogendoerfer Cc: "James E.J. Bottomley" Cc: Helge Deller Cc: Chris Zankel Cc: Max Filippov Cc: Mike Kravetz Cc: Peter Xu Cc: Rolf Eike Beer Cc: Ram Pai Cc: Shuah Khan Signed-off-by: Andrew Morton --- arch/alpha/include/uapi/asm/mman.h | 3 + arch/mips/include/uapi/asm/mman.h | 3 + arch/parisc/include/uapi/asm/mman.h | 3 + arch/xtensa/include/uapi/asm/mman.h | 3 + include/uapi/asm-generic/mman-common.h | 3 + mm/gup.c | 58 ++++++++++++++++++++ mm/internal.h | 3 + mm/madvise.c | 66 +++++++++++++++++++++++ 8 files changed, 142 insertions(+) --- a/arch/alpha/include/uapi/asm/mman.h~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/arch/alpha/include/uapi/asm/mman.h @@ -71,6 +71,9 @@ #define MADV_COLD 20 /* deactivate these pages */ #define MADV_PAGEOUT 21 /* reclaim these pages */ +#define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ +#define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ + /* compatibility flags */ #define MAP_FILE 0 --- a/arch/mips/include/uapi/asm/mman.h~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/arch/mips/include/uapi/asm/mman.h @@ -98,6 +98,9 @@ #define MADV_COLD 20 /* deactivate these pages */ #define MADV_PAGEOUT 21 /* reclaim these pages */ +#define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ +#define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ + /* compatibility flags */ #define MAP_FILE 0 --- a/arch/parisc/include/uapi/asm/mman.h~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/arch/parisc/include/uapi/asm/mman.h @@ -52,6 +52,9 @@ #define MADV_COLD 20 /* deactivate these pages */ #define MADV_PAGEOUT 21 /* reclaim these pages */ +#define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ +#define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ + #define MADV_MERGEABLE 65 /* KSM may merge identical pages */ #define MADV_UNMERGEABLE 66 /* KSM may not merge identical pages */ --- a/arch/xtensa/include/uapi/asm/mman.h~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/arch/xtensa/include/uapi/asm/mman.h @@ -106,6 +106,9 @@ #define MADV_COLD 20 /* deactivate these pages */ #define MADV_PAGEOUT 21 /* reclaim these pages */ +#define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ +#define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ + /* compatibility flags */ #define MAP_FILE 0 --- a/include/uapi/asm-generic/mman-common.h~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/include/uapi/asm-generic/mman-common.h @@ -72,6 +72,9 @@ #define MADV_COLD 20 /* deactivate these pages */ #define MADV_PAGEOUT 21 /* reclaim these pages */ +#define MADV_POPULATE_READ 22 /* populate (prefault) page tables readable */ +#define MADV_POPULATE_WRITE 23 /* populate (prefault) page tables writable */ + /* compatibility flags */ #define MAP_FILE 0 --- a/mm/gup.c~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/mm/gup.c @@ -1501,6 +1501,64 @@ long populate_vma_page_range(struct vm_a } /* + * faultin_vma_page_range() - populate (prefault) page tables inside the + * given VMA range readable/writable + * + * This takes care of mlocking the pages, too, if VM_LOCKED is set. + * + * @vma: target vma + * @start: start address + * @end: end address + * @write: whether to prefault readable or writable + * @locked: whether the mmap_lock is still held + * + * Returns either number of processed pages in the vma, or a negative error + * code on error (see __get_user_pages()). + * + * vma->vm_mm->mmap_lock must be held. The range must be page-aligned and + * covered by the VMA. + * + * If @locked is NULL, it may be held for read or write and will be unperturbed. + * + * If @locked is non-NULL, it must held for read only and may be released. If + * it's released, *@locked will be set to 0. + */ +long faultin_vma_page_range(struct vm_area_struct *vma, unsigned long start, + unsigned long end, bool write, int *locked) +{ + struct mm_struct *mm = vma->vm_mm; + unsigned long nr_pages = (end - start) / PAGE_SIZE; + int gup_flags; + + VM_BUG_ON(!PAGE_ALIGNED(start)); + VM_BUG_ON(!PAGE_ALIGNED(end)); + VM_BUG_ON_VMA(start < vma->vm_start, vma); + VM_BUG_ON_VMA(end > vma->vm_end, vma); + mmap_assert_locked(mm); + + /* + * FOLL_TOUCH: Mark page accessed and thereby young; will also mark + * the page dirty with FOLL_WRITE -- which doesn't make a + * difference with !FOLL_FORCE, because the page is writable + * in the page table. + * FOLL_HWPOISON: Return -EHWPOISON instead of -EFAULT when we hit + * a poisoned page. + * FOLL_POPULATE: Always populate memory with VM_LOCKONFAULT. + * !FOLL_FORCE: Require proper access permissions. + */ + gup_flags = FOLL_TOUCH | FOLL_POPULATE | FOLL_MLOCK | FOLL_HWPOISON; + if (write) + gup_flags |= FOLL_WRITE; + + /* + * See check_vma_flags(): Will return -EFAULT on incompatible mappings + * or with insufficient permissions. + */ + return __get_user_pages(mm, start, nr_pages, gup_flags, + NULL, NULL, locked); +} + +/* * __mm_populate - populate and/or mlock pages within a range of address space. * * This is used to implement mlock() and the MAP_POPULATE / MAP_LOCKED mmap --- a/mm/internal.h~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/mm/internal.h @@ -345,6 +345,9 @@ void __vma_unlink_list(struct mm_struct #ifdef CONFIG_MMU extern long populate_vma_page_range(struct vm_area_struct *vma, unsigned long start, unsigned long end, int *locked); +extern long faultin_vma_page_range(struct vm_area_struct *vma, + unsigned long start, unsigned long end, + bool write, int *locked); extern void munlock_vma_pages_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); static inline void munlock_vma_pages_all(struct vm_area_struct *vma) --- a/mm/madvise.c~mm-madvise-introduce-madv_populate_readwrite-to-prefault-page-tables +++ a/mm/madvise.c @@ -53,6 +53,8 @@ static int madvise_need_mmap_write(int b case MADV_COLD: case MADV_PAGEOUT: case MADV_FREE: + case MADV_POPULATE_READ: + case MADV_POPULATE_WRITE: return 0; default: /* be safe, default to 1. list exceptions explicitly */ @@ -822,6 +824,61 @@ static long madvise_dontneed_free(struct return -EINVAL; } +static long madvise_populate(struct vm_area_struct *vma, + struct vm_area_struct **prev, + unsigned long start, unsigned long end, + int behavior) +{ + const bool write = behavior == MADV_POPULATE_WRITE; + struct mm_struct *mm = vma->vm_mm; + unsigned long tmp_end; + int locked = 1; + long pages; + + *prev = vma; + + while (start < end) { + /* + * We might have temporarily dropped the lock. For example, + * our VMA might have been split. + */ + if (!vma || start >= vma->vm_end) { + vma = find_vma(mm, start); + if (!vma || start < vma->vm_start) + return -ENOMEM; + } + + tmp_end = min_t(unsigned long, end, vma->vm_end); + /* Populate (prefault) page tables readable/writable. */ + pages = faultin_vma_page_range(vma, start, tmp_end, write, + &locked); + if (!locked) { + mmap_read_lock(mm); + locked = 1; + *prev = NULL; + vma = NULL; + } + if (pages < 0) { + switch (pages) { + case -EINTR: + return -EINTR; + case -EFAULT: /* Incompatible mappings / permissions. */ + return -EINVAL; + case -EHWPOISON: + return -EHWPOISON; + default: + pr_warn_once("%s: unhandled return value: %ld\n", + __func__, pages); + fallthrough; + case -ENOMEM: + return -ENOMEM; + } + } + start += pages * PAGE_SIZE; + } + return 0; +} + /* * Application wants to free up the pages and associated backing store. * This is effectively punching a hole into the middle of a file. @@ -935,6 +992,9 @@ madvise_vma(struct vm_area_struct *vma, case MADV_FREE: case MADV_DONTNEED: return madvise_dontneed_free(vma, prev, start, end, behavior); + case MADV_POPULATE_READ: + case MADV_POPULATE_WRITE: + return madvise_populate(vma, prev, start, end, behavior); default: return madvise_behavior(vma, prev, start, end, behavior); } @@ -955,6 +1015,8 @@ madvise_behavior_valid(int behavior) case MADV_FREE: case MADV_COLD: case MADV_PAGEOUT: + case MADV_POPULATE_READ: + case MADV_POPULATE_WRITE: #ifdef CONFIG_KSM case MADV_MERGEABLE: case MADV_UNMERGEABLE: @@ -1042,6 +1104,10 @@ process_madvise_behavior_valid(int behav * easily if memory pressure happens. * MADV_PAGEOUT - the application is not expected to use this memory soon, * page out the pages in this range immediately. + * MADV_POPULATE_READ - populate (prefault) page tables readable by + * triggering read faults if required + * MADV_POPULATE_WRITE - populate (prefault) page tables writable by + * triggering write faults if required * * return values: * zero - success From patchwork Thu Jul 1 01:52:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C58DAC11F69 for ; Thu, 1 Jul 2021 01:52:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 75FC46146D for ; Thu, 1 Jul 2021 01:52:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75FC46146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 023FC8D0237; Wed, 30 Jun 2021 21:52:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3CB58D0236; Wed, 30 Jun 2021 21:52:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E04668D0237; Wed, 30 Jun 2021 21:52:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id BDA1B8D0236 for ; Wed, 30 Jun 2021 21:52:34 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 88CD4250AF for ; Thu, 1 Jul 2021 01:52:34 +0000 (UTC) X-FDA: 78312344628.38.695E2FA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 438C6B00009D for ; Thu, 1 Jul 2021 01:52:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A087661469; Thu, 1 Jul 2021 01:52:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104353; bh=ZNxVERlcr2LCNKpnWm4DVoTM9z6dFw/JmTX8WyB3LAI=; h=Date:From:To:Subject:In-Reply-To:From; b=RqO58282e0VwBmCbqjery53x40u3qCsh5EJFio7SQp0g6rkTxE013HSuRkrtaof+B CEgMdKFBvZDnCHsIfAXeJ8YEqpAeIz7UyRFiZt+buU/fu5aiI7PxzdwKVh533COk9L fpwN7Qpf8lnwH3O6BqqVpVGLom5OdoAAESlDcmTM= Date: Wed, 30 Jun 2021 18:52:32 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, chris@zankel.net, dave.hansen@intel.com, david@redhat.com, deller@gmx.de, eike-kernel@sf-tec.de, hughd@google.com, ink@jurassic.park.msu.ru, James.Bottomley@HansenPartnership.com, jannh@google.com, jcmvbkbc@gmail.com, jgg@ziepe.ca, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, linuxram@us.ibm.com, mattst88@gmail.com, mhocko@suse.com, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, mst@redhat.com, osalvador@suse.de, peterx@redhat.com, riel@surriel.com, rppt@linux.ibm.com, rth@twiddle.net, shuah@kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vbabka@suse.cz, willy@infradead.org Subject: [patch 100/192] MAINTAINERS: add tools/testing/selftests/vm/ to MEMORY MANAGEMENT Message-ID: <20210701015232.9MuCJeYCJ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=RqO58282; dmarc=none; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: 57cfyzzfmosh4i77mdsoxystipupc4pb X-Rspamd-Queue-Id: 438C6B00009D X-HE-Tag: 1625104354-691270 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: MAINTAINERS: add tools/testing/selftests/vm/ to MEMORY MANAGEMENT MEMORY MANAGEMENT seems to be a good fit. Link: https://lkml.kernel.org/r/20210419135443.12822-4-david@redhat.com Signed-off-by: David Hildenbrand Acked-by: Mike Rapoport Cc: Michal Hocko Cc: Oscar Salvador Cc: Jason Gunthorpe Cc: Peter Xu Cc: Shuah Khan Cc: Andrea Arcangeli Cc: Arnd Bergmann Cc: Chris Zankel Cc: Dave Hansen Cc: Helge Deller Cc: Hugh Dickins Cc: Ivan Kokshaysky Cc: "James E.J. Bottomley" Cc: Jann Horn Cc: Kirill A. Shutemov Cc: Matthew Wilcox (Oracle) Cc: Matt Turner Cc: Max Filippov Cc: Michael S. Tsirkin Cc: Mike Kravetz Cc: Minchan Kim Cc: Ram Pai Cc: Richard Henderson Cc: Rik van Riel Cc: Rolf Eike Beer Cc: Thomas Bogendoerfer Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) --- a/MAINTAINERS~maintainers-add-tools-testing-selftests-vm-to-memory-management +++ a/MAINTAINERS @@ -11828,6 +11828,7 @@ F: include/linux/mmzone.h F: include/linux/pagewalk.h F: include/linux/vmalloc.h F: mm/ +F: tools/testing/selftests/vm/ MEMORY TECHNOLOGY DEVICES (MTD) M: Miquel Raynal From patchwork Thu Jul 1 01:52:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE686C11F69 for ; Thu, 1 Jul 2021 01:52:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 65EA561476 for ; Thu, 1 Jul 2021 01:52:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 65EA561476 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E2DD28D0238; Wed, 30 Jun 2021 21:52:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E04658D0236; Wed, 30 Jun 2021 21:52:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCC828D0238; Wed, 30 Jun 2021 21:52:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id AA6968D0236 for ; Wed, 30 Jun 2021 21:52:38 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 842E718035454 for ; Thu, 1 Jul 2021 01:52:38 +0000 (UTC) X-FDA: 78312344796.11.BA8A822 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 3C2539000328 for ; Thu, 1 Jul 2021 01:52:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9185F6146D; Thu, 1 Jul 2021 01:52:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104357; bh=RTUhqYV1WD2G2rFObE1mtcRv7J3g9N/bdf5NQpakank=; h=Date:From:To:Subject:In-Reply-To:From; b=AvqPlZEFbb0BVSsEiI2WGe4/gAv+3ayBXklaBG1ODTmDvPZ1wvNrfmxtVw6ejVqZM 4yNTvWphuAZtJldd1TrSxgzU3OzmC9SZncvNXk5sidYne7+8PzAIrbgMIrLGf78I3p 4D/O1b/Hxe4RhBWjuah1g/zE9yaWgbuA9fo/qClY= Date: Wed, 30 Jun 2021 18:52:36 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, chris@zankel.net, dave.hansen@intel.com, david@redhat.com, deller@gmx.de, eike-kernel@sf-tec.de, hughd@google.com, ink@jurassic.park.msu.ru, James.Bottomley@HansenPartnership.com, jannh@google.com, jcmvbkbc@gmail.com, jgg@ziepe.ca, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, linuxram@us.ibm.com, mattst88@gmail.com, mhocko@suse.com, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, mst@redhat.com, osalvador@suse.de, peterx@redhat.com, riel@surriel.com, rth@twiddle.net, shuah@kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vbabka@suse.cz, willy@infradead.org Subject: [patch 101/192] selftests/vm: add protection_keys_32 / protection_keys_64 to gitignore Message-ID: <20210701015236.2NNlPctlq%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3C2539000328 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=AvqPlZEF; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: b4tqc6uay5uz3e73rr1bq61m5yc7rfod X-HE-Tag: 1625104358-131396 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: selftests/vm: add protection_keys_32 / protection_keys_64 to gitignore We missed adding two binaries to gitignore. Link: https://lkml.kernel.org/r/20210419135443.12822-5-david@redhat.com Signed-off-by: David Hildenbrand Cc: Michal Hocko Cc: Oscar Salvador Cc: Jason Gunthorpe Cc: Peter Xu Cc: Ram Pai Cc: Shuah Khan Cc: Andrea Arcangeli Cc: Arnd Bergmann Cc: Chris Zankel Cc: Dave Hansen Cc: Helge Deller Cc: Hugh Dickins Cc: Ivan Kokshaysky Cc: "James E.J. Bottomley" Cc: Jann Horn Cc: Kirill A. Shutemov Cc: Matthew Wilcox (Oracle) Cc: Matt Turner Cc: Max Filippov Cc: Michael S. Tsirkin Cc: Mike Kravetz Cc: Minchan Kim Cc: Richard Henderson Cc: Rik van Riel Cc: Rolf Eike Beer Cc: Thomas Bogendoerfer Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/.gitignore | 2 ++ 1 file changed, 2 insertions(+) --- a/tools/testing/selftests/vm/.gitignore~selftests-vm-add-protection_keys_32-protection_keys_64-to-gitignore +++ a/tools/testing/selftests/vm/.gitignore @@ -12,6 +12,8 @@ mremap_test on-fault-limit transhuge-stress protection_keys +protection_keys_32 +protection_keys_64 userfaultfd mlock-intersect-test mlock-random-test From patchwork Thu Jul 1 01:52:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FC02C11F66 for ; Thu, 1 Jul 2021 01:52:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2073F61241 for ; Thu, 1 Jul 2021 01:52:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2073F61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 999B38D0239; Wed, 30 Jun 2021 21:52:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9702D8D0236; Wed, 30 Jun 2021 21:52:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7EA448D0239; Wed, 30 Jun 2021 21:52:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0105.hostedemail.com [216.40.44.105]) by kanga.kvack.org (Postfix) with ESMTP id 5C28C8D0236 for ; Wed, 30 Jun 2021 21:52:42 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 35BED1801AB45 for ; Thu, 1 Jul 2021 01:52:42 +0000 (UTC) X-FDA: 78312344964.09.C0CDD77 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id CDAB9D00014A for ; Thu, 1 Jul 2021 01:52:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3C6C061474; Thu, 1 Jul 2021 01:52:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104361; bh=c/Y1mJ1bDy9XB7FRR3tGdy+svrQVGzmuLOwn5i6wdLk=; h=Date:From:To:Subject:In-Reply-To:From; b=TrLWobpCT9DcY4rI8C6q5R6kDM1aN1WDE+RSwp3xcfacatuBKd3bIFNV6h5cAxz4o wt4sUZVgFAbQ1zWDph3SYRZf83tWPsdAYGquBODGTir2GExTogjBqRD04pml2Dh/Fm KuQ8cpS/Jo0h/tOcvD22ec0Y7uvOeiI3Q5g+a1s8= Date: Wed, 30 Jun 2021 18:52:39 -0700 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, arnd@arndb.de, chris@zankel.net, dave.hansen@intel.com, david@redhat.com, deller@gmx.de, eike-kernel@sf-tec.de, hughd@google.com, ink@jurassic.park.msu.ru, James.Bottomley@HansenPartnership.com, jannh@google.com, jcmvbkbc@gmail.com, jgg@ziepe.ca, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, linuxram@us.ibm.com, mattst88@gmail.com, mhocko@suse.com, mike.kravetz@oracle.com, minchan@kernel.org, mm-commits@vger.kernel.org, mst@redhat.com, osalvador@suse.de, peterx@redhat.com, riel@surriel.com, rth@twiddle.net, shuah@kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vbabka@suse.cz, willy@infradead.org Subject: [patch 102/192] selftests/vm: add test for MADV_POPULATE_(READ|WRITE) Message-ID: <20210701015239.xq-bk-BfO%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=TrLWobpC; dmarc=none; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: 4hi759zn3cyofaueeo6zf6jxeuxxtbox X-Rspamd-Queue-Id: CDAB9D00014A X-HE-Tag: 1625104361-827255 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: selftests/vm: add test for MADV_POPULATE_(READ|WRITE) Let's add a simple test for MADV_POPULATE_READ and MADV_POPULATE_WRITE, verifying some error handling, that population works, and that softdirty tracking works as expected. For now, limit the test to private anonymous memory. Link: https://lkml.kernel.org/r/20210419135443.12822-6-david@redhat.com Signed-off-by: David Hildenbrand Cc: Arnd Bergmann Cc: Michal Hocko Cc: Oscar Salvador Cc: Matthew Wilcox (Oracle) Cc: Andrea Arcangeli Cc: Minchan Kim Cc: Jann Horn Cc: Jason Gunthorpe Cc: Dave Hansen Cc: Hugh Dickins Cc: Rik van Riel Cc: Michael S. Tsirkin Cc: Kirill A. Shutemov Cc: Vlastimil Babka Cc: Richard Henderson Cc: Ivan Kokshaysky Cc: Matt Turner Cc: Thomas Bogendoerfer Cc: "James E.J. Bottomley" Cc: Helge Deller Cc: Chris Zankel Cc: Max Filippov Cc: Mike Kravetz Cc: Peter Xu Cc: Rolf Eike Beer Cc: Shuah Khan Cc: Ram Pai Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/.gitignore | 1 tools/testing/selftests/vm/Makefile | 1 tools/testing/selftests/vm/madv_populate.c | 342 +++++++++++++++++++ tools/testing/selftests/vm/run_vmtests.sh | 16 4 files changed, 360 insertions(+) --- a/tools/testing/selftests/vm/.gitignore~selftests-vm-add-test-for-madv_populate_readwrite +++ a/tools/testing/selftests/vm/.gitignore @@ -14,6 +14,7 @@ transhuge-stress protection_keys protection_keys_32 protection_keys_64 +madv_populate userfaultfd mlock-intersect-test mlock-random-test --- /dev/null +++ a/tools/testing/selftests/vm/madv_populate.c @@ -0,0 +1,342 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * MADV_POPULATE_READ and MADV_POPULATE_WRITE tests + * + * Copyright 2021, Red Hat, Inc. + * + * Author(s): David Hildenbrand + */ +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include + +#include "../kselftest.h" + +#if defined(MADV_POPULATE_READ) && defined(MADV_POPULATE_WRITE) + +/* + * For now, we're using 2 MiB of private anonymous memory for all tests. + */ +#define SIZE (2 * 1024 * 1024) + +static size_t pagesize; + +static uint64_t pagemap_get_entry(int fd, char *start) +{ + const unsigned long pfn = (unsigned long)start / pagesize; + uint64_t entry; + int ret; + + ret = pread(fd, &entry, sizeof(entry), pfn * sizeof(entry)); + if (ret != sizeof(entry)) + ksft_exit_fail_msg("reading pagemap failed\n"); + return entry; +} + +static bool pagemap_is_populated(int fd, char *start) +{ + uint64_t entry = pagemap_get_entry(fd, start); + + /* Present or swapped. */ + return entry & 0xc000000000000000ull; +} + +static bool pagemap_is_softdirty(int fd, char *start) +{ + uint64_t entry = pagemap_get_entry(fd, start); + + return entry & 0x0080000000000000ull; +} + +static void sense_support(void) +{ + char *addr; + int ret; + + addr = mmap(0, pagesize, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (!addr) + ksft_exit_fail_msg("mmap failed\n"); + + ret = madvise(addr, pagesize, MADV_POPULATE_READ); + if (ret) + ksft_exit_skip("MADV_POPULATE_READ is not available\n"); + + ret = madvise(addr, pagesize, MADV_POPULATE_WRITE); + if (ret) + ksft_exit_skip("MADV_POPULATE_WRITE is not available\n"); + + munmap(addr, pagesize); +} + +static void test_prot_read(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(!ret, "MADV_POPULATE_READ with PROT_READ\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == EINVAL, + "MADV_POPULATE_WRITE with PROT_READ\n"); + + munmap(addr, SIZE); +} + +static void test_prot_write(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == EINVAL, + "MADV_POPULATE_READ with PROT_WRITE\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(!ret, "MADV_POPULATE_WRITE with PROT_WRITE\n"); + + munmap(addr, SIZE); +} + +static void test_holes(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + ret = munmap(addr + pagesize, pagesize); + if (ret) + ksft_exit_fail_msg("munmap failed\n"); + + /* Hole in the middle */ + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_READ with holes in the middle\n"); + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_WRITE with holes in the middle\n"); + + /* Hole at end */ + ret = madvise(addr, 2 * pagesize, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_READ with holes at the end\n"); + ret = madvise(addr, 2 * pagesize, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_WRITE with holes at the end\n"); + + /* Hole at beginning */ + ret = madvise(addr + pagesize, pagesize, MADV_POPULATE_READ); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_READ with holes at the beginning\n"); + ret = madvise(addr + pagesize, pagesize, MADV_POPULATE_WRITE); + ksft_test_result(ret == -1 && errno == ENOMEM, + "MADV_POPULATE_WRITE with holes at the beginning\n"); + + munmap(addr, SIZE); +} + +static bool range_is_populated(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (!pagemap_is_populated(fd, start)) + ret = false; + close(fd); + return ret; +} + +static bool range_is_not_populated(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (pagemap_is_populated(fd, start)) + ret = false; + close(fd); + return ret; +} + +static void test_populate_read(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + ksft_test_result(range_is_not_populated(addr, SIZE), + "range initially not populated\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(!ret, "MADV_POPULATE_READ\n"); + ksft_test_result(range_is_populated(addr, SIZE), + "range is populated\n"); + + munmap(addr, SIZE); +} + +static void test_populate_write(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + ksft_test_result(range_is_not_populated(addr, SIZE), + "range initially not populated\n"); + + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(!ret, "MADV_POPULATE_WRITE\n"); + ksft_test_result(range_is_populated(addr, SIZE), + "range is populated\n"); + + munmap(addr, SIZE); +} + +static bool range_is_softdirty(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (!pagemap_is_softdirty(fd, start)) + ret = false; + close(fd); + return ret; +} + +static bool range_is_not_softdirty(char *start, ssize_t size) +{ + int fd = open("/proc/self/pagemap", O_RDONLY); + bool ret = true; + + if (fd < 0) + ksft_exit_fail_msg("opening pagemap failed\n"); + for (; size > 0 && ret; size -= pagesize, start += pagesize) + if (pagemap_is_softdirty(fd, start)) + ret = false; + close(fd); + return ret; +} + +static void clear_softdirty(void) +{ + int fd = open("/proc/self/clear_refs", O_WRONLY); + const char *ctrl = "4"; + int ret; + + if (fd < 0) + ksft_exit_fail_msg("opening clear_refs failed\n"); + ret = write(fd, ctrl, strlen(ctrl)); + if (ret != strlen(ctrl)) + ksft_exit_fail_msg("writing clear_refs failed\n"); + close(fd); +} + +static void test_softdirty(void) +{ + char *addr; + int ret; + + ksft_print_msg("[RUN] %s\n", __func__); + + addr = mmap(0, SIZE, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, 0, 0); + if (addr == MAP_FAILED) + ksft_exit_fail_msg("mmap failed\n"); + + /* Clear any softdirty bits. */ + clear_softdirty(); + ksft_test_result(range_is_not_softdirty(addr, SIZE), + "range is not softdirty\n"); + + /* Populating READ should set softdirty. */ + ret = madvise(addr, SIZE, MADV_POPULATE_READ); + ksft_test_result(!ret, "MADV_POPULATE_READ\n"); + ksft_test_result(range_is_not_softdirty(addr, SIZE), + "range is not softdirty\n"); + + /* Populating WRITE should set softdirty. */ + ret = madvise(addr, SIZE, MADV_POPULATE_WRITE); + ksft_test_result(!ret, "MADV_POPULATE_WRITE\n"); + ksft_test_result(range_is_softdirty(addr, SIZE), + "range is softdirty\n"); + + munmap(addr, SIZE); +} + +int main(int argc, char **argv) +{ + int err; + + pagesize = getpagesize(); + + ksft_print_header(); + ksft_set_plan(21); + + sense_support(); + test_prot_read(); + test_prot_write(); + test_holes(); + test_populate_read(); + test_populate_write(); + test_softdirty(); + + err = ksft_get_fail_cnt(); + if (err) + ksft_exit_fail_msg("%d out of %d tests failed\n", + err, ksft_test_num()); + return ksft_exit_pass(); +} + +#else /* defined(MADV_POPULATE_READ) && defined(MADV_POPULATE_WRITE) */ + +#warning "missing MADV_POPULATE_READ or MADV_POPULATE_WRITE definition" + +int main(int argc, char **argv) +{ + ksft_print_header(); + ksft_exit_skip("MADV_POPULATE_READ or MADV_POPULATE_WRITE not defined\n"); +} + +#endif /* defined(MADV_POPULATE_READ) && defined(MADV_POPULATE_WRITE) */ --- a/tools/testing/selftests/vm/Makefile~selftests-vm-add-test-for-madv_populate_readwrite +++ a/tools/testing/selftests/vm/Makefile @@ -31,6 +31,7 @@ TEST_GEN_FILES += hmm-tests TEST_GEN_FILES += hugepage-mmap TEST_GEN_FILES += hugepage-shm TEST_GEN_FILES += khugepaged +TEST_GEN_FILES += madv_populate TEST_GEN_FILES += map_fixed_noreplace TEST_GEN_FILES += map_hugetlb TEST_GEN_FILES += map_populate --- a/tools/testing/selftests/vm/run_vmtests.sh~selftests-vm-add-test-for-madv_populate_readwrite +++ a/tools/testing/selftests/vm/run_vmtests.sh @@ -346,4 +346,20 @@ else exitcode=1 fi +echo "--------------------------------------------------------" +echo "running MADV_POPULATE_READ and MADV_POPULATE_WRITE tests" +echo "--------------------------------------------------------" +./madv_populate +ret_val=$? + +if [ $ret_val -eq 0 ]; then + echo "[PASS]" +elif [ $ret_val -eq $ksft_skip ]; then + echo "[SKIP]" + exitcode=$ksft_skip +else + echo "[FAIL]" + exitcode=1 +fi + exit $exitcode From patchwork Thu Jul 1 01:52:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D8CEC11F69 for ; Thu, 1 Jul 2021 01:52:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E0C9561476 for ; Thu, 1 Jul 2021 01:52:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0C9561476 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 665968D023A; Wed, 30 Jun 2021 21:52:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 615748D0236; Wed, 30 Jun 2021 21:52:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 504718D023A; Wed, 30 Jun 2021 21:52:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id 287778D0236 for ; Wed, 30 Jun 2021 21:52:45 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 03FA31801AB45 for ; Thu, 1 Jul 2021 01:52:45 +0000 (UTC) X-FDA: 78312345090.23.1A30AA4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id BA57F6001AA8 for ; Thu, 1 Jul 2021 01:52:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C06CD61241; Thu, 1 Jul 2021 01:52:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104364; bh=TFh44qdMZt9Vja3LMR2t+j58k4ehpx3rEYrJZb1pg1M=; h=Date:From:To:Subject:In-Reply-To:From; b=WW78ydZWlDXTQkA2PKxV4ffAyp3EMMsB3nX2jtYQjtzlaTqSfvolprwjmftNFIiHu PLuT6nkdDjhpNfybLT5GyP9xcBJ/5N8SBV3gY36CrLfpGhnou1BYvzHUVPlc2onqu8 qugndc9nJOHwgcNc9C6U6HdNW8E0o/QpgEBadVmY= Date: Wed, 30 Jun 2021 18:52:43 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, georgi.djakov@linaro.org, linux-mm@kvack.org, lmark@codeaurora.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 103/192] mm/memory_hotplug: rate limit page migration warnings Message-ID: <20210701015243.6hfM4X7tH%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: BA57F6001AA8 X-Stat-Signature: 5jfwprn5gg5eobtzh8baboiaxbjtubqb Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=WW78ydZW; dmarc=none; spf=pass (imf10.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104364-235637 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Liam Mark Subject: mm/memory_hotplug: rate limit page migration warnings When offlining memory the system can attempt to migrate a lot of pages, if there are problems with migration this can flood the logs. Printing all the data hogs the CPU and cause some RT threads to run for a long time, which may have some bad consequences. Rate limit the page migration warnings in order to avoid this. Link: https://lkml.kernel.org/r/20210505140542.24935-1-georgi.djakov@linaro.org Signed-off-by: Liam Mark Signed-off-by: Georgi Djakov Cc: David Hildenbrand Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) --- a/mm/memory_hotplug.c~mm-memory_hotplug-rate-limit-page-migration-warnings +++ a/mm/memory_hotplug.c @@ -1406,6 +1406,8 @@ do_migrate_range(unsigned long start_pfn struct page *page, *head; int ret = 0; LIST_HEAD(source); + static DEFINE_RATELIMIT_STATE(migrate_rs, DEFAULT_RATELIMIT_INTERVAL, + DEFAULT_RATELIMIT_BURST); for (pfn = start_pfn; pfn < end_pfn; pfn++) { if (!pfn_valid(pfn)) @@ -1452,8 +1454,10 @@ do_migrate_range(unsigned long start_pfn page_is_file_lru(page)); } else { - pr_warn("failed to isolate pfn %lx\n", pfn); - dump_page(page, "isolation failed"); + if (__ratelimit(&migrate_rs)) { + pr_warn("failed to isolate pfn %lx\n", pfn); + dump_page(page, "isolation failed"); + } } put_page(page); } @@ -1482,9 +1486,11 @@ do_migrate_range(unsigned long start_pfn (unsigned long)&mtc, MIGRATE_SYNC, MR_MEMORY_HOTPLUG); if (ret) { list_for_each_entry(page, &source, lru) { - pr_warn("migrating pfn %lx failed ret:%d ", - page_to_pfn(page), ret); - dump_page(page, "migration failure"); + if (__ratelimit(&migrate_rs)) { + pr_warn("migrating pfn %lx failed ret:%d\n", + page_to_pfn(page), ret); + dump_page(page, "migration failure"); + } } putback_movable_pages(&source); } From patchwork Thu Jul 1 01:52:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45E8AC11F68 for ; Thu, 1 Jul 2021 01:52:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F023B6147D for ; Thu, 1 Jul 2021 01:52:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F023B6147D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 79D488D023B; Wed, 30 Jun 2021 21:52:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7744E8D0236; Wed, 30 Jun 2021 21:52:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63C918D023B; Wed, 30 Jun 2021 21:52:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 4188E8D0236 for ; Wed, 30 Jun 2021 21:52:48 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0CF0B181AEF3F for ; Thu, 1 Jul 2021 01:52:48 +0000 (UTC) X-FDA: 78312345216.29.F7C0CF2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id B17837001A08 for ; Thu, 1 Jul 2021 01:52:47 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B187D61241; Thu, 1 Jul 2021 01:52:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104367; bh=mjkNQm7rxwA+vLTC+sk6AtUNMVSW2ID/DtgskL/YYF0=; h=Date:From:To:Subject:In-Reply-To:From; b=kDoHjj2jeM2BO36go2AUAiEKc3DOOR1Fc4O1i67XQv1oU9hNJ/3JZOp2T+lTQqeal z177Bhssar169fD504rKZxo271idGSX8e1KDWVtF1Io5cygz6n2KF90YL8TLVtGDCv h1NfGCcHbeXyZcWtM6zmjViLT+b/Qipc2GLwByZ0= Date: Wed, 30 Jun 2021 18:52:46 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, david@redhat.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, osalvador@suse.de, pasha.tatashin@soleen.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 104/192] mm,memory_hotplug: drop unneeded locking Message-ID: <20210701015246.RMwpXg9uK%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=kDoHjj2j; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B17837001A08 X-Stat-Signature: 7781o98ww7mwirmxkbf78797ze3ro73g X-HE-Tag: 1625104367-530485 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Oscar Salvador Subject: mm,memory_hotplug: drop unneeded locking Currently, memory-hotplug code takes zone's span_writelock and pgdat's resize_lock when resizing the node/zone's spanned pages via {move_pfn_range_to_zone(),remove_pfn_range_from_zone()} and when resizing node and zone's present pages via adjust_present_page_count(). These locks are also taken during the initialization of the system at boot time, where it protects parallel struct page initialization, but they should not really be needed in memory-hotplug where all operations are a) synchronized on device level and b) serialized by the mem_hotplug_lock lock. [akpm@linux-foundation.org: remove now-unused locals] Link: https://lkml.kernel.org/r/20210531093958.15021-1-osalvador@suse.de Signed-off-by: Oscar Salvador Acked-by: David Hildenbrand Acked-by: Michal Hocko Cc: Anshuman Khandual Cc: Vlastimil Babka Cc: Pavel Tatashin Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-) --- a/mm/memory_hotplug.c~mmmemory_hotplug-drop-unneeded-locking +++ a/mm/memory_hotplug.c @@ -329,7 +329,6 @@ static void shrink_zone_span(struct zone unsigned long pfn; int nid = zone_to_nid(zone); - zone_span_writelock(zone); if (zone->zone_start_pfn == start_pfn) { /* * If the section is smallest section in the zone, it need @@ -362,7 +361,6 @@ static void shrink_zone_span(struct zone zone->spanned_pages = 0; } } - zone_span_writeunlock(zone); } static void update_pgdat_span(struct pglist_data *pgdat) @@ -399,7 +397,7 @@ void __ref remove_pfn_range_from_zone(st { const unsigned long end_pfn = start_pfn + nr_pages; struct pglist_data *pgdat = zone->zone_pgdat; - unsigned long pfn, cur_nr_pages, flags; + unsigned long pfn, cur_nr_pages; /* Poison struct pages because they are now uninitialized again. */ for (pfn = start_pfn; pfn < end_pfn; pfn += cur_nr_pages) { @@ -424,10 +422,8 @@ void __ref remove_pfn_range_from_zone(st clear_zone_contiguous(zone); - pgdat_resize_lock(zone->zone_pgdat, &flags); shrink_zone_span(zone, start_pfn, start_pfn + nr_pages); update_pgdat_span(pgdat); - pgdat_resize_unlock(zone->zone_pgdat, &flags); set_zone_contiguous(zone); } @@ -634,19 +630,13 @@ void __ref move_pfn_range_to_zone(struct { struct pglist_data *pgdat = zone->zone_pgdat; int nid = pgdat->node_id; - unsigned long flags; clear_zone_contiguous(zone); - /* TODO Huh pgdat is irqsave while zone is not. It used to be like that before */ - pgdat_resize_lock(pgdat, &flags); - zone_span_writelock(zone); if (zone_is_empty(zone)) init_currently_empty_zone(zone, start_pfn, nr_pages); resize_zone_range(zone, start_pfn, nr_pages); - zone_span_writeunlock(zone); resize_pgdat_range(pgdat, start_pfn, nr_pages); - pgdat_resize_unlock(pgdat, &flags); /* * Subsection population requires care in pfn_to_online_page(). @@ -736,12 +726,8 @@ struct zone *zone_for_pfn_range(int onli */ void adjust_present_page_count(struct zone *zone, long nr_pages) { - unsigned long flags; - zone->present_pages += nr_pages; - pgdat_resize_lock(zone->zone_pgdat, &flags); zone->zone_pgdat->node_present_pages += nr_pages; - pgdat_resize_unlock(zone->zone_pgdat, &flags); } int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages, From patchwork Thu Jul 1 01:52:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353267 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51C67C11F69 for ; Thu, 1 Jul 2021 01:52:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 08CE061476 for ; Thu, 1 Jul 2021 01:52:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 08CE061476 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 887CC8D023C; Wed, 30 Jun 2021 21:52:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 85E518D0236; Wed, 30 Jun 2021 21:52:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FFA68D023C; Wed, 30 Jun 2021 21:52:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0004.hostedemail.com [216.40.44.4]) by kanga.kvack.org (Postfix) with ESMTP id 4DBE78D0236 for ; Wed, 30 Jun 2021 21:52:51 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1F74C181AEF3F for ; Thu, 1 Jul 2021 01:52:51 +0000 (UTC) X-FDA: 78312345342.27.C519453 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id D38E1D0000A0 for ; Thu, 1 Jul 2021 01:52:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B182561477; Thu, 1 Jul 2021 01:52:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104370; bh=CuZGpym98MNTmM4CbOTAGVfaql3a/d9kPHOZYP3C3o0=; h=Date:From:To:Subject:In-Reply-To:From; b=OP6tSmkg3KzDXFLeBuYqCOAlb5j7JrFBPc/y/swjsr+eBb+iOPT3EfW/iIK4GCqpP H0ni+D/jqwHfVr0Ma9bhHJYtIT2m8jnERP3L5o5IZ42nG2GCeq6wFw/loTAxKx8+dm yYsQH0r4vW+gBW5FE2PmvxDkKM6U45A74Im8emuQ= Date: Wed, 30 Jun 2021 18:52:49 -0700 From: Andrew Morton To: akpm@linux-foundation.org, bigeasy@linutronix.de, colin.king@canonical.com, ddstreet@ieee.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, nathan@kernel.org, sjenning@redhat.com, tiantao6@hisilicon.com, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 105/192] mm/zswap.c: remove unused function zswap_debugfs_exit() Message-ID: <20210701015249.mnF3Kg_Tt%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D38E1D0000A0 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=OP6tSmkg; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 5qp8odjxynpeorzbmwj46cy7qjgz95zq X-HE-Tag: 1625104370-584474 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zswap.c: remove unused function zswap_debugfs_exit() Patch series "Cleanup and fixup for zswap". This series contains cleanups to remove unused function and avoid unnecessary copy-in at map time. Also this fixes two bugs in the function zswap_writeback_entry(). More details can be found in the respective changelogs. This patch (of 3): zswap_debugfs_exit() is unused, remove it. Link: https://lkml.kernel.org/r/20210522092242.3233191-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210522092242.3233191-2-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Seth Jennings Cc: Dan Streetman Cc: Vitaly Wool Cc: Sebastian Andrzej Siewior Cc: Nathan Chancellor Cc: Colin Ian King Cc: Tian Tao Signed-off-by: Andrew Morton --- mm/zswap.c | 7 ------- 1 file changed, 7 deletions(-) --- a/mm/zswap.c~mm-zswapc-remove-unused-function-zswap_debugfs_exit +++ a/mm/zswap.c @@ -1427,18 +1427,11 @@ static int __init zswap_debugfs_init(voi return 0; } - -static void __exit zswap_debugfs_exit(void) -{ - debugfs_remove_recursive(zswap_debugfs_root); -} #else static int __init zswap_debugfs_init(void) { return 0; } - -static void __exit zswap_debugfs_exit(void) { } #endif /********************************* From patchwork Thu Jul 1 01:52:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47C1DC11F68 for ; Thu, 1 Jul 2021 01:52:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F35406105A for ; Thu, 1 Jul 2021 01:52:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F35406105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 74F308D023D; Wed, 30 Jun 2021 21:52:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 726AA8D0236; Wed, 30 Jun 2021 21:52:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C82A8D023D; Wed, 30 Jun 2021 21:52:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id 3638D8D0236 for ; Wed, 30 Jun 2021 21:52:54 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0F0A9250B8 for ; Thu, 1 Jul 2021 01:52:54 +0000 (UTC) X-FDA: 78312345468.20.2ECC355 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id C64236001A82 for ; Thu, 1 Jul 2021 01:52:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C89736147D; Thu, 1 Jul 2021 01:52:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104373; bh=AuzkxrQVo8R8WrkG+pnqM9i67wekf2KlHU1uS/jlPIQ=; h=Date:From:To:Subject:In-Reply-To:From; b=ILKSKGVsyj+9F1ktVGspZpsLsZByqik1j2gsFNLJ7U36WhN6so7MioOUsb8Rl9fhF 4jDcx2oCqcaLn6Pii3r+2283ioLqTWbHbSOYHyyKvuugtZLSObWv24fkue9bXD2HtS hYZuMTN0FK1JbCaMnBovKWmM/7jIyP6uJe8065cA= Date: Wed, 30 Jun 2021 18:52:52 -0700 From: Andrew Morton To: akpm@linux-foundation.org, bigeasy@linutronix.de, colin.king@canonical.com, ddstreet@ieee.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, nathan@kernel.org, sjenning@redhat.com, tiantao6@hisilicon.com, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 106/192] mm/zswap.c: avoid unnecessary copy-in at map time Message-ID: <20210701015252.gEmynGXv0%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ILKSKGVs; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 4mc6dft4q7im96bzd9nko8bbbfkyc4if X-Rspamd-Queue-Id: C64236001A82 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104373-309930 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zswap.c: avoid unnecessary copy-in at map time The buf mapped via zpool_map_handle() is only used to store compressed page buffer and there is no information to extract from it. So we could use ZPOOL_MM_WO instead to avoid unnecessary copy-in at map time. Link: https://lkml.kernel.org/r/20210522092242.3233191-3-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Colin Ian King Cc: Dan Streetman Cc: Nathan Chancellor Cc: Sebastian Andrzej Siewior Cc: Seth Jennings Cc: Tian Tao Cc: Vitaly Wool Signed-off-by: Andrew Morton --- mm/zswap.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/zswap.c~mm-zswapc-avoid-unnecessary-copy-in-at-map-time +++ a/mm/zswap.c @@ -1203,7 +1203,7 @@ static int zswap_frontswap_store(unsigne zswap_reject_alloc_fail++; goto put_dstmem; } - buf = zpool_map_handle(entry->pool->zpool, handle, ZPOOL_MM_RW); + buf = zpool_map_handle(entry->pool->zpool, handle, ZPOOL_MM_WO); memcpy(buf, &zhdr, hlen); memcpy(buf + hlen, dst, dlen); zpool_unmap_handle(entry->pool->zpool, handle); From patchwork Thu Jul 1 01:52:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353271 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86429C11F68 for ; Thu, 1 Jul 2021 01:52:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3E78361468 for ; Thu, 1 Jul 2021 01:52:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E78361468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BE3748D023E; Wed, 30 Jun 2021 21:52:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BBA238D0236; Wed, 30 Jun 2021 21:52:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A82E28D023E; Wed, 30 Jun 2021 21:52:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 84EDC8D0236 for ; Wed, 30 Jun 2021 21:52:57 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 59C2282D35F4 for ; Thu, 1 Jul 2021 01:52:57 +0000 (UTC) X-FDA: 78312345594.30.6D998A2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 0A1561982 for ; Thu, 1 Jul 2021 01:52:56 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DF4186105A; Thu, 1 Jul 2021 01:52:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104376; bh=6MWdX56fY3D94DuzhXP11oAIC2y/aPv8BRRKi5jD3bA=; h=Date:From:To:Subject:In-Reply-To:From; b=kQRCOUQ5EWJ7OZjQPf4cCk1xaALDzsKIr5GQTW5tIWdPTyzUhTUiRdpjEISKzzKBn Okm9eHf9aDgqpqVjgQ0wHkB6+jd28tA4E5pzCEYULp+ZvRalTWuLCTqB2PzklkhCF8 70Jllkk5+JduW7BgWFWVJJW7V5WHGx7gAgnFrNj4= Date: Wed, 30 Jun 2021 18:52:55 -0700 From: Andrew Morton To: akpm@linux-foundation.org, bigeasy@linutronix.de, colin.king@canonical.com, ddstreet@ieee.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, nathan@kernel.org, sjenning@redhat.com, tiantao6@hisilicon.com, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 107/192] mm/zswap.c: fix two bugs in zswap_writeback_entry() Message-ID: <20210701015255.wWKmg3P-p%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=kQRCOUQ5; dmarc=none; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: zcyzwn51a6t8m1t99dwub3kegcjt6hdc X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0A1561982 X-HE-Tag: 1625104376-100220 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zswap.c: fix two bugs in zswap_writeback_entry() In the ZSWAP_SWAPCACHE_FAIL and ZSWAP_SWAPCACHE_EXIST case, we forgot to call zpool_unmap_handle() when zpool can't sleep. And we might sleep in zswap_get_swap_cache_page() while zpool can't sleep. To fix all of these, zpool_unmap_handle() should be done before zswap_get_swap_cache_page() when zpool can't sleep. Link: https://lkml.kernel.org/r/20210522092242.3233191-4-linmiaohe@huawei.com Fixes: fc6697a89f56 ("mm/zswap: add the flag can_sleep_mapped") Signed-off-by: Miaohe Lin Cc: Colin Ian King Cc: Dan Streetman Cc: Nathan Chancellor Cc: Sebastian Andrzej Siewior Cc: Seth Jennings Cc: Tian Tao Cc: Vitaly Wool Signed-off-by: Andrew Morton --- mm/zswap.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) --- a/mm/zswap.c~mm-zswapc-fix-two-bugs-in-zswap_writeback_entry +++ a/mm/zswap.c @@ -967,6 +967,13 @@ static int zswap_writeback_entry(struct spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -982,17 +989,7 @@ static int zswap_writeback_entry(struct case ZSWAP_SWAPCACHE_NEW: /* page is locked */ /* decompress */ acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); - dlen = PAGE_SIZE; - src = (u8 *)zhdr + sizeof(struct zswap_header); - - if (!zpool_can_sleep_mapped(pool)) { - - memcpy(tmp, src, entry->length); - src = tmp; - - zpool_unmap_handle(pool, handle); - } mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); From patchwork Thu Jul 1 01:52:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44377C11F66 for ; Thu, 1 Jul 2021 01:53:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE4856105A for ; Thu, 1 Jul 2021 01:53:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE4856105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 776588D023F; Wed, 30 Jun 2021 21:53:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74DAB8D0236; Wed, 30 Jun 2021 21:53:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 615748D023F; Wed, 30 Jun 2021 21:53:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id 3EFD08D0236 for ; Wed, 30 Jun 2021 21:53:00 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1DB5B22BEA for ; Thu, 1 Jul 2021 01:53:00 +0000 (UTC) X-FDA: 78312345720.32.57DAF62 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id CF69B801AB00 for ; Thu, 1 Jul 2021 01:52:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EC51161241; Thu, 1 Jul 2021 01:52:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104379; bh=3mmcgWfzdxHM+eG6w1Er59n7ndos17yQennCj2C0N1s=; h=Date:From:To:Subject:In-Reply-To:From; b=zyZStW2v4TgpiRsaqRlyCmfeuxnVtbMLGjMDMNj9w3aljI3q97jEc82/fXWb0eN9D 85MPGqp0Mh9VpOTVvGAImoZi8hPduDA/Adce95vfui7LQ9gfXkUsP4XjwzFNJPRCae iMHlMhK9siJaA4hMbG+74/HU88N+KI1qUvqg9dFo= Date: Wed, 30 Jun 2021 18:52:58 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, senozhatsky@chromium.org, torvalds@linux-foundation.org, zhaoyang.huang@unisoc.com Subject: [patch 108/192] mm: zram: amend SLAB_RECLAIM_ACCOUNT on zspage_cachep Message-ID: <20210701015258.BrxjIzdE1%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=zyZStW2v; spf=pass (imf06.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: xztnckyqu3t7i4zdzf8netsgwh61ernb X-Rspamd-Queue-Id: CF69B801AB00 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104379-953825 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhaoyang Huang Subject: mm: zram: amend SLAB_RECLAIM_ACCOUNT on zspage_cachep Zspage_cachep is found be merged with other kmem cache during test, which is not good for debug things (zs_pool->zspage_cachep present to be another kmem cache in memory dumpfile). It is also neccessary to do so as shrinker has been registered for zspage. Amending this flag can help kernel to calculate SLAB_RECLAIMBLE correctly. Link: https://lkml.kernel.org/r/1623137297-29685-1-git-send-email-huangzhaoyang@gmail.com Signed-off-by: Zhaoyang Huang Cc: Minchan Kim Cc: Sergey Senozhatsky Signed-off-by: Andrew Morton --- mm/zsmalloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/zsmalloc.c~mm-zram-amend-slab_reclaim_account-on-zspage_cachep +++ a/mm/zsmalloc.c @@ -328,7 +328,7 @@ static int create_cache(struct zs_pool * return 1; pool->zspage_cachep = kmem_cache_create("zspage", sizeof(struct zspage), - 0, 0, NULL); + 0, SLAB_RECLAIM_ACCOUNT, NULL); if (!pool->zspage_cachep) { kmem_cache_destroy(pool->handle_cachep); pool->handle_cachep = NULL; From patchwork Thu Jul 1 01:53:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70139C11F66 for ; Thu, 1 Jul 2021 01:53:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2720561468 for ; Thu, 1 Jul 2021 01:53:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2720561468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A5C3A8D0240; Wed, 30 Jun 2021 21:53:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A33408D0236; Wed, 30 Jun 2021 21:53:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FB548D0240; Wed, 30 Jun 2021 21:53:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id 6E1DD8D0236 for ; Wed, 30 Jun 2021 21:53:03 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 40225181AEF3F for ; Thu, 1 Jul 2021 01:53:03 +0000 (UTC) X-FDA: 78312345846.37.C582D0A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id E80EFF000128 for ; Thu, 1 Jul 2021 01:53:02 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D79926105A; Thu, 1 Jul 2021 01:53:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104382; bh=M43RQCLRu9wA7tddapQP5/1Byd2HZw83UfDEII0Ul6A=; h=Date:From:To:Subject:In-Reply-To:From; b=Sxeb7AJjQLm5yRyJX2qlJnorgGzclyuvHaDFYhsSfbV0Avgiq0De9MR20W33VjXpq 1/W9NU++xxqjPIkWicTUmqlIcPO5ix9CMo8e9MdmCc61F8TjnsmaHY3un5qPD6AXx+ pDD3rqDDo2s5bAT5FDvw2AcOOIEvLOZPGuOIyWRM= Date: Wed, 30 Jun 2021 18:53:01 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, torvalds@linux-foundation.org Subject: [patch 109/192] mm/zsmalloc.c: remove confusing code in obj_free() Message-ID: <20210701015301.tcFA0cOau%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Sxeb7AJj; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: jsjp4urnp13dbwniqp8zxpc1ydz8wenz X-Rspamd-Queue-Id: E80EFF000128 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104382-540230 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zsmalloc.c: remove confusing code in obj_free() Patch series "Cleanup for zsmalloc". This series contains cleanups to remove confusing code in obj_free(), combine two atomic ops and improve readability for async_free_zspage(). More details can be found in the respective changelogs. This patch (of 2): OBJ_ALLOCATED_TAG is only set for handle to indicate allocated object. It's irrelevant with obj. So remove this misleading code to improve readability. Link: https://lkml.kernel.org/r/20210624123930.1769093-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210624123930.1769093-2-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Minchan Kim Cc: Nitin Gupta Cc: Sergey Senozhatsky Signed-off-by: Andrew Morton --- mm/zsmalloc.c | 1 - 1 file changed, 1 deletion(-) --- a/mm/zsmalloc.c~mm-zsmallocc-remove-confusing-code-in-obj_free +++ a/mm/zsmalloc.c @@ -1471,7 +1471,6 @@ static void obj_free(struct size_class * unsigned int f_objidx; void *vaddr; - obj &= ~OBJ_ALLOCATED_TAG; obj_to_location(obj, &f_page, &f_objidx); f_offset = (class->size * f_objidx) & ~PAGE_MASK; zspage = get_zspage(f_page); From patchwork Thu Jul 1 01:53:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B240C11F69 for ; Thu, 1 Jul 2021 01:53:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EC29961469 for ; Thu, 1 Jul 2021 01:53:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EC29961469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 74ED38D0241; Wed, 30 Jun 2021 21:53:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6FE948D0236; Wed, 30 Jun 2021 21:53:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A0038D0241; Wed, 30 Jun 2021 21:53:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id 35BCA8D0236 for ; Wed, 30 Jun 2021 21:53:06 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0ED1E1801E378 for ; Thu, 1 Jul 2021 01:53:06 +0000 (UTC) X-FDA: 78312345972.31.8BF5B7E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id BA91DB0000A2 for ; Thu, 1 Jul 2021 01:53:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D442B61468; Thu, 1 Jul 2021 01:53:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104385; bh=eYEx21DF4KneT+T4IkxLcj2uBlslKKRnWzc2TU2u6Z8=; h=Date:From:To:Subject:In-Reply-To:From; b=Ggxm7ZGcS5lGzcj4UWczdGAAqwlVxz3UKhYvbG/oKlyioorewaV0lMXS446j+wgeM vzHm21xIF5jKXDKR234z8HRe2u6K2M8t4WfDC8QNvAZ7y/p1Q7gCY/TFAwQP9hTuSF oI47TlC/6aHYzfqKGyyjQtb9FRSEML8dgRLf+58g= Date: Wed, 30 Jun 2021 18:53:04 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, torvalds@linux-foundation.org Subject: [patch 110/192] mm/zsmalloc.c: improve readability for async_free_zspage() Message-ID: <20210701015304.6sZ8amjNt%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Ggxm7ZGc; dmarc=none; spf=pass (imf19.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: pxuuxh3gk7npqt7nt6joa1xjky1ck7wo X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BA91DB0000A2 X-HE-Tag: 1625104385-565122 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/zsmalloc.c: improve readability for async_free_zspage() The class is extracted from pool->size_class[class_idx] again before calling __free_zspage(). It looks like class will change after we fetch the class lock. But this is misleading as class will stay unchanged. Link: https://lkml.kernel.org/r/20210624123930.1769093-4-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Minchan Kim Cc: Nitin Gupta Cc: Sergey Senozhatsky Signed-off-by: Andrew Morton --- mm/zsmalloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/zsmalloc.c~mm-zsmallocc-improve-readability-for-async_free_zspage +++ a/mm/zsmalloc.c @@ -2162,7 +2162,7 @@ static void async_free_zspage(struct wor VM_BUG_ON(fullness != ZS_EMPTY); class = pool->size_class[class_idx]; spin_lock(&class->lock); - __free_zspage(pool, pool->size_class[class_idx], zspage); + __free_zspage(pool, class, zspage); spin_unlock(&class->lock); } }; From patchwork Thu Jul 1 01:53:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46458C11F66 for ; Thu, 1 Jul 2021 01:53:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0B2F761468 for ; Thu, 1 Jul 2021 01:53:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B2F761468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8AEFC8D0242; Wed, 30 Jun 2021 21:53:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 886908D0236; Wed, 30 Jun 2021 21:53:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74D358D0242; Wed, 30 Jun 2021 21:53:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 50F188D0236 for ; Wed, 30 Jun 2021 21:53:09 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2B62C250CF for ; Thu, 1 Jul 2021 01:53:09 +0000 (UTC) X-FDA: 78312346098.35.C614FA3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id DCD935000098 for ; Thu, 1 Jul 2021 01:53:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BEFF36105A; Thu, 1 Jul 2021 01:53:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104388; bh=qRlUoZqlfmnGmyW2QHh7SLhKVxgwA/JqiVj3PKTOkRg=; h=Date:From:To:Subject:In-Reply-To:From; b=t0sy5+tBcrORpQOFMKI85BgpTEmT4w3nYcK/PGkSiQ/h8IBvJG+b0a/ggAa6Sz7iL 96AtpS+L1YVWyHk2+cQzp/IstcdxweoEOSchXgiym/HI6za7Noha2VJSjhJte2Wmr3 SaTO8A3AXfFyZ6hOxVmS7HSKtXfeYMBvhg1kkPY4= Date: Wed, 30 Jun 2021 18:53:07 -0700 From: Andrew Morton To: akpm@linux-foundation.org, huyue2@yulong.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, senozhatsky@chromium.org, sergey.senozhatsky@gmail.com, torvalds@linux-foundation.org Subject: [patch 111/192] zram: move backing_dev under macro CONFIG_ZRAM_WRITEBACK Message-ID: <20210701015307.EtNdJcvyZ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: DCD935000098 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=t0sy5+tB; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: kmczgnjqxgjbou4yjmt6ikx5dfy445ad X-HE-Tag: 1625104388-649178 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yue Hu Subject: zram: move backing_dev under macro CONFIG_ZRAM_WRITEBACK backing_dev is never used when not enable CONFIG_ZRAM_WRITEBACK and it's introduced from writeback feature. So it's needless also affect readability in that case. Link: https://lkml.kernel.org/r/20210521060544.2385-1-zbestahu@gmail.com Signed-off-by: Yue Hu Reviewed-by: Sergey Senozhatsky Acked-by: Minchan Kim Cc: Sergey Senozhatsky Signed-off-by: Andrew Morton --- drivers/block/zram/zram_drv.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/drivers/block/zram/zram_drv.h~zram-move-backing_dev-under-macro-config_zram_writeback +++ a/drivers/block/zram/zram_drv.h @@ -113,8 +113,8 @@ struct zram { * zram is claimed so open request will be failed */ bool claim; /* Protected by bdev->bd_mutex */ - struct file *backing_dev; #ifdef CONFIG_ZRAM_WRITEBACK + struct file *backing_dev; spinlock_t wb_limit_lock; bool wb_limit_enable; u64 bd_wb_limit; From patchwork Thu Jul 1 01:53:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 002CFC11F66 for ; Thu, 1 Jul 2021 01:53:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AF09A61241 for ; Thu, 1 Jul 2021 01:53:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF09A61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 37E018D0243; Wed, 30 Jun 2021 21:53:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32E018D0236; Wed, 30 Jun 2021 21:53:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CFCF8D0243; Wed, 30 Jun 2021 21:53:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id EFA3F8D0236 for ; Wed, 30 Jun 2021 21:53:11 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C0F09181AEF3F for ; Thu, 1 Jul 2021 01:53:11 +0000 (UTC) X-FDA: 78312346182.21.0D9775E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 80312400208D for ; Thu, 1 Jul 2021 01:53:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A9B9E6147E; Thu, 1 Jul 2021 01:53:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104390; bh=P/lhfLrruUR7vtC0Xbkq/dwoe2sxwcgaLGDzRg4CgxY=; h=Date:From:To:Subject:In-Reply-To:From; b=Cd0fRRu+kl6eFlt0Gyt3xHBHsU6Rl541UgzqkbdS7rLlVMs6pFVyBs3bJcbo5w3O9 6lzNpysZ7E2w5AzbHtLOZ75LD4SGebTYYqX0Q+kPuQJfReRN/SYBaF+FQ3PnubaQsF QvNMgpiaujJSl69VK9Wa2KPo/L34t2A/zdZBVa6U= Date: Wed, 30 Jun 2021 18:53:10 -0700 From: Andrew Morton To: 42.hyeyoo@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 112/192] mm: fix typos and grammar error in comments Message-ID: <20210701015310.w1sgX-eEm%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Cd0fRRu+; dmarc=none; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: rr74i1kst8fnyzy8acexxanrm5wywc3a X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 80312400208D X-HE-Tag: 1625104391-706073 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hyeonggon Yoo <42.hyeyoo@gmail.com> Subject: mm: fix typos and grammar error in comments We moves tha -> We move that in mm/swap.c statments -> statements in include/linux/mm.h Link: https://lkml.kernel.org/r/20210509063444.GA24745@hyeyoo Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Andrew Morton --- include/linux/mm.h | 2 +- mm/swap.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) --- a/include/linux/mm.h~mm-fix-typos-and-grammar-error-in-comments +++ a/include/linux/mm.h @@ -155,7 +155,7 @@ extern int mmap_rnd_compat_bits __read_m /* This function must be updated when the size of struct page grows above 80 * or reduces below 56. The idea that compiler optimizes out switch() * statement, and only leaves move/store instructions. Also the compiler can - * combine write statments if they are both assignments and can be reordered, + * combine write statements if they are both assignments and can be reordered, * this can result in several of the writes here being dropped. */ #define mm_zero_struct_page(pp) __mm_zero_struct_page(pp) --- a/mm/swap.c~mm-fix-typos-and-grammar-error-in-comments +++ a/mm/swap.c @@ -554,7 +554,7 @@ static void lru_deactivate_file_fn(struc } else { /* * The page's writeback ends up during pagevec - * We moves tha page into tail of inactive. + * We move that page into tail of inactive. */ add_page_to_lru_list_tail(page, lruvec); __count_vm_events(PGROTATED, nr_pages); From patchwork Thu Jul 1 01:53:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28D05C11F69 for ; Thu, 1 Jul 2021 01:53:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C0C8761468 for ; Thu, 1 Jul 2021 01:53:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0C8761468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 469B28D0244; Wed, 30 Jun 2021 21:53:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43FB68D0236; Wed, 30 Jun 2021 21:53:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E1638D0244; Wed, 30 Jun 2021 21:53:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id 097178D0236 for ; Wed, 30 Jun 2021 21:53:16 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D485F1695A for ; Thu, 1 Jul 2021 01:53:15 +0000 (UTC) X-FDA: 78312346350.01.9F5EC77 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 75E7A10000A3 for ; Thu, 1 Jul 2021 01:53:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D13A661241; Thu, 1 Jul 2021 01:53:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104394; bh=uyDHtDrGupOFb+2HVgXC9zD9OofsCQnTpjdX1+9+ui8=; h=Date:From:To:Subject:In-Reply-To:From; b=JJWww/vsq0BvxP/d+r02Ea83b510HkVp0CNn/E7eYVHWSSVPy7YRAZPHeHXshtdc3 EkAvKVzg2N8dDmvg6p9eGigy/cvoYXcqz/XXdVssDEHY+o0T69xKEerqTaPjsbOS5g O7oqi968P7ZS13jo8a4VrM006eLNc7symlb4FKMw= Date: Wed, 30 Jun 2021 18:53:13 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, bcain@codeaurora.org, catalin.marinas@arm.com, chris@zankel.net, christophe.leroy@csgroup.eu, davem@davemloft.net, geert@linux-m68k.org, guoren@kernel.org, hca@linux.ibm.com, James.Bottomley@HansenPartnership.com, jdike@addtoit.com, jonas@southpole.se, ley.foon.tan@intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, monstr@monstr.eu, mpe@ellerman.id.au, palmerdabbelt@google.com, paul.walmsley@sifive.com, rppt@linux.ibm.com, rth@twiddle.net, shorne@gmail.com, stefan.kristiansson@saunalahti.fi, tglx@linutronix.de, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vgupta@synopsys.com, will@kernel.org, ysato@users.sourceforge.jp Subject: [patch 113/192] mm: define default value for FIRST_USER_ADDRESS Message-ID: <20210701015313.pnbtd746z%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="JJWww/vs"; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: i7xgoos6h1758w8pxsetuwc6qwu9fa74 X-Rspamd-Queue-Id: 75E7A10000A3 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104395-288485 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm: define default value for FIRST_USER_ADDRESS Currently most platforms define FIRST_USER_ADDRESS as 0UL duplication the same code all over. Instead just define a generic default value (i.e 0UL) for FIRST_USER_ADDRESS and let the platforms override when required. This makes it much cleaner with reduced code. The default FIRST_USER_ADDRESS here would be skipped in when the given platform overrides its value via . Link: https://lkml.kernel.org/r/1620615725-24623-1-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Acked-by: Geert Uytterhoeven [m68k] Acked-by: Guo Ren [csky] Acked-by: Stafford Horne [openrisc] Acked-by: Catalin Marinas [arm64] Acked-by: Mike Rapoport Acked-by: Palmer Dabbelt [RISC-V] Cc: Richard Henderson Cc: Vineet Gupta Cc: Catalin Marinas Cc: Will Deacon Cc: Guo Ren Cc: Brian Cain Cc: Geert Uytterhoeven Cc: Michal Simek Cc: Thomas Bogendoerfer Cc: Ley Foon Tan Cc: Jonas Bonn Cc: Stefan Kristiansson Cc: Stafford Horne Cc: "James E.J. Bottomley" Cc: Michael Ellerman Cc: Christophe Leroy Cc: Paul Walmsley Cc: Heiko Carstens Cc: Yoshinori Sato Cc: "David S. Miller" Cc: Jeff Dike Cc: Thomas Gleixner Cc: Chris Zankel Signed-off-by: Andrew Morton --- arch/alpha/include/asm/pgtable.h | 1 - arch/arc/include/asm/pgtable.h | 6 ------ arch/arm64/include/asm/pgtable.h | 2 -- arch/csky/include/asm/pgtable.h | 1 - arch/hexagon/include/asm/pgtable.h | 3 --- arch/ia64/include/asm/pgtable.h | 1 - arch/m68k/include/asm/pgtable_mm.h | 1 - arch/microblaze/include/asm/pgtable.h | 2 -- arch/mips/include/asm/pgtable-32.h | 1 - arch/mips/include/asm/pgtable-64.h | 1 - arch/nios2/include/asm/pgtable.h | 2 -- arch/openrisc/include/asm/pgtable.h | 1 - arch/parisc/include/asm/pgtable.h | 2 -- arch/powerpc/include/asm/book3s/pgtable.h | 1 - arch/powerpc/include/asm/nohash/32/pgtable.h | 1 - arch/powerpc/include/asm/nohash/64/pgtable.h | 2 -- arch/riscv/include/asm/pgtable.h | 2 -- arch/s390/include/asm/pgtable.h | 2 -- arch/sh/include/asm/pgtable.h | 2 -- arch/sparc/include/asm/pgtable_32.h | 1 - arch/sparc/include/asm/pgtable_64.h | 3 --- arch/um/include/asm/pgtable-2level.h | 1 - arch/um/include/asm/pgtable-3level.h | 1 - arch/x86/include/asm/pgtable_types.h | 2 -- arch/xtensa/include/asm/pgtable.h | 1 - include/linux/pgtable.h | 9 +++++++++ 26 files changed, 9 insertions(+), 43 deletions(-) --- a/arch/alpha/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/alpha/include/asm/pgtable.h @@ -46,7 +46,6 @@ struct vm_area_struct; #define PTRS_PER_PMD (1UL << (PAGE_SHIFT-3)) #define PTRS_PER_PGD (1UL << (PAGE_SHIFT-3)) #define USER_PTRS_PER_PGD (TASK_SIZE / PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL /* Number of pointers that fit on a page: this will go away. */ #define PTRS_PER_PAGE (1UL << (PAGE_SHIFT-3)) --- a/arch/arc/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/arc/include/asm/pgtable.h @@ -222,12 +222,6 @@ */ #define USER_PTRS_PER_PGD (TASK_SIZE / PGDIR_SIZE) -/* - * No special requirements for lowest virtual address we permit any user space - * mapping to be mapped at. - */ -#define FIRST_USER_ADDRESS 0UL - /**************************************************************** * Bucket load of VM Helpers --- a/arch/arm64/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/arm64/include/asm/pgtable.h @@ -26,8 +26,6 @@ #define vmemmap ((struct page *)VMEMMAP_START - (memstart_addr >> PAGE_SHIFT)) -#define FIRST_USER_ADDRESS 0UL - #ifndef __ASSEMBLY__ #include --- a/arch/csky/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/csky/include/asm/pgtable.h @@ -14,7 +14,6 @@ #define PGDIR_MASK (~(PGDIR_SIZE-1)) #define USER_PTRS_PER_PGD (PAGE_OFFSET/PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL /* * C-SKY is two-level paging structure: --- a/arch/hexagon/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/hexagon/include/asm/pgtable.h @@ -155,9 +155,6 @@ extern unsigned long _dflt_cache_att; extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; /* located in head.S */ -/* Seems to be zero even in architectures where the zero page is firewalled? */ -#define FIRST_USER_ADDRESS 0UL - /* HUGETLB not working currently */ #ifdef CONFIG_HUGETLB_PAGE #define pte_mkhuge(pte) __pte((pte_val(pte) & ~0x3) | HVM_HUGEPAGE_SIZE) --- a/arch/ia64/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/ia64/include/asm/pgtable.h @@ -128,7 +128,6 @@ #define PTRS_PER_PGD_SHIFT PTRS_PER_PTD_SHIFT #define PTRS_PER_PGD (1UL << PTRS_PER_PGD_SHIFT) #define USER_PTRS_PER_PGD (5*PTRS_PER_PGD/8) /* regions 0-4 are user regions */ -#define FIRST_USER_ADDRESS 0UL /* * All the normal masks have the "page accessed" bits on, as any time --- a/arch/m68k/include/asm/pgtable_mm.h~mm-define-default-value-for-first_user_address +++ a/arch/m68k/include/asm/pgtable_mm.h @@ -72,7 +72,6 @@ #define PTRS_PER_PGD 128 #endif #define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL /* Virtual address region for use by kernel_map() */ #ifdef CONFIG_SUN3 --- a/arch/microblaze/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/microblaze/include/asm/pgtable.h @@ -25,8 +25,6 @@ extern int mem_init_done; #include #include -#define FIRST_USER_ADDRESS 0UL - extern unsigned long va_to_phys(unsigned long address); extern pte_t *va_to_pte(unsigned long address); --- a/arch/mips/include/asm/pgtable-32.h~mm-define-default-value-for-first_user_address +++ a/arch/mips/include/asm/pgtable-32.h @@ -93,7 +93,6 @@ extern int add_temporary_entry(unsigned #endif #define USER_PTRS_PER_PGD (0x80000000UL/PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL #define VMALLOC_START MAP_BASE --- a/arch/mips/include/asm/pgtable-64.h~mm-define-default-value-for-first_user_address +++ a/arch/mips/include/asm/pgtable-64.h @@ -137,7 +137,6 @@ #define PTRS_PER_PTE ((PAGE_SIZE << PTE_ORDER) / sizeof(pte_t)) #define USER_PTRS_PER_PGD ((TASK_SIZE64 / PGDIR_SIZE)?(TASK_SIZE64 / PGDIR_SIZE):1) -#define FIRST_USER_ADDRESS 0UL /* * TLB refill handlers also map the vmalloc area into xuseg. Avoid --- a/arch/nios2/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/nios2/include/asm/pgtable.h @@ -24,8 +24,6 @@ #include #include -#define FIRST_USER_ADDRESS 0UL - #define VMALLOC_START CONFIG_NIOS2_KERNEL_MMU_REGION_BASE #define VMALLOC_END (CONFIG_NIOS2_KERNEL_REGION_BASE - 1) --- a/arch/openrisc/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/openrisc/include/asm/pgtable.h @@ -73,7 +73,6 @@ extern void paging_init(void); */ #define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL /* * Kernels own virtual memory area. --- a/arch/parisc/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/parisc/include/asm/pgtable.h @@ -171,8 +171,6 @@ static inline void purge_tlb_entries(str * pgd entries used up by user/kernel: */ -#define FIRST_USER_ADDRESS 0UL - /* NB: The tlb miss handlers make certain assumptions about the order */ /* of the following bits, so be careful (One example, bits 25-31 */ /* are moved together in one instruction). */ --- a/arch/powerpc/include/asm/book3s/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/powerpc/include/asm/book3s/pgtable.h @@ -8,7 +8,6 @@ #include #endif -#define FIRST_USER_ADDRESS 0UL #ifndef __ASSEMBLY__ /* Insert a PTE, top-level function is out of line. It uses an inline * low level function in the respective pgtable-* files --- a/arch/powerpc/include/asm/nohash/32/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/powerpc/include/asm/nohash/32/pgtable.h @@ -54,7 +54,6 @@ extern int icache_44x_need_flush; #define PGD_MASKED_BITS 0 #define USER_PTRS_PER_PGD (TASK_SIZE / PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL #define pte_ERROR(e) \ pr_err("%s:%d: bad pte %llx.\n", __FILE__, __LINE__, \ --- a/arch/powerpc/include/asm/nohash/64/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/powerpc/include/asm/nohash/64/pgtable.h @@ -12,8 +12,6 @@ #include #include -#define FIRST_USER_ADDRESS 0UL - /* * Size of EA range mapped by our pagetables. */ --- a/arch/riscv/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/riscv/include/asm/pgtable.h @@ -536,8 +536,6 @@ void setup_bootmem(void); void paging_init(void); void misc_mem_init(void); -#define FIRST_USER_ADDRESS 0 - /* * ZERO_PAGE is a global shared page that is always zero, * used for zero-mapped memory areas, etc. --- a/arch/s390/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/s390/include/asm/pgtable.h @@ -65,8 +65,6 @@ extern unsigned long zero_page_mask; /* TODO: s390 cannot support io_remap_pfn_range... */ -#define FIRST_USER_ADDRESS 0UL - #define pte_ERROR(e) \ printk("%s:%d: bad pte %p.\n", __FILE__, __LINE__, (void *) pte_val(e)) #define pmd_ERROR(e) \ --- a/arch/sh/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/sh/include/asm/pgtable.h @@ -59,8 +59,6 @@ static inline unsigned long long neff_si /* Entries per level */ #define PTRS_PER_PTE (PAGE_SIZE / (1 << PTE_MAGNITUDE)) -#define FIRST_USER_ADDRESS 0UL - #define PHYS_ADDR_MASK29 0x1fffffff #define PHYS_ADDR_MASK32 0xffffffff --- a/arch/sparc/include/asm/pgtable_32.h~mm-define-default-value-for-first_user_address +++ a/arch/sparc/include/asm/pgtable_32.h @@ -48,7 +48,6 @@ unsigned long __init bootmem_init(unsign #define PTRS_PER_PMD 64 #define PTRS_PER_PGD 256 #define USER_PTRS_PER_PGD PAGE_OFFSET / PGDIR_SIZE -#define FIRST_USER_ADDRESS 0UL #define PTE_SIZE (PTRS_PER_PTE*4) #define PAGE_NONE SRMMU_PAGE_NONE --- a/arch/sparc/include/asm/pgtable_64.h~mm-define-default-value-for-first_user_address +++ a/arch/sparc/include/asm/pgtable_64.h @@ -95,9 +95,6 @@ bool kern_addr_valid(unsigned long addr) #define PTRS_PER_PUD (1UL << PUD_BITS) #define PTRS_PER_PGD (1UL << PGDIR_BITS) -/* Kernel has a separate 44bit address space. */ -#define FIRST_USER_ADDRESS 0UL - #define pmd_ERROR(e) \ pr_err("%s:%d: bad pmd %p(%016lx) seen at (%pS)\n", \ __FILE__, __LINE__, &(e), pmd_val(e), __builtin_return_address(0)) --- a/arch/um/include/asm/pgtable-2level.h~mm-define-default-value-for-first_user_address +++ a/arch/um/include/asm/pgtable-2level.h @@ -23,7 +23,6 @@ #define PTRS_PER_PTE 1024 #define USER_PTRS_PER_PGD ((TASK_SIZE + (PGDIR_SIZE - 1)) / PGDIR_SIZE) #define PTRS_PER_PGD 1024 -#define FIRST_USER_ADDRESS 0UL #define pte_ERROR(e) \ printk("%s:%d: bad pte %p(%08lx).\n", __FILE__, __LINE__, &(e), \ --- a/arch/um/include/asm/pgtable-3level.h~mm-define-default-value-for-first_user_address +++ a/arch/um/include/asm/pgtable-3level.h @@ -41,7 +41,6 @@ #endif #define USER_PTRS_PER_PGD ((TASK_SIZE + (PGDIR_SIZE - 1)) / PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL #define pte_ERROR(e) \ printk("%s:%d: bad pte %p(%016lx).\n", __FILE__, __LINE__, &(e), \ --- a/arch/x86/include/asm/pgtable_types.h~mm-define-default-value-for-first_user_address +++ a/arch/x86/include/asm/pgtable_types.h @@ -7,8 +7,6 @@ #include -#define FIRST_USER_ADDRESS 0UL - #define _PAGE_BIT_PRESENT 0 /* is present */ #define _PAGE_BIT_RW 1 /* writeable */ #define _PAGE_BIT_USER 2 /* userspace addressable */ --- a/arch/xtensa/include/asm/pgtable.h~mm-define-default-value-for-first_user_address +++ a/arch/xtensa/include/asm/pgtable.h @@ -59,7 +59,6 @@ #define PTRS_PER_PGD 1024 #define PGD_ORDER 0 #define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE) -#define FIRST_USER_ADDRESS 0UL #define FIRST_USER_PGD_NR (FIRST_USER_ADDRESS >> PGDIR_SHIFT) #ifdef CONFIG_MMU --- a/include/linux/pgtable.h~mm-define-default-value-for-first_user_address +++ a/include/linux/pgtable.h @@ -29,6 +29,15 @@ #endif /* + * This defines the first usable user address. Platforms + * can override its value with custom FIRST_USER_ADDRESS + * defined in their respective . + */ +#ifndef FIRST_USER_ADDRESS +#define FIRST_USER_ADDRESS 0UL +#endif + +/* * A page table page can be thought of an array like this: pXd_t[PTRS_PER_PxD] * * The pXx_index() functions return the index of the entry in the page From patchwork Thu Jul 1 01:53:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFDCDC11F66 for ; Thu, 1 Jul 2021 01:53:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 862976105A for ; Thu, 1 Jul 2021 01:53:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 862976105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 10D7E8D0245; Wed, 30 Jun 2021 21:53:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E4A68D0236; Wed, 30 Jun 2021 21:53:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EEE9D8D0245; Wed, 30 Jun 2021 21:53:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id CDF6E8D0236 for ; Wed, 30 Jun 2021 21:53:18 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9888580A9AAD for ; Thu, 1 Jul 2021 01:53:18 +0000 (UTC) X-FDA: 78312346476.03.E1AEFA4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 441D8D00014E for ; Thu, 1 Jul 2021 01:53:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 657EC61469; Thu, 1 Jul 2021 01:53:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104397; bh=xoHdhu8U3qtCFTOif3SW2iv5iK5pM95qFm0DCSPcCNk=; h=Date:From:To:Subject:In-Reply-To:From; b=X5PiS7C+FJYD8twMY5o9a7zPuEfQ8O6OYELSJZSL7UHU+opwnVN6i8ya5bqdse1yY yfV3YPVD5KfOjuzgdjyAwHjZNO1XvRv73NiwOIbQ8U22MLBgvfauF/hOtdGN+eLB2B 1CCgAV4JNzebc0aBIGrOOUY/FcMnExXRCW8f0auk= Date: Wed, 30 Jun 2021 18:53:17 -0700 From: Andrew Morton To: akpm@linux-foundation.org, jrdr.linux@gmail.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, thunder.leizhen@huawei.com, torvalds@linux-foundation.org Subject: [patch 114/192] mm: fix spelling mistakes Message-ID: <20210701015317.LWzFP6RnZ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=X5PiS7C+; dmarc=none; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: idio1ud3ke1wy1ji591typ1zz7k4884a X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 441D8D00014E X-HE-Tag: 1625104398-162608 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhen Lei Subject: mm: fix spelling mistakes Fix some spelling mistakes in comments: each having differents usage ==> each has a different usage statments ==> statements adresses ==> addresses aggresive ==> aggressive datas ==> data posion ==> poison higer ==> higher precisly ==> precisely wont ==> won't We moves tha ==> We move the endianess ==> endianness Link: https://lkml.kernel.org/r/20210519065853.7723-2-thunder.leizhen@huawei.com Signed-off-by: Zhen Lei Reviewed-by: Souptick Joarder Signed-off-by: Andrew Morton --- include/linux/memremap.h | 2 +- include/linux/mm_types.h | 2 +- include/linux/mmzone.h | 2 +- mm/memory-failure.c | 2 +- mm/memory_hotplug.c | 4 ++-- mm/page_alloc.c | 2 +- mm/swapfile.c | 2 +- 7 files changed, 8 insertions(+), 8 deletions(-) --- a/include/linux/memremap.h~mm-clear-spelling-mistakes +++ a/include/linux/memremap.h @@ -26,7 +26,7 @@ struct vmem_altmap { }; /* - * Specialize ZONE_DEVICE memory into multiple types each having differents + * Specialize ZONE_DEVICE memory into multiple types each has a different * usage. * * MEMORY_DEVICE_PRIVATE: --- a/include/linux/mm_types.h~mm-clear-spelling-mistakes +++ a/include/linux/mm_types.h @@ -397,7 +397,7 @@ struct mm_struct { unsigned long mmap_base; /* base of mmap area */ unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */ #ifdef CONFIG_HAVE_ARCH_COMPAT_MMAP_BASES - /* Base adresses for compatible mmap() */ + /* Base addresses for compatible mmap() */ unsigned long mmap_compat_base; unsigned long mmap_compat_legacy_base; #endif --- a/include/linux/mmzone.h~mm-clear-spelling-mistakes +++ a/include/linux/mmzone.h @@ -114,7 +114,7 @@ static inline bool free_area_empty(struc struct pglist_data; /* - * Add a wild amount of padding here to ensure datas fall into separate + * Add a wild amount of padding here to ensure data fall into separate * cachelines. There are very few zone structures in the machine, so space * consumption is not a concern here. */ --- a/mm/memory-failure.c~mm-clear-spelling-mistakes +++ a/mm/memory-failure.c @@ -1340,7 +1340,7 @@ static bool hwpoison_user_mappings(struc * could potentially call huge_pmd_unshare. Because of * this, take semaphore in write mode here and set * TTU_RMAP_LOCKED to indicate we have taken the lock - * at this higer level. + * at this higher level. */ mapping = hugetlb_page_mapping_lock_write(hpage); if (mapping) { --- a/mm/memory_hotplug.c~mm-clear-spelling-mistakes +++ a/mm/memory_hotplug.c @@ -783,7 +783,7 @@ int __ref online_pages(unsigned long pfn /* * {on,off}lining is constrained to full memory sections (or more - * precisly to memory blocks from the user space POV). + * precisely to memory blocks from the user space POV). * memmap_on_memory is an exception because it reserves initial part * of the physical memory space for vmemmaps. That space is pageblock * aligned. @@ -1580,7 +1580,7 @@ int __ref offline_pages(unsigned long st /* * {on,off}lining is constrained to full memory sections (or more - * precisly to memory blocks from the user space POV). + * precisely to memory blocks from the user space POV). * memmap_on_memory is an exception because it reserves initial part * of the physical memory space for vmemmaps. That space is pageblock * aligned. --- a/mm/page_alloc.c~mm-clear-spelling-mistakes +++ a/mm/page_alloc.c @@ -3180,7 +3180,7 @@ static void __drain_all_pages(struct zon int cpu; /* - * Allocate in the BSS so we wont require allocation in + * Allocate in the BSS so we won't require allocation in * direct reclaim path for CONFIG_CPUMASK_OFFSTACK=y */ static cpumask_t cpus_with_pcps; --- a/mm/swapfile.c~mm-clear-spelling-mistakes +++ a/mm/swapfile.c @@ -2967,7 +2967,7 @@ static unsigned long read_swap_header(st return 0; } - /* swap partition endianess hack... */ + /* swap partition endianness hack... */ if (swab32(swap_header->info.version) == 1) { swab32s(&swap_header->info.version); swab32s(&swap_header->info.last_page); From patchwork Thu Jul 1 01:53:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB9E4C11F66 for ; Thu, 1 Jul 2021 01:53:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7BE2B61468 for ; Thu, 1 Jul 2021 01:53:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7BE2B61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F3E6A8D0246; Wed, 30 Jun 2021 21:53:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F14D68D0236; Wed, 30 Jun 2021 21:53:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDEBA8D0246; Wed, 30 Jun 2021 21:53:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0006.hostedemail.com [216.40.44.6]) by kanga.kvack.org (Postfix) with ESMTP id B8C258D0236 for ; Wed, 30 Jun 2021 21:53:21 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8F4F2804163D for ; Thu, 1 Jul 2021 01:53:21 +0000 (UTC) X-FDA: 78312346602.28.B3ADEC0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 5B778B0000A0 for ; Thu, 1 Jul 2021 01:53:21 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 52EDF61476; Thu, 1 Jul 2021 01:53:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104400; bh=0+k8hs9ibwYdtLOuFHRuoU7n23xeI1sRZIOL6RVMXtg=; h=Date:From:To:Subject:In-Reply-To:From; b=Eyyog4ffAXg92KptTjfNsbZLC+0t8l2/2i6vd887hhbGzB3bdhX9k1ZgZ3XEr6Em7 bAl1KbZIBrhmDb2ym9MsA3Y9mWzXIWWGvaCfv03OpLr0XXKoq1IEZKzPqFm6AVN8sW tsN2oshs2/P0YZkiQuUQSrIAa2JQmrVZDmLVb5Z4= Date: Wed, 30 Jun 2021 18:53:19 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 115/192] mm/vmscan: remove kerneldoc-like comment from isolate_lru_pages Message-ID: <20210701015319.iIB49_A3s%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 5B778B0000A0 Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Eyyog4ff; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: hnt9n7ytta174jh1oadcax4zm1cfk4a7 X-HE-Tag: 1625104401-805318 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/vmscan: remove kerneldoc-like comment from isolate_lru_pages Patch series "Clean W=1 build warnings for mm/". This is a janitorial only. During development of a tool to catch build warnings early to avoid tripping the Intel lkp-robot, I noticed that mm/ is not clean for W=1. This is generally harmless but there is no harm in cleaning it up. It disrupts git blame a little but on relatively obvious lines that are unlikely to be git blame targets. This patch (of 13): make W=1 generates the following warning for vmscan.c mm/vmscan.c:1814: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst It is not a kerneldoc comment and isolate_lru_pages() is a static function. While the detailed comment is nice, it does not need to be exposed via kernel-doc. Link: https://lkml.kernel.org/r/20210520084809.8576-1-mgorman@techsingularity.net Link: https://lkml.kernel.org/r/20210520084809.8576-2-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Michal Hocko Cc: David Hildenbrand Cc: Dan Streetman Signed-off-by: Andrew Morton --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/vmscan.c~mm-vmscan-remove-kerneldoc-like-comment-from-isolate_lru_pages +++ a/mm/vmscan.c @@ -1821,7 +1821,7 @@ static __always_inline void update_lru_s } -/** +/* * Isolating page from the lruvec to fill in @dst list by nr_to_scan times. * * lruvec->lru_lock is heavily contended. Some of the functions that From patchwork Thu Jul 1 01:53:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9587C11F68 for ; Thu, 1 Jul 2021 01:53:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 839CB61241 for ; Thu, 1 Jul 2021 01:53:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 839CB61241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0BF6A8D0247; Wed, 30 Jun 2021 21:53:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 096BD8D0236; Wed, 30 Jun 2021 21:53:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA0718D0247; Wed, 30 Jun 2021 21:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id C4EF98D0236 for ; Wed, 30 Jun 2021 21:53:24 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A027E20BC4 for ; Thu, 1 Jul 2021 01:53:24 +0000 (UTC) X-FDA: 78312346728.10.D897142 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP id 56FF2B000378 for ; Thu, 1 Jul 2021 01:53:24 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5630561476; Thu, 1 Jul 2021 01:53:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104403; bh=eBroULm2pAEqTjm9JDywaTz26OAWVjL6idC4njnk0C0=; h=Date:From:To:Subject:In-Reply-To:From; b=VaAfunb0EalMhB8Rxg9uHfFKztnl83LcGy450u7uvWNK5jtYcuIkV6mYyLtzBXKAA UKv1pwh9yFsOzkSWoqQXGfglGjRx0RK/Imdd+WLntIHBgOii7IIALbHFPoBtMW2BjF ZOD1n6txUdiJkM9bFzaKIA6C2O6//cbmxZGvaQCc= Date: Wed, 30 Jun 2021 18:53:23 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 116/192] mm/vmalloc: include header for prototype of set_iounmap_nonlazy Message-ID: <20210701015323.jW-xhsvCw%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 56FF2B000378 X-Stat-Signature: 3dzuwmr6j5ips1unk6hh3rkk9p14hyxm Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=VaAfunb0; dmarc=none; spf=pass (imf25.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104404-934484 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/vmalloc: include header for prototype of set_iounmap_nonlazy make W=1 generates the following warning for mm/vmalloc.c mm/vmalloc.c:1599:6: warning: no previous prototype for `set_iounmap_nonlazy' [-Wmissing-prototypes] void set_iounmap_nonlazy(void) ^~~~~~~~~~~~~~~~~~~ This is an arch-generic function only used by x86. On other arches, it's dead code. Include the header with the definition and make it x86-64 specific. Link: https://lkml.kernel.org/r/20210520084809.8576-3-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/vmalloc.c | 3 +++ 1 file changed, 3 insertions(+) --- a/mm/vmalloc.c~mm-vmalloc-include-header-for-prototype-of-set_iounmap_nonlazy +++ a/mm/vmalloc.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include #include @@ -1607,6 +1608,7 @@ static DEFINE_MUTEX(vmap_purge_lock); /* for per-CPU blocks */ static void purge_fragmented_blocks_allcpus(void); +#ifdef CONFIG_X86_64 /* * called before a call to iounmap() if the caller wants vm_area_struct's * immediately freed. @@ -1615,6 +1617,7 @@ void set_iounmap_nonlazy(void) { atomic_long_set(&vmap_lazy_nr, lazy_max_pages()+1); } +#endif /* CONFIG_X86_64 */ /* * Purges all lazily-freed vmap areas. From patchwork Thu Jul 1 01:53:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9EA1C11F69 for ; Thu, 1 Jul 2021 01:53:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 811F361241 for ; Thu, 1 Jul 2021 01:53:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 811F361241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0987C8D0248; Wed, 30 Jun 2021 21:53:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 06F9E8D0236; Wed, 30 Jun 2021 21:53:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E79EC8D0248; Wed, 30 Jun 2021 21:53:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id BFAAF8D0236 for ; Wed, 30 Jun 2021 21:53:27 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9C9E81AEE7 for ; Thu, 1 Jul 2021 01:53:27 +0000 (UTC) X-FDA: 78312346854.08.91510B5 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 5F5AAF0000AC for ; Thu, 1 Jul 2021 01:53:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 563866105A; Thu, 1 Jul 2021 01:53:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104406; bh=75MpHzPBdLRs4LUJDA84iDhYm7fUQ5Jxo0mDUgQnjgI=; h=Date:From:To:Subject:In-Reply-To:From; b=bzsy12g79gC8d0yE3P6He3dBT8+a9wpYaPTVjsO5F2Y2RrAqYr/bVrN1GQ6u/O6QS 0IPvOMlMKTypF2AIDBmJfThSGEAUr/IyrHvYnG3IDi01F+YiG/GcZXqp6izS8Ue778 EsqW6oVoPZAjTKhkAY5+YzUssIfF0TeXEsN16dFk= Date: Wed, 30 Jun 2021 18:53:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 117/192] mm/page_alloc: make should_fail_alloc_page() static Message-ID: <20210701015325.rUmsTZN2H%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=bzsy12g7; dmarc=none; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: fxgd4dxds98yknjmffthkt3ij5n6tneu X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5F5AAF0000AC X-HE-Tag: 1625104407-36279 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/page_alloc: make should_fail_alloc_page() static make W=1 generates the following warning for mm/page_alloc.c mm/page_alloc.c:3651:15: warning: no previous prototype for `should_fail_alloc_page' [-Wmissing-prototypes] noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order) ^~~~~~~~~~~~~~~~~~~~~~ This function is deliberately split out for BPF to allow errors to be injected. The function is not used anywhere else so it is local to the file. Make it static which should still allow error injection to be used similar to how block/blk-core.c:should_fail_bio() works. Link: https://lkml.kernel.org/r/20210520084809.8576-4-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/page_alloc.c~mm-page_alloc-make-should_fail_alloc_page-a-static-function-should_fail_alloc_page-static +++ a/mm/page_alloc.c @@ -3819,7 +3819,7 @@ static inline bool __should_fail_alloc_p #endif /* CONFIG_FAIL_PAGE_ALLOC */ -noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order) +static noinline bool should_fail_alloc_page(gfp_t gfp_mask, unsigned int order) { return __should_fail_alloc_page(gfp_mask, order); } From patchwork Thu Jul 1 01:53:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D11E3C11F69 for ; Thu, 1 Jul 2021 01:53:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 803096105A for ; Thu, 1 Jul 2021 01:53:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 803096105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 097F98D0249; Wed, 30 Jun 2021 21:53:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 06F7A8D0236; Wed, 30 Jun 2021 21:53:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9FB08D0249; Wed, 30 Jun 2021 21:53:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220]) by kanga.kvack.org (Postfix) with ESMTP id C47228D0236 for ; Wed, 30 Jun 2021 21:53:30 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9EDC518017888 for ; Thu, 1 Jul 2021 01:53:30 +0000 (UTC) X-FDA: 78312346980.01.4F5DE52 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 579591984 for ; Thu, 1 Jul 2021 01:53:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5E2E861468; Thu, 1 Jul 2021 01:53:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104409; bh=WPR0A5ldhcr/Rg+H/hGL/pWm5PGPcZZqm+EdFgccASo=; h=Date:From:To:Subject:In-Reply-To:From; b=iWAvY5S30A1DSuNvCDeE89n0bf9nnZ7MR8ABqZerBALHNuP7M0iIXq2As79bqx8XH FwVJ9fts9mbP5WZVyhuvycCLU3LtGUhayhMg1DmADVIwQi9pRT3U2cZdj1QJjfPaWk KkrwtzR52SP1PLFTHK68QkD1dMfkhrOJucv42HM4= Date: Wed, 30 Jun 2021 18:53:29 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 118/192] mm/mapping_dirty_helpers: remove double Note in kerneldoc Message-ID: <20210701015329.whOMTfvhF%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=iWAvY5S3; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: fz7w1cgg9gppgbzxuswjs3z5puk9kui6 X-Rspamd-Queue-Id: 579591984 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104410-147946 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/mapping_dirty_helpers: remove double Note in kerneldoc make W=1 generates the following warning for mm/mapping_dirty_helpers.c mm/mapping_dirty_helpers.c:325: warning: duplicate section name 'Note' The helper function is very specific to one driver -- vmwgfx. While the two notes are separate, all of it needs to be taken into account when using the helper so make it one note. Link: https://lkml.kernel.org/r/20210520084809.8576-5-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/mapping_dirty_helpers.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/mapping_dirty_helpers.c~mm-mapping_dirty_helpers-remove-double-note-in-kerneldoc +++ a/mm/mapping_dirty_helpers.c @@ -317,7 +317,7 @@ EXPORT_SYMBOL_GPL(wp_shared_mapping_rang * pfn_mkwrite(). And then after a TLB flush following the write-protection * pick up all dirty bits. * - * Note: This function currently skips transhuge page-table entries, since + * This function currently skips transhuge page-table entries, since * it's intended for dirty-tracking on the PTE level. It will warn on * encountering transhuge dirty entries, though, and can easily be extended * to handle them as well. From patchwork Thu Jul 1 01:53:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0882AC11F66 for ; Thu, 1 Jul 2021 01:53:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C0E3B61468 for ; Thu, 1 Jul 2021 01:53:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0E3B61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3CC008D024A; Wed, 30 Jun 2021 21:53:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 37C578D0236; Wed, 30 Jun 2021 21:53:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26B4C8D024A; Wed, 30 Jun 2021 21:53:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0160.hostedemail.com [216.40.44.160]) by kanga.kvack.org (Postfix) with ESMTP id F0FFB8D0236 for ; Wed, 30 Jun 2021 21:53:33 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id CACB5181AEF3F for ; Thu, 1 Jul 2021 01:53:33 +0000 (UTC) X-FDA: 78312347106.11.CE4DC51 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP id 86425B000376 for ; Thu, 1 Jul 2021 01:53:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6DA646105A; Thu, 1 Jul 2021 01:53:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104412; bh=fuMkmBU7yPyKzy8r5TqtczMmTrAuGmdg+ZdJKBaeL9k=; h=Date:From:To:Subject:In-Reply-To:From; b=kvwO2WjwX4rRYPUkahrJXxx80rd2ZTrw2D5iIOIRUw63ZEyey6ReEy7nLZUoEMd7W M1wTBroDVmNun+cQf/yrIPvZ1ycivmV0j1vrBRK3fIIxM5pv0hgqjbdW4nlQF41mZB o+duBxWVQKlCdqaf1xpJ2NAUYJDPvM+2CNUPiPg0= Date: Wed, 30 Jun 2021 18:53:32 -0700 From: Andrew Morton To: akpm@linux-foundation.org, chris@chrisdown.name, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 119/192] mm/memcontrol.c: fix kerneldoc comment for mem_cgroup_calculate_protection Message-ID: <20210701015332.GbAUI-EaM%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=kvwO2Wjw; spf=pass (imf25.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: t1jdywocthpqyjzirxmmur3b98izs4zr X-Rspamd-Queue-Id: 86425B000376 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104413-508303 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/memcontrol.c: fix kerneldoc comment for mem_cgroup_calculate_protection make W=1 generates the following warning for mem_cgroup_calculate_protection mm/memcontrol.c:6468: warning: expecting prototype for mem_cgroup_protected(). Prototype was for mem_cgroup_calculate_protection() instead Commit 45c7f7e1ef17 ("mm, memcg: decouple e{low,min} state mutations from protection checks") changed the function definition but not the associated kerneldoc comment. Link: https://lkml.kernel.org/r/20210520084809.8576-7-mgorman@techsingularity.net Fixes: 45c7f7e1ef17 ("mm, memcg: decouple e{low,min} state mutations from protection checks") Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Chris Down Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memcontrol.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/memcontrol.c~mm-memcontrolc-fix-kerneldoc-comment-for-mem_cgroup_calculate_protection +++ a/mm/memcontrol.c @@ -6639,7 +6639,7 @@ static unsigned long effective_protectio } /** - * mem_cgroup_protected - check if memory consumption is in the normal range + * mem_cgroup_calculate_protection - check if memory consumption is in the normal range * @root: the top ancestor of the sub-tree being checked * @memcg: the memory cgroup to check * From patchwork Thu Jul 1 01:53:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7786DC11F69 for ; Thu, 1 Jul 2021 01:53:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3116761241 for ; Thu, 1 Jul 2021 01:53:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3116761241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AD0C08D024B; Wed, 30 Jun 2021 21:53:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A80868D0236; Wed, 30 Jun 2021 21:53:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 948958D024B; Wed, 30 Jun 2021 21:53:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0191.hostedemail.com [216.40.44.191]) by kanga.kvack.org (Postfix) with ESMTP id 6F84F8D0236 for ; Wed, 30 Jun 2021 21:53:37 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CDF3B22C15 for ; Thu, 1 Jul 2021 01:53:36 +0000 (UTC) X-FDA: 78312347232.23.6B71135 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 811E730002A8 for ; Thu, 1 Jul 2021 01:53:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7A4B66146D; Thu, 1 Jul 2021 01:53:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104415; bh=EpZ92Ktg0wuV7IHC93KuUtlQENqr3fTNHuENEJu4dIA=; h=Date:From:To:Subject:In-Reply-To:From; b=IGlh+Cbp0lIVr27Aqdqag3YqXI4cAxHKatvKUx5FuKy8YOJ/mveAM8G7QZkFmoMYj esVFqPJQARFGGQSnuvcKyfDFDB1CljOGawC/rx2PtEBY1doPZomjgPaxN93J+GgATq ShN+H8WJwgx6RA7n/7ouOIABBDhpNoUnaMrwm96E= Date: Wed, 30 Jun 2021 18:53:35 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 120/192] mm/memory_hotplug: fix kerneldoc comment for __try_online_node Message-ID: <20210701015335.Zrg-ltgp7%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=IGlh+Cbp; dmarc=none; spf=pass (imf09.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: 49tq61comdj5tt4m33sw9x79zsjohta1 X-Rspamd-Queue-Id: 811E730002A8 X-HE-Tag: 1625104416-22142 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/memory_hotplug: fix kerneldoc comment for __try_online_node make W=1 generates the following warning for try_online_node mm/memory_hotplug.c:1087: warning: expecting prototype for try_online_node(). Prototype was for __try_online_node() instead Commit b9ff036082cd ("mm/memory_hotplug.c: make add_memory_resource use __try_online_node") renamed the function but did not update the associated kerneldoc. The function is static and somewhat specialised in nature so it's not clear it warrants being a kerneldoc by moving the comment to try_online_node. Hence, leave the comment of the internal helper in place but leave it out of kerneldoc and correct the function name in the comment. Link: https://lkml.kernel.org/r/20210520084809.8576-8-mgorman@techsingularity.net Fixes: Commit b9ff036082cd ("mm/memory_hotplug.c: make add_memory_resource use __try_online_node") Signed-off-by: Mel Gorman Reviewed-by: David Hildenbrand Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/memory_hotplug.c~mm-memory_hotplug-fix-kerneldoc-comment-for-__try_online_node +++ a/mm/memory_hotplug.c @@ -942,8 +942,8 @@ static void rollback_node_hotadd(int nid } -/** - * try_online_node - online a node if offlined +/* + * __try_online_node - online a node if offlined * @nid: the node ID * @set_node_online: Whether we want to online the node * called by cpu_up() to online a node without onlined memory. From patchwork Thu Jul 1 01:53:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2903FC11F68 for ; Thu, 1 Jul 2021 01:53:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C9CFE6105A for ; Thu, 1 Jul 2021 01:53:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9CFE6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4412C8D024C; Wed, 30 Jun 2021 21:53:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CA778D0236; Wed, 30 Jun 2021 21:53:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26AC18D024C; Wed, 30 Jun 2021 21:53:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 01C278D0236 for ; Wed, 30 Jun 2021 21:53:39 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D0F03824C42F for ; Thu, 1 Jul 2021 01:53:39 +0000 (UTC) X-FDA: 78312347358.24.640D388 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 7BF1F1981 for ; Thu, 1 Jul 2021 01:53:39 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8324761468; Thu, 1 Jul 2021 01:53:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104418; bh=nHpuFln55P7Tu52NwndGlGtNQfEQEfrO2nZ0woL7Hew=; h=Date:From:To:Subject:In-Reply-To:From; b=cuPfYgdzjpfbwz7Hqceor22TabrAhdIY4bex/8QLA9uVGsw9vNpFMeV4aBKZs00EB roTuPiMuzYdzJTNgfGXUrSb90z0z/Fl6XaVXmj4wBt28odE1XvLt+MnVdPfx0PYk7p pUEnfo/lUx7IMugGW1UrTepHj/brhfBNv/dbJoDA= Date: Wed, 30 Jun 2021 18:53:38 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 121/192] mm/memory_hotplug: fix kerneldoc comment for __remove_memory Message-ID: <20210701015338.pSbMy1HM9%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=cuPfYgdz; dmarc=none; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: z35pofjeizdosodc65s1o5a8npubk34f X-Rspamd-Queue-Id: 7BF1F1981 X-HE-Tag: 1625104419-400364 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/memory_hotplug: fix kerneldoc comment for __remove_memory make W=1 generates the following warning for __remove_memory mm/memory_hotplug.c:2044: warning: expecting prototype for remove_memory(). Prototype was for __remove_memory() instead Commit eca499ab3749 ("mm/hotplug: make remove_memory() interface usable") introduced the kerneldoc comment and function but the kerneldoc name and function name did not match. Link: https://lkml.kernel.org/r/20210520084809.8576-9-mgorman@techsingularity.net Fixes: eca499ab3749 ("mm/hotplug: make remove_memory() interface usable") Signed-off-by: Mel Gorman Reviewed-by: David Hildenbrand Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/memory_hotplug.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/memory_hotplug.c~mm-memory_hotplug-fix-kerneldoc-comment-for-__remove_memory +++ a/mm/memory_hotplug.c @@ -1908,7 +1908,7 @@ static int __ref try_remove_memory(int n } /** - * remove_memory + * __remove_memory - Remove memory if every memory block is offline * @nid: the node ID * @start: physical address of the region to remove * @size: size of the region to remove From patchwork Thu Jul 1 01:53:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22925C11F68 for ; Thu, 1 Jul 2021 01:53:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CD1B961468 for ; Thu, 1 Jul 2021 01:53:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CD1B961468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 57AB28D024D; Wed, 30 Jun 2021 21:53:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 502648D0236; Wed, 30 Jun 2021 21:53:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37C808D024D; Wed, 30 Jun 2021 21:53:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 149CE8D0236 for ; Wed, 30 Jun 2021 21:53:43 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DBF1C824C42F for ; Thu, 1 Jul 2021 01:53:42 +0000 (UTC) X-FDA: 78312347484.03.E578DF2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 989F710000A3 for ; Thu, 1 Jul 2021 01:53:42 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9485261469; Thu, 1 Jul 2021 01:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104421; bh=cj7TYLKQL7pFq+fI+egyyyGA5+4BIr7/uLEGZrlzhNw=; h=Date:From:To:Subject:In-Reply-To:From; b=gCikUNVTldA/pmKcUVSdMGIiO/gowAGJQTDCBkKqR73DEBrg8xKxRQEqB/8qndTQJ 9/0JAr1q2N21IOaCO1qcXI04mgLpW7WH+ZsUfhxp8aNcAajixWElpQ+6ewMsmZpRPI 2tyCKE/BomItaybyK9mxaodOxLj+KxxKiw5TuAJc= Date: Wed, 30 Jun 2021 18:53:41 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 122/192] mm/zbud: add kerneldoc fields for zbud_pool Message-ID: <20210701015341.LKJP3M0fn%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=gCikUNVT; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 989F710000A3 X-Stat-Signature: uhkc8rgb3wxy4u8r3ayjqo5oq5p3ry3m X-HE-Tag: 1625104422-992326 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/zbud: add kerneldoc fields for zbud_pool make W=1 generates the following warning for zbud_pool mm/zbud.c:105: warning: Function parameter or member 'zpool' not described in 'zbud_pool' mm/zbud.c:105: warning: Function parameter or member 'zpool_ops' not described in 'zbud_pool' Commit 479305fd7172 ("zpool: remove zpool_evict()") removed the zpool_evict helper and added the associated zpool and operations structure in struct zbud_pool but did not add documentation for the fields. Add rudimentary documentation. Link: https://lkml.kernel.org/r/20210520084809.8576-10-mgorman@techsingularity.net Fixes: 479305fd7172 ("zpool: remove zpool_evict()") Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/zbud.c | 2 ++ 1 file changed, 2 insertions(+) --- a/mm/zbud.c~mm-zbud-add-kerneldoc-fields-for-zbud_pool +++ a/mm/zbud.c @@ -92,6 +92,8 @@ struct zbud_ops { * @pages_nr: number of zbud pages in the pool. * @ops: pointer to a structure of user defined operations specified at * pool creation time. + * @zpool: zpool driver + * @zpool_ops: zpool operations structure with an evict callback * * This structure is allocated at pool creation time and maintains metadata * pertaining to a particular zbud pool. From patchwork Thu Jul 1 01:53:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353303 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E6FCC11F69 for ; Thu, 1 Jul 2021 01:53:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E3C4C6105A for ; Thu, 1 Jul 2021 01:53:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E3C4C6105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6D98D8D024E; Wed, 30 Jun 2021 21:53:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63B208D0236; Wed, 30 Jun 2021 21:53:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B4AE8D024E; Wed, 30 Jun 2021 21:53:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id 258FE8D0236 for ; Wed, 30 Jun 2021 21:53:46 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id EDC1A180260DE for ; Thu, 1 Jul 2021 01:53:45 +0000 (UTC) X-FDA: 78312347610.16.AD745AC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id A9584D000081 for ; Thu, 1 Jul 2021 01:53:45 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9E9DD61468; Thu, 1 Jul 2021 01:53:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104424; bh=A4GM+AdHH/VL3TLMEqI+tQ7OFTv2WyhwXCiTcnfvg3E=; h=Date:From:To:Subject:In-Reply-To:From; b=cGCdFp0vEPjSulc60KjauW3MmQPdfReiv2Df5uWr+7BJtcEaZ5bMu14armPU9wrJS 1MFDKBd0XLs89iUb0Q6Z3lGcidYsjkwiVesbtKb3cfdh+QmT+FU5bLxpnqPv3yJmcx qPgGrMOpg/Hah3n40tDOBEAJ5aPJ7JGkMo9Fsp6I= Date: Wed, 30 Jun 2021 18:53:44 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 123/192] mm/z3fold: add kerneldoc fields for z3fold_pool Message-ID: <20210701015344.aNl5evPur%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A9584D000081 X-Stat-Signature: tyjf8819se5zkmu5t7jbksc8kwxnt449 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=cGCdFp0v; dmarc=none; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104425-16361 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/z3fold: add kerneldoc fields for z3fold_pool make W=1 generates the following warning for z3fold_pool mm/z3fold.c:171: warning: Function parameter or member 'zpool' not described in 'z3fold_pool' mm/z3fold.c:171: warning: Function parameter or member 'zpool_ops' not described in 'z3fold_pool' Commit 9a001fc19ccc ("z3fold: the 3-fold allocator for compressed pages") simply did not document the fields at the time. Add rudimentary documentation. Link: https://lkml.kernel.org/r/20210520084809.8576-11-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/z3fold.c | 2 ++ 1 file changed, 2 insertions(+) --- a/mm/z3fold.c~mm-z3fold-add-kerneldoc-fields-for-z3fold_pool +++ a/mm/z3fold.c @@ -144,6 +144,8 @@ struct z3fold_header { * @c_handle: cache for z3fold_buddy_slots allocation * @ops: pointer to a structure of user defined operations specified at * pool creation time. + * @zpool: zpool driver + * @zpool_ops: zpool operations structure with an evict callback * @compact_wq: workqueue for page layout background optimization * @release_wq: workqueue for safe page release * @work: work_struct for safe page release From patchwork Thu Jul 1 01:53:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353305 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49171C11F69 for ; Thu, 1 Jul 2021 01:53:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EBC3B61469 for ; Thu, 1 Jul 2021 01:53:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EBC3B61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6D9558D0236; Wed, 30 Jun 2021 21:53:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 63B8C8D024F; Wed, 30 Jun 2021 21:53:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EAD48D0236; Wed, 30 Jun 2021 21:53:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0001.hostedemail.com [216.40.44.1]) by kanga.kvack.org (Postfix) with ESMTP id 15F938D0236 for ; Wed, 30 Jun 2021 21:53:49 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DB9E2253D3 for ; Thu, 1 Jul 2021 01:53:48 +0000 (UTC) X-FDA: 78312347736.26.A649812 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 9595930000A5 for ; Thu, 1 Jul 2021 01:53:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A0D396105A; Thu, 1 Jul 2021 01:53:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104427; bh=/3fOwR+y7gNIuC95wBosbKEfM6/BZOtJo/SRA+Distg=; h=Date:From:To:Subject:In-Reply-To:From; b=MADWIe6EXIEsLt+lq2w53ABYRlu2L8mP0DSTcAmndwe65dBqxMLCWpMt7d0+wFrwD DnuvJyprZlSTyR3OlHsanlr6h7VD7Q26wjPU7mLVHgjbxxQR6EuG+GosomB/GaQlet 7eDcAcciAjNRif5LUFBgtd3IbtLssaYtmBQHmr1E= Date: Wed, 30 Jun 2021 18:53:47 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 124/192] mm/swap: make swap_address_space an inline function Message-ID: <20210701015347.bJuHZiNgP%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 9595930000A5 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=MADWIe6E; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: dtxyfc8bxmyqwcanhn3r5bm8959p6hkg X-HE-Tag: 1625104428-688759 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/swap: make swap_address_space an inline function make W=1 generates the following warning in page_mapping() for allnoconfig mm/util.c:700:15: warning: variable `entry' set but not used [-Wunused-but-set-variable] swp_entry_t entry; ^~~~~ swap_address is a #define on !CONFIG_SWAP configurations. Make the helper an inline function to suppress the warning, add type checking and to apply any side-effects in the parameter list. Link: https://lkml.kernel.org/r/20210520084809.8576-12-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/swap.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) --- a/include/linux/swap.h~mm-swap-make-swap_address_space-an-inline-function +++ a/include/linux/swap.h @@ -537,7 +537,11 @@ static inline void put_swap_device(struc { } -#define swap_address_space(entry) (NULL) +static inline struct address_space *swap_address_space(swp_entry_t entry) +{ + return NULL; +} + #define get_nr_swap_pages() 0L #define total_swap_pages 0L #define total_swapcache_pages() 0UL From patchwork Thu Jul 1 01:53:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353307 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F60DC11F69 for ; Thu, 1 Jul 2021 01:53:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 179A661468 for ; Thu, 1 Jul 2021 01:53:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 179A661468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8FB8D8D0250; Wed, 30 Jun 2021 21:53:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 884998D024F; Wed, 30 Jun 2021 21:53:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FEBE8D0250; Wed, 30 Jun 2021 21:53:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id 4181F8D024F for ; Wed, 30 Jun 2021 21:53:52 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1FA1E180206F4 for ; Thu, 1 Jul 2021 01:53:52 +0000 (UTC) X-FDA: 78312347904.30.39558D2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id BD652F0000AD for ; Thu, 1 Jul 2021 01:53:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B19706105A; Thu, 1 Jul 2021 01:53:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104431; bh=bVIvW+JLqWb4sQiz+oLgqfnUcl0MzXt8nuqEgyFuraw=; h=Date:From:To:Subject:In-Reply-To:From; b=Fpegt29Lw33bUM+pZSYkP6wq8SAznOgPxMkG3Ukx6gvkUVflnOERCsktZKw1YWjHq BOoGi8oKc3sfOajnowxsn+Eux1xZtHQjzsviLn596dTNavy7MC02kx+zjqdwZ7b8iz 192azhzHjW0XnfE9um7qg9gCJcdiqXxnGuPZ/jkk= Date: Wed, 30 Jun 2021 18:53:50 -0700 From: Andrew Morton To: akpm@linux-foundation.org, cuibixuan@huawei.com, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 125/192] mm/mmap_lock: remove dead code for !CONFIG_TRACING configurations Message-ID: <20210701015350.3lXiOCAeh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Fpegt29L; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: yj4jxwirha84h4wpbwxjngi76m1dqcnx X-Rspamd-Queue-Id: BD652F0000AD X-Rspamd-Server: rspam06 X-HE-Tag: 1625104431-996491 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/mmap_lock: remove dead code for !CONFIG_TRACING configurations make W=1 generates the following warning in mmap_lock.c for allnoconfig mm/mmap_lock.c:213:6: warning: no previous prototype for `__mmap_lock_do_trace_start_locking' [-Wmissing-prototypes] void __mmap_lock_do_trace_start_locking(struct mm_struct *mm, bool write) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ mm/mmap_lock.c:219:6: warning: no previous prototype for `__mmap_lock_do_trace_acquire_returned' [-Wmissing-prototypes] void __mmap_lock_do_trace_acquire_returned(struct mm_struct *mm, bool write, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ mm/mmap_lock.c:226:6: warning: no previous prototype for `__mmap_lock_do_trace_released' [-Wmissing-prototypes] void __mmap_lock_do_trace_released(struct mm_struct *mm, bool write) On !CONFIG_TRACING configurations, the code is dead so put it behind an #ifdef. [cuibixuan@huawei.com: fix warning when CONFIG_TRACING is not defined] Link: https://lkml.kernel.org/r/20210531033426.74031-1-cuibixuan@huawei.com Link: https://lkml.kernel.org/r/20210520084809.8576-13-mgorman@techsingularity.net Signed-off-by: Mel Gorman Signed-off-by: Bixuan Cui Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/mmap_lock.c | 59 +++++++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 27 deletions(-) --- a/mm/mmap_lock.c~mm-mmap_lock-remove-dead-code-for-config_tracing-configurations +++ a/mm/mmap_lock.c @@ -153,6 +153,37 @@ static inline void put_memcg_path_buf(vo rcu_read_unlock(); } +#define TRACE_MMAP_LOCK_EVENT(type, mm, ...) \ + do { \ + const char *memcg_path; \ + preempt_disable(); \ + memcg_path = get_mm_memcg_path(mm); \ + trace_mmap_lock_##type(mm, \ + memcg_path != NULL ? memcg_path : "", \ + ##__VA_ARGS__); \ + if (likely(memcg_path != NULL)) \ + put_memcg_path_buf(); \ + preempt_enable(); \ + } while (0) + +#else /* !CONFIG_MEMCG */ + +int trace_mmap_lock_reg(void) +{ + return 0; +} + +void trace_mmap_lock_unreg(void) +{ +} + +#define TRACE_MMAP_LOCK_EVENT(type, mm, ...) \ + trace_mmap_lock_##type(mm, "", ##__VA_ARGS__) + +#endif /* CONFIG_MEMCG */ + +#ifdef CONFIG_TRACING +#ifdef CONFIG_MEMCG /* * Write the given mm_struct's memcg path to a percpu buffer, and return a * pointer to it. If the path cannot be determined, or no buffer was available @@ -187,33 +218,6 @@ out: return buf; } -#define TRACE_MMAP_LOCK_EVENT(type, mm, ...) \ - do { \ - const char *memcg_path; \ - local_lock(&memcg_paths.lock); \ - memcg_path = get_mm_memcg_path(mm); \ - trace_mmap_lock_##type(mm, \ - memcg_path != NULL ? memcg_path : "", \ - ##__VA_ARGS__); \ - if (likely(memcg_path != NULL)) \ - put_memcg_path_buf(); \ - local_unlock(&memcg_paths.lock); \ - } while (0) - -#else /* !CONFIG_MEMCG */ - -int trace_mmap_lock_reg(void) -{ - return 0; -} - -void trace_mmap_lock_unreg(void) -{ -} - -#define TRACE_MMAP_LOCK_EVENT(type, mm, ...) \ - trace_mmap_lock_##type(mm, "", ##__VA_ARGS__) - #endif /* CONFIG_MEMCG */ /* @@ -239,3 +243,4 @@ void __mmap_lock_do_trace_released(struc TRACE_MMAP_LOCK_EVENT(released, mm, write); } EXPORT_SYMBOL(__mmap_lock_do_trace_released); +#endif /* CONFIG_TRACING */ From patchwork Thu Jul 1 01:53:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353309 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46389C11F69 for ; Thu, 1 Jul 2021 01:53:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F096661468 for ; Thu, 1 Jul 2021 01:53:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F096661468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 79D138D0251; Wed, 30 Jun 2021 21:53:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 74C038D024F; Wed, 30 Jun 2021 21:53:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5ED998D0251; Wed, 30 Jun 2021 21:53:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id 341B38D024F for ; Wed, 30 Jun 2021 21:53:55 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 081B9824C430 for ; Thu, 1 Jul 2021 01:53:55 +0000 (UTC) X-FDA: 78312348030.10.4D95D12 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id B87CA400208C for ; Thu, 1 Jul 2021 01:53:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BCC0261241; Thu, 1 Jul 2021 01:53:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104434; bh=yr/BES2r1OLP3gTpr5e8JAFHenZo65BslROVqtKgAIo=; h=Date:From:To:Subject:In-Reply-To:From; b=MqrjW8MocjoeMMezmi1b08j7SUYL0QICceE5ifjlTitE6h+wY1eCTTayCo324WfZi j9I5LNx8eRfXw1Z7+qhap03gx9ehYJXeqjjO/Ww719e3WHSz+4NPgdxhwWXfvdUCnK +KW0jjc8hMbaonW3wu5AiyLHqXtn2bQ+mjQdHptM= Date: Wed, 30 Jun 2021 18:53:53 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 126/192] mm/page_alloc: move prototype for find_suitable_fallback Message-ID: <20210701015353.P4XRjHhzz%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=MqrjW8Mo; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: wpch6pw9d3dy3fbst1yydmeox6iwu7it X-Rspamd-Queue-Id: B87CA400208C X-Rspamd-Server: rspam06 X-HE-Tag: 1625104434-256896 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/page_alloc: move prototype for find_suitable_fallback make W=1 generates the following warning in mmap_lock.c for allnoconfig mm/page_alloc.c:2670:5: warning: no previous prototype for `find_suitable_fallback' [-Wmissing-prototypes] int find_suitable_fallback(struct free_area *area, unsigned int order, ^~~~~~~~~~~~~~~~~~~~~~ find_suitable_fallback is only shared outside of page_alloc.c for CONFIG_COMPACTION but to suppress the warning, move the protype outside of CONFIG_COMPACTION. It is not worth the effort at this time to find a clever way of allowing compaction.c to share the code or avoid the use entirely as the function is called on relatively slow paths. Link: https://lkml.kernel.org/r/20210520084809.8576-14-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- mm/internal.h | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/mm/internal.h~mm-page_alloc-move-prototype-for-find_suitable_fallback +++ a/mm/internal.h @@ -274,11 +274,10 @@ isolate_freepages_range(struct compact_c int isolate_migratepages_range(struct compact_control *cc, unsigned long low_pfn, unsigned long end_pfn); +#endif int find_suitable_fallback(struct free_area *area, unsigned int order, int migratetype, bool only_stealable, bool *can_steal); -#endif - /* * This function returns the order of a free page in the buddy system. In * general, page_zone(page)->lock must be held by the caller to prevent the From patchwork Thu Jul 1 01:53:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353311 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DCBAC11F68 for ; Thu, 1 Jul 2021 01:53:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 147F26105A for ; Thu, 1 Jul 2021 01:53:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 147F26105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8D4CA8D0252; Wed, 30 Jun 2021 21:53:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 884CD8D024F; Wed, 30 Jun 2021 21:53:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FE3D8D0252; Wed, 30 Jun 2021 21:53:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id 439F18D024F for ; Wed, 30 Jun 2021 21:53:58 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1E9D6181AEF3F for ; Thu, 1 Jul 2021 01:53:58 +0000 (UTC) X-FDA: 78312348156.11.2D804C2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id D470FB0000A9 for ; Thu, 1 Jul 2021 01:53:57 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C0FEF61469; Thu, 1 Jul 2021 01:53:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104437; bh=Pbko7oyjPaPRNbxQeB+l3Mi8dtWEQl4zEHPaJst9M70=; h=Date:From:To:Subject:In-Reply-To:From; b=0H3C1x8URxWWPjeLbiYwn5GzOP+tD743k0zCLuWS7eELMduHv9FAQ7a0FY5fjtRnY GOwO1w51HfxoA3yqrzaLpDeB7gINk2pcXcxkAKCr6JsSKeuodnQC2utKeR0KlB0yPu +71hR5A8UuMOxXdZVyafIxidMpD+9m09xP8JYnNg= Date: Wed, 30 Jun 2021 18:53:56 -0700 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, ddstreet@ieee.org, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, shy828301@gmail.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 127/192] mm/swap: make NODE_DATA an inline function on CONFIG_FLATMEM Message-ID: <20210701015356.i3nr-saLs%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=0H3C1x8U; spf=pass (imf19.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: bwmhbcinwupfdhmuah6zk7btif67dkzt X-Rspamd-Queue-Id: D470FB0000A9 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104437-920069 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Subject: mm/swap: make NODE_DATA an inline function on CONFIG_FLATMEM make W=1 generates the following warning in mm/workingset.c for allnoconfig mm/workingset.c: In function `unpack_shadow': mm/workingset.c:201:15: warning: variable `nid' set but not used [-Wunused-but-set-variable] int memcgid, nid; ^~~ On FLATMEM, NODE_DATA returns a global pglist_data without dereferencing nid. Make the helper an inline function to suppress the warning, add type checking and to apply any side-effects in the parameter list. Link: https://lkml.kernel.org/r/20210520084809.8576-15-mgorman@techsingularity.net Signed-off-by: Mel Gorman Reviewed-by: Yang Shi Acked-by: Vlastimil Babka Cc: Dan Streetman Cc: David Hildenbrand Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) --- a/include/linux/mmzone.h~mm-swap-make-node_data-an-inline-function-on-config_flatmem +++ a/include/linux/mmzone.h @@ -1064,7 +1064,10 @@ extern char numa_zonelist_order[]; #ifndef CONFIG_NUMA extern struct pglist_data contig_page_data; -#define NODE_DATA(nid) (&contig_page_data) +static inline struct pglist_data *NODE_DATA(int nid) +{ + return &contig_page_data; +} #define NODE_MEM_MAP(nid) mem_map #else /* CONFIG_NUMA */ From patchwork Thu Jul 1 01:53:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353313 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.0 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CD73C11F69 for ; Thu, 1 Jul 2021 01:54:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F40661468 for ; Thu, 1 Jul 2021 01:54:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F40661468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9998F8D0253; Wed, 30 Jun 2021 21:54:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 920EA8D024F; Wed, 30 Jun 2021 21:54:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7743E8D0253; Wed, 30 Jun 2021 21:54:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 470328D024F for ; Wed, 30 Jun 2021 21:54:02 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1B043181AEF3F for ; Thu, 1 Jul 2021 01:54:02 +0000 (UTC) X-FDA: 78312348324.05.25D7734 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP id B91FAB00037C for ; Thu, 1 Jul 2021 01:54:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 20B116105A; Thu, 1 Jul 2021 01:54:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104441; bh=N8GPMZIdUJaYDtDCOBWihsKK8QEbU2F3Oyipk4s62ig=; h=Date:From:To:Subject:In-Reply-To:From; b=Pjw17y9pcpBsMux0HBkGIlEie3X99DjgOoYe1dNYe13pqzD81L8QRqqNZkWTNRHy8 ZWFahswV7nrJ1kWLh/yJVv5V+avXSpBc4Kb6X/vD2vxtbHDJ92rJPCNRKu6ezvrs6g 1eaLjmazrpYDtvExQS/K/fTIhB1sQVjztaXRlm6E= Date: Wed, 30 Jun 2021 18:53:59 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, bcain@codeaurora.org, catalin.marinas@arm.com, chris@zankel.net, christophe.leroy@csgroup.eu, davem@davemloft.net, geert@linux-m68k.org, guoren@kernel.org, hca@linux.ibm.com, James.Bottomley@HansenPartnership.com, jdike@addtoit.com, jonas@southpole.se, ley.foon.tan@intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, monstr@monstr.eu, mpe@ellerman.id.au, nickhu@andestech.com, palmer@dabbelt.com, paul.walmsley@sifive.com, rppt@linux.ibm.com, rth@twiddle.net, shorne@gmail.com, stefan.kristiansson@saunalahti.fi, tglx@linutronix.de, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, vgupta@synopsys.com, will@kernel.org, ysato@users.sourceforge.jp Subject: [patch 128/192] mm/thp: define default pmd_pgtable() Message-ID: <20210701015359.Pny1e1JSH%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Pjw17y9p; dmarc=none; spf=pass (imf25.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: eszzuqz3wgmsgdwgzuq8t6sor5wd8xhs X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B91FAB00037C X-HE-Tag: 1625104441-854330 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm/thp: define default pmd_pgtable() Currently most platforms define pmd_pgtable() as pmd_page() duplicating the same code all over. Instead just define a default value i.e pmd_page() for pmd_pgtable() and let platforms override when required via . All the existing platform that override pmd_pgtable() have been moved into their respective header in order to precede before the new generic definition. This makes it much cleaner with reduced code. Link: https://lkml.kernel.org/r/1623646133-20306-1-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Acked-by: Geert Uytterhoeven Acked-by: Mike Rapoport Cc: Nick Hu Cc: Richard Henderson Cc: Vineet Gupta Cc: Catalin Marinas Cc: Will Deacon Cc: Guo Ren Cc: Brian Cain Cc: Geert Uytterhoeven Cc: Michal Simek Cc: Thomas Bogendoerfer Cc: Ley Foon Tan Cc: Jonas Bonn Cc: Stefan Kristiansson Cc: Stafford Horne Cc: "James E.J. Bottomley" Cc: Michael Ellerman Cc: Christophe Leroy Cc: Paul Walmsley Cc: Palmer Dabbelt Cc: Heiko Carstens Cc: Yoshinori Sato Cc: "David S. Miller" Cc: Jeff Dike Cc: Thomas Gleixner Cc: Chris Zankel Signed-off-by: Andrew Morton --- arch/alpha/include/asm/pgalloc.h | 1 - arch/arc/include/asm/pgalloc.h | 2 -- arch/arc/include/asm/pgtable.h | 2 ++ arch/arm/include/asm/pgalloc.h | 1 - arch/arm64/include/asm/pgalloc.h | 1 - arch/csky/include/asm/pgalloc.h | 2 -- arch/hexagon/include/asm/pgtable.h | 1 - arch/ia64/include/asm/pgalloc.h | 1 - arch/m68k/include/asm/mcf_pgalloc.h | 2 -- arch/m68k/include/asm/mcf_pgtable.h | 2 ++ arch/m68k/include/asm/motorola_pgalloc.h | 1 - arch/m68k/include/asm/motorola_pgtable.h | 2 ++ arch/m68k/include/asm/sun3_pgalloc.h | 1 - arch/microblaze/include/asm/pgalloc.h | 2 -- arch/mips/include/asm/pgalloc.h | 1 - arch/nds32/include/asm/pgalloc.h | 5 ----- arch/nios2/include/asm/pgalloc.h | 1 - arch/openrisc/include/asm/pgalloc.h | 2 -- arch/parisc/include/asm/pgalloc.h | 1 - arch/powerpc/include/asm/pgalloc.h | 5 ----- arch/powerpc/include/asm/pgtable.h | 6 ++++++ arch/riscv/include/asm/pgalloc.h | 2 -- arch/s390/include/asm/pgalloc.h | 3 --- arch/s390/include/asm/pgtable.h | 3 +++ arch/sh/include/asm/pgalloc.h | 1 - arch/sparc/include/asm/pgalloc_32.h | 1 - arch/sparc/include/asm/pgalloc_64.h | 1 - arch/sparc/include/asm/pgtable_32.h | 2 ++ arch/sparc/include/asm/pgtable_64.h | 2 ++ arch/um/include/asm/pgalloc.h | 1 - arch/x86/include/asm/pgalloc.h | 2 -- arch/xtensa/include/asm/pgalloc.h | 2 -- include/linux/pgtable.h | 9 +++++++++ 33 files changed, 28 insertions(+), 43 deletions(-) --- a/arch/alpha/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/alpha/include/asm/pgalloc.h @@ -18,7 +18,6 @@ pmd_populate(struct mm_struct *mm, pmd_t { pmd_set(pmd, (pte_t *)(page_to_pa(pte) + PAGE_OFFSET)); } -#define pmd_pgtable(pmd) pmd_page(pmd) static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte) --- a/arch/arc/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/arc/include/asm/pgalloc.h @@ -129,6 +129,4 @@ static inline void pte_free(struct mm_st #define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, pte) -#define pmd_pgtable(pmd) ((pgtable_t) pmd_page_vaddr(pmd)) - #endif /* _ASM_ARC_PGALLOC_H */ --- a/arch/arc/include/asm/pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/arch/arc/include/asm/pgtable.h @@ -350,6 +350,8 @@ void update_mmu_cache(struct vm_area_str #define kern_addr_valid(addr) (1) +#define pmd_pgtable(pmd) ((pgtable_t) pmd_page_vaddr(pmd)) + /* * remap a physical page `pfn' of size `size' with page protection `prot' * into virtual address `from' --- a/arch/arm64/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/arm64/include/asm/pgalloc.h @@ -86,6 +86,5 @@ pmd_populate(struct mm_struct *mm, pmd_t VM_BUG_ON(mm == &init_mm); __pmd_populate(pmdp, page_to_phys(ptep), PMD_TYPE_TABLE | PMD_TABLE_PXN); } -#define pmd_pgtable(pmd) pmd_page(pmd) #endif --- a/arch/arm/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/arm/include/asm/pgalloc.h @@ -143,7 +143,6 @@ pmd_populate(struct mm_struct *mm, pmd_t __pmd_populate(pmdp, page_to_phys(ptep), prot); } -#define pmd_pgtable(pmd) pmd_page(pmd) #endif /* CONFIG_MMU */ --- a/arch/csky/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/csky/include/asm/pgalloc.h @@ -22,8 +22,6 @@ static inline void pmd_populate(struct m set_pmd(pmd, __pmd(__pa(page_address(pte)))); } -#define pmd_pgtable(pmd) pmd_page(pmd) - extern void pgd_init(unsigned long *p); static inline pte_t *pte_alloc_one_kernel(struct mm_struct *mm) --- a/arch/hexagon/include/asm/pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/arch/hexagon/include/asm/pgtable.h @@ -239,7 +239,6 @@ static inline int pmd_bad(pmd_t pmd) * pmd_page - converts a PMD entry to a page pointer */ #define pmd_page(pmd) (pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT)) -#define pmd_pgtable(pmd) pmd_page(pmd) /** * pte_none - check if pte is mapped --- a/arch/ia64/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/ia64/include/asm/pgalloc.h @@ -52,7 +52,6 @@ pmd_populate(struct mm_struct *mm, pmd_t { pmd_val(*pmd_entry) = page_to_phys(pte); } -#define pmd_pgtable(pmd) pmd_page(pmd) static inline void pmd_populate_kernel(struct mm_struct *mm, pmd_t * pmd_entry, pte_t * pte) --- a/arch/m68k/include/asm/mcf_pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/m68k/include/asm/mcf_pgalloc.h @@ -32,8 +32,6 @@ extern inline pmd_t *pmd_alloc_kernel(pg #define pmd_populate_kernel pmd_populate -#define pmd_pgtable(pmd) pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT) - static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pgtable, unsigned long address) { --- a/arch/m68k/include/asm/mcf_pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/arch/m68k/include/asm/mcf_pgtable.h @@ -150,6 +150,8 @@ #ifndef __ASSEMBLY__ +#define pmd_pgtable(pmd) pfn_to_virt(pmd_val(pmd) >> PAGE_SHIFT) + /* * Conversion functions: convert a page and protection to a page entry, * and a page entry and page directory to the page they refer to. --- a/arch/m68k/include/asm/motorola_pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/m68k/include/asm/motorola_pgalloc.h @@ -88,7 +88,6 @@ static inline void pmd_populate(struct m { pmd_set(pmd, page); } -#define pmd_pgtable(pmd) ((pgtable_t)pmd_page_vaddr(pmd)) static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd) { --- a/arch/m68k/include/asm/motorola_pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/arch/m68k/include/asm/motorola_pgtable.h @@ -105,6 +105,8 @@ extern unsigned long mm_cachebits; #define __S110 PAGE_SHARED_C #define __S111 PAGE_SHARED_C +#define pmd_pgtable(pmd) ((pgtable_t)pmd_page_vaddr(pmd)) + /* * Conversion functions: convert a page and protection to a page entry, * and a page entry and page directory to the page they refer to. --- a/arch/m68k/include/asm/sun3_pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/m68k/include/asm/sun3_pgalloc.h @@ -32,7 +32,6 @@ static inline void pmd_populate(struct m { pmd_val(*pmd) = __pa((unsigned long)page_address(page)); } -#define pmd_pgtable(pmd) pmd_page(pmd) /* * allocating and freeing a pmd is trivial: the 1-entry pmd is --- a/arch/microblaze/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/microblaze/include/asm/pgalloc.h @@ -28,8 +28,6 @@ static inline pgd_t *get_pgd(void) #define pgd_alloc(mm) get_pgd() -#define pmd_pgtable(pmd) pmd_page(pmd) - extern pte_t *pte_alloc_one_kernel(struct mm_struct *mm); #define __pte_free_tlb(tlb, pte, addr) pte_free((tlb)->mm, (pte)) --- a/arch/mips/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/mips/include/asm/pgalloc.h @@ -28,7 +28,6 @@ static inline void pmd_populate(struct m { set_pmd(pmd, __pmd((unsigned long)page_address(pte))); } -#define pmd_pgtable(pmd) pmd_page(pmd) /* * Initialize a new pmd table with invalid pointers. --- a/arch/nds32/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/nds32/include/asm/pgalloc.h @@ -12,11 +12,6 @@ #define __HAVE_ARCH_PTE_ALLOC_ONE #include /* for pte_{alloc,free}_one */ -/* - * Since we have only two-level page tables, these are trivial - */ -#define pmd_pgtable(pmd) pmd_page(pmd) - extern pgd_t *pgd_alloc(struct mm_struct *mm); extern void pgd_free(struct mm_struct *mm, pgd_t * pgd); --- a/arch/nios2/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/nios2/include/asm/pgalloc.h @@ -25,7 +25,6 @@ static inline void pmd_populate(struct m { set_pmd(pmd, __pmd((unsigned long)page_address(pte))); } -#define pmd_pgtable(pmd) pmd_page(pmd) /* * Initialize a new pmd table with invalid pointers. --- a/arch/openrisc/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/openrisc/include/asm/pgalloc.h @@ -72,6 +72,4 @@ do { \ tlb_remove_page((tlb), (pte)); \ } while (0) -#define pmd_pgtable(pmd) pmd_page(pmd) - #endif --- a/arch/parisc/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/parisc/include/asm/pgalloc.h @@ -69,6 +69,5 @@ pmd_populate_kernel(struct mm_struct *mm #define pmd_populate(mm, pmd, pte_page) \ pmd_populate_kernel(mm, pmd, page_address(pte_page)) -#define pmd_pgtable(pmd) pmd_page(pmd) #endif --- a/arch/powerpc/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/powerpc/include/asm/pgalloc.h @@ -70,9 +70,4 @@ extern struct kmem_cache *pgtable_cache[ #include #endif -static inline pgtable_t pmd_pgtable(pmd_t pmd) -{ - return (pgtable_t)pmd_page_vaddr(pmd); -} - #endif /* _ASM_POWERPC_PGALLOC_H */ --- a/arch/powerpc/include/asm/pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/arch/powerpc/include/asm/pgtable.h @@ -152,6 +152,12 @@ static inline bool p4d_is_leaf(p4d_t p4d } #endif +#define pmd_pgtable pmd_pgtable +static inline pgtable_t pmd_pgtable(pmd_t pmd) +{ + return (pgtable_t)pmd_page_vaddr(pmd); +} + #ifdef CONFIG_PPC64 #define is_ioremap_addr is_ioremap_addr static inline bool is_ioremap_addr(const void *x) --- a/arch/riscv/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/riscv/include/asm/pgalloc.h @@ -38,8 +38,6 @@ static inline void pud_populate(struct m } #endif /* __PAGETABLE_PMD_FOLDED */ -#define pmd_pgtable(pmd) pmd_page(pmd) - static inline pgd_t *pgd_alloc(struct mm_struct *mm) { pgd_t *pgd; --- a/arch/s390/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/s390/include/asm/pgalloc.h @@ -134,9 +134,6 @@ static inline void pmd_populate(struct m #define pmd_populate_kernel(mm, pmd, pte) pmd_populate(mm, pmd, pte) -#define pmd_pgtable(pmd) \ - ((pgtable_t)__va(pmd_val(pmd) & -sizeof(pte_t)*PTRS_PER_PTE)) - /* * page table entry allocation/free routines. */ --- a/arch/s390/include/asm/pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/arch/s390/include/asm/pgtable.h @@ -1709,4 +1709,7 @@ extern void s390_reset_cmma(struct mm_st #define HAVE_ARCH_UNMAPPED_AREA #define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN +#define pmd_pgtable(pmd) \ + ((pgtable_t)__va(pmd_val(pmd) & -sizeof(pte_t)*PTRS_PER_PTE)) + #endif /* _S390_PAGE_H */ --- a/arch/sh/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/sh/include/asm/pgalloc.h @@ -30,7 +30,6 @@ static inline void pmd_populate(struct m { set_pmd(pmd, __pmd((unsigned long)page_address(pte))); } -#define pmd_pgtable(pmd) pmd_page(pmd) #define __pte_free_tlb(tlb,pte,addr) \ do { \ --- a/arch/sparc/include/asm/pgalloc_32.h~mm-thp-define-default-pmd_pgtable +++ a/arch/sparc/include/asm/pgalloc_32.h @@ -51,7 +51,6 @@ static inline void free_pmd_fast(pmd_t * #define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) #define pmd_populate(mm, pmd, pte) pmd_set(pmd, pte) -#define pmd_pgtable(pmd) (pgtable_t)__pmd_page(pmd) void pmd_set(pmd_t *pmdp, pte_t *ptep); #define pmd_populate_kernel pmd_populate --- a/arch/sparc/include/asm/pgalloc_64.h~mm-thp-define-default-pmd_pgtable +++ a/arch/sparc/include/asm/pgalloc_64.h @@ -67,7 +67,6 @@ void pte_free(struct mm_struct *mm, pgta #define pmd_populate_kernel(MM, PMD, PTE) pmd_set(MM, PMD, PTE) #define pmd_populate(MM, PMD, PTE) pmd_set(MM, PMD, PTE) -#define pmd_pgtable(PMD) ((pte_t *)pmd_page_vaddr(PMD)) void pgtable_free(void *table, bool is_page); --- a/arch/sparc/include/asm/pgtable_32.h~mm-thp-define-default-pmd_pgtable +++ a/arch/sparc/include/asm/pgtable_32.h @@ -432,4 +432,6 @@ static inline int io_remap_pfn_range(str /* We provide our own get_unmapped_area to cope with VA holes for userland */ #define HAVE_ARCH_UNMAPPED_AREA +#define pmd_pgtable(pmd) ((pgtable_t)__pmd_page(pmd)) + #endif /* !(_SPARC_PGTABLE_H) */ --- a/arch/sparc/include/asm/pgtable_64.h~mm-thp-define-default-pmd_pgtable +++ a/arch/sparc/include/asm/pgtable_64.h @@ -1117,6 +1117,8 @@ extern unsigned long cmdline_memory_size asmlinkage void do_sparc64_fault(struct pt_regs *regs); +#define pmd_pgtable(PMD) ((pte_t *)pmd_page_vaddr(PMD)) + #ifdef CONFIG_HUGETLB_PAGE #define pud_leaf_size pud_leaf_size --- a/arch/um/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/um/include/asm/pgalloc.h @@ -19,7 +19,6 @@ set_pmd(pmd, __pmd(_PAGE_TABLE + \ ((unsigned long long)page_to_pfn(pte) << \ (unsigned long long) PAGE_SHIFT))) -#define pmd_pgtable(pmd) pmd_page(pmd) /* * Allocate and free page tables. --- a/arch/x86/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/x86/include/asm/pgalloc.h @@ -84,8 +84,6 @@ static inline void pmd_populate(struct m set_pmd(pmd, __pmd(((pteval_t)pfn << PAGE_SHIFT) | _PAGE_TABLE)); } -#define pmd_pgtable(pmd) pmd_page(pmd) - #if CONFIG_PGTABLE_LEVELS > 2 extern void ___pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd); --- a/arch/xtensa/include/asm/pgalloc.h~mm-thp-define-default-pmd_pgtable +++ a/arch/xtensa/include/asm/pgalloc.h @@ -25,7 +25,6 @@ (pmd_val(*(pmdp)) = ((unsigned long)ptep)) #define pmd_populate(mm, pmdp, page) \ (pmd_val(*(pmdp)) = ((unsigned long)page_to_virt(page))) -#define pmd_pgtable(pmd) pmd_page(pmd) static inline pgd_t* pgd_alloc(struct mm_struct *mm) @@ -63,7 +62,6 @@ static inline pgtable_t pte_alloc_one(st return page; } -#define pmd_pgtable(pmd) pmd_page(pmd) #endif /* CONFIG_MMU */ #endif /* _XTENSA_PGALLOC_H */ --- a/include/linux/pgtable.h~mm-thp-define-default-pmd_pgtable +++ a/include/linux/pgtable.h @@ -38,6 +38,15 @@ #endif /* + * This defines the generic helper for accessing PMD page + * table page. Although platforms can still override this + * via their respective . + */ +#ifndef pmd_pgtable +#define pmd_pgtable(pmd) pmd_page(pmd) +#endif + +/* * A page table page can be thought of an array like this: pXd_t[PTRS_PER_PxD] * * The pXx_index() functions return the index of the entry in the page From patchwork Thu Jul 1 01:54:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353315 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 441C7C11F6A for ; Thu, 1 Jul 2021 01:54:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE5A661241 for ; Thu, 1 Jul 2021 01:54:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE5A661241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7272D8D0254; Wed, 30 Jun 2021 21:54:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D64C8D024F; Wed, 30 Jun 2021 21:54:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 551AB8D0254; Wed, 30 Jun 2021 21:54:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 295C78D024F for ; Wed, 30 Jun 2021 21:54:05 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 00A13256EC for ; Thu, 1 Jul 2021 01:54:05 +0000 (UTC) X-FDA: 78312348450.01.48D2514 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id BC1DB500009F for ; Thu, 1 Jul 2021 01:54:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BB88361468; Thu, 1 Jul 2021 01:54:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104444; bh=WMUsQU4LSR8Ot+u8T5P1bgBzcgZX7jCdDE5l3q2i6+0=; h=Date:From:To:Subject:In-Reply-To:From; b=jDbVSy833ne+VhRBcrvgrz51aaq13kWLDqSPtZRTvQ+k8rWEf1ilWOELNeNFrGfnI 8ChGFC4x71bwvUbrs/EkyabMdwubuIuCKpjdCBKNRrQWd35uBdugyfmqAg9oYo8Dj5 xdS+ZQiltpnJybaymXP+TG4fRnTQPMgpFw/bIb9Q= Date: Wed, 30 Jun 2021 18:54:03 -0700 From: Andrew Morton To: akpm@linux-foundation.org, dvyukov@google.com, elver@google.com, glider@google.com, hdanton@sina.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 129/192] kfence: unconditionally use unbound work queue Message-ID: <20210701015403.uibak3sxB%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: BC1DB500009F Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=jDbVSy83; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: hfisux4nkdmgh97q3gfc3iwpjboe7xby X-HE-Tag: 1625104444-437437 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Marco Elver Subject: kfence: unconditionally use unbound work queue Unconditionally use unbound work queue, and not just if wq_power_efficient is true. Because if the system is idle, KFENCE may wait, and by being run on the unbound work queue, we permit the scheduler to make better scheduling decisions and not require pinning KFENCE to the same CPU upon waking up. Link: https://lkml.kernel.org/r/20210521111630.472579-1-elver@google.com Fixes: 36f0b35d0894 ("kfence: use power-efficient work queue to run delayed work") Signed-off-by: Marco Elver Reported-by: Hillf Danton Reviewed-by: Alexander Potapenko Cc: Dmitry Vyukov Signed-off-by: Andrew Morton --- mm/kfence/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/kfence/core.c~kfence-unconditionally-use-unbound-work-queue +++ a/mm/kfence/core.c @@ -636,7 +636,7 @@ static void toggle_allocation_gate(struc /* Disable static key and reset timer. */ static_branch_disable(&kfence_allocation_key); #endif - queue_delayed_work(system_power_efficient_wq, &kfence_timer, + queue_delayed_work(system_unbound_wq, &kfence_timer, msecs_to_jiffies(kfence_sample_interval)); } static DECLARE_DELAYED_WORK(kfence_timer, toggle_allocation_gate); @@ -666,7 +666,7 @@ void __init kfence_init(void) } WRITE_ONCE(kfence_enabled, true); - queue_delayed_work(system_power_efficient_wq, &kfence_timer, 0); + queue_delayed_work(system_unbound_wq, &kfence_timer, 0); pr_info("initialized - using %lu bytes for %d objects at 0x%p-0x%p\n", KFENCE_POOL_SIZE, CONFIG_KFENCE_NUM_OBJECTS, (void *)__kfence_pool, (void *)(__kfence_pool + KFENCE_POOL_SIZE)); From patchwork Thu Jul 1 01:54:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353317 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D564CC11F69 for ; Thu, 1 Jul 2021 01:54:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 75C6861468 for ; Thu, 1 Jul 2021 01:54:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 75C6861468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F166F8D0255; Wed, 30 Jun 2021 21:54:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC6908D024F; Wed, 30 Jun 2021 21:54:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3FED8D0255; Wed, 30 Jun 2021 21:54:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id A72B58D024F for ; Wed, 30 Jun 2021 21:54:08 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 717F3181AEF3F for ; Thu, 1 Jul 2021 01:54:08 +0000 (UTC) X-FDA: 78312348576.12.B36B713 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 0C18D6001A8C for ; Thu, 1 Jul 2021 01:54:07 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id F2AB161241; Thu, 1 Jul 2021 01:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104447; bh=IC4O+RBO83YS1hFpW9l7C7mBHNdArOW6nJqh1/I7SPs=; h=Date:From:To:Subject:In-Reply-To:From; b=eAMBHblQUZOI9i5TRQi+3YEfYIfSo3HgmYr5Gtp40K8Yli2V+Q+zVnjk6vyQAEI3m uCMRu4Ev1iE4dKBvMvw9agjagRb+a4h0V0rpMMh368GJYUagFevZqn8umhHf1/YhRi kj47hvpx0Q5gyt1Esz22HxOq4Pw9/mN5FSui7LTI= Date: Wed, 30 Jun 2021 18:54:06 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 130/192] mm: remove special swap entry functions Message-ID: <20210701015406.y630gT6le%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 0C18D6001A8C Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=eAMBHblQ; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: win41nh585c18oahed4diuq4nfat1981 X-HE-Tag: 1625104447-523249 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm: remove special swap entry functions Patch series "Add support for SVM atomics in Nouveau", v11. Introduction ============ Some devices have features such as atomic PTE bits that can be used to implement atomic access to system memory. To support atomic operations to a shared virtual memory page such a device needs access to that page which is exclusive of the CPU. This series introduces a mechanism to temporarily unmap pages granting exclusive access to a device. These changes are required to support OpenCL atomic operations in Nouveau to shared virtual memory (SVM) regions allocated with the CL_MEM_SVM_ATOMICS clSVMAlloc flag. A more complete description of the OpenCL SVM feature is available at https://www.khronos.org/registry/OpenCL/specs/3.0-unified/html/ OpenCL_API.html#_shared_virtual_memory . Implementation ============== Exclusive device access is implemented by adding a new swap entry type (SWAP_DEVICE_EXCLUSIVE) which is similar to a migration entry. The main difference is that on fault the original entry is immediately restored by the fault handler instead of waiting. Restoring the entry triggers calls to MMU notifers which allows a device driver to revoke the atomic access permission from the GPU prior to the CPU finalising the entry. Patches ======= Patches 1 & 2 refactor existing migration and device private entry functions. Patches 3 & 4 rework try_to_unmap_one() by splitting out unrelated functionality into separate functions - try_to_migrate_one() and try_to_munlock_one(). Patch 5 renames some existing code but does not introduce functionality. Patch 6 is a small clean-up to swap entry handling in copy_pte_range(). Patch 7 contains the bulk of the implementation for device exclusive memory. Patch 8 contains some additions to the HMM selftests to ensure everything works as expected. Patch 9 is a cleanup for the Nouveau SVM implementation. Patch 10 contains the implementation of atomic access for the Nouveau driver. Testing ======= This has been tested with upstream Mesa 21.1.0 and a simple OpenCL program which checks that GPU atomic accesses to system memory are atomic. Without this series the test fails as there is no way of write-protecting the page mapping which results in the device clobbering CPU writes. For reference the test is available at https://ozlabs.org/~apopple/opencl_svm_atomics/ Further testing has been performed by adding support for testing exclusive access to the hmm-tests kselftests. This patch (of 10): Remove multiple similar inline functions for dealing with different types of special swap entries. Both migration and device private swap entries use the swap offset to store a pfn. Instead of multiple inline functions to obtain a struct page for each swap entry type use a common function pfn_swap_entry_to_page(). Also open-code the various entry_to_pfn() functions as this results is shorter code that is easier to understand. Link: https://lkml.kernel.org/r/20210616105937.23201-1-apopple@nvidia.com Link: https://lkml.kernel.org/r/20210616105937.23201-2-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Ralph Campbell Reviewed-by: Christoph Hellwig Cc: "Matthew Wilcox (Oracle)" Cc: Hugh Dickins Cc: Peter Xu Cc: Shakeel Butt Cc: Ben Skeggs Cc: Jason Gunthorpe Cc: John Hubbard Signed-off-by: Andrew Morton --- arch/s390/mm/pgtable.c | 2 - fs/proc/task_mmu.c | 23 ++++-------- include/linux/swap.h | 4 +- include/linux/swapops.h | 69 ++++++++++++-------------------------- mm/hmm.c | 5 +- mm/huge_memory.c | 6 +-- mm/memcontrol.c | 2 - mm/memory.c | 10 ++--- mm/migrate.c | 6 +-- mm/page_vma_mapped.c | 6 +-- 10 files changed, 51 insertions(+), 82 deletions(-) --- a/arch/s390/mm/pgtable.c~mm-remove-special-swap-entry-functions +++ a/arch/s390/mm/pgtable.c @@ -691,7 +691,7 @@ static void ptep_zap_swap_entry(struct m if (!non_swap_entry(entry)) dec_mm_counter(mm, MM_SWAPENTS); else if (is_migration_entry(entry)) { - struct page *page = migration_entry_to_page(entry); + struct page *page = pfn_swap_entry_to_page(entry); dec_mm_counter(mm, mm_counter(page)); } --- a/fs/proc/task_mmu.c~mm-remove-special-swap-entry-functions +++ a/fs/proc/task_mmu.c @@ -514,10 +514,8 @@ static void smaps_pte_entry(pte_t *pte, } else { mss->swap_pss += (u64)PAGE_SIZE << PSS_SHIFT; } - } else if (is_migration_entry(swpent)) - page = migration_entry_to_page(swpent); - else if (is_device_private_entry(swpent)) - page = device_private_entry_to_page(swpent); + } else if (is_pfn_swap_entry(swpent)) + page = pfn_swap_entry_to_page(swpent); } else if (unlikely(IS_ENABLED(CONFIG_SHMEM) && mss->check_shmem_swap && pte_none(*pte))) { page = xa_load(&vma->vm_file->f_mapping->i_pages, @@ -549,7 +547,7 @@ static void smaps_pmd_entry(pmd_t *pmd, swp_entry_t entry = pmd_to_swp_entry(*pmd); if (is_migration_entry(entry)) - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); } if (IS_ERR_OR_NULL(page)) return; @@ -694,10 +692,8 @@ static int smaps_hugetlb_range(pte_t *pt } else if (is_swap_pte(*pte)) { swp_entry_t swpent = pte_to_swp_entry(*pte); - if (is_migration_entry(swpent)) - page = migration_entry_to_page(swpent); - else if (is_device_private_entry(swpent)) - page = device_private_entry_to_page(swpent); + if (is_pfn_swap_entry(swpent)) + page = pfn_swap_entry_to_page(swpent); } if (page) { int mapcount = page_mapcount(page); @@ -1389,11 +1385,8 @@ static pagemap_entry_t pte_to_pagemap_en frame = swp_type(entry) | (swp_offset(entry) << MAX_SWAPFILES_SHIFT); flags |= PM_SWAP; - if (is_migration_entry(entry)) - page = migration_entry_to_page(entry); - - if (is_device_private_entry(entry)) - page = device_private_entry_to_page(entry); + if (is_pfn_swap_entry(entry)) + page = pfn_swap_entry_to_page(entry); } if (page && !PageAnon(page)) @@ -1454,7 +1447,7 @@ static int pagemap_pmd_range(pmd_t *pmdp if (pmd_swp_uffd_wp(pmd)) flags |= PM_UFFD_WP; VM_BUG_ON(!is_pmd_migration_entry(pmd)); - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); } #endif --- a/include/linux/swap.h~mm-remove-special-swap-entry-functions +++ a/include/linux/swap.h @@ -564,8 +564,8 @@ static inline void show_swap_cache_info( { } -#define free_swap_and_cache(e) ({(is_migration_entry(e) || is_device_private_entry(e));}) -#define swapcache_prepare(e) ({(is_migration_entry(e) || is_device_private_entry(e));}) +/* used to sanity check ptes in zap_pte_range when CONFIG_SWAP=0 */ +#define free_swap_and_cache(e) is_pfn_swap_entry(e) static inline int add_swap_count_continuation(swp_entry_t swp, gfp_t gfp_mask) { --- a/include/linux/swapops.h~mm-remove-special-swap-entry-functions +++ a/include/linux/swapops.h @@ -128,16 +128,6 @@ static inline bool is_write_device_priva { return unlikely(swp_type(entry) == SWP_DEVICE_WRITE); } - -static inline unsigned long device_private_entry_to_pfn(swp_entry_t entry) -{ - return swp_offset(entry); -} - -static inline struct page *device_private_entry_to_page(swp_entry_t entry) -{ - return pfn_to_page(swp_offset(entry)); -} #else /* CONFIG_DEVICE_PRIVATE */ static inline swp_entry_t make_device_private_entry(struct page *page, bool write) { @@ -157,16 +147,6 @@ static inline bool is_write_device_priva { return false; } - -static inline unsigned long device_private_entry_to_pfn(swp_entry_t entry) -{ - return 0; -} - -static inline struct page *device_private_entry_to_page(swp_entry_t entry) -{ - return NULL; -} #endif /* CONFIG_DEVICE_PRIVATE */ #ifdef CONFIG_MIGRATION @@ -189,22 +169,6 @@ static inline int is_write_migration_ent return unlikely(swp_type(entry) == SWP_MIGRATION_WRITE); } -static inline unsigned long migration_entry_to_pfn(swp_entry_t entry) -{ - return swp_offset(entry); -} - -static inline struct page *migration_entry_to_page(swp_entry_t entry) -{ - struct page *p = pfn_to_page(swp_offset(entry)); - /* - * Any use of migration entries may only occur while the - * corresponding page is locked - */ - BUG_ON(!PageLocked(compound_head(p))); - return p; -} - static inline void make_migration_entry_read(swp_entry_t *entry) { *entry = swp_entry(SWP_MIGRATION_READ, swp_offset(*entry)); @@ -224,16 +188,6 @@ static inline int is_migration_entry(swp return 0; } -static inline unsigned long migration_entry_to_pfn(swp_entry_t entry) -{ - return 0; -} - -static inline struct page *migration_entry_to_page(swp_entry_t entry) -{ - return NULL; -} - static inline void make_migration_entry_read(swp_entry_t *entryp) { } static inline void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, spinlock_t *ptl) { } @@ -248,6 +202,29 @@ static inline int is_write_migration_ent #endif +static inline struct page *pfn_swap_entry_to_page(swp_entry_t entry) +{ + struct page *p = pfn_to_page(swp_offset(entry)); + + /* + * Any use of migration entries may only occur while the + * corresponding page is locked + */ + BUG_ON(is_migration_entry(entry) && !PageLocked(p)); + + return p; +} + +/* + * A pfn swap entry is a special type of swap entry that always has a pfn stored + * in the swap offset. They are used to represent unaddressable device memory + * and to restrict access to a page undergoing migration. + */ +static inline bool is_pfn_swap_entry(swp_entry_t entry) +{ + return is_migration_entry(entry) || is_device_private_entry(entry); +} + struct page_vma_mapped_walk; #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION --- a/mm/hmm.c~mm-remove-special-swap-entry-functions +++ a/mm/hmm.c @@ -214,7 +214,7 @@ static inline bool hmm_is_device_private swp_entry_t entry) { return is_device_private_entry(entry) && - device_private_entry_to_page(entry)->pgmap->owner == + pfn_swap_entry_to_page(entry)->pgmap->owner == range->dev_private_owner; } @@ -257,8 +257,7 @@ static int hmm_vma_handle_pte(struct mm_ cpu_flags = HMM_PFN_VALID; if (is_write_device_private_entry(entry)) cpu_flags |= HMM_PFN_WRITE; - *hmm_pfn = device_private_entry_to_pfn(entry) | - cpu_flags; + *hmm_pfn = swp_offset(entry) | cpu_flags; return 0; } --- a/mm/huge_memory.c~mm-remove-special-swap-entry-functions +++ a/mm/huge_memory.c @@ -1643,7 +1643,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, VM_BUG_ON(!is_pmd_migration_entry(orig_pmd)); entry = pmd_to_swp_entry(orig_pmd); - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); flush_needed = 0; } else WARN_ONCE(1, "Non present huge pmd without pmd migration enabled!"); @@ -2012,7 +2012,7 @@ static void __split_huge_pmd_locked(stru swp_entry_t entry; entry = pmd_to_swp_entry(old_pmd); - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); } else { page = pmd_page(old_pmd); if (!PageDirty(page) && pmd_dirty(old_pmd)) @@ -2066,7 +2066,7 @@ static void __split_huge_pmd_locked(stru swp_entry_t entry; entry = pmd_to_swp_entry(old_pmd); - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); write = is_write_migration_entry(entry); young = false; soft_dirty = pmd_swp_soft_dirty(old_pmd); --- a/mm/memcontrol.c~mm-remove-special-swap-entry-functions +++ a/mm/memcontrol.c @@ -5532,7 +5532,7 @@ static struct page *mc_handle_swap_pte(s * as special swap entry in the CPU page table. */ if (is_device_private_entry(ent)) { - page = device_private_entry_to_page(ent); + page = pfn_swap_entry_to_page(ent); /* * MEMORY_DEVICE_PRIVATE means ZONE_DEVICE page and which have * a refcount of 1 when free (unlike normal page) --- a/mm/memory.c~mm-remove-special-swap-entry-functions +++ a/mm/memory.c @@ -729,7 +729,7 @@ copy_nonpresent_pte(struct mm_struct *ds } rss[MM_SWAPENTS]++; } else if (is_migration_entry(entry)) { - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); rss[mm_counter(page)]++; @@ -748,7 +748,7 @@ copy_nonpresent_pte(struct mm_struct *ds set_pte_at(src_mm, addr, src_pte, pte); } } else if (is_device_private_entry(entry)) { - page = device_private_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); /* * Update rss count even for unaddressable pages, as @@ -1280,7 +1280,7 @@ again: entry = pte_to_swp_entry(ptent); if (is_device_private_entry(entry)) { - struct page *page = device_private_entry_to_page(entry); + struct page *page = pfn_swap_entry_to_page(entry); if (unlikely(details && details->check_mapping)) { /* @@ -1309,7 +1309,7 @@ again: else if (is_migration_entry(entry)) { struct page *page; - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); rss[mm_counter(page)]--; } if (unlikely(!free_swap_and_cache(entry))) @@ -3372,7 +3372,7 @@ vm_fault_t do_swap_page(struct vm_fault migration_entry_wait(vma->vm_mm, vmf->pmd, vmf->address); } else if (is_device_private_entry(entry)) { - vmf->page = device_private_entry_to_page(entry); + vmf->page = pfn_swap_entry_to_page(entry); ret = vmf->page->pgmap->ops->migrate_to_ram(vmf); } else if (is_hwpoison_entry(entry)) { ret = VM_FAULT_HWPOISON; --- a/mm/migrate.c~mm-remove-special-swap-entry-functions +++ a/mm/migrate.c @@ -296,7 +296,7 @@ void __migration_entry_wait(struct mm_st if (!is_migration_entry(entry)) goto out; - page = migration_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); page = compound_head(page); /* @@ -337,7 +337,7 @@ void pmd_migration_entry_wait(struct mm_ ptl = pmd_lock(mm, pmd); if (!is_pmd_migration_entry(*pmd)) goto unlock; - page = migration_entry_to_page(pmd_to_swp_entry(*pmd)); + page = pfn_swap_entry_to_page(pmd_to_swp_entry(*pmd)); if (!get_page_unless_zero(page)) goto unlock; spin_unlock(ptl); @@ -2289,7 +2289,7 @@ again: if (!is_device_private_entry(entry)) goto next; - page = device_private_entry_to_page(entry); + page = pfn_swap_entry_to_page(entry); if (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || page->pgmap->owner != migrate->pgmap_owner) --- a/mm/page_vma_mapped.c~mm-remove-special-swap-entry-functions +++ a/mm/page_vma_mapped.c @@ -96,7 +96,7 @@ static bool check_pte(struct page_vma_ma if (!is_migration_entry(entry)) return false; - pfn = migration_entry_to_pfn(entry); + pfn = swp_offset(entry); } else if (is_swap_pte(*pvmw->pte)) { swp_entry_t entry; @@ -105,7 +105,7 @@ static bool check_pte(struct page_vma_ma if (!is_device_private_entry(entry)) return false; - pfn = device_private_entry_to_pfn(entry); + pfn = swp_offset(entry); } else { if (!pte_present(*pvmw->pte)) return false; @@ -233,7 +233,7 @@ restart: return not_found(pvmw); entry = pmd_to_swp_entry(pmde); if (!is_migration_entry(entry) || - migration_entry_to_page(entry) != page) + pfn_swap_entry_to_page(entry) != page) return not_found(pvmw); return true; } From patchwork Thu Jul 1 01:54:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353319 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 121D6C11F66 for ; Thu, 1 Jul 2021 01:54:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A5D276105A for ; Thu, 1 Jul 2021 01:54:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5D276105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2BA6F8D0256; Wed, 30 Jun 2021 21:54:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26AEF8D024F; Wed, 30 Jun 2021 21:54:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E39D8D0256; Wed, 30 Jun 2021 21:54:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id CF8448D024F for ; Wed, 30 Jun 2021 21:54:11 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id AF8D423E79 for ; Thu, 1 Jul 2021 01:54:11 +0000 (UTC) X-FDA: 78312348702.09.C9386B0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 630A55000097 for ; Thu, 1 Jul 2021 01:54:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3AE1B61469; Thu, 1 Jul 2021 01:54:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104450; bh=P4y0dyyBJRJDYB6A9auJpv0I7cb/npsaYJc8ctwBA2o=; h=Date:From:To:Subject:In-Reply-To:From; b=v2ekvM+XUirjUCFaL35kRFEIoRsNESce+JR6dsa/6zFdFJOlQlHB1sqppbaJYewJh cuavGEao5MGj+czeKm0ToBZb8Ihu63erd6FEdNj5PGIswQ0Rr6/PBrhF7ilLDhCltz PNh8dUi2TgToVgLeJR5/G/gKFtyzzPTiL+usmv7c= Date: Wed, 30 Jun 2021 18:54:09 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 131/192] mm/swapops: rework swap entry manipulation code Message-ID: <20210701015409.I_QhqVa0y%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 630A55000097 X-Stat-Signature: momunpj1y8iaptdjcyf5c6o3xsox4wgd Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=v2ekvM+X; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104451-447215 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm/swapops: rework swap entry manipulation code Both migration and device private pages use special swap entries that are manipluated by a range of inline functions. The arguments to these are somewhat inconsitent so rework them to remove flag type arguments and to make the arguments similar for both read and write entry creation. Link: https://lkml.kernel.org/r/20210616105937.23201-3-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Christoph Hellwig Reviewed-by: Jason Gunthorpe Reviewed-by: Ralph Campbell Cc: Ben Skeggs Cc: Hugh Dickins Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Shakeel Butt Signed-off-by: Andrew Morton --- include/linux/swapops.h | 56 ++++++++++++++++++++------------------ mm/debug_vm_pgtable.c | 12 ++++---- mm/hmm.c | 2 - mm/huge_memory.c | 26 ++++++++++++----- mm/hugetlb.c | 10 ++++-- mm/memory.c | 10 ++++-- mm/migrate.c | 26 +++++++++++++---- mm/mprotect.c | 10 ++++-- mm/rmap.c | 10 ++++-- 9 files changed, 100 insertions(+), 62 deletions(-) --- a/include/linux/swapops.h~mm-swapops-rework-swap-entry-manipulation-code +++ a/include/linux/swapops.h @@ -107,35 +107,35 @@ static inline void *swp_to_radix_entry(s } #if IS_ENABLED(CONFIG_DEVICE_PRIVATE) -static inline swp_entry_t make_device_private_entry(struct page *page, bool write) +static inline swp_entry_t make_readable_device_private_entry(pgoff_t offset) { - return swp_entry(write ? SWP_DEVICE_WRITE : SWP_DEVICE_READ, - page_to_pfn(page)); + return swp_entry(SWP_DEVICE_READ, offset); } -static inline bool is_device_private_entry(swp_entry_t entry) +static inline swp_entry_t make_writable_device_private_entry(pgoff_t offset) { - int type = swp_type(entry); - return type == SWP_DEVICE_READ || type == SWP_DEVICE_WRITE; + return swp_entry(SWP_DEVICE_WRITE, offset); } -static inline void make_device_private_entry_read(swp_entry_t *entry) +static inline bool is_device_private_entry(swp_entry_t entry) { - *entry = swp_entry(SWP_DEVICE_READ, swp_offset(*entry)); + int type = swp_type(entry); + return type == SWP_DEVICE_READ || type == SWP_DEVICE_WRITE; } -static inline bool is_write_device_private_entry(swp_entry_t entry) +static inline bool is_writable_device_private_entry(swp_entry_t entry) { return unlikely(swp_type(entry) == SWP_DEVICE_WRITE); } #else /* CONFIG_DEVICE_PRIVATE */ -static inline swp_entry_t make_device_private_entry(struct page *page, bool write) +static inline swp_entry_t make_readable_device_private_entry(pgoff_t offset) { return swp_entry(0, 0); } -static inline void make_device_private_entry_read(swp_entry_t *entry) +static inline swp_entry_t make_writable_device_private_entry(pgoff_t offset) { + return swp_entry(0, 0); } static inline bool is_device_private_entry(swp_entry_t entry) @@ -143,35 +143,32 @@ static inline bool is_device_private_ent return false; } -static inline bool is_write_device_private_entry(swp_entry_t entry) +static inline bool is_writable_device_private_entry(swp_entry_t entry) { return false; } #endif /* CONFIG_DEVICE_PRIVATE */ #ifdef CONFIG_MIGRATION -static inline swp_entry_t make_migration_entry(struct page *page, int write) -{ - BUG_ON(!PageLocked(compound_head(page))); - - return swp_entry(write ? SWP_MIGRATION_WRITE : SWP_MIGRATION_READ, - page_to_pfn(page)); -} - static inline int is_migration_entry(swp_entry_t entry) { return unlikely(swp_type(entry) == SWP_MIGRATION_READ || swp_type(entry) == SWP_MIGRATION_WRITE); } -static inline int is_write_migration_entry(swp_entry_t entry) +static inline int is_writable_migration_entry(swp_entry_t entry) { return unlikely(swp_type(entry) == SWP_MIGRATION_WRITE); } -static inline void make_migration_entry_read(swp_entry_t *entry) +static inline swp_entry_t make_readable_migration_entry(pgoff_t offset) { - *entry = swp_entry(SWP_MIGRATION_READ, swp_offset(*entry)); + return swp_entry(SWP_MIGRATION_READ, offset); +} + +static inline swp_entry_t make_writable_migration_entry(pgoff_t offset) +{ + return swp_entry(SWP_MIGRATION_WRITE, offset); } extern void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, @@ -181,21 +178,28 @@ extern void migration_entry_wait(struct extern void migration_entry_wait_huge(struct vm_area_struct *vma, struct mm_struct *mm, pte_t *pte); #else +static inline swp_entry_t make_readable_migration_entry(pgoff_t offset) +{ + return swp_entry(0, 0); +} + +static inline swp_entry_t make_writable_migration_entry(pgoff_t offset) +{ + return swp_entry(0, 0); +} -#define make_migration_entry(page, write) swp_entry(0, 0) static inline int is_migration_entry(swp_entry_t swp) { return 0; } -static inline void make_migration_entry_read(swp_entry_t *entryp) { } static inline void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, spinlock_t *ptl) { } static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address) { } static inline void migration_entry_wait_huge(struct vm_area_struct *vma, struct mm_struct *mm, pte_t *pte) { } -static inline int is_write_migration_entry(swp_entry_t entry) +static inline int is_writable_migration_entry(swp_entry_t entry) { return 0; } --- a/mm/debug_vm_pgtable.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/debug_vm_pgtable.c @@ -843,17 +843,17 @@ static void __init swap_migration_tests( * locked, otherwise it stumbles upon a BUG_ON(). */ __SetPageLocked(page); - swp = make_migration_entry(page, 1); + swp = make_writable_migration_entry(page_to_pfn(page)); WARN_ON(!is_migration_entry(swp)); - WARN_ON(!is_write_migration_entry(swp)); + WARN_ON(!is_writable_migration_entry(swp)); - make_migration_entry_read(&swp); + swp = make_readable_migration_entry(swp_offset(swp)); WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_write_migration_entry(swp)); + WARN_ON(is_writable_migration_entry(swp)); - swp = make_migration_entry(page, 0); + swp = make_readable_migration_entry(page_to_pfn(page)); WARN_ON(!is_migration_entry(swp)); - WARN_ON(is_write_migration_entry(swp)); + WARN_ON(is_writable_migration_entry(swp)); __ClearPageLocked(page); __free_page(page); } --- a/mm/hmm.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/hmm.c @@ -255,7 +255,7 @@ static int hmm_vma_handle_pte(struct mm_ */ if (hmm_is_device_private_entry(range, entry)) { cpu_flags = HMM_PFN_VALID; - if (is_write_device_private_entry(entry)) + if (is_writable_device_private_entry(entry)) cpu_flags |= HMM_PFN_WRITE; *hmm_pfn = swp_offset(entry) | cpu_flags; return 0; --- a/mm/huge_memory.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/huge_memory.c @@ -1054,8 +1054,9 @@ int copy_huge_pmd(struct mm_struct *dst_ swp_entry_t entry = pmd_to_swp_entry(pmd); VM_BUG_ON(!is_pmd_migration_entry(pmd)); - if (is_write_migration_entry(entry)) { - make_migration_entry_read(&entry); + if (is_writable_migration_entry(entry)) { + entry = make_readable_migration_entry( + swp_offset(entry)); pmd = swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*src_pmd)) pmd = pmd_swp_mksoft_dirty(pmd); @@ -1772,13 +1773,14 @@ int change_huge_pmd(struct vm_area_struc swp_entry_t entry = pmd_to_swp_entry(*pmd); VM_BUG_ON(!is_pmd_migration_entry(*pmd)); - if (is_write_migration_entry(entry)) { + if (is_writable_migration_entry(entry)) { pmd_t newpmd; /* * A protection check is difficult so * just be safe and disable write */ - make_migration_entry_read(&entry); + entry = make_readable_migration_entry( + swp_offset(entry)); newpmd = swp_entry_to_pmd(entry); if (pmd_swp_soft_dirty(*pmd)) newpmd = pmd_swp_mksoft_dirty(newpmd); @@ -2067,7 +2069,7 @@ static void __split_huge_pmd_locked(stru entry = pmd_to_swp_entry(old_pmd); page = pfn_swap_entry_to_page(entry); - write = is_write_migration_entry(entry); + write = is_writable_migration_entry(entry); young = false; soft_dirty = pmd_swp_soft_dirty(old_pmd); uffd_wp = pmd_swp_uffd_wp(old_pmd); @@ -2099,7 +2101,12 @@ static void __split_huge_pmd_locked(stru */ if (freeze || pmd_migration) { swp_entry_t swp_entry; - swp_entry = make_migration_entry(page + i, write); + if (write) + swp_entry = make_writable_migration_entry( + page_to_pfn(page + i)); + else + swp_entry = make_readable_migration_entry( + page_to_pfn(page + i)); entry = swp_entry_to_pte(swp_entry); if (soft_dirty) entry = pte_swp_mksoft_dirty(entry); @@ -3171,7 +3178,10 @@ void set_pmd_migration_entry(struct page pmdval = pmdp_invalidate(vma, address, pvmw->pmd); if (pmd_dirty(pmdval)) set_page_dirty(page); - entry = make_migration_entry(page, pmd_write(pmdval)); + if (pmd_write(pmdval)) + entry = make_writable_migration_entry(page_to_pfn(page)); + else + entry = make_readable_migration_entry(page_to_pfn(page)); pmdswp = swp_entry_to_pmd(entry); if (pmd_soft_dirty(pmdval)) pmdswp = pmd_swp_mksoft_dirty(pmdswp); @@ -3197,7 +3207,7 @@ void remove_migration_pmd(struct page_vm pmde = pmd_mkold(mk_huge_pmd(new, vma->vm_page_prot)); if (pmd_swp_soft_dirty(*pvmw->pmd)) pmde = pmd_mksoft_dirty(pmde); - if (is_write_migration_entry(entry)) + if (is_writable_migration_entry(entry)) pmde = maybe_pmd_mkwrite(pmde, vma); if (pmd_swp_uffd_wp(*pvmw->pmd)) pmde = pmd_wrprotect(pmd_mkuffd_wp(pmde)); --- a/mm/hugetlb.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/hugetlb.c @@ -4242,12 +4242,13 @@ again: is_hugetlb_entry_hwpoisoned(entry))) { swp_entry_t swp_entry = pte_to_swp_entry(entry); - if (is_write_migration_entry(swp_entry) && cow) { + if (is_writable_migration_entry(swp_entry) && cow) { /* * COW mappings require pages in both * parent and child to be set to read. */ - make_migration_entry_read(&swp_entry); + swp_entry = make_readable_migration_entry( + swp_offset(swp_entry)); entry = swp_entry_to_pte(swp_entry); set_huge_swap_pte_at(src, addr, src_pte, entry, sz); @@ -5532,10 +5533,11 @@ unsigned long hugetlb_change_protection( if (unlikely(is_hugetlb_entry_migration(pte))) { swp_entry_t entry = pte_to_swp_entry(pte); - if (is_write_migration_entry(entry)) { + if (is_writable_migration_entry(entry)) { pte_t newpte; - make_migration_entry_read(&entry); + entry = make_readable_migration_entry( + swp_offset(entry)); newpte = swp_entry_to_pte(entry); set_huge_swap_pte_at(mm, address, ptep, newpte, huge_page_size(h)); --- a/mm/memory.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/memory.c @@ -733,13 +733,14 @@ copy_nonpresent_pte(struct mm_struct *ds rss[mm_counter(page)]++; - if (is_write_migration_entry(entry) && + if (is_writable_migration_entry(entry) && is_cow_mapping(vm_flags)) { /* * COW mappings require pages in both * parent and child to be set to read. */ - make_migration_entry_read(&entry); + entry = make_readable_migration_entry( + swp_offset(entry)); pte = swp_entry_to_pte(entry); if (pte_swp_soft_dirty(*src_pte)) pte = pte_swp_mksoft_dirty(pte); @@ -770,9 +771,10 @@ copy_nonpresent_pte(struct mm_struct *ds * when a device driver is involved (you cannot easily * save and restore device driver state). */ - if (is_write_device_private_entry(entry) && + if (is_writable_device_private_entry(entry) && is_cow_mapping(vm_flags)) { - make_device_private_entry_read(&entry); + entry = make_readable_device_private_entry( + swp_offset(entry)); pte = swp_entry_to_pte(entry); if (pte_swp_uffd_wp(*src_pte)) pte = pte_swp_mkuffd_wp(pte); --- a/mm/migrate.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/migrate.c @@ -210,13 +210,18 @@ static bool remove_migration_pte(struct * Recheck VMA as permissions can change since migration started */ entry = pte_to_swp_entry(*pvmw.pte); - if (is_write_migration_entry(entry)) + if (is_writable_migration_entry(entry)) pte = maybe_mkwrite(pte, vma); else if (pte_swp_uffd_wp(*pvmw.pte)) pte = pte_mkuffd_wp(pte); if (unlikely(is_device_private_page(new))) { - entry = make_device_private_entry(new, pte_write(pte)); + if (pte_write(pte)) + entry = make_writable_device_private_entry( + page_to_pfn(new)); + else + entry = make_readable_device_private_entry( + page_to_pfn(new)); pte = swp_entry_to_pte(entry); if (pte_swp_soft_dirty(*pvmw.pte)) pte = pte_swp_mksoft_dirty(pte); @@ -2297,7 +2302,7 @@ again: mpfn = migrate_pfn(page_to_pfn(page)) | MIGRATE_PFN_MIGRATE; - if (is_write_device_private_entry(entry)) + if (is_writable_device_private_entry(entry)) mpfn |= MIGRATE_PFN_WRITE; } else { if (!(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM)) @@ -2343,8 +2348,12 @@ again: ptep_get_and_clear(mm, addr, ptep); /* Setup special migration page table entry */ - entry = make_migration_entry(page, mpfn & - MIGRATE_PFN_WRITE); + if (mpfn & MIGRATE_PFN_WRITE) + entry = make_writable_migration_entry( + page_to_pfn(page)); + else + entry = make_readable_migration_entry( + page_to_pfn(page)); swp_pte = swp_entry_to_pte(entry); if (pte_present(pte)) { if (pte_soft_dirty(pte)) @@ -2817,7 +2826,12 @@ static void migrate_vma_insert_page(stru if (is_device_private_page(page)) { swp_entry_t swp_entry; - swp_entry = make_device_private_entry(page, vma->vm_flags & VM_WRITE); + if (vma->vm_flags & VM_WRITE) + swp_entry = make_writable_device_private_entry( + page_to_pfn(page)); + else + swp_entry = make_readable_device_private_entry( + page_to_pfn(page)); entry = swp_entry_to_pte(swp_entry); } else { /* --- a/mm/mprotect.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/mprotect.c @@ -143,23 +143,25 @@ static unsigned long change_pte_range(st swp_entry_t entry = pte_to_swp_entry(oldpte); pte_t newpte; - if (is_write_migration_entry(entry)) { + if (is_writable_migration_entry(entry)) { /* * A protection check is difficult so * just be safe and disable write */ - make_migration_entry_read(&entry); + entry = make_readable_migration_entry( + swp_offset(entry)); newpte = swp_entry_to_pte(entry); if (pte_swp_soft_dirty(oldpte)) newpte = pte_swp_mksoft_dirty(newpte); if (pte_swp_uffd_wp(oldpte)) newpte = pte_swp_mkuffd_wp(newpte); - } else if (is_write_device_private_entry(entry)) { + } else if (is_writable_device_private_entry(entry)) { /* * We do not preserve soft-dirtiness. See * copy_one_pte() for explanation. */ - make_device_private_entry_read(&entry); + entry = make_readable_device_private_entry( + swp_offset(entry)); newpte = swp_entry_to_pte(entry); if (pte_swp_uffd_wp(oldpte)) newpte = pte_swp_mkuffd_wp(newpte); --- a/mm/rmap.c~mm-swapops-rework-swap-entry-manipulation-code +++ a/mm/rmap.c @@ -1533,7 +1533,7 @@ static bool try_to_unmap_one(struct page * pte. do_swap_page() will wait until the migration * pte is removed and then restart fault handling. */ - entry = make_migration_entry(page, 0); + entry = make_readable_migration_entry(page_to_pfn(page)); swp_pte = swp_entry_to_pte(entry); /* @@ -1629,8 +1629,12 @@ static bool try_to_unmap_one(struct page * pte. do_swap_page() will wait until the migration * pte is removed and then restart fault handling. */ - entry = make_migration_entry(subpage, - pte_write(pteval)); + if (pte_write(pteval)) + entry = make_writable_migration_entry( + page_to_pfn(subpage)); + else + entry = make_readable_migration_entry( + page_to_pfn(subpage)); swp_pte = swp_entry_to_pte(entry); if (pte_soft_dirty(pteval)) swp_pte = pte_swp_mksoft_dirty(swp_pte); From patchwork Thu Jul 1 01:54:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353321 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18834C11F69 for ; Thu, 1 Jul 2021 01:54:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BE2126105A for ; Thu, 1 Jul 2021 01:54:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BE2126105A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 440478D0257; Wed, 30 Jun 2021 21:54:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C9258D024F; Wed, 30 Jun 2021 21:54:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F5108D0257; Wed, 30 Jun 2021 21:54:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id EB5E98D024F for ; Wed, 30 Jun 2021 21:54:14 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C2DFD8248076 for ; Thu, 1 Jul 2021 01:54:14 +0000 (UTC) X-FDA: 78312348828.20.5BF3EB9 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 6922C6001AA8 for ; Thu, 1 Jul 2021 01:54:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5187E61241; Thu, 1 Jul 2021 01:54:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104453; bh=1mLf1zR8rpOL8LdHJ3sDV6pUmT6Sb3F5WfGGkH6Jm3A=; h=Date:From:To:Subject:In-Reply-To:From; b=UyVGImAnXZ3QTISysKsOh2fFKqX5Xpj+tX1PxTLM/WzHnwmLb+kCIQFkWB1sSBeG0 7nypZwEou0/1x5DA/+5QkVnoi01ZtVIyHhVm/+sOQkpwv7devusVGFbkgr4q1zLy33 VA3pyfyPNiAV84xTKDqiMQY5gklJFwwTDs51RalE= Date: Wed, 30 Jun 2021 18:54:12 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 132/192] mm/rmap: split try_to_munlock from try_to_unmap Message-ID: <20210701015412.snaTuyrO6%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=UyVGImAn; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: 81antbctefaz4wy7eurfz8zgkq83neif X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6922C6001AA8 X-HE-Tag: 1625104454-969328 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm/rmap: split try_to_munlock from try_to_unmap The behaviour of try_to_unmap_one() is difficult to follow because it performs different operations based on a fairly large set of flags used in different combinations. TTU_MUNLOCK is one such flag. However it is exclusively used by try_to_munlock() which specifies no other flags. Therefore rather than overload try_to_unmap_one() with unrelated behaviour split this out into it's own function and remove the flag. Link: https://lkml.kernel.org/r/20210616105937.23201-4-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Ralph Campbell Reviewed-by: Christoph Hellwig Cc: Ben Skeggs Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Shakeel Butt Signed-off-by: Andrew Morton --- Documentation/vm/unevictable-lru.rst | 33 ++++-------- include/linux/rmap.h | 3 - mm/mlock.c | 12 ++-- mm/rmap.c | 66 ++++++++++++++++++------- 4 files changed, 69 insertions(+), 45 deletions(-) --- a/Documentation/vm/unevictable-lru.rst~mm-rmap-split-try_to_munlock-from-try_to_unmap +++ a/Documentation/vm/unevictable-lru.rst @@ -389,14 +389,14 @@ mlocked, munlock_vma_page() updates that mlocked pages. Note, however, that at this point we haven't checked whether the page is mapped by other VM_LOCKED VMAs. -We can't call try_to_munlock(), the function that walks the reverse map to +We can't call page_mlock(), the function that walks the reverse map to check for other VM_LOCKED VMAs, without first isolating the page from the LRU. -try_to_munlock() is a variant of try_to_unmap() and thus requires that the page +page_mlock() is a variant of try_to_unmap() and thus requires that the page not be on an LRU list [more on these below]. However, the call to -isolate_lru_page() could fail, in which case we couldn't try_to_munlock(). So, +isolate_lru_page() could fail, in which case we can't call page_mlock(). So, we go ahead and clear PG_mlocked up front, as this might be the only chance we -have. If we can successfully isolate the page, we go ahead and -try_to_munlock(), which will restore the PG_mlocked flag and update the zone +have. If we can successfully isolate the page, we go ahead and call +page_mlock(), which will restore the PG_mlocked flag and update the zone page statistics if it finds another VMA holding the page mlocked. If we fail to isolate the page, we'll have left a potentially mlocked page on the LRU. This is fine, because we'll catch it later if and if vmscan tries to reclaim @@ -545,31 +545,24 @@ munlock or munmap system calls, mm teard holepunching, and truncation of file pages and their anonymous COWed pages. -try_to_munlock() Reverse Map Scan +page_mlock() Reverse Map Scan --------------------------------- -.. warning:: - [!] TODO/FIXME: a better name might be page_mlocked() - analogous to the - page_referenced() reverse map walker. - When munlock_vma_page() [see section :ref:`munlock()/munlockall() System Call Handling ` above] tries to munlock a page, it needs to determine whether or not the page is mapped by any VM_LOCKED VMA without actually attempting to unmap all PTEs from the page. For this purpose, the unevictable/mlock infrastructure -introduced a variant of try_to_unmap() called try_to_munlock(). +introduced a variant of try_to_unmap() called page_mlock(). -try_to_munlock() calls the same functions as try_to_unmap() for anonymous and -mapped file and KSM pages with a flag argument specifying unlock versus unmap -processing. Again, these functions walk the respective reverse maps looking -for VM_LOCKED VMAs. When such a VMA is found, as in the try_to_unmap() case, -the functions mlock the page via mlock_vma_page() and return SWAP_MLOCK. This -undoes the pre-clearing of the page's PG_mlocked done by munlock_vma_page. +page_mlock() walks the respective reverse maps looking for VM_LOCKED VMAs. When +such a VMA is found the page is mlocked via mlock_vma_page(). This undoes the +pre-clearing of the page's PG_mlocked done by munlock_vma_page. -Note that try_to_munlock()'s reverse map walk must visit every VMA in a page's +Note that page_mlock()'s reverse map walk must visit every VMA in a page's reverse map to determine that a page is NOT mapped into any VM_LOCKED VMA. However, the scan can terminate when it encounters a VM_LOCKED VMA. -Although try_to_munlock() might be called a great many times when munlocking a +Although page_mlock() might be called a great many times when munlocking a large region or tearing down a large address space that has been mlocked via mlockall(), overall this is a fairly rare event. @@ -602,7 +595,7 @@ inactive lists to the appropriate node's shrink_inactive_list() should only see SHM_LOCK'd pages that became SHM_LOCK'd after shrink_active_list() had moved them to the inactive list, or pages mapped into VM_LOCKED VMAs that munlock_vma_page() couldn't isolate from the LRU to -recheck via try_to_munlock(). shrink_inactive_list() won't notice the latter, +recheck via page_mlock(). shrink_inactive_list() won't notice the latter, but will pass on to shrink_page_list(). shrink_page_list() again culls obviously unevictable pages that it could --- a/include/linux/rmap.h~mm-rmap-split-try_to_munlock-from-try_to_unmap +++ a/include/linux/rmap.h @@ -87,7 +87,6 @@ struct anon_vma_chain { enum ttu_flags { TTU_MIGRATION = 0x1, /* migration mode */ - TTU_MUNLOCK = 0x2, /* munlock mode */ TTU_SPLIT_HUGE_PMD = 0x4, /* split huge PMD if any */ TTU_IGNORE_MLOCK = 0x8, /* ignore mlock */ @@ -240,7 +239,7 @@ int page_mkclean(struct page *); * called in munlock()/munmap() path to check for other vmas holding * the page mlocked. */ -void try_to_munlock(struct page *); +void page_mlock(struct page *page); void remove_migration_ptes(struct page *old, struct page *new, bool locked); --- a/mm/mlock.c~mm-rmap-split-try_to_munlock-from-try_to_unmap +++ a/mm/mlock.c @@ -108,7 +108,7 @@ void mlock_vma_page(struct page *page) /* * Finish munlock after successful page isolation * - * Page must be locked. This is a wrapper for try_to_munlock() + * Page must be locked. This is a wrapper for page_mlock() * and putback_lru_page() with munlock accounting. */ static void __munlock_isolated_page(struct page *page) @@ -118,7 +118,7 @@ static void __munlock_isolated_page(stru * and we don't need to check all the other vmas. */ if (page_mapcount(page) > 1) - try_to_munlock(page); + page_mlock(page); /* Did try_to_unlock() succeed or punt? */ if (!PageMlocked(page)) @@ -158,7 +158,7 @@ static void __munlock_isolation_failed(s * munlock()ed or munmap()ed, we want to check whether other vmas hold the * page locked so that we can leave it on the unevictable lru list and not * bother vmscan with it. However, to walk the page's rmap list in - * try_to_munlock() we must isolate the page from the LRU. If some other + * page_mlock() we must isolate the page from the LRU. If some other * task has removed the page from the LRU, we won't be able to do that. * So we clear the PageMlocked as we might not get another chance. If we * can't isolate the page, we leave it for putback_lru_page() and vmscan @@ -168,7 +168,7 @@ unsigned int munlock_vma_page(struct pag { int nr_pages; - /* For try_to_munlock() and to serialize with page migration */ + /* For page_mlock() and to serialize with page migration */ BUG_ON(!PageLocked(page)); VM_BUG_ON_PAGE(PageTail(page), page); @@ -205,7 +205,7 @@ static int __mlock_posix_error_return(lo * * The fast path is available only for evictable pages with single mapping. * Then we can bypass the per-cpu pvec and get better performance. - * when mapcount > 1 we need try_to_munlock() which can fail. + * when mapcount > 1 we need page_mlock() which can fail. * when !page_evictable(), we need the full redo logic of putback_lru_page to * avoid leaving evictable page in unevictable list. * @@ -414,7 +414,7 @@ static unsigned long __munlock_pagevec_f * * We don't save and restore VM_LOCKED here because pages are * still on lru. In unmap path, pages might be scanned by reclaim - * and re-mlocked by try_to_{munlock|unmap} before we unmap and + * and re-mlocked by page_mlock/try_to_unmap before we unmap and * free them. This will result in freeing mlocked pages. */ void munlock_vma_pages_range(struct vm_area_struct *vma, --- a/mm/rmap.c~mm-rmap-split-try_to_munlock-from-try_to_unmap +++ a/mm/rmap.c @@ -1411,10 +1411,6 @@ static bool try_to_unmap_one(struct page if (flags & TTU_SYNC) pvmw.flags = PVMW_SYNC; - /* munlock has nothing to gain from examining un-locked vmas */ - if ((flags & TTU_MUNLOCK) && !(vma->vm_flags & VM_LOCKED)) - return true; - if (IS_ENABLED(CONFIG_MIGRATION) && (flags & TTU_MIGRATION) && is_zone_device_page(page) && !is_device_private_page(page)) return true; @@ -1476,8 +1472,6 @@ static bool try_to_unmap_one(struct page page_vma_mapped_walk_done(&pvmw); break; } - if (flags & TTU_MUNLOCK) - continue; } /* Unexpected PMD-mapped THP? */ @@ -1790,20 +1784,58 @@ void try_to_unmap(struct page *page, enu rmap_walk(page, &rwc); } +/* + * Walks the vma's mapping a page and mlocks the page if any locked vma's are + * found. Once one is found the page is locked and the scan can be terminated. + */ +static bool page_mlock_one(struct page *page, struct vm_area_struct *vma, + unsigned long address, void *unused) +{ + struct page_vma_mapped_walk pvmw = { + .page = page, + .vma = vma, + .address = address, + }; + + /* An un-locked vma doesn't have any pages to lock, continue the scan */ + if (!(vma->vm_flags & VM_LOCKED)) + return true; + + while (page_vma_mapped_walk(&pvmw)) { + /* + * Need to recheck under the ptl to serialise with + * __munlock_pagevec_fill() after VM_LOCKED is cleared in + * munlock_vma_pages_range(). + */ + if (vma->vm_flags & VM_LOCKED) { + /* PTE-mapped THP are never mlocked */ + if (!PageTransCompound(page)) + mlock_vma_page(page); + page_vma_mapped_walk_done(&pvmw); + } + + /* + * no need to continue scanning other vma's if the page has + * been locked. + */ + return false; + } + + return true; +} + /** - * try_to_munlock - try to munlock a page - * @page: the page to be munlocked + * page_mlock - try to mlock a page + * @page: the page to be mlocked * - * Called from munlock code. Checks all of the VMAs mapping the page - * to make sure nobody else has this page mlocked. The page will be - * returned with PG_mlocked cleared if no other vmas have it mlocked. + * Called from munlock code. Checks all of the VMAs mapping the page and mlocks + * the page if any are found. The page will be returned with PG_mlocked cleared + * if it is not mapped by any locked vmas. */ - -void try_to_munlock(struct page *page) +void page_mlock(struct page *page) { struct rmap_walk_control rwc = { - .rmap_one = try_to_unmap_one, - .arg = (void *)TTU_MUNLOCK, + .rmap_one = page_mlock_one, .done = page_not_mapped, .anon_lock = page_lock_anon_vma_read, @@ -1855,7 +1887,7 @@ static struct anon_vma *rmap_walk_anon_l * Find all the mappings of a page using the mapping pointer and the vma chains * contained in the anon_vma struct it points to. * - * When called from try_to_munlock(), the mmap_lock of the mm containing the vma + * When called from page_mlock(), the mmap_lock of the mm containing the vma * where the page was found will be held for write. So, we won't recheck * vm_flags for that VMA. That should be OK, because that vma shouldn't be * LOCKED. @@ -1908,7 +1940,7 @@ static void rmap_walk_anon(struct page * * Find all the mappings of a page using the mapping pointer and the vma chains * contained in the address_space struct it points to. * - * When called from try_to_munlock(), the mmap_lock of the mm containing the vma + * When called from page_mlock(), the mmap_lock of the mm containing the vma * where the page was found will be held for write. So, we won't recheck * vm_flags for that VMA. That should be OK, because that vma shouldn't be * LOCKED. From patchwork Thu Jul 1 01:54:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81EFDC11F66 for ; Thu, 1 Jul 2021 01:54:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 23EB361241 for ; Thu, 1 Jul 2021 01:54:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 23EB361241 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A0CF98D0258; Wed, 30 Jun 2021 21:54:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BCA98D024F; Wed, 30 Jun 2021 21:54:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80F948D0258; Wed, 30 Jun 2021 21:54:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 507FD8D024F for ; Wed, 30 Jun 2021 21:54:18 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2549F22C15 for ; Thu, 1 Jul 2021 01:54:18 +0000 (UTC) X-FDA: 78312348996.29.0D76BFA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id C4C98F000128 for ; Thu, 1 Jul 2021 01:54:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8369161468; Thu, 1 Jul 2021 01:54:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104456; bh=WUC+p9meDlC/sRCLDrS9MmF4rEZKaCVk61c/nbQRKEE=; h=Date:From:To:Subject:In-Reply-To:From; b=g2fi2rUWTvPovqja1A+sICpqqtbGJNC09FJRAqJPfivKQHa1dKyNjG4dMpLNTYlVu /I4qJ4wnAzcwBMmy1rCLEyrD5uza7Xz4bM2j7lecUgjdCYEk3ilmyNxliTnNWnhJ2e EmUBWynI7btgwdv9LWp8U56mlgVdrg9uAHg4yJPU= Date: Wed, 30 Jun 2021 18:54:16 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 133/192] mm/rmap: split migration into its own function Message-ID: <20210701015416.0t4MkxtDR%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C4C98F000128 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=g2fi2rUW; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: kykp5az7rp3dshu8to313wc4u991tn8d X-HE-Tag: 1625104457-786418 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm/rmap: split migration into its own function Migration is currently implemented as a mode of operation for try_to_unmap_one() generally specified by passing the TTU_MIGRATION flag or in the case of splitting a huge anonymous page TTU_SPLIT_FREEZE. However it does not have much in common with the rest of the unmap functionality of try_to_unmap_one() and thus splitting it into a separate function reduces the complexity of try_to_unmap_one() making it more readable. Several simplifications can also be made in try_to_migrate_one() based on the following observations: - All users of TTU_MIGRATION also set TTU_IGNORE_MLOCK. - No users of TTU_MIGRATION ever set TTU_IGNORE_HWPOISON. - No users of TTU_MIGRATION ever set TTU_BATCH_FLUSH. TTU_SPLIT_FREEZE is a special case of migration used when splitting an anonymous page. This is most easily dealt with by calling the correct function from unmap_page() in mm/huge_memory.c - either try_to_migrate() for PageAnon or try_to_unmap(). Link: https://lkml.kernel.org/r/20210616105937.23201-5-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Christoph Hellwig Reviewed-by: Ralph Campbell Cc: Ben Skeggs Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Shakeel Butt Signed-off-by: Andrew Morton --- include/linux/rmap.h | 4 mm/huge_memory.c | 16 + mm/migrate.c | 9 - mm/rmap.c | 367 ++++++++++++++++++++++++++++++----------- 4 files changed, 289 insertions(+), 107 deletions(-) --- a/include/linux/rmap.h~mm-rmap-split-migration-into-its-own-function +++ a/include/linux/rmap.h @@ -86,8 +86,6 @@ struct anon_vma_chain { }; enum ttu_flags { - TTU_MIGRATION = 0x1, /* migration mode */ - TTU_SPLIT_HUGE_PMD = 0x4, /* split huge PMD if any */ TTU_IGNORE_MLOCK = 0x8, /* ignore mlock */ TTU_SYNC = 0x10, /* avoid racy checks with PVMW_SYNC */ @@ -97,7 +95,6 @@ enum ttu_flags { * do a final flush if necessary */ TTU_RMAP_LOCKED = 0x80, /* do not grab rmap lock: * caller holds it */ - TTU_SPLIT_FREEZE = 0x100, /* freeze pte under splitting thp */ }; #ifdef CONFIG_MMU @@ -194,6 +191,7 @@ static inline void page_dup_rmap(struct int page_referenced(struct page *, int is_locked, struct mem_cgroup *memcg, unsigned long *vm_flags); +void try_to_migrate(struct page *page, enum ttu_flags flags); void try_to_unmap(struct page *, enum ttu_flags flags); /* Avoid racy checks */ --- a/mm/huge_memory.c~mm-rmap-split-migration-into-its-own-function +++ a/mm/huge_memory.c @@ -2309,16 +2309,20 @@ void vma_adjust_trans_huge(struct vm_are static void unmap_page(struct page *page) { - enum ttu_flags ttu_flags = TTU_IGNORE_MLOCK | TTU_SYNC | - TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD; + enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD | + TTU_SYNC; VM_BUG_ON_PAGE(!PageHead(page), page); - /* If TTU_SPLIT_FREEZE is ever extended to file, update remap_page() */ + /* + * Anon pages need migration entries to preserve them, but file + * pages can simply be left unmapped, then faulted back on demand. + * If that is ever changed (perhaps for mlock), update remap_page(). + */ if (PageAnon(page)) - ttu_flags |= TTU_SPLIT_FREEZE; - - try_to_unmap(page, ttu_flags); + try_to_migrate(page, ttu_flags); + else + try_to_unmap(page, ttu_flags | TTU_IGNORE_MLOCK); VM_WARN_ON_ONCE_PAGE(page_mapped(page), page); } --- a/mm/migrate.c~mm-rmap-split-migration-into-its-own-function +++ a/mm/migrate.c @@ -1109,7 +1109,7 @@ static int __unmap_and_move(struct page /* Establish migration ptes */ VM_BUG_ON_PAGE(PageAnon(page) && !PageKsm(page) && !anon_vma, page); - try_to_unmap(page, TTU_MIGRATION|TTU_IGNORE_MLOCK); + try_to_migrate(page, 0); page_was_mapped = 1; } @@ -1311,7 +1311,7 @@ static int unmap_and_move_huge_page(new_ if (page_mapped(hpage)) { bool mapping_locked = false; - enum ttu_flags ttu = TTU_MIGRATION|TTU_IGNORE_MLOCK; + enum ttu_flags ttu = 0; if (!PageAnon(hpage)) { /* @@ -1328,7 +1328,7 @@ static int unmap_and_move_huge_page(new_ ttu |= TTU_RMAP_LOCKED; } - try_to_unmap(hpage, ttu); + try_to_migrate(hpage, ttu); page_was_mapped = 1; if (mapping_locked) @@ -2602,7 +2602,6 @@ static void migrate_vma_prepare(struct m */ static void migrate_vma_unmap(struct migrate_vma *migrate) { - int flags = TTU_MIGRATION | TTU_IGNORE_MLOCK; const unsigned long npages = migrate->npages; const unsigned long start = migrate->start; unsigned long addr, i, restore = 0; @@ -2614,7 +2613,7 @@ static void migrate_vma_unmap(struct mig continue; if (page_mapped(page)) { - try_to_unmap(page, flags); + try_to_migrate(page, 0); if (page_mapped(page)) goto restore; } --- a/mm/rmap.c~mm-rmap-split-migration-into-its-own-function +++ a/mm/rmap.c @@ -1411,14 +1411,8 @@ static bool try_to_unmap_one(struct page if (flags & TTU_SYNC) pvmw.flags = PVMW_SYNC; - if (IS_ENABLED(CONFIG_MIGRATION) && (flags & TTU_MIGRATION) && - is_zone_device_page(page) && !is_device_private_page(page)) - return true; - - if (flags & TTU_SPLIT_HUGE_PMD) { - split_huge_pmd_address(vma, address, - flags & TTU_SPLIT_FREEZE, page); - } + if (flags & TTU_SPLIT_HUGE_PMD) + split_huge_pmd_address(vma, address, false, page); /* * For THP, we have to assume the worse case ie pmd for invalidation. @@ -1443,16 +1437,6 @@ static bool try_to_unmap_one(struct page mmu_notifier_invalidate_range_start(&range); while (page_vma_mapped_walk(&pvmw)) { -#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION - /* PMD-mapped THP migration entry */ - if (!pvmw.pte && (flags & TTU_MIGRATION)) { - VM_BUG_ON_PAGE(PageHuge(page) || !PageTransCompound(page), page); - - set_pmd_migration_entry(&pvmw, page); - continue; - } -#endif - /* * If the page is mlock()d, we cannot swap it out. * If it's recently referenced (perhaps page_referenced @@ -1514,46 +1498,6 @@ static bool try_to_unmap_one(struct page } } - if (IS_ENABLED(CONFIG_MIGRATION) && - (flags & TTU_MIGRATION) && - is_zone_device_page(page)) { - swp_entry_t entry; - pte_t swp_pte; - - pteval = ptep_get_and_clear(mm, pvmw.address, pvmw.pte); - - /* - * Store the pfn of the page in a special migration - * pte. do_swap_page() will wait until the migration - * pte is removed and then restart fault handling. - */ - entry = make_readable_migration_entry(page_to_pfn(page)); - swp_pte = swp_entry_to_pte(entry); - - /* - * pteval maps a zone device page and is therefore - * a swap pte. - */ - if (pte_swp_soft_dirty(pteval)) - swp_pte = pte_swp_mksoft_dirty(swp_pte); - if (pte_swp_uffd_wp(pteval)) - swp_pte = pte_swp_mkuffd_wp(swp_pte); - set_pte_at(mm, pvmw.address, pvmw.pte, swp_pte); - /* - * No need to invalidate here it will synchronize on - * against the special swap migration pte. - * - * The assignment to subpage above was computed from a - * swap PTE which results in an invalid pointer. - * Since only PAGE_SIZE pages can currently be - * migrated, just set it to page. This will need to be - * changed when hugepage migrations to device private - * memory are supported. - */ - subpage = page; - goto discard; - } - /* Nuke the page table entry. */ flush_cache_page(vma, address, pte_pfn(*pvmw.pte)); if (should_defer_flush(mm, flags)) { @@ -1606,39 +1550,6 @@ static bool try_to_unmap_one(struct page /* We have to invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); - } else if (IS_ENABLED(CONFIG_MIGRATION) && - (flags & (TTU_MIGRATION|TTU_SPLIT_FREEZE))) { - swp_entry_t entry; - pte_t swp_pte; - - if (arch_unmap_one(mm, vma, address, pteval) < 0) { - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - break; - } - - /* - * Store the pfn of the page in a special migration - * pte. do_swap_page() will wait until the migration - * pte is removed and then restart fault handling. - */ - if (pte_write(pteval)) - entry = make_writable_migration_entry( - page_to_pfn(subpage)); - else - entry = make_readable_migration_entry( - page_to_pfn(subpage)); - swp_pte = swp_entry_to_pte(entry); - if (pte_soft_dirty(pteval)) - swp_pte = pte_swp_mksoft_dirty(swp_pte); - if (pte_uffd_wp(pteval)) - swp_pte = pte_swp_mkuffd_wp(swp_pte); - set_pte_at(mm, address, pvmw.pte, swp_pte); - /* - * No need to invalidate here it will synchronize on - * against the special swap migration pte. - */ } else if (PageAnon(page)) { swp_entry_t entry = { .val = page_private(subpage) }; pte_t swp_pte; @@ -1766,6 +1677,277 @@ void try_to_unmap(struct page *page, enu .anon_lock = page_lock_anon_vma_read, }; + if (flags & TTU_RMAP_LOCKED) + rmap_walk_locked(page, &rwc); + else + rmap_walk(page, &rwc); +} + +/* + * @arg: enum ttu_flags will be passed to this argument. + * + * If TTU_SPLIT_HUGE_PMD is specified any PMD mappings will be split into PTEs + * containing migration entries. This and TTU_RMAP_LOCKED are the only supported + * flags. + */ +static bool try_to_migrate_one(struct page *page, struct vm_area_struct *vma, + unsigned long address, void *arg) +{ + struct mm_struct *mm = vma->vm_mm; + struct page_vma_mapped_walk pvmw = { + .page = page, + .vma = vma, + .address = address, + }; + pte_t pteval; + struct page *subpage; + bool ret = true; + struct mmu_notifier_range range; + enum ttu_flags flags = (enum ttu_flags)(long)arg; + + if (is_zone_device_page(page) && !is_device_private_page(page)) + return true; + + /* + * When racing against e.g. zap_pte_range() on another cpu, + * in between its ptep_get_and_clear_full() and page_remove_rmap(), + * try_to_migrate() may return before page_mapped() has become false, + * if page table locking is skipped: use TTU_SYNC to wait for that. + */ + if (flags & TTU_SYNC) + pvmw.flags = PVMW_SYNC; + + /* + * unmap_page() in mm/huge_memory.c is the only user of migration with + * TTU_SPLIT_HUGE_PMD and it wants to freeze. + */ + if (flags & TTU_SPLIT_HUGE_PMD) + split_huge_pmd_address(vma, address, true, page); + + /* + * For THP, we have to assume the worse case ie pmd for invalidation. + * For hugetlb, it could be much worse if we need to do pud + * invalidation in the case of pmd sharing. + * + * Note that the page can not be free in this function as call of + * try_to_unmap() must hold a reference on the page. + */ + range.end = PageKsm(page) ? + address + PAGE_SIZE : vma_address_end(page, vma); + mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, + address, range.end); + if (PageHuge(page)) { + /* + * If sharing is possible, start and end will be adjusted + * accordingly. + */ + adjust_range_if_pmd_sharing_possible(vma, &range.start, + &range.end); + } + mmu_notifier_invalidate_range_start(&range); + + while (page_vma_mapped_walk(&pvmw)) { +#ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION + /* PMD-mapped THP migration entry */ + if (!pvmw.pte) { + VM_BUG_ON_PAGE(PageHuge(page) || + !PageTransCompound(page), page); + + set_pmd_migration_entry(&pvmw, page); + continue; + } +#endif + + /* Unexpected PMD-mapped THP? */ + VM_BUG_ON_PAGE(!pvmw.pte, page); + + subpage = page - page_to_pfn(page) + pte_pfn(*pvmw.pte); + address = pvmw.address; + + if (PageHuge(page) && !PageAnon(page)) { + /* + * To call huge_pmd_unshare, i_mmap_rwsem must be + * held in write mode. Caller needs to explicitly + * do this outside rmap routines. + */ + VM_BUG_ON(!(flags & TTU_RMAP_LOCKED)); + if (huge_pmd_unshare(mm, vma, &address, pvmw.pte)) { + /* + * huge_pmd_unshare unmapped an entire PMD + * page. There is no way of knowing exactly + * which PMDs may be cached for this mm, so + * we must flush them all. start/end were + * already adjusted above to cover this range. + */ + flush_cache_range(vma, range.start, range.end); + flush_tlb_range(vma, range.start, range.end); + mmu_notifier_invalidate_range(mm, range.start, + range.end); + + /* + * The ref count of the PMD page was dropped + * which is part of the way map counting + * is done for shared PMDs. Return 'true' + * here. When there is no other sharing, + * huge_pmd_unshare returns false and we will + * unmap the actual page and drop map count + * to zero. + */ + page_vma_mapped_walk_done(&pvmw); + break; + } + } + + /* Nuke the page table entry. */ + flush_cache_page(vma, address, pte_pfn(*pvmw.pte)); + pteval = ptep_clear_flush(vma, address, pvmw.pte); + + /* Move the dirty bit to the page. Now the pte is gone. */ + if (pte_dirty(pteval)) + set_page_dirty(page); + + /* Update high watermark before we lower rss */ + update_hiwater_rss(mm); + + if (is_zone_device_page(page)) { + swp_entry_t entry; + pte_t swp_pte; + + /* + * Store the pfn of the page in a special migration + * pte. do_swap_page() will wait until the migration + * pte is removed and then restart fault handling. + */ + entry = make_readable_migration_entry( + page_to_pfn(page)); + swp_pte = swp_entry_to_pte(entry); + + /* + * pteval maps a zone device page and is therefore + * a swap pte. + */ + if (pte_swp_soft_dirty(pteval)) + swp_pte = pte_swp_mksoft_dirty(swp_pte); + if (pte_swp_uffd_wp(pteval)) + swp_pte = pte_swp_mkuffd_wp(swp_pte); + set_pte_at(mm, pvmw.address, pvmw.pte, swp_pte); + /* + * No need to invalidate here it will synchronize on + * against the special swap migration pte. + * + * The assignment to subpage above was computed from a + * swap PTE which results in an invalid pointer. + * Since only PAGE_SIZE pages can currently be + * migrated, just set it to page. This will need to be + * changed when hugepage migrations to device private + * memory are supported. + */ + subpage = page; + } else if (PageHWPoison(page)) { + pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); + if (PageHuge(page)) { + hugetlb_count_sub(compound_nr(page), mm); + set_huge_swap_pte_at(mm, address, + pvmw.pte, pteval, + vma_mmu_pagesize(vma)); + } else { + dec_mm_counter(mm, mm_counter(page)); + set_pte_at(mm, address, pvmw.pte, pteval); + } + + } else if (pte_unused(pteval) && !userfaultfd_armed(vma)) { + /* + * The guest indicated that the page content is of no + * interest anymore. Simply discard the pte, vmscan + * will take care of the rest. + * A future reference will then fault in a new zero + * page. When userfaultfd is active, we must not drop + * this page though, as its main user (postcopy + * migration) will not expect userfaults on already + * copied pages. + */ + dec_mm_counter(mm, mm_counter(page)); + /* We have to invalidate as we cleared the pte */ + mmu_notifier_invalidate_range(mm, address, + address + PAGE_SIZE); + } else { + swp_entry_t entry; + pte_t swp_pte; + + if (arch_unmap_one(mm, vma, address, pteval) < 0) { + set_pte_at(mm, address, pvmw.pte, pteval); + ret = false; + page_vma_mapped_walk_done(&pvmw); + break; + } + + /* + * Store the pfn of the page in a special migration + * pte. do_swap_page() will wait until the migration + * pte is removed and then restart fault handling. + */ + if (pte_write(pteval)) + entry = make_writable_migration_entry( + page_to_pfn(subpage)); + else + entry = make_readable_migration_entry( + page_to_pfn(subpage)); + + swp_pte = swp_entry_to_pte(entry); + if (pte_soft_dirty(pteval)) + swp_pte = pte_swp_mksoft_dirty(swp_pte); + if (pte_uffd_wp(pteval)) + swp_pte = pte_swp_mkuffd_wp(swp_pte); + set_pte_at(mm, address, pvmw.pte, swp_pte); + /* + * No need to invalidate here it will synchronize on + * against the special swap migration pte. + */ + } + + /* + * No need to call mmu_notifier_invalidate_range() it has be + * done above for all cases requiring it to happen under page + * table lock before mmu_notifier_invalidate_range_end() + * + * See Documentation/vm/mmu_notifier.rst + */ + page_remove_rmap(subpage, PageHuge(page)); + put_page(page); + } + + mmu_notifier_invalidate_range_end(&range); + + return ret; +} + +/** + * try_to_migrate - try to replace all page table mappings with swap entries + * @page: the page to replace page table entries for + * @flags: action and flags + * + * Tries to remove all the page table entries which are mapping this page and + * replace them with special swap entries. Caller must hold the page lock. + * + * If is successful, return true. Otherwise, false. + */ +void try_to_migrate(struct page *page, enum ttu_flags flags) +{ + struct rmap_walk_control rwc = { + .rmap_one = try_to_migrate_one, + .arg = (void *)flags, + .done = page_not_mapped, + .anon_lock = page_lock_anon_vma_read, + }; + + /* + * Migration always ignores mlock and only supports TTU_RMAP_LOCKED and + * TTU_SPLIT_HUGE_PMD and TTU_SYNC flags. + */ + if (WARN_ON_ONCE(flags & ~(TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD | + TTU_SYNC))) + return; + /* * During exec, a temporary VMA is setup and later moved. * The VMA is moved under the anon_vma lock but not the @@ -1774,8 +1956,7 @@ void try_to_unmap(struct page *page, enu * locking requirements of exec(), migration skips * temporary VMAs until after exec() completes. */ - if ((flags & (TTU_MIGRATION|TTU_SPLIT_FREEZE)) - && !PageKsm(page) && PageAnon(page)) + if (!PageKsm(page) && PageAnon(page)) rwc.invalid_vma = invalid_migration_vma; if (flags & TTU_RMAP_LOCKED) From patchwork Thu Jul 1 01:54:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353325 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D8EEC11F66 for ; Thu, 1 Jul 2021 01:54:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 19AAE6146D for ; Thu, 1 Jul 2021 01:54:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 19AAE6146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 949A88D0259; Wed, 30 Jun 2021 21:54:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F8AE8D024F; Wed, 30 Jun 2021 21:54:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 74C5A8D0259; Wed, 30 Jun 2021 21:54:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id 491878D024F for ; Wed, 30 Jun 2021 21:54:21 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 24DE922C15 for ; Thu, 1 Jul 2021 01:54:21 +0000 (UTC) X-FDA: 78312349122.21.EF0DDD1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id C79866001A8C for ; Thu, 1 Jul 2021 01:54:20 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B605C61469; Thu, 1 Jul 2021 01:54:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104460; bh=vqaGm00wTqjI9ezwYLUFsDzCwVDvHVyGbUBbSFxL6fU=; h=Date:From:To:Subject:In-Reply-To:From; b=cpliDzJ+tB4Mm6N0RaJuh70ZqbbObU3ysu5Dh/YstlDZA7KMnEXKim1tHpM+Wd34U 8Sn8mATm2vr/DS8AOu4+kvAm8tW2LUgVcIUzCI1S7z60a3a+heBpLwHRbsm8GMmsKX v5jUjogjrbb5/BhNPK9mRJari9oTaE8LgPp5jDOg= Date: Wed, 30 Jun 2021 18:54:19 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 134/192] mm: rename migrate_pgmap_owner Message-ID: <20210701015419.pNG4jYiLs%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C79866001A8C X-Stat-Signature: 6fpa8gtgt1oxsr3gpdr399xxkokw369h Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=cpliDzJ+; dmarc=none; spf=pass (imf14.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104460-746749 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm: rename migrate_pgmap_owner MMU notifier ranges have a migrate_pgmap_owner field which is used by drivers to store a pointer. This is subsequently used by the driver callback to filter MMU_NOTIFY_MIGRATE events. Other notifier event types can also benefit from this filtering, so rename the 'migrate_pgmap_owner' field to 'owner' and create a new notifier initialisation function to initialise this field. Link: https://lkml.kernel.org/r/20210616105937.23201-6-apopple@nvidia.com Signed-off-by: Alistair Popple Suggested-by: Peter Xu Reviewed-by: Peter Xu Cc: Ben Skeggs Cc: Christoph Hellwig Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Ralph Campbell Cc: Shakeel Butt Signed-off-by: Andrew Morton --- Documentation/vm/hmm.rst | 2 +- drivers/gpu/drm/nouveau/nouveau_svm.c | 2 +- include/linux/mmu_notifier.h | 20 ++++++++++---------- lib/test_hmm.c | 2 +- mm/migrate.c | 10 +++++----- 5 files changed, 18 insertions(+), 18 deletions(-) --- a/Documentation/vm/hmm.rst~mm-rename-migrate_pgmap_owner +++ a/Documentation/vm/hmm.rst @@ -332,7 +332,7 @@ between device driver specific code and walks to fill in the ``args->src`` array with PFNs to be migrated. The ``invalidate_range_start()`` callback is passed a ``struct mmu_notifier_range`` with the ``event`` field set to - ``MMU_NOTIFY_MIGRATE`` and the ``migrate_pgmap_owner`` field set to + ``MMU_NOTIFY_MIGRATE`` and the ``owner`` field set to the ``args->pgmap_owner`` field passed to migrate_vma_setup(). This is allows the device driver to skip the invalidation callback and only invalidate device private MMU mappings that are actually migrating. --- a/drivers/gpu/drm/nouveau/nouveau_svm.c~mm-rename-migrate_pgmap_owner +++ a/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -265,7 +265,7 @@ nouveau_svmm_invalidate_range_start(stru * the invalidation is handled as part of the migration process. */ if (update->event == MMU_NOTIFY_MIGRATE && - update->migrate_pgmap_owner == svmm->vmm->cli->drm->dev) + update->owner == svmm->vmm->cli->drm->dev) goto out; if (limit > svmm->unmanaged.start && start < svmm->unmanaged.limit) { --- a/include/linux/mmu_notifier.h~mm-rename-migrate_pgmap_owner +++ a/include/linux/mmu_notifier.h @@ -41,7 +41,7 @@ struct mmu_interval_notifier; * * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal * a device driver to possibly ignore the invalidation if the - * migrate_pgmap_owner field matches the driver's device private pgmap owner. + * owner field matches the driver's device private pgmap owner. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -269,7 +269,7 @@ struct mmu_notifier_range { unsigned long end; unsigned flags; enum mmu_notifier_event event; - void *migrate_pgmap_owner; + void *owner; }; static inline int mm_has_notifiers(struct mm_struct *mm) @@ -521,14 +521,14 @@ static inline void mmu_notifier_range_in range->flags = flags; } -static inline void mmu_notifier_range_init_migrate( - struct mmu_notifier_range *range, unsigned int flags, +static inline void mmu_notifier_range_init_owner( + struct mmu_notifier_range *range, + enum mmu_notifier_event event, unsigned int flags, struct vm_area_struct *vma, struct mm_struct *mm, - unsigned long start, unsigned long end, void *pgmap) + unsigned long start, unsigned long end, void *owner) { - mmu_notifier_range_init(range, MMU_NOTIFY_MIGRATE, flags, vma, mm, - start, end); - range->migrate_pgmap_owner = pgmap; + mmu_notifier_range_init(range, event, flags, vma, mm, start, end); + range->owner = owner; } #define ptep_clear_flush_young_notify(__vma, __address, __ptep) \ @@ -655,8 +655,8 @@ static inline void _mmu_notifier_range_i #define mmu_notifier_range_init(range,event,flags,vma,mm,start,end) \ _mmu_notifier_range_init(range, start, end) -#define mmu_notifier_range_init_migrate(range, flags, vma, mm, start, end, \ - pgmap) \ +#define mmu_notifier_range_init_owner(range, event, flags, vma, mm, start, \ + end, owner) \ _mmu_notifier_range_init(range, start, end) static inline bool --- a/lib/test_hmm.c~mm-rename-migrate_pgmap_owner +++ a/lib/test_hmm.c @@ -218,7 +218,7 @@ static bool dmirror_interval_invalidate( * the invalidation is handled as part of the migration process. */ if (range->event == MMU_NOTIFY_MIGRATE && - range->migrate_pgmap_owner == dmirror->mdevice) + range->owner == dmirror->mdevice) return true; if (mmu_notifier_range_blockable(range)) --- a/mm/migrate.c~mm-rename-migrate_pgmap_owner +++ a/mm/migrate.c @@ -2416,8 +2416,8 @@ static void migrate_vma_collect(struct m * that the registered device driver can skip invalidating device * private page mappings that won't be migrated. */ - mmu_notifier_range_init_migrate(&range, 0, migrate->vma, - migrate->vma->vm_mm, migrate->start, migrate->end, + mmu_notifier_range_init_owner(&range, MMU_NOTIFY_MIGRATE, 0, + migrate->vma, migrate->vma->vm_mm, migrate->start, migrate->end, migrate->pgmap_owner); mmu_notifier_invalidate_range_start(&range); @@ -2927,9 +2927,9 @@ void migrate_vma_pages(struct migrate_vm if (!notified) { notified = true; - mmu_notifier_range_init_migrate(&range, 0, - migrate->vma, migrate->vma->vm_mm, - addr, migrate->end, + mmu_notifier_range_init_owner(&range, + MMU_NOTIFY_MIGRATE, 0, migrate->vma, + migrate->vma->vm_mm, addr, migrate->end, migrate->pgmap_owner); mmu_notifier_invalidate_range_start(&range); } From patchwork Thu Jul 1 01:54:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353327 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 972C8C11F69 for ; Thu, 1 Jul 2021 01:54:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4D81761474 for ; Thu, 1 Jul 2021 01:54:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D81761474 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C7DC08D025A; Wed, 30 Jun 2021 21:54:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C06488D024F; Wed, 30 Jun 2021 21:54:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7FDD8D025A; Wed, 30 Jun 2021 21:54:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id 7A95C8D024F for ; Wed, 30 Jun 2021 21:54:24 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5307C1802EC0A for ; Thu, 1 Jul 2021 01:54:24 +0000 (UTC) X-FDA: 78312349248.15.89BBC4E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id 11984F000129 for ; Thu, 1 Jul 2021 01:54:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E12E96146D; Thu, 1 Jul 2021 01:54:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104463; bh=6Pji3cq4i9A9CHhjXB2IEQFixwrlJYwoxrsU8PnJmPs=; h=Date:From:To:Subject:In-Reply-To:From; b=Az/a3r2eq3VcZvK3iLX/T5N4nM28HBIlDAIuiT3L+YrmJxHQ3S5wLW6kdVfyCsBxk GyvlR6NHOxZlAAOp4LZImFeBZDj3creGwfXShSx7dXHf/k/QENMjdIbRp9z3+kNxKN J9AEj4ixgJzwvADxBf8u8rXHowu8fyuAiGHbxWvM= Date: Wed, 30 Jun 2021 18:54:22 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 135/192] mm/memory.c: allow different return codes for copy_nonpresent_pte() Message-ID: <20210701015422.fnRcl9cvh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="Az/a3r2e"; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: w1eoxgsi9tg3zfhx7wefz1pmemoowpw6 X-Rspamd-Queue-Id: 11984F000129 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104463-360888 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm/memory.c: allow different return codes for copy_nonpresent_pte() Currently if copy_nonpresent_pte() returns a non-zero value it is assumed to be a swap entry which requires further processing outside the loop in copy_pte_range() after dropping locks. This prevents other values being returned to signal conditions such as failure which a subsequent change requires. Instead make copy_nonpresent_pte() return an error code if further processing is required and read the value for the swap entry in the main loop under the ptl. Link: https://lkml.kernel.org/r/20210616105937.23201-7-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Peter Xu Cc: Ben Skeggs Cc: Christoph Hellwig Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Ralph Campbell Cc: Shakeel Butt Signed-off-by: Andrew Morton --- mm/memory.c | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) --- a/mm/memory.c~mm-memoryc-allow-different-return-codes-for-copy_nonpresent_pte +++ a/mm/memory.c @@ -717,7 +717,7 @@ copy_nonpresent_pte(struct mm_struct *ds if (likely(!non_swap_entry(entry))) { if (swap_duplicate(entry) < 0) - return entry.val; + return -EIO; /* make sure dst_mm is on swapoff's mmlist. */ if (unlikely(list_empty(&dst_mm->mmlist))) { @@ -973,12 +973,14 @@ again: continue; } if (unlikely(!pte_present(*src_pte))) { - entry.val = copy_nonpresent_pte(dst_mm, src_mm, - dst_pte, src_pte, - dst_vma, src_vma, - addr, rss); - if (entry.val) + ret = copy_nonpresent_pte(dst_mm, src_mm, + dst_pte, src_pte, + dst_vma, src_vma, + addr, rss); + if (ret == -EIO) { + entry = pte_to_swp_entry(*src_pte); break; + } progress += 8; continue; } @@ -1011,20 +1013,24 @@ again: pte_unmap_unlock(orig_dst_pte, dst_ptl); cond_resched(); - if (entry.val) { + if (ret == -EIO) { + VM_WARN_ON_ONCE(!entry.val); if (add_swap_count_continuation(entry, GFP_KERNEL) < 0) { ret = -ENOMEM; goto out; } entry.val = 0; - } else if (ret) { - WARN_ON_ONCE(ret != -EAGAIN); + } else if (ret == -EAGAIN) { prealloc = page_copy_prealloc(src_mm, src_vma, addr); if (!prealloc) return -ENOMEM; - /* We've captured and resolved the error. Reset, try again. */ - ret = 0; + } else if (ret) { + VM_WARN_ON_ONCE(1); } + + /* We've captured and resolved the error. Reset, try again. */ + ret = 0; + if (addr != end) goto again; out: From patchwork Thu Jul 1 01:54:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C670C11F69 for ; Thu, 1 Jul 2021 01:54:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B353761420 for ; Thu, 1 Jul 2021 01:54:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B353761420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 356258D025B; Wed, 30 Jun 2021 21:54:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 306C78D024F; Wed, 30 Jun 2021 21:54:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 158D28D025B; Wed, 30 Jun 2021 21:54:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0115.hostedemail.com [216.40.44.115]) by kanga.kvack.org (Postfix) with ESMTP id D7DE98D024F for ; Wed, 30 Jun 2021 21:54:27 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A859A256E0 for ; Thu, 1 Jul 2021 01:54:27 +0000 (UTC) X-FDA: 78312349374.23.2083177 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 3CF7D30000A5 for ; Thu, 1 Jul 2021 01:54:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2236D6141A; Thu, 1 Jul 2021 01:54:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104466; bh=flEN5Lq1aIxmebrOpmr+PlLHpnPVPBe9vkcVWxIhzzs=; h=Date:From:To:Subject:In-Reply-To:From; b=Zx9Ad+tFCzowbggr9P6IHt7kFGHhFI/E5ugsuRW49o/zvVCp3bn8WwQmwwku0fBmF DKHlt1VTGUF32CRQz4OaYuTimV+ublQO9vrVexMz9oW+LPXplQWjP3HtzgBzRFjvoM li3S7QcM2rKiid1vPGfP7S/lv5Vl9DPXU5nqBvMQ= Date: Wed, 30 Jun 2021 18:54:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, dan.carpenter@oracle.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 136/192] mm: device exclusive memory access Message-ID: <20210701015425.tR5PCMTvv%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Zx9Ad+tF; dmarc=none; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: py7eh17x36icaqqofw3uokaezjp6zqup X-Rspamd-Queue-Id: 3CF7D30000A5 X-HE-Tag: 1625104467-534753 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm: device exclusive memory access Some devices require exclusive write access to shared virtual memory (SVM) ranges to perform atomic operations on that memory. This requires CPU page tables to be updated to deny access whilst atomic operations are occurring. In order to do this introduce a new swap entry type (SWP_DEVICE_EXCLUSIVE). When a SVM range needs to be marked for exclusive access by a device all page table mappings for the particular range are replaced with device exclusive swap entries. This causes any CPU access to the page to result in a fault. Faults are resovled by replacing the faulting entry with the original mapping. This results in MMU notifiers being called which a driver uses to update access permissions such as revoking atomic access. After notifiers have been called the device will no longer have exclusive access to the region. Walking of the page tables to find the target pages is handled by get_user_pages() rather than a direct page table walk. A direct page table walk similar to what migrate_vma_collect()/unmap() does could also have been utilised. However this resulted in more code similar in functionality to what get_user_pages() provides as page faulting is required to make the PTEs present and to break COW. [dan.carpenter@oracle.com: fix signedness bug in make_device_exclusive_range()] Link: https://lkml.kernel.org/r/YNIz5NVnZ5GiZ3u1@mwanda Link: https://lkml.kernel.org/r/20210616105937.23201-8-apopple@nvidia.com Signed-off-by: Alistair Popple Signed-off-by: Dan Carpenter Reviewed-by: Christoph Hellwig Cc: Ben Skeggs Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Ralph Campbell Cc: Shakeel Butt Signed-off-by: Andrew Morton --- Documentation/vm/hmm.rst | 17 +++ include/linux/mmu_notifier.h | 6 + include/linux/rmap.h | 4 include/linux/swap.h | 9 + include/linux/swapops.h | 44 +++++++ mm/hmm.c | 5 mm/memory.c | 127 +++++++++++++++++++++- mm/mprotect.c | 8 + mm/page_vma_mapped.c | 9 + mm/rmap.c | 186 +++++++++++++++++++++++++++++++++ 10 files changed, 405 insertions(+), 10 deletions(-) --- a/Documentation/vm/hmm.rst~mm-device-exclusive-memory-access +++ a/Documentation/vm/hmm.rst @@ -405,6 +405,23 @@ between device driver specific code and The lock can now be released. +Exclusive access memory +======================= + +Some devices have features such as atomic PTE bits that can be used to implement +atomic access to system memory. To support atomic operations to a shared virtual +memory page such a device needs access to that page which is exclusive of any +userspace access from the CPU. The ``make_device_exclusive_range()`` function +can be used to make a memory range inaccessible from userspace. + +This replaces all mappings for pages in the given range with special swap +entries. Any attempt to access the swap entry results in a fault which is +resovled by replacing the entry with the original mapping. A driver gets +notified that the mapping has been changed by MMU notifiers, after which point +it will no longer have exclusive access to the page. Exclusive access is +guranteed to last until the driver drops the page lock and page reference, at +which point any CPU faults on the page may proceed as described. + Memory cgroup (memcg) and rss accounting ======================================== --- a/include/linux/mmu_notifier.h~mm-device-exclusive-memory-access +++ a/include/linux/mmu_notifier.h @@ -42,6 +42,11 @@ struct mmu_interval_notifier; * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal * a device driver to possibly ignore the invalidation if the * owner field matches the driver's device private pgmap owner. + * + * @MMU_NOTIFY_EXCLUSIVE: to signal a device driver that the device will no + * longer have exclusive access to the page. When sent during creation of an + * exclusive range the owner will be initialised to the value provided by the + * caller of make_device_exclusive_range(), otherwise the owner will be NULL. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -51,6 +56,7 @@ enum mmu_notifier_event { MMU_NOTIFY_SOFT_DIRTY, MMU_NOTIFY_RELEASE, MMU_NOTIFY_MIGRATE, + MMU_NOTIFY_EXCLUSIVE, }; #define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) --- a/include/linux/rmap.h~mm-device-exclusive-memory-access +++ a/include/linux/rmap.h @@ -194,6 +194,10 @@ int page_referenced(struct page *, int i void try_to_migrate(struct page *page, enum ttu_flags flags); void try_to_unmap(struct page *, enum ttu_flags flags); +int make_device_exclusive_range(struct mm_struct *mm, unsigned long start, + unsigned long end, struct page **pages, + void *arg); + /* Avoid racy checks */ #define PVMW_SYNC (1 << 0) /* Look for migarion entries rather than present PTEs */ --- a/include/linux/swap.h~mm-device-exclusive-memory-access +++ a/include/linux/swap.h @@ -62,12 +62,17 @@ static inline int current_is_kswapd(void * migrate part of a process memory to device memory. * * When a page is migrated from CPU to device, we set the CPU page table entry - * to a special SWP_DEVICE_* entry. + * to a special SWP_DEVICE_{READ|WRITE} entry. + * + * When a page is mapped by the device for exclusive access we set the CPU page + * table entries to special SWP_DEVICE_EXCLUSIVE_* entries. */ #ifdef CONFIG_DEVICE_PRIVATE -#define SWP_DEVICE_NUM 2 +#define SWP_DEVICE_NUM 4 #define SWP_DEVICE_WRITE (MAX_SWAPFILES+SWP_HWPOISON_NUM+SWP_MIGRATION_NUM) #define SWP_DEVICE_READ (MAX_SWAPFILES+SWP_HWPOISON_NUM+SWP_MIGRATION_NUM+1) +#define SWP_DEVICE_EXCLUSIVE_WRITE (MAX_SWAPFILES+SWP_HWPOISON_NUM+SWP_MIGRATION_NUM+2) +#define SWP_DEVICE_EXCLUSIVE_READ (MAX_SWAPFILES+SWP_HWPOISON_NUM+SWP_MIGRATION_NUM+3) #else #define SWP_DEVICE_NUM 0 #endif --- a/include/linux/swapops.h~mm-device-exclusive-memory-access +++ a/include/linux/swapops.h @@ -127,6 +127,27 @@ static inline bool is_writable_device_pr { return unlikely(swp_type(entry) == SWP_DEVICE_WRITE); } + +static inline swp_entry_t make_readable_device_exclusive_entry(pgoff_t offset) +{ + return swp_entry(SWP_DEVICE_EXCLUSIVE_READ, offset); +} + +static inline swp_entry_t make_writable_device_exclusive_entry(pgoff_t offset) +{ + return swp_entry(SWP_DEVICE_EXCLUSIVE_WRITE, offset); +} + +static inline bool is_device_exclusive_entry(swp_entry_t entry) +{ + return swp_type(entry) == SWP_DEVICE_EXCLUSIVE_READ || + swp_type(entry) == SWP_DEVICE_EXCLUSIVE_WRITE; +} + +static inline bool is_writable_device_exclusive_entry(swp_entry_t entry) +{ + return unlikely(swp_type(entry) == SWP_DEVICE_EXCLUSIVE_WRITE); +} #else /* CONFIG_DEVICE_PRIVATE */ static inline swp_entry_t make_readable_device_private_entry(pgoff_t offset) { @@ -147,6 +168,26 @@ static inline bool is_writable_device_pr { return false; } + +static inline swp_entry_t make_readable_device_exclusive_entry(pgoff_t offset) +{ + return swp_entry(0, 0); +} + +static inline swp_entry_t make_writable_device_exclusive_entry(pgoff_t offset) +{ + return swp_entry(0, 0); +} + +static inline bool is_device_exclusive_entry(swp_entry_t entry) +{ + return false; +} + +static inline bool is_writable_device_exclusive_entry(swp_entry_t entry) +{ + return false; +} #endif /* CONFIG_DEVICE_PRIVATE */ #ifdef CONFIG_MIGRATION @@ -226,7 +267,8 @@ static inline struct page *pfn_swap_entr */ static inline bool is_pfn_swap_entry(swp_entry_t entry) { - return is_migration_entry(entry) || is_device_private_entry(entry); + return is_migration_entry(entry) || is_device_private_entry(entry) || + is_device_exclusive_entry(entry); } struct page_vma_mapped_walk; --- a/mm/hmm.c~mm-device-exclusive-memory-access +++ a/mm/hmm.c @@ -26,6 +26,8 @@ #include #include +#include "internal.h" + struct hmm_vma_walk { struct hmm_range *range; unsigned long last; @@ -271,6 +273,9 @@ static int hmm_vma_handle_pte(struct mm_ if (!non_swap_entry(entry)) goto fault; + if (is_device_exclusive_entry(entry)) + goto fault; + if (is_migration_entry(entry)) { pte_unmap(ptep); hmm_vma_walk->last = addr; --- a/mm/memory.c~mm-device-exclusive-memory-access +++ a/mm/memory.c @@ -699,6 +699,68 @@ out: } #endif +static void restore_exclusive_pte(struct vm_area_struct *vma, + struct page *page, unsigned long address, + pte_t *ptep) +{ + pte_t pte; + swp_entry_t entry; + + pte = pte_mkold(mk_pte(page, READ_ONCE(vma->vm_page_prot))); + if (pte_swp_soft_dirty(*ptep)) + pte = pte_mksoft_dirty(pte); + + entry = pte_to_swp_entry(*ptep); + if (pte_swp_uffd_wp(*ptep)) + pte = pte_mkuffd_wp(pte); + else if (is_writable_device_exclusive_entry(entry)) + pte = maybe_mkwrite(pte_mkdirty(pte), vma); + + set_pte_at(vma->vm_mm, address, ptep, pte); + + /* + * No need to take a page reference as one was already + * created when the swap entry was made. + */ + if (PageAnon(page)) + page_add_anon_rmap(page, vma, address, false); + else + /* + * Currently device exclusive access only supports anonymous + * memory so the entry shouldn't point to a filebacked page. + */ + WARN_ON_ONCE(!PageAnon(page)); + + if (vma->vm_flags & VM_LOCKED) + mlock_vma_page(page); + + /* + * No need to invalidate - it was non-present before. However + * secondary CPUs may have mappings that need invalidating. + */ + update_mmu_cache(vma, address, ptep); +} + +/* + * Tries to restore an exclusive pte if the page lock can be acquired without + * sleeping. + */ +static int +try_restore_exclusive_pte(pte_t *src_pte, struct vm_area_struct *vma, + unsigned long addr) +{ + swp_entry_t entry = pte_to_swp_entry(*src_pte); + struct page *page = pfn_swap_entry_to_page(entry); + + if (trylock_page(page)) { + restore_exclusive_pte(vma, page, addr, src_pte); + unlock_page(page); + return 0; + } + + return -EBUSY; +} + /* * copy one vm_area from one task to the other. Assumes the page tables * already present in the new task to be cleared in the whole range @@ -780,6 +842,17 @@ copy_nonpresent_pte(struct mm_struct *ds pte = pte_swp_mkuffd_wp(pte); set_pte_at(src_mm, addr, src_pte, pte); } + } else if (is_device_exclusive_entry(entry)) { + /* + * Make device exclusive entries present by restoring the + * original entry then copying as for a present pte. Device + * exclusive entries currently only support private writable + * (ie. COW) mappings. + */ + VM_BUG_ON(!is_cow_mapping(src_vma->vm_flags)); + if (try_restore_exclusive_pte(src_pte, src_vma, addr)) + return -EBUSY; + return -ENOENT; } if (!userfaultfd_wp(dst_vma)) pte = pte_swp_clear_uffd_wp(pte); @@ -980,9 +1053,18 @@ again: if (ret == -EIO) { entry = pte_to_swp_entry(*src_pte); break; + } else if (ret == -EBUSY) { + break; + } else if (!ret) { + progress += 8; + continue; } - progress += 8; - continue; + + /* + * Device exclusive entry restored, continue by copying + * the now present pte. + */ + WARN_ON_ONCE(ret != -ENOENT); } /* copy_present_pte() will clear `*prealloc' if consumed */ ret = copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, @@ -1020,6 +1102,8 @@ again: goto out; } entry.val = 0; + } else if (ret == -EBUSY) { + goto out; } else if (ret == -EAGAIN) { prealloc = page_copy_prealloc(src_mm, src_vma, addr); if (!prealloc) @@ -1287,7 +1371,8 @@ again: } entry = pte_to_swp_entry(ptent); - if (is_device_private_entry(entry)) { + if (is_device_private_entry(entry) || + is_device_exclusive_entry(entry)) { struct page *page = pfn_swap_entry_to_page(entry); if (unlikely(details && details->check_mapping)) { @@ -1303,7 +1388,10 @@ again: pte_clear_not_present_full(mm, addr, pte, tlb->fullmm); rss[mm_counter(page)]--; - page_remove_rmap(page, false); + + if (is_device_private_entry(entry)) + page_remove_rmap(page, false); + put_page(page); continue; } @@ -3352,6 +3440,34 @@ void unmap_mapping_range(struct address_ EXPORT_SYMBOL(unmap_mapping_range); /* + * Restore a potential device exclusive pte to a working pte entry + */ +static vm_fault_t remove_device_exclusive_entry(struct vm_fault *vmf) +{ + struct page *page = vmf->page; + struct vm_area_struct *vma = vmf->vma; + struct mmu_notifier_range range; + + if (!lock_page_or_retry(page, vma->vm_mm, vmf->flags)) + return VM_FAULT_RETRY; + mmu_notifier_range_init_owner(&range, MMU_NOTIFY_EXCLUSIVE, 0, vma, + vma->vm_mm, vmf->address & PAGE_MASK, + (vmf->address & PAGE_MASK) + PAGE_SIZE, NULL); + mmu_notifier_invalidate_range_start(&range); + + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, + &vmf->ptl); + if (likely(pte_same(*vmf->pte, vmf->orig_pte))) + restore_exclusive_pte(vma, page, vmf->address, vmf->pte); + + pte_unmap_unlock(vmf->pte, vmf->ptl); + unlock_page(page); + + mmu_notifier_invalidate_range_end(&range); + return 0; +} + +/* * We enter with non-exclusive mmap_lock (to exclude vma changes, * but allow concurrent faults), and pte mapped but not yet locked. * We return with pte unmapped and unlocked. @@ -3379,6 +3495,9 @@ vm_fault_t do_swap_page(struct vm_fault if (is_migration_entry(entry)) { migration_entry_wait(vma->vm_mm, vmf->pmd, vmf->address); + } else if (is_device_exclusive_entry(entry)) { + vmf->page = pfn_swap_entry_to_page(entry); + ret = remove_device_exclusive_entry(vmf); } else if (is_device_private_entry(entry)) { vmf->page = pfn_swap_entry_to_page(entry); ret = vmf->page->pgmap->ops->migrate_to_ram(vmf); --- a/mm/mprotect.c~mm-device-exclusive-memory-access +++ a/mm/mprotect.c @@ -165,6 +165,14 @@ static unsigned long change_pte_range(st newpte = swp_entry_to_pte(entry); if (pte_swp_uffd_wp(oldpte)) newpte = pte_swp_mkuffd_wp(newpte); + } else if (is_writable_device_exclusive_entry(entry)) { + entry = make_readable_device_exclusive_entry( + swp_offset(entry)); + newpte = swp_entry_to_pte(entry); + if (pte_swp_soft_dirty(oldpte)) + newpte = pte_swp_mksoft_dirty(newpte); + if (pte_swp_uffd_wp(oldpte)) + newpte = pte_swp_mkuffd_wp(newpte); } else { newpte = oldpte; } --- a/mm/page_vma_mapped.c~mm-device-exclusive-memory-access +++ a/mm/page_vma_mapped.c @@ -41,7 +41,8 @@ static bool map_pte(struct page_vma_mapp /* Handle un-addressable ZONE_DEVICE memory */ entry = pte_to_swp_entry(*pvmw->pte); - if (!is_device_private_entry(entry)) + if (!is_device_private_entry(entry) && + !is_device_exclusive_entry(entry)) return false; } else if (!pte_present(*pvmw->pte)) return false; @@ -93,7 +94,8 @@ static bool check_pte(struct page_vma_ma return false; entry = pte_to_swp_entry(*pvmw->pte); - if (!is_migration_entry(entry)) + if (!is_migration_entry(entry) && + !is_device_exclusive_entry(entry)) return false; pfn = swp_offset(entry); @@ -102,7 +104,8 @@ static bool check_pte(struct page_vma_ma /* Handle un-addressable ZONE_DEVICE memory */ entry = pte_to_swp_entry(*pvmw->pte); - if (!is_device_private_entry(entry)) + if (!is_device_private_entry(entry) && + !is_device_exclusive_entry(entry)) return false; pfn = swp_offset(entry); --- a/mm/rmap.c~mm-device-exclusive-memory-access +++ a/mm/rmap.c @@ -2028,6 +2028,192 @@ void page_mlock(struct page *page) rmap_walk(page, &rwc); } +#ifdef CONFIG_DEVICE_PRIVATE +struct make_exclusive_args { + struct mm_struct *mm; + unsigned long address; + void *owner; + bool valid; +}; + +static bool page_make_device_exclusive_one(struct page *page, + struct vm_area_struct *vma, unsigned long address, void *priv) +{ + struct mm_struct *mm = vma->vm_mm; + struct page_vma_mapped_walk pvmw = { + .page = page, + .vma = vma, + .address = address, + }; + struct make_exclusive_args *args = priv; + pte_t pteval; + struct page *subpage; + bool ret = true; + struct mmu_notifier_range range; + swp_entry_t entry; + pte_t swp_pte; + + mmu_notifier_range_init_owner(&range, MMU_NOTIFY_EXCLUSIVE, 0, vma, + vma->vm_mm, address, min(vma->vm_end, + address + page_size(page)), args->owner); + mmu_notifier_invalidate_range_start(&range); + + while (page_vma_mapped_walk(&pvmw)) { + /* Unexpected PMD-mapped THP? */ + VM_BUG_ON_PAGE(!pvmw.pte, page); + + if (!pte_present(*pvmw.pte)) { + ret = false; + page_vma_mapped_walk_done(&pvmw); + break; + } + + subpage = page - page_to_pfn(page) + pte_pfn(*pvmw.pte); + address = pvmw.address; + + /* Nuke the page table entry. */ + flush_cache_page(vma, address, pte_pfn(*pvmw.pte)); + pteval = ptep_clear_flush(vma, address, pvmw.pte); + + /* Move the dirty bit to the page. Now the pte is gone. */ + if (pte_dirty(pteval)) + set_page_dirty(page); + + /* + * Check that our target page is still mapped at the expected + * address. + */ + if (args->mm == mm && args->address == address && + pte_write(pteval)) + args->valid = true; + + /* + * Store the pfn of the page in a special migration + * pte. do_swap_page() will wait until the migration + * pte is removed and then restart fault handling. + */ + if (pte_write(pteval)) + entry = make_writable_device_exclusive_entry( + page_to_pfn(subpage)); + else + entry = make_readable_device_exclusive_entry( + page_to_pfn(subpage)); + swp_pte = swp_entry_to_pte(entry); + if (pte_soft_dirty(pteval)) + swp_pte = pte_swp_mksoft_dirty(swp_pte); + if (pte_uffd_wp(pteval)) + swp_pte = pte_swp_mkuffd_wp(swp_pte); + + set_pte_at(mm, address, pvmw.pte, swp_pte); + + /* + * There is a reference on the page for the swap entry which has + * been removed, so shouldn't take another. + */ + page_remove_rmap(subpage, false); + } + + mmu_notifier_invalidate_range_end(&range); + + return ret; +} + +/** + * page_make_device_exclusive - mark the page exclusively owned by a device + * @page: the page to replace page table entries for + * @mm: the mm_struct where the page is expected to be mapped + * @address: address where the page is expected to be mapped + * @owner: passed to MMU_NOTIFY_EXCLUSIVE range notifier callbacks + * + * Tries to remove all the page table entries which are mapping this page and + * replace them with special device exclusive swap entries to grant a device + * exclusive access to the page. Caller must hold the page lock. + * + * Returns false if the page is still mapped, or if it could not be unmapped + * from the expected address. Otherwise returns true (success). + */ +static bool page_make_device_exclusive(struct page *page, struct mm_struct *mm, + unsigned long address, void *owner) +{ + struct make_exclusive_args args = { + .mm = mm, + .address = address, + .owner = owner, + .valid = false, + }; + struct rmap_walk_control rwc = { + .rmap_one = page_make_device_exclusive_one, + .done = page_not_mapped, + .anon_lock = page_lock_anon_vma_read, + .arg = &args, + }; + + /* + * Restrict to anonymous pages for now to avoid potential writeback + * issues. Also tail pages shouldn't be passed to rmap_walk so skip + * those. + */ + if (!PageAnon(page) || PageTail(page)) + return false; + + rmap_walk(page, &rwc); + + return args.valid && !page_mapcount(page); +} + +/** + * make_device_exclusive_range() - Mark a range for exclusive use by a device + * @mm: mm_struct of assoicated target process + * @start: start of the region to mark for exclusive device access + * @end: end address of region + * @pages: returns the pages which were successfully marked for exclusive access + * @owner: passed to MMU_NOTIFY_EXCLUSIVE range notifier to allow filtering + * + * Returns: number of pages found in the range by GUP. A page is marked for + * exclusive access only if the page pointer is non-NULL. + * + * This function finds ptes mapping page(s) to the given address range, locks + * them and replaces mappings with special swap entries preventing userspace CPU + * access. On fault these entries are replaced with the original mapping after + * calling MMU notifiers. + * + * A driver using this to program access from a device must use a mmu notifier + * critical section to hold a device specific lock during programming. Once + * programming is complete it should drop the page lock and reference after + * which point CPU access to the page will revoke the exclusive access. + */ +int make_device_exclusive_range(struct mm_struct *mm, unsigned long start, + unsigned long end, struct page **pages, + void *owner) +{ + long npages = (end - start) >> PAGE_SHIFT; + long i; + + npages = get_user_pages_remote(mm, start, npages, + FOLL_GET | FOLL_WRITE | FOLL_SPLIT_PMD, + pages, NULL, NULL); + if (npages < 0) + return npages; + + for (i = 0; i < npages; i++, start += PAGE_SIZE) { + if (!trylock_page(pages[i])) { + put_page(pages[i]); + pages[i] = NULL; + continue; + } + + if (!page_make_device_exclusive(pages[i], mm, start, owner)) { + unlock_page(pages[i]); + put_page(pages[i]); + pages[i] = NULL; + } + } + + return npages; +} +EXPORT_SYMBOL_GPL(make_device_exclusive_range); +#endif + void __put_anon_vma(struct anon_vma *anon_vma) { struct anon_vma *root = anon_vma->root; From patchwork Thu Jul 1 01:54:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C9D9C11F69 for ; Thu, 1 Jul 2021 01:54:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DDC4961418 for ; Thu, 1 Jul 2021 01:54:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDC4961418 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 662E38D025C; Wed, 30 Jun 2021 21:54:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 612CE8D024F; Wed, 30 Jun 2021 21:54:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48C818D025C; Wed, 30 Jun 2021 21:54:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0242.hostedemail.com [216.40.44.242]) by kanga.kvack.org (Postfix) with ESMTP id 1DF6A8D024F for ; Wed, 30 Jun 2021 21:54:31 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E6DF8181AF5C1 for ; Thu, 1 Jul 2021 01:54:30 +0000 (UTC) X-FDA: 78312349500.10.E503A1A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 807229000326 for ; Thu, 1 Jul 2021 01:54:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5029061420; Thu, 1 Jul 2021 01:54:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104469; bh=esm/4w6sD6DGj+2dJ4fvAY5Q3TxX9ss+3KXi4YNqK2w=; h=Date:From:To:Subject:In-Reply-To:From; b=eiI808sr8W4Alh+xfdyiM71eQP+ezdSZxVP1nIhe7PnwtYCQPQ1kFO+DAMJCy6E1a fLxC5NtvLQxRKhBplYzDw97GaIbseIqxJGNB/kFcjgMKEgucA+z3+jdG26KGn2xkxd UwDG5y9xY8f7errQBEn+h5ZASaOXqAPEiwAKptyc= Date: Wed, 30 Jun 2021 18:54:28 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 137/192] mm: selftests for exclusive device memory Message-ID: <20210701015428.z3NdM9BbH%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=eiI808sr; dmarc=none; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: 3b7ozzttqmh3xrgi8s74r94bx8ari4h5 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 807229000326 X-HE-Tag: 1625104470-495508 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: mm: selftests for exclusive device memory Adds some selftests for exclusive device memory. Link: https://lkml.kernel.org/r/20210616105937.23201-9-apopple@nvidia.com Signed-off-by: Alistair Popple Acked-by: Jason Gunthorpe Tested-by: Ralph Campbell Reviewed-by: Ralph Campbell Cc: Ben Skeggs Cc: Christoph Hellwig Cc: Hugh Dickins Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Shakeel Butt Signed-off-by: Andrew Morton --- lib/test_hmm.c | 125 ++++++++++++++++++ lib/test_hmm_uapi.h | 2 tools/testing/selftests/vm/hmm-tests.c | 158 +++++++++++++++++++++++ 3 files changed, 285 insertions(+) --- a/lib/test_hmm.c~mm-selftests-for-exclusive-device-memory +++ a/lib/test_hmm.c @@ -25,6 +25,7 @@ #include #include #include +#include #include "test_hmm_uapi.h" @@ -46,6 +47,7 @@ struct dmirror_bounce { unsigned long cpages; }; +#define DPT_XA_TAG_ATOMIC 1UL #define DPT_XA_TAG_WRITE 3UL /* @@ -619,6 +621,54 @@ static void dmirror_migrate_alloc_and_co } } +static int dmirror_check_atomic(struct dmirror *dmirror, unsigned long start, + unsigned long end) +{ + unsigned long pfn; + + for (pfn = start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++) { + void *entry; + struct page *page; + + entry = xa_load(&dmirror->pt, pfn); + page = xa_untag_pointer(entry); + if (xa_pointer_tag(entry) == DPT_XA_TAG_ATOMIC) + return -EPERM; + } + + return 0; +} + +static int dmirror_atomic_map(unsigned long start, unsigned long end, + struct page **pages, struct dmirror *dmirror) +{ + unsigned long pfn, mapped = 0; + int i; + + /* Map the migrated pages into the device's page tables. */ + mutex_lock(&dmirror->mutex); + + for (i = 0, pfn = start >> PAGE_SHIFT; pfn < (end >> PAGE_SHIFT); pfn++, i++) { + void *entry; + + if (!pages[i]) + continue; + + entry = pages[i]; + entry = xa_tag_pointer(entry, DPT_XA_TAG_ATOMIC); + entry = xa_store(&dmirror->pt, pfn, entry, GFP_ATOMIC); + if (xa_is_err(entry)) { + mutex_unlock(&dmirror->mutex); + return xa_err(entry); + } + + mapped++; + } + + mutex_unlock(&dmirror->mutex); + return mapped; +} + static int dmirror_migrate_finalize_and_map(struct migrate_vma *args, struct dmirror *dmirror) { @@ -661,6 +711,72 @@ static int dmirror_migrate_finalize_and_ return 0; } +static int dmirror_exclusive(struct dmirror *dmirror, + struct hmm_dmirror_cmd *cmd) +{ + unsigned long start, end, addr; + unsigned long size = cmd->npages << PAGE_SHIFT; + struct mm_struct *mm = dmirror->notifier.mm; + struct page *pages[64]; + struct dmirror_bounce bounce; + unsigned long next; + int ret; + + start = cmd->addr; + end = start + size; + if (end < start) + return -EINVAL; + + /* Since the mm is for the mirrored process, get a reference first. */ + if (!mmget_not_zero(mm)) + return -EINVAL; + + mmap_read_lock(mm); + for (addr = start; addr < end; addr = next) { + unsigned long mapped; + int i; + + if (end < addr + (ARRAY_SIZE(pages) << PAGE_SHIFT)) + next = end; + else + next = addr + (ARRAY_SIZE(pages) << PAGE_SHIFT); + + ret = make_device_exclusive_range(mm, addr, next, pages, NULL); + mapped = dmirror_atomic_map(addr, next, pages, dmirror); + for (i = 0; i < ret; i++) { + if (pages[i]) { + unlock_page(pages[i]); + put_page(pages[i]); + } + } + + if (addr + (mapped << PAGE_SHIFT) < next) { + mmap_read_unlock(mm); + mmput(mm); + return -EBUSY; + } + } + mmap_read_unlock(mm); + mmput(mm); + + /* Return the migrated data for verification. */ + ret = dmirror_bounce_init(&bounce, start, size); + if (ret) + return ret; + mutex_lock(&dmirror->mutex); + ret = dmirror_do_read(dmirror, start, end, &bounce); + mutex_unlock(&dmirror->mutex); + if (ret == 0) { + if (copy_to_user(u64_to_user_ptr(cmd->ptr), bounce.ptr, + bounce.size)) + ret = -EFAULT; + } + + cmd->cpages = bounce.cpages; + dmirror_bounce_fini(&bounce); + return ret; +} + static int dmirror_migrate(struct dmirror *dmirror, struct hmm_dmirror_cmd *cmd) { @@ -948,6 +1064,15 @@ static long dmirror_fops_unlocked_ioctl( ret = dmirror_migrate(dmirror, &cmd); break; + case HMM_DMIRROR_EXCLUSIVE: + ret = dmirror_exclusive(dmirror, &cmd); + break; + + case HMM_DMIRROR_CHECK_EXCLUSIVE: + ret = dmirror_check_atomic(dmirror, cmd.addr, + cmd.addr + (cmd.npages << PAGE_SHIFT)); + break; + case HMM_DMIRROR_SNAPSHOT: ret = dmirror_snapshot(dmirror, &cmd); break; --- a/lib/test_hmm_uapi.h~mm-selftests-for-exclusive-device-memory +++ a/lib/test_hmm_uapi.h @@ -33,6 +33,8 @@ struct hmm_dmirror_cmd { #define HMM_DMIRROR_WRITE _IOWR('H', 0x01, struct hmm_dmirror_cmd) #define HMM_DMIRROR_MIGRATE _IOWR('H', 0x02, struct hmm_dmirror_cmd) #define HMM_DMIRROR_SNAPSHOT _IOWR('H', 0x03, struct hmm_dmirror_cmd) +#define HMM_DMIRROR_EXCLUSIVE _IOWR('H', 0x04, struct hmm_dmirror_cmd) +#define HMM_DMIRROR_CHECK_EXCLUSIVE _IOWR('H', 0x05, struct hmm_dmirror_cmd) /* * Values returned in hmm_dmirror_cmd.ptr for HMM_DMIRROR_SNAPSHOT. --- a/tools/testing/selftests/vm/hmm-tests.c~mm-selftests-for-exclusive-device-memory +++ a/tools/testing/selftests/vm/hmm-tests.c @@ -1485,4 +1485,162 @@ TEST_F(hmm2, double_map) hmm_buffer_free(buffer); } +/* + * Basic check of exclusive faulting. + */ +TEST_F(hmm, exclusive) +{ + struct hmm_buffer *buffer; + unsigned long npages; + unsigned long size; + unsigned long i; + int *ptr; + int ret; + + npages = ALIGN(HMM_BUFFER_SIZE, self->page_size) >> self->page_shift; + ASSERT_NE(npages, 0); + size = npages << self->page_shift; + + buffer = malloc(sizeof(*buffer)); + ASSERT_NE(buffer, NULL); + + buffer->fd = -1; + buffer->size = size; + buffer->mirror = malloc(size); + ASSERT_NE(buffer->mirror, NULL); + + buffer->ptr = mmap(NULL, size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + buffer->fd, 0); + ASSERT_NE(buffer->ptr, MAP_FAILED); + + /* Initialize buffer in system memory. */ + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ptr[i] = i; + + /* Map memory exclusively for device access. */ + ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_EXCLUSIVE, buffer, npages); + ASSERT_EQ(ret, 0); + ASSERT_EQ(buffer->cpages, npages); + + /* Check what the device read. */ + for (i = 0, ptr = buffer->mirror; i < size / sizeof(*ptr); ++i) + ASSERT_EQ(ptr[i], i); + + /* Fault pages back to system memory and check them. */ + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ASSERT_EQ(ptr[i]++, i); + + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ASSERT_EQ(ptr[i], i+1); + + /* Check atomic access revoked */ + ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_CHECK_EXCLUSIVE, buffer, npages); + ASSERT_EQ(ret, 0); + + hmm_buffer_free(buffer); +} + +TEST_F(hmm, exclusive_mprotect) +{ + struct hmm_buffer *buffer; + unsigned long npages; + unsigned long size; + unsigned long i; + int *ptr; + int ret; + + npages = ALIGN(HMM_BUFFER_SIZE, self->page_size) >> self->page_shift; + ASSERT_NE(npages, 0); + size = npages << self->page_shift; + + buffer = malloc(sizeof(*buffer)); + ASSERT_NE(buffer, NULL); + + buffer->fd = -1; + buffer->size = size; + buffer->mirror = malloc(size); + ASSERT_NE(buffer->mirror, NULL); + + buffer->ptr = mmap(NULL, size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + buffer->fd, 0); + ASSERT_NE(buffer->ptr, MAP_FAILED); + + /* Initialize buffer in system memory. */ + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ptr[i] = i; + + /* Map memory exclusively for device access. */ + ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_EXCLUSIVE, buffer, npages); + ASSERT_EQ(ret, 0); + ASSERT_EQ(buffer->cpages, npages); + + /* Check what the device read. */ + for (i = 0, ptr = buffer->mirror; i < size / sizeof(*ptr); ++i) + ASSERT_EQ(ptr[i], i); + + ret = mprotect(buffer->ptr, size, PROT_READ); + ASSERT_EQ(ret, 0); + + /* Simulate a device writing system memory. */ + ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_WRITE, buffer, npages); + ASSERT_EQ(ret, -EPERM); + + hmm_buffer_free(buffer); +} + +/* + * Check copy-on-write works. + */ +TEST_F(hmm, exclusive_cow) +{ + struct hmm_buffer *buffer; + unsigned long npages; + unsigned long size; + unsigned long i; + int *ptr; + int ret; + + npages = ALIGN(HMM_BUFFER_SIZE, self->page_size) >> self->page_shift; + ASSERT_NE(npages, 0); + size = npages << self->page_shift; + + buffer = malloc(sizeof(*buffer)); + ASSERT_NE(buffer, NULL); + + buffer->fd = -1; + buffer->size = size; + buffer->mirror = malloc(size); + ASSERT_NE(buffer->mirror, NULL); + + buffer->ptr = mmap(NULL, size, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, + buffer->fd, 0); + ASSERT_NE(buffer->ptr, MAP_FAILED); + + /* Initialize buffer in system memory. */ + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ptr[i] = i; + + /* Map memory exclusively for device access. */ + ret = hmm_dmirror_cmd(self->fd, HMM_DMIRROR_EXCLUSIVE, buffer, npages); + ASSERT_EQ(ret, 0); + ASSERT_EQ(buffer->cpages, npages); + + fork(); + + /* Fault pages back to system memory and check them. */ + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ASSERT_EQ(ptr[i]++, i); + + for (i = 0, ptr = buffer->ptr; i < size / sizeof(*ptr); ++i) + ASSERT_EQ(ptr[i], i+1); + + hmm_buffer_free(buffer); +} + TEST_HARNESS_MAIN From patchwork Thu Jul 1 01:54:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D0AAC11F69 for ; Thu, 1 Jul 2021 01:54:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D8D2861468 for ; Thu, 1 Jul 2021 01:54:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D8D2861468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5EDA88D025D; Wed, 30 Jun 2021 21:54:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59D988D024F; Wed, 30 Jun 2021 21:54:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 417598D025D; Wed, 30 Jun 2021 21:54:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id 178C18D024F for ; Wed, 30 Jun 2021 21:54:34 -0400 (EDT) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id EBCB91803671E for ; Thu, 1 Jul 2021 01:54:33 +0000 (UTC) X-FDA: 78312349626.19.7D82AA7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 9D5CB10004D4 for ; Thu, 1 Jul 2021 01:54:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 795836146D; Thu, 1 Jul 2021 01:54:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104472; bh=r99XeJ8JRxdtj8fvIQBT8w0Bq+c3mzVfV+c20U9dDv4=; h=Date:From:To:Subject:In-Reply-To:From; b=CiOCy/+w/BaVaURMLhBTNvcxZAT91teTwWYgWA1dAAq0QFSqxuHrV52+LtH2qyVmd xVIbIj0dKj/187s/9a3HYwux3Lw75jQ5K9TCdQp/WID07avsxkIOAvxyTSG8cFtiMK h1aHfM4jFkaLern1+ZKyzj8tNF99uKm83vntGkIs= Date: Wed, 30 Jun 2021 18:54:32 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 138/192] nouveau/svm: refactor nouveau_range_fault Message-ID: <20210701015432.n88XEOWNe%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="CiOCy/+w"; spf=pass (imf07.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: oekdxo3yd3n7iem1yimux7tyngic1ksa X-Rspamd-Queue-Id: 9D5CB10004D4 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104473-824630 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: nouveau/svm: refactor nouveau_range_fault Call mmu_interval_notifier_insert() as part of nouveau_range_fault(). This doesn't introduce any functional change but makes it easier for a subsequent patch to alter the behaviour of nouveau_range_fault() to support GPU atomic operations. Link: https://lkml.kernel.org/r/20210616105937.23201-10-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Ben Skeggs Cc: Christoph Hellwig Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Ralph Campbell Cc: Shakeel Butt Signed-off-by: Andrew Morton --- drivers/gpu/drm/nouveau/nouveau_svm.c | 34 ++++++++++++++---------- 1 file changed, 20 insertions(+), 14 deletions(-) --- a/drivers/gpu/drm/nouveau/nouveau_svm.c~nouveau-svm-refactor-nouveau_range_fault +++ a/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -567,18 +567,27 @@ static int nouveau_range_fault(struct no unsigned long hmm_pfns[1]; struct hmm_range range = { .notifier = ¬ifier->notifier, - .start = notifier->notifier.interval_tree.start, - .end = notifier->notifier.interval_tree.last + 1, .default_flags = hmm_flags, .hmm_pfns = hmm_pfns, .dev_private_owner = drm->dev, }; - struct mm_struct *mm = notifier->notifier.mm; + struct mm_struct *mm = svmm->notifier.mm; int ret; + ret = mmu_interval_notifier_insert(¬ifier->notifier, mm, + args->p.addr, args->p.size, + &nouveau_svm_mni_ops); + if (ret) + return ret; + + range.start = notifier->notifier.interval_tree.start; + range.end = notifier->notifier.interval_tree.last + 1; + while (true) { - if (time_after(jiffies, timeout)) - return -EBUSY; + if (time_after(jiffies, timeout)) { + ret = -EBUSY; + goto out; + } range.notifier_seq = mmu_interval_read_begin(range.notifier); mmap_read_lock(mm); @@ -587,7 +596,7 @@ static int nouveau_range_fault(struct no if (ret) { if (ret == -EBUSY) continue; - return ret; + goto out; } mutex_lock(&svmm->mutex); @@ -606,6 +615,9 @@ static int nouveau_range_fault(struct no svmm->vmm->vmm.object.client->super = false; mutex_unlock(&svmm->mutex); +out: + mmu_interval_notifier_remove(¬ifier->notifier); + return ret; } @@ -727,14 +739,8 @@ nouveau_svm_fault(struct nvif_notify *no } notifier.svmm = svmm; - ret = mmu_interval_notifier_insert(¬ifier.notifier, mm, - args.i.p.addr, args.i.p.size, - &nouveau_svm_mni_ops); - if (!ret) { - ret = nouveau_range_fault(svmm, svm->drm, &args.i, - sizeof(args), hmm_flags, ¬ifier); - mmu_interval_notifier_remove(¬ifier.notifier); - } + ret = nouveau_range_fault(svmm, svm->drm, &args.i, + sizeof(args), hmm_flags, ¬ifier); mmput(mm); limit = args.i.p.addr + args.i.p.size; From patchwork Thu Jul 1 01:54:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79911C11F68 for ; Thu, 1 Jul 2021 01:54:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2576C61420 for ; Thu, 1 Jul 2021 01:54:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2576C61420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A0D098D025E; Wed, 30 Jun 2021 21:54:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BBE68D024F; Wed, 30 Jun 2021 21:54:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 835EF8D025E; Wed, 30 Jun 2021 21:54:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0245.hostedemail.com [216.40.44.245]) by kanga.kvack.org (Postfix) with ESMTP id 570E28D024F for ; Wed, 30 Jun 2021 21:54:37 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2A969231DC for ; Thu, 1 Jul 2021 01:54:37 +0000 (UTC) X-FDA: 78312349794.30.587D6DA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id CE9FE700009B for ; Thu, 1 Jul 2021 01:54:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AC74961474; Thu, 1 Jul 2021 01:54:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104476; bh=P4ldzissIWG7QBIrggsSPPcAanASakZDmFmnn2e/gmE=; h=Date:From:To:Subject:In-Reply-To:From; b=JE5XTCDiH2bpcEYLS3x2bf/KpMGOkYoKcZK6YQxhmio7MAQvm5UT0JBIayEXeClLf a84XVneGohg+IxWwMqOjL91BgB+sMTADfpXaSXDff6PUx6s1liviUwGDqhRE+dpvdP 6a0uPf6p8R9x5UBMLHfLyje+Bo1eRcV8Eo+QkR4g= Date: Wed, 30 Jun 2021 18:54:35 -0700 From: Andrew Morton To: akpm@linux-foundation.org, apopple@nvidia.com, bskeggs@redhat.com, hch@lst.de, hughd@google.com, jgg@nvidia.com, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, rcampbell@nvidia.com, shakeelb@google.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 139/192] nouveau/svm: implement atomic SVM access Message-ID: <20210701015435.8OcjE5dDr%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=JE5XTCDi; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 7odqck78ucdpor14izrz9q6dce7smb7g X-Rspamd-Queue-Id: CE9FE700009B X-Rspamd-Server: rspam06 X-HE-Tag: 1625104476-622266 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alistair Popple Subject: nouveau/svm: implement atomic SVM access Some NVIDIA GPUs do not support direct atomic access to system memory via PCIe. Instead this must be emulated by granting the GPU exclusive access to the memory. This is achieved by replacing CPU page table entries with special swap entries that fault on userspace access. The driver then grants the GPU permission to update the page undergoing atomic access via the GPU page tables. When CPU access to the page is required a CPU fault is raised which calls into the device driver via MMU notifiers to revoke the atomic access. The original page table entries are then restored allowing CPU access to proceed. Link: https://lkml.kernel.org/r/20210616105937.23201-11-apopple@nvidia.com Signed-off-by: Alistair Popple Reviewed-by: Ben Skeggs Cc: Christoph Hellwig Cc: Hugh Dickins Cc: Jason Gunthorpe Cc: John Hubbard Cc: "Matthew Wilcox (Oracle)" Cc: Peter Xu Cc: Ralph Campbell Cc: Shakeel Butt Signed-off-by: Andrew Morton --- drivers/gpu/drm/nouveau/include/nvif/if000c.h | 1 drivers/gpu/drm/nouveau/nouveau_svm.c | 126 ++++++++++- drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h | 1 drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c | 6 4 files changed, 123 insertions(+), 11 deletions(-) --- a/drivers/gpu/drm/nouveau/include/nvif/if000c.h~nouveau-svm-implement-atomic-svm-access +++ a/drivers/gpu/drm/nouveau/include/nvif/if000c.h @@ -77,6 +77,7 @@ struct nvif_vmm_pfnmap_v0 { #define NVIF_VMM_PFNMAP_V0_APER 0x00000000000000f0ULL #define NVIF_VMM_PFNMAP_V0_HOST 0x0000000000000000ULL #define NVIF_VMM_PFNMAP_V0_VRAM 0x0000000000000010ULL +#define NVIF_VMM_PFNMAP_V0_A 0x0000000000000004ULL #define NVIF_VMM_PFNMAP_V0_W 0x0000000000000002ULL #define NVIF_VMM_PFNMAP_V0_V 0x0000000000000001ULL #define NVIF_VMM_PFNMAP_V0_NONE 0x0000000000000000ULL --- a/drivers/gpu/drm/nouveau/nouveau_svm.c~nouveau-svm-implement-atomic-svm-access +++ a/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -35,6 +35,7 @@ #include #include #include +#include struct nouveau_svm { struct nouveau_drm *drm; @@ -67,6 +68,11 @@ struct nouveau_svm { } buffer[1]; }; +#define FAULT_ACCESS_READ 0 +#define FAULT_ACCESS_WRITE 1 +#define FAULT_ACCESS_ATOMIC 2 +#define FAULT_ACCESS_PREFETCH 3 + #define SVM_DBG(s,f,a...) NV_DEBUG((s)->drm, "svm: "f"\n", ##a) #define SVM_ERR(s,f,a...) NV_WARN((s)->drm, "svm: "f"\n", ##a) @@ -412,6 +418,24 @@ nouveau_svm_fault_cancel_fault(struct no } static int +nouveau_svm_fault_priority(u8 fault) +{ + switch (fault) { + case FAULT_ACCESS_PREFETCH: + return 0; + case FAULT_ACCESS_READ: + return 1; + case FAULT_ACCESS_WRITE: + return 2; + case FAULT_ACCESS_ATOMIC: + return 3; + default: + WARN_ON_ONCE(1); + return -1; + } +} + +static int nouveau_svm_fault_cmp(const void *a, const void *b) { const struct nouveau_svm_fault *fa = *(struct nouveau_svm_fault **)a; @@ -421,9 +445,8 @@ nouveau_svm_fault_cmp(const void *a, con return ret; if ((ret = (s64)fa->addr - fb->addr)) return ret; - /*XXX: atomic? */ - return (fa->access == 0 || fa->access == 3) - - (fb->access == 0 || fb->access == 3); + return nouveau_svm_fault_priority(fa->access) - + nouveau_svm_fault_priority(fb->access); } static void @@ -487,6 +510,10 @@ static bool nouveau_svm_range_invalidate struct svm_notifier *sn = container_of(mni, struct svm_notifier, notifier); + if (range->event == MMU_NOTIFY_EXCLUSIVE && + range->owner == sn->svmm->vmm->cli->drm->dev) + return true; + /* * serializes the update to mni->invalidate_seq done by caller and * prevents invalidation of the PTE from progressing while HW is being @@ -555,6 +582,71 @@ static void nouveau_hmm_convert_pfn(stru args->p.phys[0] |= NVIF_VMM_PFNMAP_V0_W; } +static int nouveau_atomic_range_fault(struct nouveau_svmm *svmm, + struct nouveau_drm *drm, + struct nouveau_pfnmap_args *args, u32 size, + struct svm_notifier *notifier) +{ + unsigned long timeout = + jiffies + msecs_to_jiffies(HMM_RANGE_DEFAULT_TIMEOUT); + struct mm_struct *mm = svmm->notifier.mm; + struct page *page; + unsigned long start = args->p.addr; + unsigned long notifier_seq; + int ret = 0; + + ret = mmu_interval_notifier_insert(¬ifier->notifier, mm, + args->p.addr, args->p.size, + &nouveau_svm_mni_ops); + if (ret) + return ret; + + while (true) { + if (time_after(jiffies, timeout)) { + ret = -EBUSY; + goto out; + } + + notifier_seq = mmu_interval_read_begin(¬ifier->notifier); + mmap_read_lock(mm); + ret = make_device_exclusive_range(mm, start, start + PAGE_SIZE, + &page, drm->dev); + mmap_read_unlock(mm); + if (ret <= 0 || !page) { + ret = -EINVAL; + goto out; + } + + mutex_lock(&svmm->mutex); + if (!mmu_interval_read_retry(¬ifier->notifier, + notifier_seq)) + break; + mutex_unlock(&svmm->mutex); + } + + /* Map the page on the GPU. */ + args->p.page = 12; + args->p.size = PAGE_SIZE; + args->p.addr = start; + args->p.phys[0] = page_to_phys(page) | + NVIF_VMM_PFNMAP_V0_V | + NVIF_VMM_PFNMAP_V0_W | + NVIF_VMM_PFNMAP_V0_A | + NVIF_VMM_PFNMAP_V0_HOST; + + svmm->vmm->vmm.object.client->super = true; + ret = nvif_object_ioctl(&svmm->vmm->vmm.object, args, size, NULL); + svmm->vmm->vmm.object.client->super = false; + mutex_unlock(&svmm->mutex); + + unlock_page(page); + put_page(page); + +out: + mmu_interval_notifier_remove(¬ifier->notifier); + return ret; +} + static int nouveau_range_fault(struct nouveau_svmm *svmm, struct nouveau_drm *drm, struct nouveau_pfnmap_args *args, u32 size, @@ -637,7 +729,7 @@ nouveau_svm_fault(struct nvif_notify *no unsigned long hmm_flags; u64 inst, start, limit; int fi, fn; - int replay = 0, ret; + int replay = 0, atomic = 0, ret; /* Parse available fault buffer entries into a cache, and update * the GET pointer so HW can reuse the entries. @@ -718,12 +810,14 @@ nouveau_svm_fault(struct nvif_notify *no /* * Determine required permissions based on GPU fault * access flags. - * XXX: atomic? */ switch (buffer->fault[fi]->access) { case 0: /* READ. */ hmm_flags = HMM_PFN_REQ_FAULT; break; + case 2: /* ATOMIC. */ + atomic = true; + break; case 3: /* PREFETCH. */ hmm_flags = 0; break; @@ -739,8 +833,14 @@ nouveau_svm_fault(struct nvif_notify *no } notifier.svmm = svmm; - ret = nouveau_range_fault(svmm, svm->drm, &args.i, - sizeof(args), hmm_flags, ¬ifier); + if (atomic) + ret = nouveau_atomic_range_fault(svmm, svm->drm, + &args.i, sizeof(args), + ¬ifier); + else + ret = nouveau_range_fault(svmm, svm->drm, &args.i, + sizeof(args), hmm_flags, + ¬ifier); mmput(mm); limit = args.i.p.addr + args.i.p.size; @@ -756,11 +856,15 @@ nouveau_svm_fault(struct nvif_notify *no */ if (buffer->fault[fn]->svmm != svmm || buffer->fault[fn]->addr >= limit || - (buffer->fault[fi]->access == 0 /* READ. */ && + (buffer->fault[fi]->access == FAULT_ACCESS_READ && !(args.phys[0] & NVIF_VMM_PFNMAP_V0_V)) || - (buffer->fault[fi]->access != 0 /* READ. */ && - buffer->fault[fi]->access != 3 /* PREFETCH. */ && - !(args.phys[0] & NVIF_VMM_PFNMAP_V0_W))) + (buffer->fault[fi]->access != FAULT_ACCESS_READ && + buffer->fault[fi]->access != FAULT_ACCESS_PREFETCH && + !(args.phys[0] & NVIF_VMM_PFNMAP_V0_W)) || + (buffer->fault[fi]->access != FAULT_ACCESS_READ && + buffer->fault[fi]->access != FAULT_ACCESS_WRITE && + buffer->fault[fi]->access != FAULT_ACCESS_PREFETCH && + !(args.phys[0] & NVIF_VMM_PFNMAP_V0_A))) break; } --- a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c~nouveau-svm-implement-atomic-svm-access +++ a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.c @@ -88,6 +88,9 @@ gp100_vmm_pgt_pfn(struct nvkm_vmm *vmm, if (!(*map->pfn & NVKM_VMM_PFN_W)) data |= BIT_ULL(6); /* RO. */ + if (!(*map->pfn & NVKM_VMM_PFN_A)) + data |= BIT_ULL(7); /* Atomic disable. */ + if (!(*map->pfn & NVKM_VMM_PFN_VRAM)) { addr = *map->pfn >> NVKM_VMM_PFN_ADDR_SHIFT; addr = dma_map_page(dev, pfn_to_page(addr), 0, @@ -322,6 +325,9 @@ gp100_vmm_pd0_pfn(struct nvkm_vmm *vmm, if (!(*map->pfn & NVKM_VMM_PFN_W)) data |= BIT_ULL(6); /* RO. */ + if (!(*map->pfn & NVKM_VMM_PFN_A)) + data |= BIT_ULL(7); /* Atomic disable. */ + if (!(*map->pfn & NVKM_VMM_PFN_VRAM)) { addr = *map->pfn >> NVKM_VMM_PFN_ADDR_SHIFT; addr = dma_map_page(dev, pfn_to_page(addr), 0, --- a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h~nouveau-svm-implement-atomic-svm-access +++ a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.h @@ -178,6 +178,7 @@ void nvkm_vmm_unmap_region(struct nvkm_v #define NVKM_VMM_PFN_APER 0x00000000000000f0ULL #define NVKM_VMM_PFN_HOST 0x0000000000000000ULL #define NVKM_VMM_PFN_VRAM 0x0000000000000010ULL +#define NVKM_VMM_PFN_A 0x0000000000000004ULL #define NVKM_VMM_PFN_W 0x0000000000000002ULL #define NVKM_VMM_PFN_V 0x0000000000000001ULL #define NVKM_VMM_PFN_NONE 0x0000000000000000ULL From patchwork Thu Jul 1 01:54:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 142ADC11F66 for ; Thu, 1 Jul 2021 01:55:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BBAA06141A for ; Thu, 1 Jul 2021 01:55:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBAA06141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D67D38D026A; Wed, 30 Jun 2021 21:55:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C7B998D024F; Wed, 30 Jun 2021 21:55:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA6818D026A; Wed, 30 Jun 2021 21:55:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 704978D024F for ; Wed, 30 Jun 2021 21:55:40 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 40B4A181AF5CA for ; Thu, 1 Jul 2021 01:55:40 +0000 (UTC) X-FDA: 78312352440.25.42F2C29 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id E2EC1E00304F for ; Thu, 1 Jul 2021 01:54:39 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DF7EB61418; Thu, 1 Jul 2021 01:54:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104479; bh=fRvBNWulQUko4aPaQqRgmopGLIhUL/UCkKdhIKskUfc=; h=Date:From:To:Subject:In-Reply-To:From; b=blizlUI6AJeKi8buonZMv6YsGoqH6P/ttq+yyg1qw9H1itlzVXRvFXxCf/wzvBQSF 4bpNWA2CaD5oPBmTf9bsThQePY+g/djfoZt2dxXMmhoQqGK9Yol0Oh8yhuiH2u7185 0e6ev8Nc2X9Ei6/zsCNifQ8aUxYGemvM3JJD9H2I= Date: Wed, 30 Jun 2021 18:54:38 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, cascardo@canonical.com, christian.brauner@ubuntu.com, ddiss@suse.de, deller@gmx.de, linux-mm@kvack.org, lstoakes@gmail.com, marcelo.cerri@canonical.com, mm-commits@vger.kernel.org, oleg@redhat.com, torvalds@linux-foundation.org, walken@google.com Subject: [patch 140/192] proc: Avoid mixing integer types in mem_rw() Message-ID: <20210701015438.0zNDxEvkA%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E2EC1E00304F Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=blizlUI6; spf=pass (imf30.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: k83ohr9hr4tq636k8wsfgmi51zwr5uyr X-HE-Tag: 1625104479-323729 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Marcelo Henrique Cerri Subject: proc: Avoid mixing integer types in mem_rw() Use size_t when capping the count argument received by mem_rw(). Since count is size_t, using min_t(int, ...) can lead to a negative value that will later be passed to access_remote_vm(), which can cause unexpected behavior. Since we are capping the value to at maximum PAGE_SIZE, the conversion from size_t to int when passing it to access_remote_vm() as "len" shouldn't be a problem. Link: https://lkml.kernel.org/r/20210512125215.3348316-1-marcelo.cerri@canonical.com Reviewed-by: David Disseldorp Signed-off-by: Thadeu Lima de Souza Cascardo Signed-off-by: Marcelo Henrique Cerri Cc: Alexey Dobriyan Cc: Souza Cascardo Cc: Christian Brauner Cc: Michel Lespinasse Cc: Helge Deller Cc: Oleg Nesterov Cc: Lorenzo Stoakes Signed-off-by: Andrew Morton --- fs/proc/base.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/fs/proc/base.c~proc-avoid-mixing-integer-types-in-mem_rw +++ a/fs/proc/base.c @@ -854,7 +854,7 @@ static ssize_t mem_rw(struct file *file, flags = FOLL_FORCE | (write ? FOLL_WRITE : 0); while (count > 0) { - int this_len = min_t(int, count, PAGE_SIZE); + size_t this_len = min_t(size_t, count, PAGE_SIZE); if (write && copy_from_user(page, buf, this_len)) { copied = -EFAULT; From patchwork Thu Jul 1 01:54:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6F5FC11F68 for ; Thu, 1 Jul 2021 01:54:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6ECE561468 for ; Thu, 1 Jul 2021 01:54:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6ECE561468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E79C78D025F; Wed, 30 Jun 2021 21:54:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E28BD8D024F; Wed, 30 Jun 2021 21:54:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7C488D025F; Wed, 30 Jun 2021 21:54:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0039.hostedemail.com [216.40.44.39]) by kanga.kvack.org (Postfix) with ESMTP id 9DD4D8D024F for ; Wed, 30 Jun 2021 21:54:43 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 6F233253D3 for ; Thu, 1 Jul 2021 01:54:43 +0000 (UTC) X-FDA: 78312350046.24.F924E54 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 21CE5D000156 for ; Thu, 1 Jul 2021 01:54:43 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 09C6661468; Thu, 1 Jul 2021 01:54:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104482; bh=Ysq8AwRTzvxW44GufXmZ/F+f8EYx6JCP+ijynxjjfLk=; h=Date:From:To:Subject:In-Reply-To:From; b=yrwGkqVSsGLX+zBXd78FhveuK/lYhE7Zieic/zpvra1kLUBuqGLnzud6o6yyVlCio xYbIVTKsDwy+j9Sq9xOFdut4EPDqGovZoRcjjqMW1jbvYuw3ycod9St6Tg6TxU6NAM xmnvv3iLHk3XxxBKmGMKM+sPignuyA0yAU5Ctgl4= Date: Wed, 30 Jun 2021 18:54:41 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, chenying.kernel@bytedance.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@kernel.org, songmuchun@bytedance.com, torvalds@linux-foundation.org, zhouchengming@bytedance.com, zhoufeng.zf@bytedance.com Subject: [patch 141/192] fs/proc/kcore.c: add mmap interface Message-ID: <20210701015441.snfkDnNcO%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=yrwGkqVS; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 21CE5D000156 X-Stat-Signature: k1e9oo1xjkrczjsjwqacbx7wysixkx9j X-HE-Tag: 1625104483-983202 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: ZHOUFENG Subject: fs/proc/kcore.c: add mmap interface When we do the kernel monitor, use the DRGN (https://github.com/osandov/drgn) access to kernel data structures, found that the system calls a lot. DRGN is implemented by reading /proc/kcore. After looking at the kcore code, it is found that kcore does not implement mmap, resulting in frequent context switching triggered by read. Therefore, we want to add mmap interface to optimize performance. Since vmalloc and module areas will change with allocation and release, consistency cannot be guaranteed, so mmap interface only maps KCORE_TEXT and KCORE_RAM. The test results: 1. the default version of kcore real 11.00 user 8.53 sys 3.59 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 99.64 128.578319 12 11168701 pread64 ... ------ ----------- ----------- --------- --------- ---------------- 100.00 129.042853 11193748 966 total 2. added kcore for the mmap interface real 6.44 user 7.32 sys 0.24 % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 32.94 0.130120 24 5317 315 futex 11.66 0.046077 21 2231 1 lstat 9.23 0.036449 177 206 mmap ... ------ ----------- ----------- --------- --------- ---------------- 100.00 0.395077 25435 971 total The test results show that the number of system calls and time consumption are significantly reduced. Thanks to Andrew Morton for your advice. [akpm@linux-foundation.org: KCORE_REMAP is no more] Link: https://lkml.kernel.org/r/20210601082241.13378-1-zhoufeng.zf@bytedance.com Co-developed-by: CHENYING Signed-off-by: CHENYING Signed-off-by: ZHOUFENG Cc: Alexey Dobriyan Cc: Mike Rapoport Cc: Muchun Song Cc: Chengming Zhou Signed-off-by: Andrew Morton --- fs/proc/kcore.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) --- a/fs/proc/kcore.c~fs-proc-kcorec-add-mmap-interface +++ a/fs/proc/kcore.c @@ -614,11 +614,78 @@ static int release_kcore(struct inode *i return 0; } +static vm_fault_t mmap_kcore_fault(struct vm_fault *vmf) +{ + return VM_FAULT_SIGBUS; +} + +static const struct vm_operations_struct kcore_mmap_ops = { + .fault = mmap_kcore_fault, +}; + +static int mmap_kcore(struct file *file, struct vm_area_struct *vma) +{ + size_t size = vma->vm_end - vma->vm_start; + u64 start, pfn; + int nphdr; + size_t data_offset; + size_t phdrs_len, notes_len; + struct kcore_list *m = NULL; + int ret = 0; + + down_read(&kclist_lock); + + get_kcore_size(&nphdr, &phdrs_len, ¬es_len, &data_offset); + + start = kc_offset_to_vaddr(((u64)vma->vm_pgoff << PAGE_SHIFT) - + ((data_offset >> PAGE_SHIFT) << PAGE_SHIFT)); + + list_for_each_entry(m, &kclist_head, list) { + if (start >= m->addr && size <= m->size) + break; + } + + if (&m->list == &kclist_head) { + ret = -EINVAL; + goto out; + } + + if (vma->vm_flags & (VM_WRITE | VM_EXEC)) { + ret = -EPERM; + goto out; + } + + vma->vm_flags &= ~(VM_MAYWRITE | VM_MAYEXEC); + vma->vm_flags |= VM_MIXEDMAP; + vma->vm_ops = &kcore_mmap_ops; + + if (kern_addr_valid(start)) { + if (m->type == KCORE_RAM) + pfn = __pa(start) >> PAGE_SHIFT; + else if (m->type == KCORE_TEXT) + pfn = __pa_symbol(start) >> PAGE_SHIFT; + else { + ret = -EFAULT; + goto out; + } + + ret = remap_pfn_range(vma, vma->vm_start, pfn, size, + vma->vm_page_prot); + } else { + ret = -EFAULT; + } + +out: + up_read(&kclist_lock); + return ret; +} + static const struct proc_ops kcore_proc_ops = { .proc_read = read_kcore, .proc_open = open_kcore, .proc_release = release_kcore, .proc_lseek = default_llseek, + .proc_mmap = mmap_kcore, }; /* just remember that we have to update kcore */ From patchwork Thu Jul 1 01:54:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353339 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27348C11F68 for ; Thu, 1 Jul 2021 01:54:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CE4616141A for ; Thu, 1 Jul 2021 01:54:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE4616141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 578418D0260; Wed, 30 Jun 2021 21:54:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 500CD8D024F; Wed, 30 Jun 2021 21:54:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A2258D0260; Wed, 30 Jun 2021 21:54:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 104A88D024F for ; Wed, 30 Jun 2021 21:54:47 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DC09282DBD41 for ; Thu, 1 Jul 2021 01:54:46 +0000 (UTC) X-FDA: 78312350172.01.44AC5C3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id 862ECF000123 for ; Thu, 1 Jul 2021 01:54:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 26F2B61420; Thu, 1 Jul 2021 01:54:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104486; bh=kfGV9OQf9RWeVwp+jTzprYxY+fIDam5q/fW18WdXoZo=; h=Date:From:To:Subject:In-Reply-To:From; b=RRunjc7U6adkbi5Zitxc9mq99UtwuQE7CbJnCjayNhuYM3vzgEq5EjQScQUfb6B8U nbHvVkstuQhAGQF/wZylGGRkBFunworJ0mL6n2g+9HwALflp6i5B3j6DHTxaUhUXlJ sMu1YRpXS1DHRT62ltzIgX3KaAC41PxghdE3f0qo= Date: Wed, 30 Jun 2021 18:54:44 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, avagin@gmail.com, bernd.edlinger@hotmail.de, christian.brauner@ubuntu.com, christian.koenig@amd.com, corbet@lwn.net, deller@gmx.de, ebiederm@xmission.com, gladkov.alexey@gmail.com, hridya@google.com, jamorris@linux.microsoft.com, jannh@google.com, jeffv@google.com, kaleshsingh@google.com, keescook@chromium.org, linux-mm@kvack.org, mchehab+huawei@kernel.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, surenb@google.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, walken@google.com, willy@infradead.org Subject: [patch 142/192] procfs: allow reading fdinfo with PTRACE_MODE_READ Message-ID: <20210701015444.ZOZaFPX0b%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 862ECF000123 X-Stat-Signature: wfqpb97jiagctz5wmr9equ5izutcd4f7 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=RRunjc7U; dmarc=none; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104486-366496 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Kalesh Singh Subject: procfs: allow reading fdinfo with PTRACE_MODE_READ Android captures per-process system memory state when certain low memory events (e.g a foreground app kill) occur, to identify potential memory hoggers. In order to measure how much memory a process actually consumes, it is necessary to include the DMA buffer sizes for that process in the memory accounting. Since the handle to DMA buffers are raw FDs, it is important to be able to identify which processes have FD references to a DMA buffer. Currently, DMA buffer FDs can be accounted using /proc//fd/* and /proc//fdinfo -- both are only readable by the process owner, as follows: 1. Do a readlink on each FD. 2. If the target path begins with "/dmabuf", then the FD is a dmabuf FD. 3. stat the file to get the dmabuf inode number. 4. Read/ proc//fdinfo/, to get the DMA buffer size. Accessing other processes' fdinfo requires root privileges. This limits the use of the interface to debugging environments and is not suitable for production builds. Granting root privileges even to a system process increases the attack surface and is highly undesirable. Since fdinfo doesn't permit reading process memory and manipulating process state, allow accessing fdinfo under PTRACE_MODE_READ_FSCRED. Link: https://lkml.kernel.org/r/20210308170651.919148-1-kaleshsingh@google.com Signed-off-by: Kalesh Singh Suggested-by: Jann Horn Acked-by: Christian König Cc: Alexander Viro Cc: Alexey Dobriyan Cc: Alexey Gladkov Cc: Andrei Vagin Cc: Bernd Edlinger Cc: Christian Brauner Cc: Eric W. Biederman Cc: Helge Deller Cc: Hridya Valsaraju Cc: James Morris Cc: Jeff Vander Stoep Cc: Jonathan Corbet Cc: Kees Cook Cc: Matthew Wilcox Cc: Mauro Carvalho Chehab Cc: Michal Hocko Cc: Michel Lespinasse Cc: Minchan Kim Cc: Randy Dunlap Cc: Suren Baghdasaryan Cc: Szabolcs Nagy Signed-off-by: Andrew Morton Acked-by: Christian Brauner --- fs/proc/base.c | 4 ++-- fs/proc/fd.c | 15 ++++++++++++++- 2 files changed, 16 insertions(+), 3 deletions(-) --- a/fs/proc/base.c~procfs-allow-reading-fdinfo-with-ptrace_mode_read +++ a/fs/proc/base.c @@ -3172,7 +3172,7 @@ static const struct pid_entry tgid_base_ DIR("task", S_IRUGO|S_IXUGO, proc_task_inode_operations, proc_task_operations), DIR("fd", S_IRUSR|S_IXUSR, proc_fd_inode_operations, proc_fd_operations), DIR("map_files", S_IRUSR|S_IXUSR, proc_map_files_inode_operations, proc_map_files_operations), - DIR("fdinfo", S_IRUSR|S_IXUSR, proc_fdinfo_inode_operations, proc_fdinfo_operations), + DIR("fdinfo", S_IRUGO|S_IXUGO, proc_fdinfo_inode_operations, proc_fdinfo_operations), DIR("ns", S_IRUSR|S_IXUGO, proc_ns_dir_inode_operations, proc_ns_dir_operations), #ifdef CONFIG_NET DIR("net", S_IRUGO|S_IXUGO, proc_net_inode_operations, proc_net_operations), @@ -3517,7 +3517,7 @@ static const struct inode_operations pro */ static const struct pid_entry tid_base_stuff[] = { DIR("fd", S_IRUSR|S_IXUSR, proc_fd_inode_operations, proc_fd_operations), - DIR("fdinfo", S_IRUSR|S_IXUSR, proc_fdinfo_inode_operations, proc_fdinfo_operations), + DIR("fdinfo", S_IRUGO|S_IXUGO, proc_fdinfo_inode_operations, proc_fdinfo_operations), DIR("ns", S_IRUSR|S_IXUGO, proc_ns_dir_inode_operations, proc_ns_dir_operations), #ifdef CONFIG_NET DIR("net", S_IRUGO|S_IXUGO, proc_net_inode_operations, proc_net_operations), --- a/fs/proc/fd.c~procfs-allow-reading-fdinfo-with-ptrace_mode_read +++ a/fs/proc/fd.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include @@ -72,6 +73,18 @@ out: static int seq_fdinfo_open(struct inode *inode, struct file *file) { + bool allowed = false; + struct task_struct *task = get_proc_task(inode); + + if (!task) + return -ESRCH; + + allowed = ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS); + put_task_struct(task); + + if (!allowed) + return -EACCES; + return single_open(file, seq_show, inode); } @@ -308,7 +321,7 @@ static struct dentry *proc_fdinfo_instan struct proc_inode *ei; struct inode *inode; - inode = proc_pid_make_inode(dentry->d_sb, task, S_IFREG | S_IRUSR); + inode = proc_pid_make_inode(dentry->d_sb, task, S_IFREG | S_IRUGO); if (!inode) return ERR_PTR(-ENOENT); From patchwork Thu Jul 1 01:54:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0483DC11F69 for ; Thu, 1 Jul 2021 01:54:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0D4061476 for ; Thu, 1 Jul 2021 01:54:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0D4061476 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3A3BF8D0261; Wed, 30 Jun 2021 21:54:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 352DB8D024F; Wed, 30 Jun 2021 21:54:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F4B98D0261; Wed, 30 Jun 2021 21:54:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id EB1978D024F for ; Wed, 30 Jun 2021 21:54:51 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C05A8250D9 for ; Thu, 1 Jul 2021 01:54:51 +0000 (UTC) X-FDA: 78312350382.13.6262E02 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 66F051000160 for ; Thu, 1 Jul 2021 01:54:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CD33961468; Thu, 1 Jul 2021 01:54:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104490; bh=j9bb7KCpT+WlaIRcv9TsKm+fpCcSh6Z1jLwWQXqKbUQ=; h=Date:From:To:Subject:In-Reply-To:From; b=opFHlEd/NqhKB5Oz+D4QcKsQ/qQG1gMDK/SzN1JmkIgWiflagzt1qwT+yOBnifKLM /ArzE2lux5fG1RtPQH9zDUoi4F7IssKjflkebn4k6iaZTtJmfrHf+M18k1x/8dT/jb IXGyk6Wgc12fc+j7xPsHippdrOTMjugk+qPu7ecU= Date: Wed, 30 Jun 2021 18:54:49 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, avagin@gmail.com, bernd.edlinger@hotmail.de, christian.brauner@ubuntu.com, christian.koenig@amd.com, corbet@lwn.net, deller@gmx.de, ebiederm@xmission.com, gladkov.alexey@gmail.com, hridya@google.com, jamorris@linux.microsoft.com, jannh@google.com, jeffv@google.com, kaleshsingh@google.com, keescook@chromium.org, linux-mm@kvack.org, mchehab+huawei@kernel.org, mhocko@suse.com, minchan@kernel.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, surenb@google.com, szabolcs.nagy@arm.com, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, walken@google.com, willy@infradead.org Subject: [patch 143/192] procfs/dmabuf: add inode number to /proc/*/fdinfo Message-ID: <20210701015449.VAmBDscYI%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="opFHlEd/"; dmarc=none; spf=pass (imf13.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: p7moh1rbmqh7y9tidzjhzhg1aa8ab93x X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 66F051000160 X-HE-Tag: 1625104491-651560 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Kalesh Singh Subject: procfs/dmabuf: add inode number to /proc/*/fdinfo And 'ino' field to /proc//fdinfo/ and /proc//task//fdinfo/. The inode numbers can be used to uniquely identify DMA buffers in user space and avoids a dependency on /proc//fd/* when accounting per-process DMA buffer sizes. Link: https://lkml.kernel.org/r/20210308170651.919148-2-kaleshsingh@google.com Signed-off-by: Kalesh Singh Acked-by: Randy Dunlap Acked-by: Christian König Cc: Jann Horn Cc: Jeff Vander Stoep Cc: Kees Cook Cc: Suren Baghdasaryan Cc: Minchan Kim Cc: Hridya Valsaraju Cc: Matthew Wilcox Cc: Alexander Viro Cc: Kalesh Singh Cc: Alexey Dobriyan Cc: Jonathan Corbet Cc: Mauro Carvalho Chehab Cc: Michal Hocko Cc: Alexey Gladkov Cc: Szabolcs Nagy Cc: Eric W. Biederman Cc: Christian Brauner Cc: Michel Lespinasse Cc: Bernd Edlinger Cc: Andrei Vagin Cc: Helge Deller Cc: James Morris Signed-off-by: Andrew Morton --- Documentation/filesystems/proc.rst | 37 ++++++++++++++++++++++----- fs/proc/fd.c | 5 ++- 2 files changed, 34 insertions(+), 8 deletions(-) --- a/Documentation/filesystems/proc.rst~procfs-dmabuf-add-inode-number-to-proc-fdinfo +++ a/Documentation/filesystems/proc.rst @@ -1920,18 +1920,20 @@ if precise results are needed. 3.8 /proc//fdinfo/ - Information about opened file --------------------------------------------------------------- This file provides information associated with an opened file. The regular -files have at least three fields -- 'pos', 'flags' and 'mnt_id'. The 'pos' -represents the current offset of the opened file in decimal form [see lseek(2) -for details], 'flags' denotes the octal O_xxx mask the file has been -created with [see open(2) for details] and 'mnt_id' represents mount ID of -the file system containing the opened file [see 3.5 /proc//mountinfo -for details]. +files have at least four fields -- 'pos', 'flags', 'mnt_id' and 'ino'. +The 'pos' represents the current offset of the opened file in decimal +form [see lseek(2) for details], 'flags' denotes the octal O_xxx mask the +file has been created with [see open(2) for details] and 'mnt_id' represents +mount ID of the file system containing the opened file [see 3.5 +/proc//mountinfo for details]. 'ino' represents the inode number of +the file. A typical output is:: pos: 0 flags: 0100002 mnt_id: 19 + ino: 63107 All locks associated with a file descriptor are shown in its fdinfo too:: @@ -1948,6 +1950,7 @@ Eventfd files pos: 0 flags: 04002 mnt_id: 9 + ino: 63107 eventfd-count: 5a where 'eventfd-count' is hex value of a counter. @@ -1960,6 +1963,7 @@ Signalfd files pos: 0 flags: 04002 mnt_id: 9 + ino: 63107 sigmask: 0000000000000200 where 'sigmask' is hex value of the signal mask associated @@ -1973,6 +1977,7 @@ Epoll files pos: 0 flags: 02 mnt_id: 9 + ino: 63107 tfd: 5 events: 1d data: ffffffffffffffff pos:0 ino:61af sdev:7 where 'tfd' is a target file descriptor number in decimal form, @@ -1989,6 +1994,8 @@ For inotify files the format is the foll pos: 0 flags: 02000000 + mnt_id: 9 + ino: 63107 inotify wd:3 ino:9e7e sdev:800013 mask:800afce ignored_mask:0 fhandle-bytes:8 fhandle-type:1 f_handle:7e9e0000640d1b6d where 'wd' is a watch descriptor in decimal form, i.e. a target file @@ -2011,6 +2018,7 @@ For fanotify files the format is:: pos: 0 flags: 02 mnt_id: 9 + ino: 63107 fanotify flags:10 event-flags:0 fanotify mnt_id:12 mflags:40 mask:38 ignored_mask:40000003 fanotify ino:4f969 sdev:800013 mflags:0 mask:3b ignored_mask:40000000 fhandle-bytes:8 fhandle-type:1 f_handle:69f90400c275b5b4 @@ -2035,6 +2043,7 @@ Timerfd files pos: 0 flags: 02 mnt_id: 9 + ino: 63107 clockid: 0 ticks: 0 settime flags: 01 @@ -2049,6 +2058,22 @@ details]. 'it_value' is remaining time u with TIMER_ABSTIME option which will be shown in 'settime flags', but 'it_value' still exhibits timer's remaining time. +DMA Buffer files +~~~~~~~~~~~~~~~~ + +:: + + pos: 0 + flags: 04002 + mnt_id: 9 + ino: 63107 + size: 32768 + count: 2 + exp_name: system-heap + +where 'size' is the size of the DMA buffer in bytes. 'count' is the file count of +the DMA buffer file. 'exp_name' is the name of the DMA buffer exporter. + 3.9 /proc//map_files - Information about memory mapped files --------------------------------------------------------------------- This directory contains symbolic links which represent memory mapped files --- a/fs/proc/fd.c~procfs-dmabuf-add-inode-number-to-proc-fdinfo +++ a/fs/proc/fd.c @@ -54,9 +54,10 @@ static int seq_show(struct seq_file *m, if (ret) return ret; - seq_printf(m, "pos:\t%lli\nflags:\t0%o\nmnt_id:\t%i\n", + seq_printf(m, "pos:\t%lli\nflags:\t0%o\nmnt_id:\t%i\nino:\t%lu\n", (long long)file->f_pos, f_flags, - real_mount(file->f_path.mnt)->mnt_id); + real_mount(file->f_path.mnt)->mnt_id, + file_inode(file)->i_ino); /* show_fd_locks() never deferences files so a stale value is safe */ show_fd_locks(m, file, files); From patchwork Thu Jul 1 01:54:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E11DC11F68 for ; Thu, 1 Jul 2021 01:54:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C7E6D61469 for ; Thu, 1 Jul 2021 01:54:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7E6D61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4DB608D0262; Wed, 30 Jun 2021 21:54:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 497868D024F; Wed, 30 Jun 2021 21:54:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3057F8D0262; Wed, 30 Jun 2021 21:54:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 051918D024F for ; Wed, 30 Jun 2021 21:54:54 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D752982E090B for ; Thu, 1 Jul 2021 01:54:54 +0000 (UTC) X-FDA: 78312350508.01.7992B85 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 91731D000099 for ; Thu, 1 Jul 2021 01:54:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6B7F56141A; Thu, 1 Jul 2021 01:54:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104493; bh=Zay68I8W1NAS0TDg5JlNAJU7hfz/RJOB6jjX9bKT2uI=; h=Date:From:To:Subject:In-Reply-To:From; b=HwREGdCIKEqqsfayp5uk6HRm8706rc9zyZm1wypRcw6Hs8HswczK7L1TOLbHv9k4S MutIMAy+zoxE10a5X4RRrgstWAYfFEfHscHAWGrKz/NuptjQ0Gv+KXgY023d37Tr0d Lk3T8vGC9/8HgX41BcNjbLgU9rz+hwtmZztVIl/M= Date: Wed, 30 Jun 2021 18:54:53 -0700 From: Andrew Morton To: abaci@linux.alibaba.com, akpm@linux-foundation.org, andrii@kernel.org, ast@kernel.org, daniel@iogearbox.net, jiapeng.chong@linux.alibaba.com, john.fastabend@gmail.com, kafai@fb.com, keescook@chromium.org, kpsingh@kernel.org, linux-mm@kvack.org, mcgrof@kernel.org, mm-commits@vger.kernel.org, songliubraving@fb.com, torvalds@linux-foundation.org, yhs@fb.com, yzaikin@google.com Subject: [patch 144/192] sysctl: remove redundant assignment to first Message-ID: <20210701015453.ABOmnCugY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=HwREGdCI; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: c7145t1pug898wt7m6uxz75oipucer7y X-Rspamd-Queue-Id: 91731D000099 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104494-43351 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jiapeng Chong Subject: sysctl: remove redundant assignment to first Variable first is set to '0', but this value is never read as it is not used later on, hence it is a redundant assignment and can be removed. Clean up the following clang-analyzer warning: kernel/sysctl.c:1562:4: warning: Value stored to 'first' is never read [clang-analyzer-deadcode.DeadStores]. Link: https://lkml.kernel.org/r/1620469990-22182-1-git-send-email-jiapeng.chong@linux.alibaba.com Signed-off-by: Jiapeng Chong Reported-by: Abaci Robot Acked-by: Luis Chamberlain Cc: Kees Cook Cc: Iurii Zaikin Cc: Alexei Starovoitov Cc: Daniel Borkmann Cc: Andrii Nakryiko Cc: Martin KaFai Lau Cc: Song Liu Cc: Yonghong Song Cc: John Fastabend Cc: KP Singh Signed-off-by: Andrew Morton --- kernel/sysctl.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/kernel/sysctl.c~sysctl-remove-redundant-assignment-to-first +++ a/kernel/sysctl.c @@ -1494,7 +1494,6 @@ int proc_do_large_bitmap(struct ctl_tabl void *buffer, size_t *lenp, loff_t *ppos) { int err = 0; - bool first = 1; size_t left = *lenp; unsigned long bitmap_len = table->maxlen; unsigned long *bitmap = *(unsigned long **) table->data; @@ -1579,12 +1578,12 @@ int proc_do_large_bitmap(struct ctl_tabl } bitmap_set(tmp_bitmap, val_a, val_b - val_a + 1); - first = 0; proc_skip_char(&p, &left, '\n'); } left += skipped; } else { unsigned long bit_a, bit_b = 0; + bool first = 1; while (left) { bit_a = find_next_bit(bitmap, bitmap_len, bit_b); From patchwork Thu Jul 1 01:54:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF3D7C11F69 for ; Thu, 1 Jul 2021 01:54:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AF8466146D for ; Thu, 1 Jul 2021 01:54:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF8466146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 37BEB8D0263; Wed, 30 Jun 2021 21:54:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 306888D024F; Wed, 30 Jun 2021 21:54:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CDAE8D0263; Wed, 30 Jun 2021 21:54:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0114.hostedemail.com [216.40.44.114]) by kanga.kvack.org (Postfix) with ESMTP id E67758D024F for ; Wed, 30 Jun 2021 21:54:57 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BE5AA1802611B for ; Thu, 1 Jul 2021 01:54:57 +0000 (UTC) X-FDA: 78312350634.15.B2F609D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 8761BF00008C for ; Thu, 1 Jul 2021 01:54:57 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8E87061469; Thu, 1 Jul 2021 01:54:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104496; bh=1r4RZLNo8SkhQuLOS8DJMsoMAgtwTPXj8MVi3tBh+n8=; h=Date:From:To:Subject:In-Reply-To:From; b=IHXpJN4DUG6rbER2O4H8tNuWGWqDtifNcQF6Uv8ogQmYw45IXIN19av2Zl5z5KIJd nf2ZEQ90II6OZyiztwKPsiEwlGl6eBy8rnHAjLWb/r/FUPBgre1BW5ilTy7lXP3M21 MxBm5K48LUer+FT+lx3i8wIGlF8yaOX5h4i9AAbk= Date: Wed, 30 Jun 2021 18:54:56 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, jani.nikula@intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 145/192] drm: include only needed headers in ascii85.h Message-ID: <20210701015456.fEwxQa2nY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=IHXpJN4D; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8761BF00008C X-Stat-Signature: 4bfmntooxhfpy71yc3uits44e5syfae3 X-HE-Tag: 1625104497-196779 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: drm: include only needed headers in ascii85.h The ascii85.h is user of exactly two headers, i.e. math.h and types.h. There is no need to carry on entire kernel.h. Link: https://lkml.kernel.org/r/20210611185915.44181-1-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Reviewed-by: Jani Nikula Signed-off-by: Andrew Morton --- include/linux/ascii85.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/include/linux/ascii85.h~drm-include-only-needed-headers-in-ascii85h +++ a/include/linux/ascii85.h @@ -8,7 +8,8 @@ #ifndef _ASCII85_H_ #define _ASCII85_H_ -#include +#include +#include #define ASCII85_BUFSZ 6 From patchwork Thu Jul 1 01:54:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 048E9C11F69 for ; Thu, 1 Jul 2021 01:55:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8F2ED61476 for ; Thu, 1 Jul 2021 01:55:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F2ED61476 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 159366B00BC; Wed, 30 Jun 2021 21:55:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 109C88D0264; Wed, 30 Jun 2021 21:55:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E77258D024F; Wed, 30 Jun 2021 21:55:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id AF5886B00BC for ; Wed, 30 Jun 2021 21:55:01 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4AAF6181AF5C1 for ; Thu, 1 Jul 2021 01:55:01 +0000 (UTC) X-FDA: 78312350802.22.7D3ABE7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id E710A30000A6 for ; Thu, 1 Jul 2021 01:55:00 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AC6FC61469; Thu, 1 Jul 2021 01:54:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104500; bh=xyQsZifqOTOPqjHYoCWAJzwkqgUGBbf2vhRGqG8rrJU=; h=Date:From:To:Subject:In-Reply-To:From; b=aRJUWtbwaXBsSwTwH/ZTcPPextRhr+SSwjJLWkG25iQOrnIC4bb6oFFIfgiDZj6ha yHenyFSVQ1YzLKgcy6unrnWvkAXeNhjwX9Qn6kuMvs0m2MvqyCztxeBIm867md94e4 tFU/t0J2mJVVcfVd7V0c215RwJyALAMqeBc/Zs98= Date: Wed, 30 Jun 2021 18:54:59 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, arnd@arndb.de, bjorn.andersson@linaro.org, christian.brauner@ubuntu.com, cminyard@mvista.com, deller@gmx.de, keescook@chromium.org, linux-mm@kvack.org, linux@rasmusvillemoes.dk, mcgrof@kernel.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, sboyd@kernel.org, sre@kernel.org, torvalds@linux-foundation.org, tsbogend@alpha.franken.de, wei.liu@kernel.org Subject: [patch 146/192] kernel.h: split out panic and oops helpers Message-ID: <20210701015459.RNbWvwfIP%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=aRJUWtbw; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: gf6iqe3kijs4i58jai96fdfrquhy7dod X-Rspamd-Queue-Id: E710A30000A6 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104500-846979 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: kernel.h: split out panic and oops helpers kernel.h is being used as a dump for all kinds of stuff for a long time. Here is the attempt to start cleaning it up by splitting out panic and oops helpers. There are several purposes of doing this: - dropping dependency in bug.h - dropping a loop by moving out panic_notifier.h - unload kernel.h from something which has its own domain At the same time convert users tree-wide to use new headers, although for the time being include new header back to kernel.h to avoid twisted indirected includes for existing users. [akpm@linux-foundation.org: thread_info.h needs limits.h] [andriy.shevchenko@linux.intel.com: ia64 fix] Link: https://lkml.kernel.org/r/20210520130557.55277-1-andriy.shevchenko@linux.intel.com Link: https://lkml.kernel.org/r/20210511074137.33666-1-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Reviewed-by: Bjorn Andersson Co-developed-by: Andrew Morton Acked-by: Mike Rapoport Acked-by: Corey Minyard Acked-by: Christian Brauner Acked-by: Arnd Bergmann Acked-by: Kees Cook Acked-by: Wei Liu Acked-by: Rasmus Villemoes Signed-off-by: Andrew Morton Acked-by: Sebastian Reichel Acked-by: Luis Chamberlain Acked-by: Stephen Boyd Acked-by: Thomas Bogendoerfer Acked-by: Helge Deller # parisc Signed-off-by: Andrew Morton --- arch/alpha/kernel/setup.c | 2 arch/arm64/kernel/setup.c | 1 arch/ia64/include/asm/pal.h | 1 arch/mips/kernel/relocate.c | 1 arch/mips/sgi-ip22/ip22-reset.c | 1 arch/mips/sgi-ip32/ip32-reset.c | 1 arch/parisc/kernel/pdc_chassis.c | 1 arch/powerpc/kernel/setup-common.c | 1 arch/s390/kernel/ipl.c | 1 arch/sparc/kernel/sstate.c | 1 arch/um/drivers/mconsole_kern.c | 1 arch/um/kernel/um_arch.c | 1 arch/x86/include/asm/desc.h | 1 arch/x86/kernel/cpu/mshyperv.c | 1 arch/x86/kernel/setup.c | 1 arch/x86/purgatory/purgatory.c | 2 arch/x86/xen/enlighten.c | 1 arch/xtensa/platforms/iss/setup.c | 1 drivers/bus/brcmstb_gisb.c | 1 drivers/char/ipmi/ipmi_msghandler.c | 1 drivers/clk/analogbits/wrpll-cln28hpc.c | 4 drivers/edac/altera_edac.c | 1 drivers/firmware/google/gsmi.c | 1 drivers/hv/vmbus_drv.c | 1 drivers/hwtracing/coresight/coresight-cpu-debug.c | 1 drivers/leds/trigger/ledtrig-activity.c | 1 drivers/leds/trigger/ledtrig-heartbeat.c | 1 drivers/leds/trigger/ledtrig-panic.c | 1 drivers/misc/bcm-vk/bcm_vk_dev.c | 1 drivers/misc/ibmasm/heartbeat.c | 1 drivers/misc/pvpanic/pvpanic.c | 1 drivers/net/ipa/ipa_smp2p.c | 1 drivers/parisc/power.c | 1 drivers/power/reset/ltc2952-poweroff.c | 1 drivers/remoteproc/remoteproc_core.c | 1 drivers/s390/char/con3215.c | 1 drivers/s390/char/con3270.c | 1 drivers/s390/char/sclp.c | 1 drivers/s390/char/sclp_con.c | 1 drivers/s390/char/sclp_vt220.c | 1 drivers/s390/char/zcore.c | 1 drivers/soc/bcm/brcmstb/pm/pm-arm.c | 1 drivers/staging/olpc_dcon/olpc_dcon.c | 1 drivers/video/fbdev/hyperv_fb.c | 1 include/asm-generic/bug.h | 3 include/linux/kernel.h | 84 ---------- include/linux/panic.h | 98 ++++++++++++ include/linux/panic_notifier.h | 12 + include/linux/thread_info.h | 1 kernel/hung_task.c | 1 kernel/kexec_core.c | 1 kernel/panic.c | 1 kernel/rcu/tree.c | 2 kernel/sysctl.c | 1 kernel/trace/trace.c | 1 55 files changed, 169 insertions(+), 85 deletions(-) --- a/arch/alpha/kernel/setup.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/alpha/kernel/setup.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -46,7 +47,6 @@ #include #include -extern struct atomic_notifier_head panic_notifier_list; static int alpha_panic_event(struct notifier_block *, unsigned long, void *); static struct notifier_block alpha_panic_block = { alpha_panic_event, --- a/arch/arm64/kernel/setup.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/arm64/kernel/setup.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include --- a/arch/ia64/include/asm/pal.h~kernelh-split-out-panic-and-oops-helpers +++ a/arch/ia64/include/asm/pal.h @@ -99,6 +99,7 @@ #include #include +#include /* * Data types needed to pass information into PAL procedures and --- a/arch/mips/kernel/relocate.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/mips/kernel/relocate.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include --- a/arch/mips/sgi-ip22/ip22-reset.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/mips/sgi-ip22/ip22-reset.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include --- a/arch/mips/sgi-ip32/ip32-reset.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/mips/sgi-ip32/ip32-reset.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include --- a/arch/parisc/kernel/pdc_chassis.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/parisc/kernel/pdc_chassis.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include --- a/arch/powerpc/kernel/setup-common.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/powerpc/kernel/setup-common.c @@ -9,6 +9,7 @@ #undef DEBUG #include +#include #include #include #include --- a/arch/s390/kernel/ipl.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/s390/kernel/ipl.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include --- a/arch/sparc/kernel/sstate.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/sparc/kernel/sstate.c @@ -6,6 +6,7 @@ #include #include +#include #include #include --- a/arch/um/drivers/mconsole_kern.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/um/drivers/mconsole_kern.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include --- a/arch/um/kernel/um_arch.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/um/kernel/um_arch.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include #include --- a/arch/x86/include/asm/desc.h~kernelh-split-out-panic-and-oops-helpers +++ a/arch/x86/include/asm/desc.h @@ -9,6 +9,7 @@ #include #include +#include #include #include --- a/arch/x86/kernel/cpu/mshyperv.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/x86/kernel/cpu/mshyperv.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include --- a/arch/x86/kernel/setup.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/x86/kernel/setup.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include --- a/arch/x86/purgatory/purgatory.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/x86/purgatory/purgatory.c @@ -9,6 +9,8 @@ */ #include +#include +#include #include #include --- a/arch/x86/xen/enlighten.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/x86/xen/enlighten.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include --- a/arch/xtensa/platforms/iss/setup.c~kernelh-split-out-panic-and-oops-helpers +++ a/arch/xtensa/platforms/iss/setup.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include --- a/drivers/bus/brcmstb_gisb.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/bus/brcmstb_gisb.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include --- a/drivers/char/ipmi/ipmi_msghandler.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/char/ipmi/ipmi_msghandler.c @@ -16,6 +16,7 @@ #include #include +#include #include #include #include --- a/drivers/clk/analogbits/wrpll-cln28hpc.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/clk/analogbits/wrpll-cln28hpc.c @@ -23,8 +23,12 @@ #include #include +#include #include #include +#include +#include + #include /* MIN_INPUT_FREQ: minimum input clock frequency, in Hz (Fref_min) */ --- a/drivers/edac/altera_edac.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/edac/altera_edac.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include --- a/drivers/firmware/google/gsmi.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/firmware/google/gsmi.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include --- a/drivers/hv/vmbus_drv.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/hv/vmbus_drv.c @@ -25,6 +25,7 @@ #include #include +#include #include #include #include --- a/drivers/hwtracing/coresight/coresight-cpu-debug.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/hwtracing/coresight/coresight-cpu-debug.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include --- a/drivers/leds/trigger/ledtrig-activity.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/leds/trigger/ledtrig-activity.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include --- a/drivers/leds/trigger/ledtrig-heartbeat.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/leds/trigger/ledtrig-heartbeat.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include --- a/drivers/leds/trigger/ledtrig-panic.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/leds/trigger/ledtrig-panic.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include "../leds.h" --- a/drivers/misc/bcm-vk/bcm_vk_dev.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/misc/bcm-vk/bcm_vk_dev.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include --- a/drivers/misc/ibmasm/heartbeat.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/misc/ibmasm/heartbeat.c @@ -9,6 +9,7 @@ */ #include +#include #include "ibmasm.h" #include "dot_command.h" #include "lowlevel.h" --- a/drivers/misc/pvpanic/pvpanic.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/misc/pvpanic/pvpanic.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include --- a/drivers/net/ipa/ipa_smp2p.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/net/ipa/ipa_smp2p.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include --- a/drivers/parisc/power.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/parisc/power.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include #include --- a/drivers/power/reset/ltc2952-poweroff.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/power/reset/ltc2952-poweroff.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include #include --- a/drivers/remoteproc/remoteproc_core.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/remoteproc/remoteproc_core.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include #include --- a/drivers/s390/char/con3215.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/s390/char/con3215.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include /* ASYNC_* flags */ #include --- a/drivers/s390/char/con3270.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/s390/char/con3270.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include --- a/drivers/s390/char/sclp.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/s390/char/sclp.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include --- a/drivers/s390/char/sclp_con.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/s390/char/sclp_con.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include --- a/drivers/s390/char/sclp_vt220.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/s390/char/sclp_vt220.c @@ -9,6 +9,7 @@ #include #include +#include #include #include #include --- a/drivers/s390/char/zcore.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/s390/char/zcore.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include --- a/drivers/soc/bcm/brcmstb/pm/pm-arm.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/soc/bcm/brcmstb/pm/pm-arm.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include --- a/drivers/staging/olpc_dcon/olpc_dcon.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/staging/olpc_dcon/olpc_dcon.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include --- a/drivers/video/fbdev/hyperv_fb.c~kernelh-split-out-panic-and-oops-helpers +++ a/drivers/video/fbdev/hyperv_fb.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #include --- a/include/asm-generic/bug.h~kernelh-split-out-panic-and-oops-helpers +++ a/include/asm-generic/bug.h @@ -17,7 +17,8 @@ #endif #ifndef __ASSEMBLY__ -#include +#include +#include #ifdef CONFIG_BUG --- a/include/linux/kernel.h~kernelh-split-out-panic-and-oops-helpers +++ a/include/linux/kernel.h @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -72,7 +73,6 @@ #define lower_32_bits(n) ((u32)((n) & 0xffffffff)) struct completion; -struct pt_regs; struct user; #ifdef CONFIG_PREEMPT_VOLUNTARY @@ -177,14 +177,6 @@ void __might_fault(const char *file, int static inline void might_fault(void) { } #endif -extern struct atomic_notifier_head panic_notifier_list; -extern long (*panic_blink)(int state); -__printf(1, 2) -void panic(const char *fmt, ...) __noreturn __cold; -void nmi_panic(struct pt_regs *regs, const char *msg); -extern void oops_enter(void); -extern void oops_exit(void); -extern bool oops_may_print(void); void do_exit(long error_code) __noreturn; void complete_and_exit(struct completion *, long) __noreturn; @@ -372,52 +364,8 @@ extern int __kernel_text_address(unsigne extern int kernel_text_address(unsigned long addr); extern int func_ptr_is_kernel_text(void *ptr); -#ifdef CONFIG_SMP -extern unsigned int sysctl_oops_all_cpu_backtrace; -#else -#define sysctl_oops_all_cpu_backtrace 0 -#endif /* CONFIG_SMP */ - extern void bust_spinlocks(int yes); -extern int panic_timeout; -extern unsigned long panic_print; -extern int panic_on_oops; -extern int panic_on_unrecovered_nmi; -extern int panic_on_io_nmi; -extern int panic_on_warn; -extern unsigned long panic_on_taint; -extern bool panic_on_taint_nousertaint; -extern int sysctl_panic_on_rcu_stall; -extern int sysctl_max_rcu_stall_to_panic; -extern int sysctl_panic_on_stackoverflow; - -extern bool crash_kexec_post_notifiers; - -/* - * panic_cpu is used for synchronizing panic() and crash_kexec() execution. It - * holds a CPU number which is executing panic() currently. A value of - * PANIC_CPU_INVALID means no CPU has entered panic() or crash_kexec(). - */ -extern atomic_t panic_cpu; -#define PANIC_CPU_INVALID -1 -/* - * Only to be used by arch init code. If the user over-wrote the default - * CONFIG_PANIC_TIMEOUT, honor it. - */ -static inline void set_arch_panic_timeout(int timeout, int arch_default_timeout) -{ - if (panic_timeout == arch_default_timeout) - panic_timeout = timeout; -} -extern const char *print_tainted(void); -enum lockdep_ok { - LOCKDEP_STILL_OK, - LOCKDEP_NOW_UNRELIABLE -}; -extern void add_taint(unsigned flag, enum lockdep_ok); -extern int test_taint(unsigned flag); -extern unsigned long get_taint(void); extern int root_mountflags; extern bool early_boot_irqs_disabled; @@ -436,36 +384,6 @@ extern enum system_states { SYSTEM_SUSPEND, } system_state; -/* This cannot be an enum because some may be used in assembly source. */ -#define TAINT_PROPRIETARY_MODULE 0 -#define TAINT_FORCED_MODULE 1 -#define TAINT_CPU_OUT_OF_SPEC 2 -#define TAINT_FORCED_RMMOD 3 -#define TAINT_MACHINE_CHECK 4 -#define TAINT_BAD_PAGE 5 -#define TAINT_USER 6 -#define TAINT_DIE 7 -#define TAINT_OVERRIDDEN_ACPI_TABLE 8 -#define TAINT_WARN 9 -#define TAINT_CRAP 10 -#define TAINT_FIRMWARE_WORKAROUND 11 -#define TAINT_OOT_MODULE 12 -#define TAINT_UNSIGNED_MODULE 13 -#define TAINT_SOFTLOCKUP 14 -#define TAINT_LIVEPATCH 15 -#define TAINT_AUX 16 -#define TAINT_RANDSTRUCT 17 -#define TAINT_FLAGS_COUNT 18 -#define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1) - -struct taint_flag { - char c_true; /* character printed when tainted */ - char c_false; /* character printed when not tainted */ - bool module; /* also show as a per-module taint flag */ -}; - -extern const struct taint_flag taint_flags[TAINT_FLAGS_COUNT]; - extern const char hex_asc[]; #define hex_asc_lo(x) hex_asc[((x) & 0x0f)] #define hex_asc_hi(x) hex_asc[((x) & 0xf0) >> 4] --- /dev/null +++ a/include/linux/panic.h @@ -0,0 +1,98 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_PANIC_H +#define _LINUX_PANIC_H + +#include +#include + +struct pt_regs; + +extern long (*panic_blink)(int state); +__printf(1, 2) +void panic(const char *fmt, ...) __noreturn __cold; +void nmi_panic(struct pt_regs *regs, const char *msg); +extern void oops_enter(void); +extern void oops_exit(void); +extern bool oops_may_print(void); + +#ifdef CONFIG_SMP +extern unsigned int sysctl_oops_all_cpu_backtrace; +#else +#define sysctl_oops_all_cpu_backtrace 0 +#endif /* CONFIG_SMP */ + +extern int panic_timeout; +extern unsigned long panic_print; +extern int panic_on_oops; +extern int panic_on_unrecovered_nmi; +extern int panic_on_io_nmi; +extern int panic_on_warn; + +extern unsigned long panic_on_taint; +extern bool panic_on_taint_nousertaint; + +extern int sysctl_panic_on_rcu_stall; +extern int sysctl_max_rcu_stall_to_panic; +extern int sysctl_panic_on_stackoverflow; + +extern bool crash_kexec_post_notifiers; + +/* + * panic_cpu is used for synchronizing panic() and crash_kexec() execution. It + * holds a CPU number which is executing panic() currently. A value of + * PANIC_CPU_INVALID means no CPU has entered panic() or crash_kexec(). + */ +extern atomic_t panic_cpu; +#define PANIC_CPU_INVALID -1 + +/* + * Only to be used by arch init code. If the user over-wrote the default + * CONFIG_PANIC_TIMEOUT, honor it. + */ +static inline void set_arch_panic_timeout(int timeout, int arch_default_timeout) +{ + if (panic_timeout == arch_default_timeout) + panic_timeout = timeout; +} + +/* This cannot be an enum because some may be used in assembly source. */ +#define TAINT_PROPRIETARY_MODULE 0 +#define TAINT_FORCED_MODULE 1 +#define TAINT_CPU_OUT_OF_SPEC 2 +#define TAINT_FORCED_RMMOD 3 +#define TAINT_MACHINE_CHECK 4 +#define TAINT_BAD_PAGE 5 +#define TAINT_USER 6 +#define TAINT_DIE 7 +#define TAINT_OVERRIDDEN_ACPI_TABLE 8 +#define TAINT_WARN 9 +#define TAINT_CRAP 10 +#define TAINT_FIRMWARE_WORKAROUND 11 +#define TAINT_OOT_MODULE 12 +#define TAINT_UNSIGNED_MODULE 13 +#define TAINT_SOFTLOCKUP 14 +#define TAINT_LIVEPATCH 15 +#define TAINT_AUX 16 +#define TAINT_RANDSTRUCT 17 +#define TAINT_FLAGS_COUNT 18 +#define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1) + +struct taint_flag { + char c_true; /* character printed when tainted */ + char c_false; /* character printed when not tainted */ + bool module; /* also show as a per-module taint flag */ +}; + +extern const struct taint_flag taint_flags[TAINT_FLAGS_COUNT]; + +enum lockdep_ok { + LOCKDEP_STILL_OK, + LOCKDEP_NOW_UNRELIABLE, +}; + +extern const char *print_tainted(void); +extern void add_taint(unsigned flag, enum lockdep_ok); +extern int test_taint(unsigned flag); +extern unsigned long get_taint(void); + +#endif /* _LINUX_PANIC_H */ --- /dev/null +++ a/include/linux/panic_notifier.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_PANIC_NOTIFIERS_H +#define _LINUX_PANIC_NOTIFIERS_H + +#include +#include + +extern struct atomic_notifier_head panic_notifier_list; + +extern bool crash_kexec_post_notifiers; + +#endif /* _LINUX_PANIC_NOTIFIERS_H */ --- a/include/linux/thread_info.h~kernelh-split-out-panic-and-oops-helpers +++ a/include/linux/thread_info.h @@ -9,6 +9,7 @@ #define _LINUX_THREAD_INFO_H #include +#include #include #include #include --- a/kernel/hung_task.c~kernelh-split-out-panic-and-oops-helpers +++ a/kernel/hung_task.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include --- a/kernel/kexec_core.c~kernelh-split-out-panic-and-oops-helpers +++ a/kernel/kexec_core.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include --- a/kernel/panic.c~kernelh-split-out-panic-and-oops-helpers +++ a/kernel/panic.c @@ -23,6 +23,7 @@ #include #include #include +#include #include #include #include --- a/kernel/rcu/tree.c~kernelh-split-out-panic-and-oops-helpers +++ a/kernel/rcu/tree.c @@ -32,6 +32,8 @@ #include #include #include +#include +#include #include #include #include --- a/kernel/sysctl.c~kernelh-split-out-panic-and-oops-helpers +++ a/kernel/sysctl.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include --- a/kernel/trace/trace.c~kernelh-split-out-panic-and-oops-helpers +++ a/kernel/trace/trace.c @@ -39,6 +39,7 @@ #include #include #include +#include #include #include #include From patchwork Thu Jul 1 01:55:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E27CC11F6A for ; Thu, 1 Jul 2021 01:55:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0A52461469 for ; Thu, 1 Jul 2021 01:55:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0A52461469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 85DDF6B00BE; Wed, 30 Jun 2021 21:55:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 80D948D0264; Wed, 30 Jun 2021 21:55:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D5CB8D024F; Wed, 30 Jun 2021 21:55:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0203.hostedemail.com [216.40.44.203]) by kanga.kvack.org (Postfix) with ESMTP id 430FB6B00BE for ; Wed, 30 Jun 2021 21:55:04 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 1564A82E220A for ; Thu, 1 Jul 2021 01:55:04 +0000 (UTC) X-FDA: 78312350928.02.C9EC3C8 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id C87661981 for ; Thu, 1 Jul 2021 01:55:03 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D8E9E61420; Thu, 1 Jul 2021 01:55:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104503; bh=lzWDZl7iHCfeefN6bqRDTgU1KA8F1T/4UU1KFbO4rfU=; h=Date:From:To:Subject:In-Reply-To:From; b=FZOS6CzwOAIQ+5gKjGLvO1z50PimgV1h+oJqEHBUJwNz1GKPfgXxV7nvqt4/cC88a xykWNpmIDqsVvBNMlY+/FUhtTcslTp4QeiXSVt9dygQMGiB1CeXU9cWwGJ1jT84NLF yoN9PGaUB9UDtm4YDOHr02EuQ8kC15Ox+tjXaGzA= Date: Wed, 30 Jun 2021 18:55:02 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, thunder.leizhen@huawei.com, torvalds@linux-foundation.org Subject: [patch 147/192] lib: decompress_bunzip2: remove an unneeded semicolon Message-ID: <20210701015502.IAHucUpyV%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C87661981 X-Stat-Signature: 7guqndzd19rm6f1xg55cf7sypacq9brd Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=FZOS6Czw; dmarc=none; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104503-114162 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhen Lei Subject: lib: decompress_bunzip2: remove an unneeded semicolon The semicolon immediately following '}' is unneeded. Link: https://lkml.kernel.org/r/20210508094926.2889-1-thunder.leizhen@huawei.com Signed-off-by: Zhen Lei Signed-off-by: Andrew Morton --- lib/decompress_bunzip2.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/lib/decompress_bunzip2.c~lib-decompress_bunzip2-remove-an-unneeded-semicolon +++ a/lib/decompress_bunzip2.c @@ -385,7 +385,7 @@ static int INIT get_next_block(struct bu bd->inbufBits = (bd->inbufBits << 8)|bd->inbuf[bd->inbufPos++]; bd->inbufBitCount += 8; - }; + } bd->inbufBitCount -= hufGroup->maxLen; j = (bd->inbufBits >> bd->inbufBitCount)& ((1 << hufGroup->maxLen)-1); From patchwork Thu Jul 1 01:55:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 399B6C11F6D for ; Thu, 1 Jul 2021 01:55:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E67BC61468 for ; Thu, 1 Jul 2021 01:55:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E67BC61468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6FEA86B00C0; Wed, 30 Jun 2021 21:55:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6AE768D0264; Wed, 30 Jun 2021 21:55:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54F488D024F; Wed, 30 Jun 2021 21:55:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0033.hostedemail.com [216.40.44.33]) by kanga.kvack.org (Postfix) with ESMTP id 278996B00C0 for ; Wed, 30 Jun 2021 21:55:07 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id F12D4181AF5C1 for ; Thu, 1 Jul 2021 01:55:06 +0000 (UTC) X-FDA: 78312351012.27.CD41139 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id A66432001708 for ; Thu, 1 Jul 2021 01:55:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BFE3261469; Thu, 1 Jul 2021 01:55:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104506; bh=pM0+FeXVkO9cDXZDX8AB6B9LS+cW/LwCvoFYQ/g2jM4=; h=Date:From:To:Subject:In-Reply-To:From; b=HPyyNrb6l38vkJ0RmVjhOvpTR+7OpYpeS4kgKvO1uPNAd+naNzATW+le/U2rANWOt 8VA4dBrAjXblYRZqFKy5b/lD469nnCI4UekHLUJvsICPjDools17P22d5XqRkSpdsL Vn+rpX9YzDvFfk7HqMLQG5lrgLEjKb4gNCnzf4b4= Date: Wed, 30 Jun 2021 18:55:05 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 148/192] lib/string_helpers: switch to use BIT() macro Message-ID: <20210701015505.B7JgnSQKB%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=HPyyNrb6; dmarc=none; spf=pass (imf26.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: 8tu13md5akxdarhimq5wqw7md1m55h6z X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A66432001708 X-HE-Tag: 1625104506-155698 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/string_helpers: switch to use BIT() macro Patch series "lib/string_helpers: get rid of ugly *_escape_mem_ascii()", v3. Get rid of ugly *_escape_mem_ascii() API since it's not flexible and has the only single user. Provide better approach based on usage of the string_escape_mem() with appropriate flags. Test cases has been expanded accordingly to cover new functionality. This patch (of 15): Switch to use BIT() macro for flag definitions. No changes implied. Link: https://lkml.kernel.org/r/20210504180819.73127-1-andriy.shevchenko@linux.intel.com Link: https://lkml.kernel.org/r/20210504180819.73127-2-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: "J. Bruce Fields" Cc: Chuck Lever Cc: Alexander Viro Signed-off-by: Andrew Morton --- include/linux/string_helpers.h | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) --- a/include/linux/string_helpers.h~lib-string_helpers-switch-to-use-bit-macro +++ a/include/linux/string_helpers.h @@ -2,6 +2,7 @@ #ifndef _LINUX_STRING_HELPERS_H_ #define _LINUX_STRING_HELPERS_H_ +#include #include #include @@ -18,10 +19,10 @@ enum string_size_units { void string_get_size(u64 size, u64 blk_size, enum string_size_units units, char *buf, int len); -#define UNESCAPE_SPACE 0x01 -#define UNESCAPE_OCTAL 0x02 -#define UNESCAPE_HEX 0x04 -#define UNESCAPE_SPECIAL 0x08 +#define UNESCAPE_SPACE BIT(0) +#define UNESCAPE_OCTAL BIT(1) +#define UNESCAPE_HEX BIT(2) +#define UNESCAPE_SPECIAL BIT(3) #define UNESCAPE_ANY \ (UNESCAPE_SPACE | UNESCAPE_OCTAL | UNESCAPE_HEX | UNESCAPE_SPECIAL) @@ -42,15 +43,15 @@ static inline int string_unescape_any_in return string_unescape_any(buf, buf, 0); } -#define ESCAPE_SPACE 0x01 -#define ESCAPE_SPECIAL 0x02 -#define ESCAPE_NULL 0x04 -#define ESCAPE_OCTAL 0x08 +#define ESCAPE_SPACE BIT(0) +#define ESCAPE_SPECIAL BIT(1) +#define ESCAPE_NULL BIT(2) +#define ESCAPE_OCTAL BIT(3) #define ESCAPE_ANY \ (ESCAPE_SPACE | ESCAPE_OCTAL | ESCAPE_SPECIAL | ESCAPE_NULL) -#define ESCAPE_NP 0x10 +#define ESCAPE_NP BIT(4) #define ESCAPE_ANY_NP (ESCAPE_ANY | ESCAPE_NP) -#define ESCAPE_HEX 0x20 +#define ESCAPE_HEX BIT(5) int string_escape_mem(const char *src, size_t isz, char *dst, size_t osz, unsigned int flags, const char *only); From patchwork Thu Jul 1 01:55:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 500B0C11F68 for ; Thu, 1 Jul 2021 01:55:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D1A361468 for ; Thu, 1 Jul 2021 01:55:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D1A361468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 885766B00C2; Wed, 30 Jun 2021 21:55:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8343F8D0264; Wed, 30 Jun 2021 21:55:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63AAB8D024F; Wed, 30 Jun 2021 21:55:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 359E16B00C2 for ; Wed, 30 Jun 2021 21:55:10 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0E8F7231DC for ; Thu, 1 Jul 2021 01:55:10 +0000 (UTC) X-FDA: 78312351180.06.81E3E6D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id B31B1900009B for ; Thu, 1 Jul 2021 01:55:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B1C1E61477; Thu, 1 Jul 2021 01:55:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104509; bh=u9LQ6uoTdjrj7hrwsKvaUjKS8Fqfglmv5172vJHajKM=; h=Date:From:To:Subject:In-Reply-To:From; b=vkEP6o7XpS2rwBBE52u1qQvZ98KzggQP6hm3JNlVI/8rZgFF7rv+rn95oD2W9pDaw 2mrcs+4VSucujhB7ixfA6ndcnO7fqZ6DdyI4VnbCbG0qbZrFItL4yiaf4mqTU+HAhy dslwVzqinL3vZuPZEbyWZoQ3ZgYcTkzsRhd/HGCs= Date: Wed, 30 Jun 2021 18:55:08 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 149/192] lib/string_helpers: move ESCAPE_NP check inside 'else' branch in a loop Message-ID: <20210701015508.STQE9tMFn%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=vkEP6o7X; spf=pass (imf23.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: n66eikzs4yd3kpba16woejfwf678rnaw X-Rspamd-Queue-Id: B31B1900009B X-Rspamd-Server: rspam06 X-HE-Tag: 1625104509-459695 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/string_helpers: move ESCAPE_NP check inside 'else' branch in a loop Refactor code to have better readability by moving ESCAPE_NP handling inside 'else' branch in the loop. No functional change intended. Link: https://lkml.kernel.org/r/20210504180819.73127-3-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- lib/string_helpers.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) --- a/lib/string_helpers.c~lib-string_helpers-move-escape_np-check-inside-else-branch-in-a-loop +++ a/lib/string_helpers.c @@ -452,10 +452,10 @@ static bool escape_hex(unsigned char c, * The process of escaping byte buffer includes several parts. They are applied * in the following sequence. * - * 1. The character is matched to the printable class, if asked, and in - * case of match it passes through to the output. - * 2. The character is not matched to the one from @only string and thus + * 1. The character is not matched to the one from @only string and thus * must go as-is to the output. + * 2. The character is matched to the printable class, if asked, and in + * case of match it passes through to the output. * 3. The character is checked if it falls into the class given by @flags. * %ESCAPE_OCTAL and %ESCAPE_HEX are going last since they cover any * character. Note that they actually can't go together, otherwise @@ -506,19 +506,22 @@ int string_escape_mem(const char *src, s /* * Apply rules in the following sequence: - * - the character is printable, when @flags has - * %ESCAPE_NP bit set * - the @only string is supplied and does not contain a * character under question + * - the character is printable, when @flags has + * %ESCAPE_NP bit set * - the character doesn't fall into a class of symbols * defined by given @flags * In these cases we just pass through a character to the * output buffer. */ - if ((flags & ESCAPE_NP && isprint(c)) || - (is_dict && !strchr(only, c))) { + if (is_dict && !strchr(only, c)) { /* do nothing */ } else { + if (isprint(c) && + flags & ESCAPE_NP && escape_passthrough(c, &p, end)) + continue; + if (flags & ESCAPE_SPACE && escape_space(c, &p, end)) continue; From patchwork Thu Jul 1 01:55:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53A76C11F69 for ; Thu, 1 Jul 2021 01:55:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0322B61469 for ; Thu, 1 Jul 2021 01:55:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0322B61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 80F476B00C4; Wed, 30 Jun 2021 21:55:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7991C6B00C6; Wed, 30 Jun 2021 21:55:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6211F6B00C4; Wed, 30 Jun 2021 21:55:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0069.hostedemail.com [216.40.44.69]) by kanga.kvack.org (Postfix) with ESMTP id 3828D6B00C4 for ; Wed, 30 Jun 2021 21:55:13 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 13C9E253D3 for ; Thu, 1 Jul 2021 01:55:13 +0000 (UTC) X-FDA: 78312351306.25.C42BC01 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id B0BDA6001A84 for ; Thu, 1 Jul 2021 01:55:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A827161468; Thu, 1 Jul 2021 01:55:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104512; bh=4Mpa2rwdbCPvVFcSJxxOL9nBYSlpttWLrsHinkDW53g=; h=Date:From:To:Subject:In-Reply-To:From; b=NUjnVtaNlp8LJUGPeCs9AYMLZayDhZfDLSQVWiBs8BGKLoepO4gzbChIIh/m+/arv sUCfTvJbiNP+n6ucbpNqIPzuhoUrm4/N38gwWIiMDBB4Skc07LfC+Ph5TVIyX5M4Ei TBX4C4+KPlIfBJmP8NJGGdAfS9gZgkwn2sDmcLzM= Date: Wed, 30 Jun 2021 18:55:11 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 150/192] lib/string_helpers: drop indentation level in string_escape_mem() Message-ID: <20210701015511.sGCCmfpPg%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: B0BDA6001A84 X-Stat-Signature: k1s8pj7ah1m8xhgf58mdwbp4jim1qi5h Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=NUjnVtaN; dmarc=none; spf=pass (imf10.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104512-380733 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/string_helpers: drop indentation level in string_escape_mem() The only one conditional is left on the upper level, move the rest to the same level and drop indentation level. No functional changes. Link: https://lkml.kernel.org/r/20210504180819.73127-4-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- lib/string_helpers.c | 46 ++++++++++++++++++++--------------------- 1 file changed, 23 insertions(+), 23 deletions(-) --- a/lib/string_helpers.c~lib-string_helpers-drop-indentation-level-in-string_escape_mem +++ a/lib/string_helpers.c @@ -515,29 +515,29 @@ int string_escape_mem(const char *src, s * In these cases we just pass through a character to the * output buffer. */ - if (is_dict && !strchr(only, c)) { - /* do nothing */ - } else { - if (isprint(c) && - flags & ESCAPE_NP && escape_passthrough(c, &p, end)) - continue; - - if (flags & ESCAPE_SPACE && escape_space(c, &p, end)) - continue; - - if (flags & ESCAPE_SPECIAL && escape_special(c, &p, end)) - continue; - - if (flags & ESCAPE_NULL && escape_null(c, &p, end)) - continue; - - /* ESCAPE_OCTAL and ESCAPE_HEX always go last */ - if (flags & ESCAPE_OCTAL && escape_octal(c, &p, end)) - continue; - - if (flags & ESCAPE_HEX && escape_hex(c, &p, end)) - continue; - } + if (is_dict && !strchr(only, c) && + escape_passthrough(c, &p, end)) + continue; + + if (isprint(c) && + flags & ESCAPE_NP && escape_passthrough(c, &p, end)) + continue; + + if (flags & ESCAPE_SPACE && escape_space(c, &p, end)) + continue; + + if (flags & ESCAPE_SPECIAL && escape_special(c, &p, end)) + continue; + + if (flags & ESCAPE_NULL && escape_null(c, &p, end)) + continue; + + /* ESCAPE_OCTAL and ESCAPE_HEX always go last */ + if (flags & ESCAPE_OCTAL && escape_octal(c, &p, end)) + continue; + + if (flags & ESCAPE_HEX && escape_hex(c, &p, end)) + continue; escape_passthrough(c, &p, end); } From patchwork Thu Jul 1 01:55:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55753C11F68 for ; Thu, 1 Jul 2021 01:55:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0B4CD61420 for ; Thu, 1 Jul 2021 01:55:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B4CD61420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 85D7C6B00C6; Wed, 30 Jun 2021 21:55:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E6756B00C7; Wed, 30 Jun 2021 21:55:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 660066B00C8; Wed, 30 Jun 2021 21:55:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 383716B00C6 for ; Wed, 30 Jun 2021 21:55:16 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0D124253D3 for ; Thu, 1 Jul 2021 01:55:16 +0000 (UTC) X-FDA: 78312351432.37.5291B47 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id AA04AD000090 for ; Thu, 1 Jul 2021 01:55:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A06F961468; Thu, 1 Jul 2021 01:55:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104514; bh=eQHdghz9R8V3nszjckKcL+tzm5ZspdBY/WJnutbaMjI=; h=Date:From:To:Subject:In-Reply-To:From; b=xiuCsvTCF+WxxEcKPGLruchFl70Jokhi+tMw0UYMpXczc99/BPv4mUutsOfiOhA4S gWNLn0ec33nLioK2Yom/y0BU1d/DbIxBxteVscx/4zOGo2qyF3mggJJub+ULZVn6bM x+SuYHy94lE+ckGW8cg5Sicg5g7/42zJxM8hLEkc= Date: Wed, 30 Jun 2021 18:55:14 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 151/192] lib/string_helpers: introduce ESCAPE_NA for escaping non-ASCII Message-ID: <20210701015514.fNmvXuUVO%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=xiuCsvTC; dmarc=none; spf=pass (imf15.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: doocwztcb761mnmp1ocfezt5a8kbgybs X-Rspamd-Queue-Id: AA04AD000090 X-HE-Tag: 1625104515-373249 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/string_helpers: introduce ESCAPE_NA for escaping non-ASCII Some users may want to have an ASCII based filter, provided by isascii() function. Here is the addition of a such. Link: https://lkml.kernel.org/r/20210504180819.73127-5-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- include/linux/string_helpers.h | 1 + lib/string_helpers.c | 21 +++++++++++++++++---- 2 files changed, 18 insertions(+), 4 deletions(-) --- a/include/linux/string_helpers.h~lib-string_helpers-introduce-escape_na-for-escaping-non-ascii +++ a/include/linux/string_helpers.h @@ -52,6 +52,7 @@ static inline int string_unescape_any_in #define ESCAPE_NP BIT(4) #define ESCAPE_ANY_NP (ESCAPE_ANY | ESCAPE_NP) #define ESCAPE_HEX BIT(5) +#define ESCAPE_NA BIT(6) int string_escape_mem(const char *src, size_t isz, char *dst, size_t osz, unsigned int flags, const char *only); --- a/lib/string_helpers.c~lib-string_helpers-introduce-escape_na-for-escaping-non-ascii +++ a/lib/string_helpers.c @@ -454,8 +454,8 @@ static bool escape_hex(unsigned char c, * * 1. The character is not matched to the one from @only string and thus * must go as-is to the output. - * 2. The character is matched to the printable class, if asked, and in - * case of match it passes through to the output. + * 2. The character is matched to the printable or ASCII class, if asked, + * and in case of match it passes through to the output. * 3. The character is checked if it falls into the class given by @flags. * %ESCAPE_OCTAL and %ESCAPE_HEX are going last since they cover any * character. Note that they actually can't go together, otherwise @@ -463,7 +463,7 @@ static bool escape_hex(unsigned char c, * * Caller must provide valid source and destination pointers. Be aware that * destination buffer will not be NULL-terminated, thus caller have to append - * it if needs. The supported flags are:: + * it if needs. The supported flags are:: * * %ESCAPE_SPACE: (special white space, not space itself) * '\f' - form feed @@ -482,11 +482,18 @@ static bool escape_hex(unsigned char c, * %ESCAPE_ANY: * all previous together * %ESCAPE_NP: - * escape only non-printable characters (checked by isprint) + * escape only non-printable characters, checked by isprint() * %ESCAPE_ANY_NP: * all previous together * %ESCAPE_HEX: * '\xHH' - byte with hexadecimal value HH (2 digits) + * %ESCAPE_NA: + * escape only non-ascii characters, checked by isascii() + * + * One notable caveat, the %ESCAPE_NP and %ESCAPE_NA have higher priority + * than the rest of the flags (%ESCAPE_NP is higher than %ESCAPE_NA). + * It doesn't make much sense to use either of them without %ESCAPE_OCTAL + * or %ESCAPE_HEX, because they cover most of the other character classes. * * Return: * The total size of the escaped output that would be generated for @@ -510,6 +517,8 @@ int string_escape_mem(const char *src, s * character under question * - the character is printable, when @flags has * %ESCAPE_NP bit set + * - the character is ASCII, when @flags has + * %ESCAPE_NA bit set * - the character doesn't fall into a class of symbols * defined by given @flags * In these cases we just pass through a character to the @@ -523,6 +532,10 @@ int string_escape_mem(const char *src, s flags & ESCAPE_NP && escape_passthrough(c, &p, end)) continue; + if (isascii(c) && + flags & ESCAPE_NA && escape_passthrough(c, &p, end)) + continue; + if (flags & ESCAPE_SPACE && escape_space(c, &p, end)) continue; From patchwork Thu Jul 1 01:55:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27EB0C11F66 for ; Thu, 1 Jul 2021 01:55:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D335D61420 for ; Thu, 1 Jul 2021 01:55:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D335D61420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 59E636B00C7; Wed, 30 Jun 2021 21:55:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 528066B00C9; Wed, 30 Jun 2021 21:55:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C8706B00CA; Wed, 30 Jun 2021 21:55:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id 0E9D46B00C7 for ; Wed, 30 Jun 2021 21:55:19 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DA6D982E090B for ; Thu, 1 Jul 2021 01:55:18 +0000 (UTC) X-FDA: 78312351516.03.49FD5CA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id 847FAF00012B for ; Thu, 1 Jul 2021 01:55:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9600D61476; Thu, 1 Jul 2021 01:55:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104517; bh=HtnzH4et/7uUBBU5CSmAMAU5ohVsctPE28Lz0R7p6/Y=; h=Date:From:To:Subject:In-Reply-To:From; b=qzIViynqW2gKtKx3TArcWNFi4f6ThoQqVvU+pKiSHYBjcVvpV7EJKcp03x0nhXccw mYAXIoHZKBDJhrEGWmECs+3StHjNCHYxGODSkWnHQgpXi2W27iRAzQpxBHg+k/iKvV M1LsFpG0b2MvLC4qpCw8ymspA60H+idmArideipw= Date: Wed, 30 Jun 2021 18:55:17 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 152/192] lib/string_helpers: introduce ESCAPE_NAP to escape non-ASCII and non-printable Message-ID: <20210701015517.qp0JG4ivY%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 847FAF00012B X-Stat-Signature: ieopcx4usopjxk66siqee9bkxkz1nq7x Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=qzIViynq; dmarc=none; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104518-261595 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/string_helpers: introduce ESCAPE_NAP to escape non-ASCII and non-printable Some users may want to have an ASCII based filter for printable only characters, provided by conjunction of isascii() and isprint() functions. Here is the addition of a such. Link: https://lkml.kernel.org/r/20210504180819.73127-6-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- include/linux/string_helpers.h | 1 + lib/string_helpers.c | 20 ++++++++++++++++---- 2 files changed, 17 insertions(+), 4 deletions(-) --- a/include/linux/string_helpers.h~lib-string_helpers-introduce-escape_nap-to-escape-non-ascii-and-non-printable +++ a/include/linux/string_helpers.h @@ -53,6 +53,7 @@ static inline int string_unescape_any_in #define ESCAPE_ANY_NP (ESCAPE_ANY | ESCAPE_NP) #define ESCAPE_HEX BIT(5) #define ESCAPE_NA BIT(6) +#define ESCAPE_NAP BIT(7) int string_escape_mem(const char *src, size_t isz, char *dst, size_t osz, unsigned int flags, const char *only); --- a/lib/string_helpers.c~lib-string_helpers-introduce-escape_nap-to-escape-non-ascii-and-non-printable +++ a/lib/string_helpers.c @@ -454,9 +454,11 @@ static bool escape_hex(unsigned char c, * * 1. The character is not matched to the one from @only string and thus * must go as-is to the output. - * 2. The character is matched to the printable or ASCII class, if asked, + * 2. The character is matched to the printable and ASCII classes, if asked, * and in case of match it passes through to the output. - * 3. The character is checked if it falls into the class given by @flags. + * 3. The character is matched to the printable or ASCII class, if asked, + * and in case of match it passes through to the output. + * 4. The character is checked if it falls into the class given by @flags. * %ESCAPE_OCTAL and %ESCAPE_HEX are going last since they cover any * character. Note that they actually can't go together, otherwise * %ESCAPE_HEX will be ignored. @@ -489,11 +491,15 @@ static bool escape_hex(unsigned char c, * '\xHH' - byte with hexadecimal value HH (2 digits) * %ESCAPE_NA: * escape only non-ascii characters, checked by isascii() + * %ESCAPE_NAP: + * escape only non-printable or non-ascii characters * - * One notable caveat, the %ESCAPE_NP and %ESCAPE_NA have higher priority - * than the rest of the flags (%ESCAPE_NP is higher than %ESCAPE_NA). + * One notable caveat, the %ESCAPE_NAP, %ESCAPE_NP and %ESCAPE_NA have the + * higher priority than the rest of the flags (%ESCAPE_NAP is the highest). * It doesn't make much sense to use either of them without %ESCAPE_OCTAL * or %ESCAPE_HEX, because they cover most of the other character classes. + * %ESCAPE_NAP can utilize %ESCAPE_SPACE or %ESCAPE_SPECIAL in addition to + * the above. * * Return: * The total size of the escaped output that would be generated for @@ -515,6 +521,8 @@ int string_escape_mem(const char *src, s * Apply rules in the following sequence: * - the @only string is supplied and does not contain a * character under question + * - the character is printable and ASCII, when @flags has + * %ESCAPE_NAP bit set * - the character is printable, when @flags has * %ESCAPE_NP bit set * - the character is ASCII, when @flags has @@ -528,6 +536,10 @@ int string_escape_mem(const char *src, s escape_passthrough(c, &p, end)) continue; + if (isascii(c) && isprint(c) && + flags & ESCAPE_NAP && escape_passthrough(c, &p, end)) + continue; + if (isprint(c) && flags & ESCAPE_NP && escape_passthrough(c, &p, end)) continue; From patchwork Thu Jul 1 01:55:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47810C11F66 for ; Thu, 1 Jul 2021 01:55:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDD9A6141A for ; Thu, 1 Jul 2021 01:55:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD9A6141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6FE376B00C9; Wed, 30 Jun 2021 21:55:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6AD8E6B00CB; Wed, 30 Jun 2021 21:55:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54EAF6B00CC; Wed, 30 Jun 2021 21:55:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0229.hostedemail.com [216.40.44.229]) by kanga.kvack.org (Postfix) with ESMTP id 2BFBA6B00C9 for ; Wed, 30 Jun 2021 21:55:22 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 0160A1802611B for ; Thu, 1 Jul 2021 01:55:22 +0000 (UTC) X-FDA: 78312351684.01.4BC7F71 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id A57F0500009F for ; Thu, 1 Jul 2021 01:55:21 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 95E7E6147E; Thu, 1 Jul 2021 01:55:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104520; bh=BLC+64tC2Oh0WoAv7v65CAfBZKCM7Oa+8avD+we9koQ=; h=Date:From:To:Subject:In-Reply-To:From; b=WNvstbqIYY42VswoY/V0cPHxUgOYHSnFe90hvfOM8yKaNME9WxiIgqOzFl3gt4kA4 qRNnqm5HeHk25/RvmWQb3xv7KilpQ/LNQGqcBUNStQja7oIGJf3k8XM2Hkty6X26bf V3P4Gd5dQuEXBpr8g+8w8gQc1oBCX3HxCX41Qqg4= Date: Wed, 30 Jun 2021 18:55:20 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 153/192] lib/string_helpers: allow to append additional characters to be escaped Message-ID: <20210701015520.3p-rGfCUh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A57F0500009F X-Stat-Signature: prcak5ae76i64eibf9yyggbjsauecg7r Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=WNvstbqI; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104521-367706 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/string_helpers: allow to append additional characters to be escaped Introduce a new flag to append additional characters, passed in 'only' parameter, to be escaped if they fall in the corresponding class. Link: https://lkml.kernel.org/r/20210504180819.73127-7-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- include/linux/string_helpers.h | 1 + lib/string_helpers.c | 19 +++++++++++++++---- 2 files changed, 16 insertions(+), 4 deletions(-) --- a/include/linux/string_helpers.h~lib-string_helpers-allow-to-append-additional-characters-to-be-escaped +++ a/include/linux/string_helpers.h @@ -54,6 +54,7 @@ static inline int string_unescape_any_in #define ESCAPE_HEX BIT(5) #define ESCAPE_NA BIT(6) #define ESCAPE_NAP BIT(7) +#define ESCAPE_APPEND BIT(8) int string_escape_mem(const char *src, size_t isz, char *dst, size_t osz, unsigned int flags, const char *only); --- a/lib/string_helpers.c~lib-string_helpers-allow-to-append-additional-characters-to-be-escaped +++ a/lib/string_helpers.c @@ -493,6 +493,11 @@ static bool escape_hex(unsigned char c, * escape only non-ascii characters, checked by isascii() * %ESCAPE_NAP: * escape only non-printable or non-ascii characters + * %ESCAPE_APPEND: + * append characters from @only to be escaped by the given classes + * + * %ESCAPE_APPEND would help to pass additional characters to the escaped, when + * one of %ESCAPE_NP, %ESCAPE_NA, or %ESCAPE_NAP is provided. * * One notable caveat, the %ESCAPE_NAP, %ESCAPE_NP and %ESCAPE_NA have the * higher priority than the rest of the flags (%ESCAPE_NAP is the highest). @@ -513,9 +518,11 @@ int string_escape_mem(const char *src, s char *p = dst; char *end = p + osz; bool is_dict = only && *only; + bool is_append = flags & ESCAPE_APPEND; while (isz--) { unsigned char c = *src++; + bool in_dict = is_dict && strchr(only, c); /* * Apply rules in the following sequence: @@ -531,20 +538,24 @@ int string_escape_mem(const char *src, s * defined by given @flags * In these cases we just pass through a character to the * output buffer. + * + * When %ESCAPE_APPEND is passed, the characters from @only + * have been excluded from the %ESCAPE_NAP, %ESCAPE_NP, and + * %ESCAPE_NA cases. */ - if (is_dict && !strchr(only, c) && + if (!(is_append || in_dict) && is_dict && escape_passthrough(c, &p, end)) continue; - if (isascii(c) && isprint(c) && + if (!(is_append && in_dict) && isascii(c) && isprint(c) && flags & ESCAPE_NAP && escape_passthrough(c, &p, end)) continue; - if (isprint(c) && + if (!(is_append && in_dict) && isprint(c) && flags & ESCAPE_NP && escape_passthrough(c, &p, end)) continue; - if (isascii(c) && + if (!(is_append && in_dict) && isascii(c) && flags & ESCAPE_NA && escape_passthrough(c, &p, end)) continue; From patchwork Thu Jul 1 01:55:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12939C11F69 for ; Thu, 1 Jul 2021 01:55:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AAF8A61476 for ; Thu, 1 Jul 2021 01:55:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAF8A61476 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 306738D0264; Wed, 30 Jun 2021 21:55:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 28F348D024F; Wed, 30 Jun 2021 21:55:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 157478D0264; Wed, 30 Jun 2021 21:55:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0162.hostedemail.com [216.40.44.162]) by kanga.kvack.org (Postfix) with ESMTP id E24048D024F for ; Wed, 30 Jun 2021 21:55:24 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id B920522C15 for ; Thu, 1 Jul 2021 01:55:24 +0000 (UTC) X-FDA: 78312351768.02.19A57C5 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 7A6B9100009D for ; Thu, 1 Jul 2021 01:55:24 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 942F96141A; Thu, 1 Jul 2021 01:55:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104523; bh=p2RllxadCbIusKUbC7ewkpNnDEyDXLUR8LAolvOg/f8=; h=Date:From:To:Subject:In-Reply-To:From; b=rOluH3KyvKMA4GXo0o761TZNY8A7Dhx8E9W/veFP+sAgI12BDc40vze0bcwZxrZC9 b3KH1MITs0OWTLYtxvHiGZ+iZ33a4pF7lsZLw0Y7etucM/JwXQXCSBeG+lXAcTYoVq yEcYQVKeamAnZudGd4i2yNoXyDJzwfiwM2kO25JA= Date: Wed, 30 Jun 2021 18:55:23 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 154/192] lib/test-string_helpers: print flags in hexadecimal format Message-ID: <20210701015523.QwafX-mc5%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=rOluH3Ky; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: iq97dimje3oktiphkbzo4ycpo51omzsn X-Rspamd-Queue-Id: 7A6B9100009D X-Rspamd-Server: rspam06 X-HE-Tag: 1625104524-598856 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/test-string_helpers: print flags in hexadecimal format Since flags are bitmapped, it's better to print them in hexadecimal format. Link: https://lkml.kernel.org/r/20210504180819.73127-8-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- lib/test-string_helpers.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/lib/test-string_helpers.c~lib-test-string_helpers-print-flags-in-hexadecimal-format +++ a/lib/test-string_helpers.c @@ -19,7 +19,7 @@ static __init bool test_string_check_buf if (q_real == q_test && !memcmp(out_test, out_real, q_test)) return true; - pr_warn("Test '%s' failed: flags = %u\n", name, flags); + pr_warn("Test '%s' failed: flags = %#x\n", name, flags); print_hex_dump(KERN_WARNING, "Input: ", DUMP_PREFIX_NONE, 16, 1, in, p, true); @@ -290,7 +290,7 @@ test_string_escape_overflow(const char * q_real = string_escape_mem(in, p, NULL, 0, flags, esc); if (q_real != q_test) - pr_warn("Test '%s' failed: flags = %u, osz = 0, expected %d, got %d\n", + pr_warn("Test '%s' failed: flags = %#x, osz = 0, expected %d, got %d\n", name, flags, q_test, q_real); } From patchwork Thu Jul 1 01:55:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0A96C11F69 for ; Thu, 1 Jul 2021 01:55:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A8CB161420 for ; Thu, 1 Jul 2021 01:55:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A8CB161420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 355108D0265; Wed, 30 Jun 2021 21:55:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 32C048D024F; Wed, 30 Jun 2021 21:55:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F3268D0265; Wed, 30 Jun 2021 21:55:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0223.hostedemail.com [216.40.44.223]) by kanga.kvack.org (Postfix) with ESMTP id E93348D024F for ; Wed, 30 Jun 2021 21:55:27 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C29E618033D29 for ; Thu, 1 Jul 2021 01:55:27 +0000 (UTC) X-FDA: 78312351894.14.23B80D3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 7E6CA1981 for ; Thu, 1 Jul 2021 01:55:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 81B4B6147D; Thu, 1 Jul 2021 01:55:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104526; bh=YABlZvrkIRQAM39pgP7bivPIaKU83C0SruaqZYLMPUw=; h=Date:From:To:Subject:In-Reply-To:From; b=I/fm0KIpfbgAJx6beU+/NtHV2NY0wdHLLg5/pcovr8kdTL3OJPyXISCZCJQucvfdx UC0qApY00UXmab+KWfjaZQpqH1ab1TWc4tTEnwKX7MtQoe5yqzlLTNrkKFQmW8Gctw g8v+i6bY/ZiLZNdIjZIXzFqKM8NJ9A3nbrA0lDhk= Date: Wed, 30 Jun 2021 18:55:26 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 155/192] lib/test-string_helpers: get rid of trailing comma in terminators Message-ID: <20210701015526.YCbwPm9CE%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="I/fm0KIp"; dmarc=none; spf=pass (imf22.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: ou6dzt13wxywcydsimxmdpw4sqatfjfw X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7E6CA1981 X-HE-Tag: 1625104527-655064 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/test-string_helpers: get rid of trailing comma in terminators Terminators by definition shouldn't accept anything behind. Make them robust by removing trailing commas. Link: https://lkml.kernel.org/r/20210504180819.73127-9-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- lib/test-string_helpers.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) --- a/lib/test-string_helpers.c~lib-test-string_helpers-get-rid-of-trailing-comma-in-terminators +++ a/lib/test-string_helpers.c @@ -136,7 +136,7 @@ static const struct test_string_2 escape .flags = ESCAPE_SPACE | ESCAPE_HEX, },{ /* terminator */ - }}, + }} },{ .in = "\\h\\\"\a\e\\", .s1 = {{ @@ -150,7 +150,7 @@ static const struct test_string_2 escape .flags = ESCAPE_SPECIAL | ESCAPE_HEX, },{ /* terminator */ - }}, + }} },{ .in = "\eb \\C\007\"\x90\r]", .s1 = {{ @@ -201,7 +201,7 @@ static const struct test_string_2 escape .flags = ESCAPE_NP | ESCAPE_HEX, },{ /* terminator */ - }}, + }} },{ /* terminator */ }}; @@ -217,7 +217,7 @@ static const struct test_string_2 escape .flags = ESCAPE_HEX, },{ /* terminator */ - }}, + }} },{ .in = "\\h\\\"\a\e\\", .s1 = {{ @@ -225,7 +225,7 @@ static const struct test_string_2 escape .flags = ESCAPE_OCTAL, },{ /* terminator */ - }}, + }} },{ .in = "\eb \\C\007\"\x90\r]", .s1 = {{ @@ -233,7 +233,7 @@ static const struct test_string_2 escape .flags = ESCAPE_OCTAL, },{ /* terminator */ - }}, + }} },{ /* terminator */ }}; From patchwork Thu Jul 1 01:55:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05E98C11F68 for ; Thu, 1 Jul 2021 01:55:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B0A3F61420 for ; Thu, 1 Jul 2021 01:55:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B0A3F61420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3A3518D0266; Wed, 30 Jun 2021 21:55:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 379F08D024F; Wed, 30 Jun 2021 21:55:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 21A988D0266; Wed, 30 Jun 2021 21:55:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0096.hostedemail.com [216.40.44.96]) by kanga.kvack.org (Postfix) with ESMTP id EAEFD8D024F for ; Wed, 30 Jun 2021 21:55:30 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B96D8181AF5C1 for ; Thu, 1 Jul 2021 01:55:30 +0000 (UTC) X-FDA: 78312352020.29.204C654 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 5CAC7700009B for ; Thu, 1 Jul 2021 01:55:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E5E96146D; Thu, 1 Jul 2021 01:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104529; bh=DVRUBa6gTcLU53zvZlPrv70B8B4tyU/yVpZLkBY91DE=; h=Date:From:To:Subject:In-Reply-To:From; b=avWv3LKTR+bYjzFg+Ejf8xiI5FzptwUI3TzWDMamDaAE7n0DzAQiLdId6Ii600imX kJ8uJTURGiMdQNhc+T1kpQSwFAxroL68xuu+vElQNkAOTSL6+xVRPbxryI2/L1CFIZ w8r/hQSgc4cyX6GPo/tZbeE+wdUOBaKPjVnH5Fuw= Date: Wed, 30 Jun 2021 18:55:29 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 156/192] lib/test-string_helpers: add test cases for new features Message-ID: <20210701015529.TnkAtUVM1%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5CAC7700009B X-Stat-Signature: q1po4q7p7me9dbi5zmk3ir47wsmx3fgp Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=avWv3LKT; dmarc=none; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104530-213705 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: lib/test-string_helpers: add test cases for new features We have got new flags and hence new features of string_escape_mem(). Add test cases for that. Link: https://lkml.kernel.org/r/20210504180819.73127-10-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- include/linux/string_helpers.h | 4 lib/test-string_helpers.c | 141 +++++++++++++++++++++++++++++-- 2 files changed, 137 insertions(+), 8 deletions(-) --- a/include/linux/string_helpers.h~lib-test-string_helpers-add-test-cases-for-new-features +++ a/include/linux/string_helpers.h @@ -26,6 +26,8 @@ void string_get_size(u64 size, u64 blk_s #define UNESCAPE_ANY \ (UNESCAPE_SPACE | UNESCAPE_OCTAL | UNESCAPE_HEX | UNESCAPE_SPECIAL) +#define UNESCAPE_ALL_MASK GENMASK(3, 0) + int string_unescape(char *src, char *dst, size_t size, unsigned int flags); static inline int string_unescape_inplace(char *buf, unsigned int flags) @@ -56,6 +58,8 @@ static inline int string_unescape_any_in #define ESCAPE_NAP BIT(7) #define ESCAPE_APPEND BIT(8) +#define ESCAPE_ALL_MASK GENMASK(8, 0) + int string_escape_mem(const char *src, size_t isz, char *dst, size_t osz, unsigned int flags, const char *only); --- a/lib/test-string_helpers.c~lib-test-string_helpers-add-test-cases-for-new-features +++ a/lib/test-string_helpers.c @@ -203,10 +203,24 @@ static const struct test_string_2 escape /* terminator */ }} },{ + .in = "\007 \eb\"\x90\xCF\r", + .s1 = {{ + .out = "\007 \eb\"\\220\\317\r", + .flags = ESCAPE_OCTAL | ESCAPE_NA, + },{ + .out = "\007 \eb\"\\x90\\xcf\r", + .flags = ESCAPE_HEX | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_NA, + },{ + /* terminator */ + }} +},{ /* terminator */ }}; -#define TEST_STRING_2_DICT_1 "b\\ \t\r" +#define TEST_STRING_2_DICT_1 "b\\ \t\r\xCF" static const struct test_string_2 escape1[] __initconst = {{ .in = "\f\\ \n\r\t\v", .s1 = {{ @@ -216,14 +230,38 @@ static const struct test_string_2 escape .out = "\f\\x5c\\x20\n\\x0d\\x09\v", .flags = ESCAPE_HEX, },{ + .out = "\f\\134\\040\n\\015\\011\v", + .flags = ESCAPE_ANY | ESCAPE_APPEND, + },{ + .out = "\\014\\134\\040\\012\\015\\011\\013", + .flags = ESCAPE_OCTAL | ESCAPE_APPEND | ESCAPE_NAP, + },{ + .out = "\\x0c\\x5c\\x20\\x0a\\x0d\\x09\\x0b", + .flags = ESCAPE_HEX | ESCAPE_APPEND | ESCAPE_NAP, + },{ + .out = "\f\\134\\040\n\\015\\011\v", + .flags = ESCAPE_OCTAL | ESCAPE_APPEND | ESCAPE_NA, + },{ + .out = "\f\\x5c\\x20\n\\x0d\\x09\v", + .flags = ESCAPE_HEX | ESCAPE_APPEND | ESCAPE_NA, + },{ /* terminator */ }} },{ - .in = "\\h\\\"\a\e\\", + .in = "\\h\\\"\a\xCF\e\\", .s1 = {{ - .out = "\\134h\\134\"\a\e\\134", + .out = "\\134h\\134\"\a\\317\e\\134", .flags = ESCAPE_OCTAL, },{ + .out = "\\134h\\134\"\a\\317\e\\134", + .flags = ESCAPE_ANY | ESCAPE_APPEND, + },{ + .out = "\\134h\\134\"\\007\\317\\033\\134", + .flags = ESCAPE_OCTAL | ESCAPE_APPEND | ESCAPE_NAP, + },{ + .out = "\\134h\\134\"\a\\317\e\\134", + .flags = ESCAPE_OCTAL | ESCAPE_APPEND | ESCAPE_NA, + },{ /* terminator */ }} },{ @@ -235,6 +273,88 @@ static const struct test_string_2 escape /* terminator */ }} },{ + .in = "\007 \eb\"\x90\xCF\r", + .s1 = {{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_SPACE | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_SPECIAL | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_SPACE | ESCAPE_SPECIAL | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\317\r", + .flags = ESCAPE_OCTAL | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\317\r", + .flags = ESCAPE_SPACE | ESCAPE_OCTAL | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\317\r", + .flags = ESCAPE_SPECIAL | ESCAPE_OCTAL | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\317\r", + .flags = ESCAPE_ANY | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\xcf\r", + .flags = ESCAPE_HEX | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\xcf\r", + .flags = ESCAPE_SPACE | ESCAPE_HEX | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\xcf\r", + .flags = ESCAPE_SPECIAL | ESCAPE_HEX | ESCAPE_NA, + },{ + .out = "\007 \eb\"\x90\\xcf\r", + .flags = ESCAPE_SPACE | ESCAPE_SPECIAL | ESCAPE_HEX | ESCAPE_NA, + },{ + /* terminator */ + }} +},{ + .in = "\007 \eb\"\x90\xCF\r", + .s1 = {{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\xCF\\r", + .flags = ESCAPE_SPACE | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\xCF\r", + .flags = ESCAPE_SPECIAL | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\xCF\\r", + .flags = ESCAPE_SPACE | ESCAPE_SPECIAL | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\317\\015", + .flags = ESCAPE_OCTAL | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\317\\r", + .flags = ESCAPE_SPACE | ESCAPE_OCTAL | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\317\\015", + .flags = ESCAPE_SPECIAL | ESCAPE_OCTAL | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\317\r", + .flags = ESCAPE_ANY | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\xcf\\x0d", + .flags = ESCAPE_HEX | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\xcf\\r", + .flags = ESCAPE_SPACE | ESCAPE_HEX | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\xcf\\x0d", + .flags = ESCAPE_SPECIAL | ESCAPE_HEX | ESCAPE_NAP, + },{ + .out = "\007 \eb\"\x90\\xcf\\r", + .flags = ESCAPE_SPACE | ESCAPE_SPECIAL | ESCAPE_HEX | ESCAPE_NAP, + },{ + /* terminator */ + }} +},{ /* terminator */ }}; @@ -315,8 +435,13 @@ static __init void test_string_escape(co /* NULL injection */ if (flags & ESCAPE_NULL) { in[p++] = '\0'; - out_test[q_test++] = '\\'; - out_test[q_test++] = '0'; + /* '\0' passes isascii() test */ + if (flags & ESCAPE_NA && !(flags & ESCAPE_APPEND && esc)) { + out_test[q_test++] = '\0'; + } else { + out_test[q_test++] = '\\'; + out_test[q_test++] = '0'; + } } /* Don't try strings that have no output */ @@ -459,17 +584,17 @@ static int __init test_string_helpers_in unsigned int i; pr_info("Running tests...\n"); - for (i = 0; i < UNESCAPE_ANY + 1; i++) + for (i = 0; i < UNESCAPE_ALL_MASK + 1; i++) test_string_unescape("unescape", i, false); test_string_unescape("unescape inplace", get_random_int() % (UNESCAPE_ANY + 1), true); /* Without dictionary */ - for (i = 0; i < (ESCAPE_ANY_NP | ESCAPE_HEX) + 1; i++) + for (i = 0; i < ESCAPE_ALL_MASK + 1; i++) test_string_escape("escape 0", escape0, i, TEST_STRING_2_DICT_0); /* With dictionary */ - for (i = 0; i < (ESCAPE_ANY_NP | ESCAPE_HEX) + 1; i++) + for (i = 0; i < ESCAPE_ALL_MASK + 1; i++) test_string_escape("escape 1", escape1, i, TEST_STRING_2_DICT_1); /* Test string_get_size() */ From patchwork Thu Jul 1 01:55:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA572C11F66 for ; Thu, 1 Jul 2021 01:55:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8965B61420 for ; Thu, 1 Jul 2021 01:55:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8965B61420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0E41F8D0267; Wed, 30 Jun 2021 21:55:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0BA368D024F; Wed, 30 Jun 2021 21:55:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9D2E8D0267; Wed, 30 Jun 2021 21:55:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0118.hostedemail.com [216.40.44.118]) by kanga.kvack.org (Postfix) with ESMTP id BDD048D024F for ; Wed, 30 Jun 2021 21:55:33 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9927525F58 for ; Thu, 1 Jul 2021 01:55:33 +0000 (UTC) X-FDA: 78312352146.09.CEA0DED Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 579225000AFB for ; Thu, 1 Jul 2021 01:55:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5D0006141A; Thu, 1 Jul 2021 01:55:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104532; bh=Xgv0iCo6p/cj/ROz+yQ0rE+g2nRQ5V5X6Lszzs53qLo=; h=Date:From:To:Subject:In-Reply-To:From; b=0VqTRTv6VlA6qNNHBe1aR+3rDpYhtspNTqfxMXuPXiZWP4lsI2vHwqgyV90wq7dBR 3t5LJtgKZEgPTz+B5apj7/mtyH9GHcIHSDW4IFDZ/Me0qPisHCeRxln4oQAwDtz6iL brhczsRkIVKsa8onOkzCLW1CPeA062OSQezOIs84= Date: Wed, 30 Jun 2021 18:55:32 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 157/192] MAINTAINERS: add myself as designated reviewer for generic string library Message-ID: <20210701015532.TM-YdEfeZ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=0VqTRTv6; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 579225000AFB X-Stat-Signature: k6pigsmq744aexsyyxas8xjis9pqddj4 X-HE-Tag: 1625104533-780167 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: MAINTAINERS: add myself as designated reviewer for generic string library Add myself as designated reviewer for generic string library. Link: https://lkml.kernel.org/r/20210504180819.73127-11-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+) --- a/MAINTAINERS~maintainers-add-myself-as-designated-reviewer-for-generic-string-library +++ a/MAINTAINERS @@ -7648,6 +7648,14 @@ L: linux-input@vger.kernel.org S: Maintained F: drivers/input/touchscreen/resistive-adc-touch.c +GENERIC STRING LIBRARY +R: Andy Shevchenko +S: Maintained +F: lib/string.c +F: lib/string_helpers.c +F: lib/test_string.c +F: lib/test-string_helpers.c + GENERIC UIO DRIVER FOR PCI DEVICES M: "Michael S. Tsirkin" L: kvm@vger.kernel.org From patchwork Thu Jul 1 01:55:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF109C11F68 for ; Thu, 1 Jul 2021 01:55:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6741961469 for ; Thu, 1 Jul 2021 01:55:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6741961469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E511B8D0268; Wed, 30 Jun 2021 21:55:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E27A78D024F; Wed, 30 Jun 2021 21:55:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D16708D0268; Wed, 30 Jun 2021 21:55:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id A87548D024F for ; Wed, 30 Jun 2021 21:55:36 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7FA2818033D29 for ; Thu, 1 Jul 2021 01:55:36 +0000 (UTC) X-FDA: 78312352272.03.DE7239E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 369F8100009D for ; Thu, 1 Jul 2021 01:55:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 47CBE61420; Thu, 1 Jul 2021 01:55:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104535; bh=6CIP4YdyeHYqf6rxoBTLM+Y1uFHyq+GNWeCSjMEm7SA=; h=Date:From:To:Subject:In-Reply-To:From; b=DTiNRv50MDwfrW/+DKk6ZLV1SwRN3c2DplidHf3q1MnL6csZrMQCeG0pzfX9t01VS 6mSpJFya4Kj3gWIsIfa5dt8CK2FNHvHdIUH0klDz9mZzEtAOqQlQFKVV4OYrbCFNNc wpuivQ0nzEKMEaMtK4YI1ScATlkTVsjuEt9jLuXU= Date: Wed, 30 Jun 2021 18:55:34 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 158/192] seq_file: introduce seq_escape_mem() Message-ID: <20210701015534.dRLur5WZy%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=DTiNRv50; dmarc=none; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: 4khwrhboxzt4c8akdiz6rr6qm3y73mta X-Rspamd-Queue-Id: 369F8100009D X-HE-Tag: 1625104536-152230 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: seq_file: introduce seq_escape_mem() Introduce seq_escape_mem() to allow users to pass additional parameters to string_escape_mem(). Link: https://lkml.kernel.org/r/20210504180819.73127-12-andriy.shevchenko@linux.intel.com Suggested-by: Al Viro Signed-off-by: Andy Shevchenko Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- fs/seq_file.c | 25 +++++++++++++++++++++++++ include/linux/seq_file.h | 2 ++ 2 files changed, 27 insertions(+) --- a/fs/seq_file.c~seq_file-introduce-seq_escape_mem +++ a/fs/seq_file.c @@ -356,6 +356,31 @@ int seq_release(struct inode *inode, str EXPORT_SYMBOL(seq_release); /** + * seq_escape_mem - print data into buffer, escaping some characters + * @m: target buffer + * @src: source buffer + * @len: size of source buffer + * @flags: flags to pass to string_escape_mem() + * @esc: set of characters that need escaping + * + * Puts data into buffer, replacing each occurrence of character from + * given class (defined by @flags and @esc) with printable escaped sequence. + * + * Use seq_has_overflowed() to check for errors. + */ +void seq_escape_mem(struct seq_file *m, const char *src, size_t len, + unsigned int flags, const char *esc) +{ + char *buf; + size_t size = seq_get_buf(m, &buf); + int ret; + + ret = string_escape_mem(src, len, buf, size, flags, esc); + seq_commit(m, ret < size ? ret : -1); +} +EXPORT_SYMBOL(seq_escape_mem); + +/** * seq_escape - print string into buffer, escaping some characters * @m: target buffer * @s: string --- a/include/linux/seq_file.h~seq_file-introduce-seq_escape_mem +++ a/include/linux/seq_file.h @@ -126,6 +126,8 @@ void seq_put_decimal_ll(struct seq_file void seq_put_hex_ll(struct seq_file *m, const char *delimiter, unsigned long long v, unsigned int width); +void seq_escape_mem(struct seq_file *m, const char *src, size_t len, + unsigned int flags, const char *esc); void seq_escape(struct seq_file *m, const char *s, const char *esc); void seq_escape_mem_ascii(struct seq_file *m, const char *src, size_t isz); From patchwork Thu Jul 1 01:55:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B60E2C11F68 for ; Thu, 1 Jul 2021 01:55:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6B0086141A for ; Thu, 1 Jul 2021 01:55:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6B0086141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E29848D0269; Wed, 30 Jun 2021 21:55:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E007D8D024F; Wed, 30 Jun 2021 21:55:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEF858D0269; Wed, 30 Jun 2021 21:55:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A50C08D024F for ; Wed, 30 Jun 2021 21:55:39 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 7E4541DE67 for ; Thu, 1 Jul 2021 01:55:39 +0000 (UTC) X-FDA: 78312352398.25.DB3ACD5 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 3E1905000097 for ; Thu, 1 Jul 2021 01:55:39 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3BC5661469; Thu, 1 Jul 2021 01:55:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104538; bh=P3jDuOZjcaumvHKBj7ainPYIEpwGa34rc1WEuesTvd0=; h=Date:From:To:Subject:In-Reply-To:From; b=KwBdUcZ97uPJHQAtawrJJ237/JkBLhQcavIFzUPBwncUxQfmIR1S5/iciQJkJzD+o NKTooYqzwNswa6RQwXJ9YE4TPcEF6GTZ/KA3r+LQt2hFtroCqVSpxMkm4++8pyCrSD iCTaxc4LitGpAW+bqqTekZlLQFlTze6YB759FHiE= Date: Wed, 30 Jun 2021 18:55:37 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 159/192] seq_file: add seq_escape_str() as replica of string_escape_str() Message-ID: <20210701015537.nylTt3S3W%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3E1905000097 X-Stat-Signature: xhk9o5gm5nqpfct55goyw59rikmhjguo Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=KwBdUcZ9; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104539-291568 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: seq_file: add seq_escape_str() as replica of string_escape_str() In some cases we want to escape characters from NULL-terminated strings. Add seq_escape_str() as replica of string_escape_str() for that. Link: https://lkml.kernel.org/r/20210504180819.73127-13-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- include/linux/seq_file.h | 7 +++++++ 1 file changed, 7 insertions(+) --- a/include/linux/seq_file.h~seq_file-add-seq_escape_str-as-replica-of-string_escape_str +++ a/include/linux/seq_file.h @@ -128,6 +128,13 @@ void seq_put_hex_ll(struct seq_file *m, void seq_escape_mem(struct seq_file *m, const char *src, size_t len, unsigned int flags, const char *esc); + +static inline void seq_escape_str(struct seq_file *m, const char *src, + unsigned int flags, const char *esc) +{ + seq_escape_mem(m, src, strlen(src), flags, esc); +} + void seq_escape(struct seq_file *m, const char *s, const char *esc); void seq_escape_mem_ascii(struct seq_file *m, const char *src, size_t isz); From patchwork Thu Jul 1 01:55:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9282C11F66 for ; Thu, 1 Jul 2021 01:55:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 710B06141A for ; Thu, 1 Jul 2021 01:55:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 710B06141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D190D8D026B; Wed, 30 Jun 2021 21:55:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C79C98D024F; Wed, 30 Jun 2021 21:55:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1AB48D026B; Wed, 30 Jun 2021 21:55:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id 85F1E8D024F for ; Wed, 30 Jun 2021 21:55:42 -0400 (EDT) Received: from smtpin39.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 58DE021960 for ; Thu, 1 Jul 2021 01:55:42 +0000 (UTC) X-FDA: 78312352524.39.ADE654C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 149307001773 for ; Thu, 1 Jul 2021 01:55:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 29C7B61420; Thu, 1 Jul 2021 01:55:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104541; bh=X3E4436PzFkfXdyT61o/JSkrqgwMOa6MEDcYQWSLNDY=; h=Date:From:To:Subject:In-Reply-To:From; b=jrzVdYCaTaK4RodLv+kJLhlbvTvQ4/tyWEKWX2FqZHsWEMuqnr1v7a4BwH5RZhJqK /FByUPCf/Fvok2E48AymaB6vykVs1H2foI7sElk0ocPHhR3wzz25R/kkIctH1xoOnj B8PGp/e4uZVbrZKU4MJWIjl/kEq+tyd4SasVxer8= Date: Wed, 30 Jun 2021 18:55:40 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 160/192] seq_file: convert seq_escape() to use seq_escape_str() Message-ID: <20210701015540.8J5zoxl9V%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=jrzVdYCa; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 149307001773 X-Stat-Signature: jc7jbit8swscjubhg3z9xj831h8bhaj5 X-HE-Tag: 1625104541-112837 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: seq_file: convert seq_escape() to use seq_escape_str() Convert seq_escape() to use seq_escape_str() rather than open coding it. Note, for now we leave it as an exported symbol due to some old code that can't tolerate ctype.h being (indirectly) included. Link: https://lkml.kernel.org/r/20210504180819.73127-14-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- fs/seq_file.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) --- a/fs/seq_file.c~seq_file-convert-seq_escape-to-use-seq_escape_str +++ a/fs/seq_file.c @@ -392,12 +392,7 @@ EXPORT_SYMBOL(seq_escape_mem); */ void seq_escape(struct seq_file *m, const char *s, const char *esc) { - char *buf; - size_t size = seq_get_buf(m, &buf); - int ret; - - ret = string_escape_str(s, buf, size, ESCAPE_OCTAL, esc); - seq_commit(m, ret < size ? ret : -1); + seq_escape_str(m, s, ESCAPE_OCTAL, esc); } EXPORT_SYMBOL(seq_escape); From patchwork Thu Jul 1 01:55:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 839CDC11F68 for ; Thu, 1 Jul 2021 01:55:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4E3B26141A for ; Thu, 1 Jul 2021 01:55:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4E3B26141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CCA0E8D026C; Wed, 30 Jun 2021 21:55:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA1158D024F; Wed, 30 Jun 2021 21:55:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B92158D026C; Wed, 30 Jun 2021 21:55:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0149.hostedemail.com [216.40.44.149]) by kanga.kvack.org (Postfix) with ESMTP id 91FA28D024F for ; Wed, 30 Jun 2021 21:55:45 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 61F0B1813F3F1 for ; Thu, 1 Jul 2021 01:55:45 +0000 (UTC) X-FDA: 78312352650.02.E0602EB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 24371D000099 for ; Thu, 1 Jul 2021 01:55:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1A27761469; Thu, 1 Jul 2021 01:55:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104544; bh=xhHpZARSknPhHNKbcTGGZInnwjH486Rov5ZB19Lj9CA=; h=Date:From:To:Subject:In-Reply-To:From; b=kxp2NBnjA9sUR515OpUW8nJ7Exxq2llykwn6PuJRebIHzQuh5xoTs9/t2QNBnBwdL pdWKo3KJYA24itdGpWerCkPr4X+gJwFOsspiv5pucIkon/ybfT0UYG10NeXzYs8r7/ MQmuLnv2Tu/5q89xlZQMJh/y8OVK5AMesCZwVTGo= Date: Wed, 30 Jun 2021 18:55:43 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 161/192] nfsd: avoid non-flexible API in seq_quote_mem() Message-ID: <20210701015543.qMe2Bv2DX%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 24371D000099 X-Stat-Signature: uht3wk4tx6uynpsjx977477mn4x63jh9 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=kxp2NBnj; dmarc=none; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104544-914625 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: nfsd: avoid non-flexible API in seq_quote_mem() The seq_escape_mem_ascii() is completely non-flexible and shouldn't be used. Replace it with properly called seq_escape_mem(). Link: https://lkml.kernel.org/r/20210504180819.73127-15-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- fs/nfsd/nfs4state.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/fs/nfsd/nfs4state.c~nfsd-avoid-non-flexible-api-in-seq_quote_mem +++ a/fs/nfsd/nfs4state.c @@ -2351,7 +2351,7 @@ static struct nfs4_client *get_nfsdfs_cl static void seq_quote_mem(struct seq_file *m, char *data, int len) { seq_printf(m, "\""); - seq_escape_mem_ascii(m, data, len); + seq_escape_mem(m, data, len, ESCAPE_HEX | ESCAPE_NAP | ESCAPE_APPEND, "\"\\"); seq_printf(m, "\""); } From patchwork Thu Jul 1 01:55:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CABEC11F68 for ; Thu, 1 Jul 2021 01:55:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4239F61420 for ; Thu, 1 Jul 2021 01:55:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4239F61420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C06788D026D; Wed, 30 Jun 2021 21:55:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BB6528D024F; Wed, 30 Jun 2021 21:55:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7F128D026D; Wed, 30 Jun 2021 21:55:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0173.hostedemail.com [216.40.44.173]) by kanga.kvack.org (Postfix) with ESMTP id 7B4388D024F for ; Wed, 30 Jun 2021 21:55:48 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 529A2802FDF8 for ; Thu, 1 Jul 2021 01:55:48 +0000 (UTC) X-FDA: 78312352776.36.DB7E815 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id F2179D000099 for ; Thu, 1 Jul 2021 01:55:47 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 11E3A61418; Thu, 1 Jul 2021 01:55:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104547; bh=PwC8MUotYe9y0h9hrqnU8rpMllTzEKvYhoIsN9UEDtc=; h=Date:From:To:Subject:In-Reply-To:From; b=BZ0dnP3JAV7v/jImvJpSqo/Fxt+0TFbx3SWLK3trDzJ+I4BGjJ9yKGKwYfkBWVb4e GrPzO308/0BdMmqshbFm93HrGJScHRFHOTgg0gFLlVNGom5iRagijYMvyEawuyATT7 4yT9SzSlKNlebeSiDfwE14nUtb6tpFzmpjIGYkos= Date: Wed, 30 Jun 2021 18:55:46 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, bfields@fieldses.org, chuck.lever@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 162/192] seq_file: drop unused *_escape_mem_ascii() Message-ID: <20210701015546.-G-crt14P%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=BZ0dnP3J; spf=pass (imf20.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: qemrtjasxaa98ryn6rktkherfjypsk5m X-Rspamd-Queue-Id: F2179D000099 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104547-387237 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: seq_file: drop unused *_escape_mem_ascii() There are no more users of the seq_escape_mem_ascii() followed by string_escape_mem_ascii(). Remove them for good. Link: https://lkml.kernel.org/r/20210504180819.73127-16-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Cc: Alexander Viro Cc: Chuck Lever Cc: "J. Bruce Fields" Signed-off-by: Andrew Morton --- fs/seq_file.c | 11 ----------- include/linux/seq_file.h | 1 - include/linux/string_helpers.h | 3 --- lib/string_helpers.c | 19 ------------------- 4 files changed, 34 deletions(-) --- a/fs/seq_file.c~seq_file-drop-unused-_escape_mem_ascii +++ a/fs/seq_file.c @@ -396,17 +396,6 @@ void seq_escape(struct seq_file *m, cons } EXPORT_SYMBOL(seq_escape); -void seq_escape_mem_ascii(struct seq_file *m, const char *src, size_t isz) -{ - char *buf; - size_t size = seq_get_buf(m, &buf); - int ret; - - ret = string_escape_mem_ascii(src, isz, buf, size); - seq_commit(m, ret < size ? ret : -1); -} -EXPORT_SYMBOL(seq_escape_mem_ascii); - void seq_vprintf(struct seq_file *m, const char *f, va_list args) { int len; --- a/include/linux/seq_file.h~seq_file-drop-unused-_escape_mem_ascii +++ a/include/linux/seq_file.h @@ -136,7 +136,6 @@ static inline void seq_escape_str(struct } void seq_escape(struct seq_file *m, const char *s, const char *esc); -void seq_escape_mem_ascii(struct seq_file *m, const char *src, size_t isz); void seq_hex_dump(struct seq_file *m, const char *prefix_str, int prefix_type, int rowsize, int groupsize, const void *buf, size_t len, --- a/include/linux/string_helpers.h~seq_file-drop-unused-_escape_mem_ascii +++ a/include/linux/string_helpers.h @@ -63,9 +63,6 @@ static inline int string_unescape_any_in int string_escape_mem(const char *src, size_t isz, char *dst, size_t osz, unsigned int flags, const char *only); -int string_escape_mem_ascii(const char *src, size_t isz, char *dst, - size_t osz); - static inline int string_escape_mem_any_np(const char *src, size_t isz, char *dst, size_t osz, const char *only) { --- a/lib/string_helpers.c~seq_file-drop-unused-_escape_mem_ascii +++ a/lib/string_helpers.c @@ -582,25 +582,6 @@ int string_escape_mem(const char *src, s } EXPORT_SYMBOL(string_escape_mem); -int string_escape_mem_ascii(const char *src, size_t isz, char *dst, - size_t osz) -{ - char *p = dst; - char *end = p + osz; - - while (isz--) { - unsigned char c = *src++; - - if (!isprint(c) || !isascii(c) || c == '"' || c == '\\') - escape_hex(c, &p, end); - else - escape_passthrough(c, &p, end); - } - - return p - dst; -} -EXPORT_SYMBOL(string_escape_mem_ascii); - /* * Return an allocated string that has been escaped of special characters * and double quotes, making it safe to log in quotes. From patchwork Thu Jul 1 01:55:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9898CC11F68 for ; Thu, 1 Jul 2021 01:55:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4CDE861468 for ; Thu, 1 Jul 2021 01:55:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4CDE861468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C55158D026E; Wed, 30 Jun 2021 21:55:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C2B9E8D024F; Wed, 30 Jun 2021 21:55:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF5868D026E; Wed, 30 Jun 2021 21:55:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0074.hostedemail.com [216.40.44.74]) by kanga.kvack.org (Postfix) with ESMTP id 8678E8D024F for ; Wed, 30 Jun 2021 21:55:51 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5B5CE181AF5C1 for ; Thu, 1 Jul 2021 01:55:51 +0000 (UTC) X-FDA: 78312352902.35.D7B42D2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 15278D000156 for ; Thu, 1 Jul 2021 01:55:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1391C6141A; Thu, 1 Jul 2021 01:55:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104550; bh=OoDZR3KoSDH45IG/Fn+nvWFNLGFWP29DBNy4VtpxdJw=; h=Date:From:To:Subject:In-Reply-To:From; b=yzMHwqaKFEsSZM4gULuviuXODu+gfz9w8M9oMJLkpRKjULGYjnek/9oY0DUyV6AtB ke5uVAtHDng+wMdEM4RSdS/a8D+PSEygwX8kdC1K7eKdCe+b8oNY++WSnMqM51/DyC DD4d01OcPNnMoyr9Lh/lzCnhhog19fxRCNVL1j8w= Date: Wed, 30 Jun 2021 18:55:49 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, dlatypov@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, oskar@scara.com, torvalds@linux-foundation.org, tpiepho@gmail.com, yguoaz@gmail.com Subject: [patch 163/192] lib/math/rational.c: fix divide by zero Message-ID: <20210701015549.0882YHh8B%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=yzMHwqaK; dmarc=none; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: 85qutram6xhnithducu5a5zh3nfxto9o X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 15278D000156 X-HE-Tag: 1625104550-208484 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Trent Piepho Subject: lib/math/rational.c: fix divide by zero If the input is out of the range of the allowed values, either larger than the largest value or closer to zero than the smallest non-zero allowed value, then a division by zero would occur. In the case of input too large, the division by zero will occur on the first iteration. The best result (largest allowed value) will be found by always choosing the semi-convergent and excluding the denominator based limit when finding it. In the case of the input too small, the division by zero will occur on the second iteration. The numerator based semi-convergent should not be calculated to avoid the division by zero. But the semi-convergent vs previous convergent test is still needed, which effectively chooses between 0 (the previous convergent) vs the smallest allowed fraction (best semi-convergent) as the result. Link: https://lkml.kernel.org/r/20210525144250.214670-1-tpiepho@gmail.com Fixes: 323dd2c3ed0 ("lib/math/rational.c: fix possible incorrect result from rational fractions helper") Signed-off-by: Trent Piepho Reported-by: Yiyuan Guo Reviewed-by: Andy Shevchenko Cc: Oskar Schirmer Cc: Daniel Latypov Signed-off-by: Andrew Morton --- lib/math/rational.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) --- a/lib/math/rational.c~lib-math-rationalc-fix-divide-by-zero +++ a/lib/math/rational.c @@ -12,6 +12,7 @@ #include #include #include +#include /* * calculate best rational approximation for a given fraction @@ -78,13 +79,18 @@ void rational_best_approximation( * found below as 't'. */ if ((n2 > max_numerator) || (d2 > max_denominator)) { - unsigned long t = min((max_numerator - n0) / n1, - (max_denominator - d0) / d1); + unsigned long t = ULONG_MAX; - /* This tests if the semi-convergent is closer - * than the previous convergent. + if (d1) + t = (max_denominator - d0) / d1; + if (n1) + t = min(t, (max_numerator - n0) / n1); + + /* This tests if the semi-convergent is closer than the previous + * convergent. If d1 is zero there is no previous convergent as this + * is the 1st iteration, so always choose the semi-convergent. */ - if (2u * t > a || (2u * t == a && d0 * dp > d1 * d)) { + if (!d1 || 2u * t > a || (2u * t == a && d0 * dp > d1 * d)) { n1 = n0 + t * n1; d1 = d0 + t * d1; } From patchwork Thu Jul 1 01:55:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A051FC11F69 for ; Thu, 1 Jul 2021 01:55:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5611C61469 for ; Thu, 1 Jul 2021 01:55:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5611C61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D3EEB8D026F; Wed, 30 Jun 2021 21:55:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D16278D024F; Wed, 30 Jun 2021 21:55:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDDD68D026F; Wed, 30 Jun 2021 21:55:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0063.hostedemail.com [216.40.44.63]) by kanga.kvack.org (Postfix) with ESMTP id 952C18D024F for ; Wed, 30 Jun 2021 21:55:54 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 672D481621D4 for ; Thu, 1 Jul 2021 01:55:54 +0000 (UTC) X-FDA: 78312353028.14.010DFAA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 15263B00009A for ; Thu, 1 Jul 2021 01:55:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0ECDD61420; Thu, 1 Jul 2021 01:55:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104553; bh=EXoz6J7Eyrsot+KAX1cnNMNIcpXWBdH51aKfbMCnwhE=; h=Date:From:To:Subject:In-Reply-To:From; b=pSd7bIJNBY9K+5mSV1f1i+L6mQea+h64NaJesE4gIW8gdLqjSo6PVBKXAtvYmI6zO u8QI3ynh5uF/e9PJPc9rIlWOuQ6AZt5illuyvYD7MvltPOi1Pqvvkdw5Dmaxu97Esj WhDpW+0ukC7JOHSHKowpg6PrHaavaHeeWRjUnGxU= Date: Wed, 30 Jun 2021 18:55:52 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, colin.king@canonical.com, dlatypov@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, oskar@scara.com, torvalds@linux-foundation.org, tpiepho@gmail.com, yguoaz@gmail.com Subject: [patch 164/192] lib/math/rational: add Kunit test cases Message-ID: <20210701015552.rpE3gozWu%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 15263B00009A Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=pSd7bIJN; spf=pass (imf24.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: 7uhppgz7c9d8s53t51phz34i7pnd63zk X-HE-Tag: 1625104553-396602 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Trent Piepho Subject: lib/math/rational: add Kunit test cases Adds a number of test cases that cover a range of possible code paths. [akpm@linux-foundation.org: remove non-ascii characters, fix whitespace] [colin.king@canonical.com: fix spelling mistake "demominator" -> "denominator"] Link: https://lkml.kernel.org/r/20210526085049.6393-1-colin.king@canonical.com Link: https://lkml.kernel.org/r/20210525144250.214670-2-tpiepho@gmail.com Signed-off-by: Trent Piepho Signed-off-by: Colin Ian King Reviewed-by: Andy Shevchenko Cc: Daniel Latypov Cc: Oskar Schirmer Cc: Yiyuan Guo Signed-off-by: Andrew Morton --- lib/Kconfig.debug | 12 +++++++ lib/math/Makefile | 1 lib/math/rational-test.c | 56 +++++++++++++++++++++++++++++++++++++ 3 files changed, 69 insertions(+) --- a/lib/Kconfig.debug~lib-math-rational-add-kunit-test-cases +++ a/lib/Kconfig.debug @@ -2444,6 +2444,18 @@ config SLUB_KUNIT_TEST If unsure, say N. +config RATIONAL_KUNIT_TEST + tristate "KUnit test for rational.c" if !KUNIT_ALL_TESTS + depends on KUNIT + select RATIONAL + default KUNIT_ALL_TESTS + help + This builds the rational math unit test. + For more information on KUnit and unit tests in general please refer + to the KUnit documentation in Documentation/dev-tools/kunit/. + + If unsure, say N. + config TEST_UDELAY tristate "udelay test driver" help --- a/lib/math/Makefile~lib-math-rational-add-kunit-test-cases +++ a/lib/math/Makefile @@ -6,3 +6,4 @@ obj-$(CONFIG_PRIME_NUMBERS) += prime_num obj-$(CONFIG_RATIONAL) += rational.o obj-$(CONFIG_TEST_DIV64) += test_div64.o +obj-$(CONFIG_RATIONAL_KUNIT_TEST) += rational-test.o --- /dev/null +++ a/lib/math/rational-test.c @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include + +#include + +struct rational_test_param { + unsigned long num, den; + unsigned long max_num, max_den; + unsigned long exp_num, exp_den; + + const char *name; +}; + +static const struct rational_test_param test_parameters[] = { + { 1230, 10, 100, 20, 100, 1, "Exceeds bounds, semi-convergent term > 1/2 last term" }, + { 34567,100, 120, 20, 120, 1, "Exceeds bounds, semi-convergent term < 1/2 last term" }, + { 1, 30, 100, 10, 0, 1, "Closest to zero" }, + { 1, 19, 100, 10, 1, 10, "Closest to smallest non-zero" }, + { 27,32, 16, 16, 11, 13, "Use convergent" }, + { 1155, 7735, 255, 255, 33, 221, "Exact answer" }, + { 87, 32, 70, 32, 68, 25, "Semiconvergent, numerator limit" }, + { 14533, 4626, 15000, 2400, 7433, 2366, "Semiconvergent, denominator limit" }, +}; + +static void get_desc(const struct rational_test_param *param, char *desc) +{ + strscpy(desc, param->name, KUNIT_PARAM_DESC_SIZE); +} + +/* Creates function rational_gen_params */ +KUNIT_ARRAY_PARAM(rational, test_parameters, get_desc); + +static void rational_test(struct kunit *test) +{ + const struct rational_test_param *param = (const struct rational_test_param *)test->param_value; + unsigned long n = 0, d = 0; + + rational_best_approximation(param->num, param->den, param->max_num, param->max_den, &n, &d); + KUNIT_EXPECT_EQ(test, n, param->exp_num); + KUNIT_EXPECT_EQ(test, d, param->exp_den); +} + +static struct kunit_case rational_test_cases[] = { + KUNIT_CASE_PARAM(rational_test, rational_gen_params), + {} +}; + +static struct kunit_suite rational_test_suite = { + .name = "rational", + .test_cases = rational_test_cases, +}; + +kunit_test_suites(&rational_test_suite); + +MODULE_LICENSE("GPL v2"); From patchwork Thu Jul 1 01:55:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8DB9C11F69 for ; Thu, 1 Jul 2021 01:55:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5B66661469 for ; Thu, 1 Jul 2021 01:55:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5B66661469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CF16D8D0270; Wed, 30 Jun 2021 21:55:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC7658D024F; Wed, 30 Jun 2021 21:55:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B8F8A8D0270; Wed, 30 Jun 2021 21:55:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id 8CBD18D024F for ; Wed, 30 Jun 2021 21:55:57 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 65A3D24386 for ; Thu, 1 Jul 2021 01:55:57 +0000 (UTC) X-FDA: 78312353154.36.7619195 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf17.hostedemail.com (Postfix) with ESMTP id 0DDBEF000129 for ; Thu, 1 Jul 2021 01:55:56 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 148C661468; Thu, 1 Jul 2021 01:55:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104556; bh=kjdDGEEzlF1joX5pjEH7jEu9uEg2WAscjy20Zs6LQe8=; h=Date:From:To:Subject:In-Reply-To:From; b=iZgg6MY7AS1SIb85JTEMpYYGE5KrRRSCOWNPQayYqlkyd05xnQ1DtjPrYhiCmIy4S gPt0T3ZoJraQQSYY9cUNXHdViWieH+vbfHUcn6ILcMEh1CLHiEfpXczVfUQwGkbPIt QMK5xCInWb/YFO7I0c6qmvU6LKlsmCnyMtk7QcQo= Date: Wed, 30 Jun 2021 18:55:55 -0700 From: Andrew Morton To: akpm@linux-foundation.org, jkosina@suse.cz, linux-mm@kvack.org, mm-commits@vger.kernel.org, thunder.leizhen@huawei.com, torvalds@linux-foundation.org, tsbogend@alpha.franken.de Subject: [patch 165/192] lib/decompressors: fix spelling mistakes Message-ID: <20210701015555.EglHOdWHp%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0DDBEF000129 X-Stat-Signature: 7bk9cmdz7u1jmhdbhm1rt9ehzxfjcemq Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=iZgg6MY7; dmarc=none; spf=pass (imf17.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104556-351362 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhen Lei Subject: lib/decompressors: fix spelling mistakes Fix some spelling mistakes in comments: sentinal ==> sentinel compresed ==> compressed dependeny ==> dependency immediatelly ==> immediately dervied ==> derived splitted ==> split nore ==> not independed ==> independent asumed ==> assumed Link: https://lkml.kernel.org/r/20210604085656.12257-1-thunder.leizhen@huawei.com Signed-off-by: Zhen Lei Cc: Jiri Kosina Cc: Thomas Bogendoerfer Signed-off-by: Andrew Morton --- lib/decompress_bunzip2.c | 4 ++-- lib/decompress_unxz.c | 2 +- lib/decompress_unzstd.c | 4 ++-- lib/xz/xz_dec_bcj.c | 2 +- lib/xz/xz_dec_lzma2.c | 8 ++++---- lib/zlib_inflate/inffast.c | 2 +- lib/zstd/huf.h | 2 +- 7 files changed, 12 insertions(+), 12 deletions(-) --- a/lib/decompress_bunzip2.c~lib-decompressors-fix-spelling-mistakes +++ a/lib/decompress_bunzip2.c @@ -80,7 +80,7 @@ /* This is what we know about each Huffman coding group */ struct group_data { - /* We have an extra slot at the end of limit[] for a sentinal value. */ + /* We have an extra slot at the end of limit[] for a sentinel value. */ int limit[MAX_HUFCODE_BITS+1]; int base[MAX_HUFCODE_BITS]; int permute[MAX_SYMBOLS]; @@ -337,7 +337,7 @@ static int INIT get_next_block(struct bu pp <<= 1; base[i+1] = pp-(t += temp[i]); } - limit[maxLen+1] = INT_MAX; /* Sentinal value for + limit[maxLen+1] = INT_MAX; /* Sentinel value for * reading next sym. */ limit[maxLen] = pp+temp[maxLen]-1; base[minLen] = 0; --- a/lib/decompress_unxz.c~lib-decompressors-fix-spelling-mistakes +++ a/lib/decompress_unxz.c @@ -23,7 +23,7 @@ * uncompressible. Thus, we must look for worst-case expansion when the * compressor is encoding uncompressible data. * - * The structure of the .xz file in case of a compresed kernel is as follows. + * The structure of the .xz file in case of a compressed kernel is as follows. * Sizes (as bytes) of the fields are in parenthesis. * * Stream Header (12) --- a/lib/decompress_unzstd.c~lib-decompressors-fix-spelling-mistakes +++ a/lib/decompress_unzstd.c @@ -16,7 +16,7 @@ * uncompressible. Thus, we must look for worst-case expansion when the * compressor is encoding uncompressible data. * - * The structure of the .zst file in case of a compresed kernel is as follows. + * The structure of the .zst file in case of a compressed kernel is as follows. * Maximum sizes (as bytes) of the fields are in parenthesis. * * Frame Header: (18) @@ -56,7 +56,7 @@ /* * Preboot environments #include "path/to/decompress_unzstd.c". * All of the source files we depend on must be #included. - * zstd's only source dependeny is xxhash, which has no source + * zstd's only source dependency is xxhash, which has no source * dependencies. * * When UNZSTD_PREBOOT is defined we declare __decompress(), which is --- a/lib/xz/xz_dec_bcj.c~lib-decompressors-fix-spelling-mistakes +++ a/lib/xz/xz_dec_bcj.c @@ -422,7 +422,7 @@ XZ_EXTERN enum xz_ret xz_dec_bcj_run(str /* * Flush pending already filtered data to the output buffer. Return - * immediatelly if we couldn't flush everything, or if the next + * immediately if we couldn't flush everything, or if the next * filter in the chain had already returned XZ_STREAM_END. */ if (s->temp.filtered > 0) { --- a/lib/xz/xz_dec_lzma2.c~lib-decompressors-fix-spelling-mistakes +++ a/lib/xz/xz_dec_lzma2.c @@ -147,8 +147,8 @@ struct lzma_dec { /* * LZMA properties or related bit masks (number of literal - * context bits, a mask dervied from the number of literal - * position bits, and a mask dervied from the number + * context bits, a mask derived from the number of literal + * position bits, and a mask derived from the number * position bits) */ uint32_t lc; @@ -484,7 +484,7 @@ static __always_inline void rc_normalize } /* - * Decode one bit. In some versions, this function has been splitted in three + * Decode one bit. In some versions, this function has been split in three * functions so that the compiler is supposed to be able to more easily avoid * an extra branch. In this particular version of the LZMA decoder, this * doesn't seem to be a good idea (tested with GCC 3.3.6, 3.4.6, and 4.3.3 @@ -761,7 +761,7 @@ static bool lzma_main(struct xz_dec_lzma } /* - * Reset the LZMA decoder and range decoder state. Dictionary is nore reset + * Reset the LZMA decoder and range decoder state. Dictionary is not reset * here, because LZMA state may be reset without resetting the dictionary. */ static void lzma_reset(struct xz_dec_lzma2 *s) --- a/lib/zlib_inflate/inffast.c~lib-decompressors-fix-spelling-mistakes +++ a/lib/zlib_inflate/inffast.c @@ -15,7 +15,7 @@ union uu { unsigned char b[2]; }; -/* Endian independed version */ +/* Endian independent version */ static inline unsigned short get_unaligned16(const unsigned short *p) { --- a/lib/zstd/huf.h~lib-decompressors-fix-spelling-mistakes +++ a/lib/zstd/huf.h @@ -134,7 +134,7 @@ typedef enum { HUF_repeat_none, /**< Cannot use the previous table */ HUF_repeat_check, /**< Can use the previous table but it must be checked. Note : The previous table must have been constructed by HUF_compress{1, 4}X_repeat */ - HUF_repeat_valid /**< Can use the previous table and it is asumed to be valid */ + HUF_repeat_valid /**< Can use the previous table and it is assumed to be valid */ } HUF_repeat; /** HUF_compress4X_repeat() : * Same as HUF_compress4X_wksp(), but considers using hufTable if *repeat != HUF_repeat_none. From patchwork Thu Jul 1 01:55:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6CB8DC11F69 for ; Thu, 1 Jul 2021 01:56:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 226F361420 for ; Thu, 1 Jul 2021 01:56:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 226F361420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A0A478D0271; Wed, 30 Jun 2021 21:56:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BA2C8D024F; Wed, 30 Jun 2021 21:56:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85BE58D0271; Wed, 30 Jun 2021 21:56:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id 5CB878D024F for ; Wed, 30 Jun 2021 21:56:00 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 2E62E21242 for ; Thu, 1 Jul 2021 01:56:00 +0000 (UTC) X-FDA: 78312353280.03.3F08943 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id CDD39E0016BE for ; Thu, 1 Jul 2021 01:55:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E567E6141A; Thu, 1 Jul 2021 01:55:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104559; bh=Mp9auJLxTLvqReqVizpP+BQ+5Se3sAu4cR3+kbDEe4g=; h=Date:From:To:Subject:In-Reply-To:From; b=Ty6bbMv2TXWVTdmUn7BxiEdV71Nm/i9xEhXzi0EC7Qoq9iq/wgX4b2Dx3ZjObeK7R 0KYyvYuVBPqSNE/5bLu8r0QDCpANwLQrDRLBZughiUb8Hey0qJOG42BR3ygBVe7Asn c2RkMmajb9a9qCc3IqkuYhkobxECuLZ30IatUlAg= Date: Wed, 30 Jun 2021 18:55:58 -0700 From: Andrew Morton To: akpm@linux-foundation.org, herbert@gondor.apana.org.au, linux-mm@kvack.org, mm-commits@vger.kernel.org, thunder.leizhen@huawei.com, torvalds@linux-foundation.org Subject: [patch 166/192] lib/mpi: fix spelling mistakes Message-ID: <20210701015558.63PZ7e5h-%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Ty6bbMv2; dmarc=none; spf=pass (imf30.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: 5typq7nba795iphddexi5okjkxk91zcw X-Rspamd-Queue-Id: CDD39E0016BE X-HE-Tag: 1625104559-101307 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhen Lei Subject: lib/mpi: fix spelling mistakes Fix some spelling mistakes in comments: flaged ==> flagged bufer ==> buffer multipler ==> multiplier MULTIPLER ==> MULTIPLIER leaset ==> least chnage ==> change Link: https://lkml.kernel.org/r/20210604074401.12198-1-thunder.leizhen@huawei.com Signed-off-by: Zhen Lei Cc: Herbert Xu Signed-off-by: Andrew Morton --- include/linux/mpi.h | 4 ++-- lib/mpi/longlong.h | 4 ++-- lib/mpi/mpicoder.c | 6 +++--- lib/mpi/mpiutil.c | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) --- a/include/linux/mpi.h~lib-mpi-fix-spelling-mistakes +++ a/include/linux/mpi.h @@ -200,7 +200,7 @@ struct mpi_ec_ctx { unsigned int nbits; /* Number of bits. */ /* Domain parameters. Note that they may not all be set and if set - * the MPIs may be flaged as constant. + * the MPIs may be flagged as constant. */ MPI p; /* Prime specifying the field GF(p). */ MPI a; /* First coefficient of the Weierstrass equation. */ @@ -267,7 +267,7 @@ int mpi_ec_curve_point(MPI_POINT point, /** * mpi_get_size() - returns max size required to store the number * - * @a: A multi precision integer for which we want to allocate a bufer + * @a: A multi precision integer for which we want to allocate a buffer * * Return: size required to store the number */ --- a/lib/mpi/longlong.h~lib-mpi-fix-spelling-mistakes +++ a/lib/mpi/longlong.h @@ -48,8 +48,8 @@ /* Define auxiliary asm macros. * - * 1) umul_ppmm(high_prod, low_prod, multipler, multiplicand) multiplies two - * UWtype integers MULTIPLER and MULTIPLICAND, and generates a two UWtype + * 1) umul_ppmm(high_prod, low_prod, multiplier, multiplicand) multiplies two + * UWtype integers MULTIPLIER and MULTIPLICAND, and generates a two UWtype * word product in HIGH_PROD and LOW_PROD. * * 2) __umulsidi3(a,b) multiplies two UWtype integers A and B, and returns a --- a/lib/mpi/mpicoder.c~lib-mpi-fix-spelling-mistakes +++ a/lib/mpi/mpicoder.c @@ -234,11 +234,11 @@ static int count_lzeros(MPI a) } /** - * mpi_read_buffer() - read MPI to a bufer provided by user (msb first) + * mpi_read_buffer() - read MPI to a buffer provided by user (msb first) * * @a: a multi precision integer - * @buf: bufer to which the output will be written to. Needs to be at - * leaset mpi_get_size(a) long. + * @buf: buffer to which the output will be written to. Needs to be at + * least mpi_get_size(a) long. * @buf_len: size of the buf. * @nbytes: receives the actual length of the data written on success and * the data to-be-written on -EOVERFLOW in case buf_len was too --- a/lib/mpi/mpiutil.c~lib-mpi-fix-spelling-mistakes +++ a/lib/mpi/mpiutil.c @@ -80,7 +80,7 @@ EXPORT_SYMBOL_GPL(mpi_const); /**************** * Note: It was a bad idea to use the number of limbs to allocate * because on a alpha the limbs are large but we normally need - * integers of n bits - So we should chnage this to bits (or bytes). + * integers of n bits - So we should change this to bits (or bytes). * * But mpi_alloc is used in a lot of places :-) */ From patchwork Thu Jul 1 01:56:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4007CC11F66 for ; Thu, 1 Jul 2021 01:56:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 050C461420 for ; Thu, 1 Jul 2021 01:56:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 050C461420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7E7718D0272; Wed, 30 Jun 2021 21:56:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7BFB38D024F; Wed, 30 Jun 2021 21:56:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AD838D0272; Wed, 30 Jun 2021 21:56:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0132.hostedemail.com [216.40.44.132]) by kanga.kvack.org (Postfix) with ESMTP id 405288D024F for ; Wed, 30 Jun 2021 21:56:03 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 12036250D9 for ; Thu, 1 Jul 2021 01:56:03 +0000 (UTC) X-FDA: 78312353406.30.345C12E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id CAE319000328 for ; Thu, 1 Jul 2021 01:56:02 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C81BB61418; Thu, 1 Jul 2021 01:56:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104562; bh=qHGXSWtOk3VLrb2L/UpN8WPAMGnTXo4Y8XT5q4kDFYU=; h=Date:From:To:Subject:In-Reply-To:From; b=EQf4cXfm/QT+FypRh+rCXDv5s+/sEMGyb8QFmUQ3hnwCi/YRQ6DucZicXvtalth2G aFsXwj/ivcX4FCI9TN1btswIyWSqfWG61WklkWz0eDQ8zElwHgVBlLku+LZxXwYLI2 t/hMUK2hcjeluS/fP04vitUfdJ5OyezpgR8LOGSQ= Date: Wed, 30 Jun 2021 18:56:01 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 167/192] lib: memscan() fixlet Message-ID: <20210701015601.Iv6tEUqqh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=EQf4cXfm; dmarc=none; spf=pass (imf29.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: apbxrdqet97mdskzgc59ymbs59pnsf7e X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: CAE319000328 X-HE-Tag: 1625104562-600497 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexey Dobriyan Subject: lib: memscan() fixlet Generic version doesn't trucate second argument to char. Older brother memchr() does as do s390, sparc and i386 assembly versions. Fortunately, no code passes c >= 256. Link: https://lkml.kernel.org/r/YLv4cCf0t5UPdyK+@localhost.localdomain Signed-off-by: Alexey Dobriyan Signed-off-by: Andrew Morton --- lib/string.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/lib/string.c~lib-memscan-fixlet +++ a/lib/string.c @@ -977,7 +977,7 @@ void *memscan(void *addr, int c, size_t unsigned char *p = addr; while (size) { - if (*p == c) + if (*p == (unsigned char)c) return (void *)p; p++; size--; From patchwork Thu Jul 1 01:56:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B0EBC11F68 for ; Thu, 1 Jul 2021 01:56:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E3A1061420 for ; Thu, 1 Jul 2021 01:56:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E3A1061420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6881F8D0273; Wed, 30 Jun 2021 21:56:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65EE98D024F; Wed, 30 Jun 2021 21:56:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54DEA8D0273; Wed, 30 Jun 2021 21:56:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 2B4338D024F for ; Wed, 30 Jun 2021 21:56:06 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id EBA6882E2339 for ; Thu, 1 Jul 2021 01:56:05 +0000 (UTC) X-FDA: 78312353490.11.7BCF6B7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 769854002087 for ; Thu, 1 Jul 2021 01:56:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A505F61418; Thu, 1 Jul 2021 01:56:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104564; bh=rZZI2bnSbL3/duuUqmmIHxEEwnOkYYkO7iPqhrm/TQc=; h=Date:From:To:Subject:In-Reply-To:From; b=IE1qrrxv7cvAwCczh8zriRfwzdelO3pE4x55J83e0a41OSqESireQfnVQA0c8W7zv al0T8nu3oA/RIO/45f1fkU6Kd9bnhozqKa4O4VL4BslAYvC1Cz5m7OFG7rB87UWtsQ +pKuWb5GeXqfpkivcvN0+Sl95Fj81T98PODuLZtc= Date: Wed, 30 Jun 2021 18:56:04 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 168/192] lib: uninline simple_strtoull() Message-ID: <20210701015604.siv4k3Srj%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=IE1qrrxv; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: zoftpsi81f8z54xrfws46uioju75sj66 X-Rspamd-Queue-Id: 769854002087 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104565-829271 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexey Dobriyan Subject: lib: uninline simple_strtoull() Gcc inlines simple_strtoull() too agressively. Given that all 4 signatures match, everything very efficiently calls or tailcalls into simple_strtoull(): ffffffff81da0240 : ffffffff81da0240: 80 3f 2d cmp BYTE PTR [rdi],0x2d ffffffff81da0243: 74 05 je ffffffff81da024a ffffffff81da0245: e9 76 ff ff ff jmp simple_strtoull ffffffff81da024a: 48 83 c7 01 add rdi,0x1 ffffffff81da024e: e8 6d ff ff ff call simple_strtoull ffffffff81da0253: 48 f7 d8 neg rax ffffffff81da0256: c3 ret Space savings (on F34-ish .config) add/remove: 0/0 grow/shrink: 1/3 up/down: 52/-313 (-261) Function old new delta vsscanf 2167 2219 +52 simple_strtoul 72 2 -70 simple_strtoll 143 23 -120 simple_strtol 143 20 -123 Link: https://lkml.kernel.org/r/YMO2zoOQk2eF34tn@localhost.localdomain Signed-off-by: Alexey Dobriyan Signed-off-by: Andrew Morton --- lib/vsprintf.c | 1 + 1 file changed, 1 insertion(+) --- a/lib/vsprintf.c~lib-uninline-simple_strtoull +++ a/lib/vsprintf.c @@ -61,6 +61,7 @@ * * This function has caveats. Please use kstrtoull instead. */ +noinline unsigned long long simple_strtoull(const char *cp, char **endp, unsigned int base) { unsigned long long result; From patchwork Thu Jul 1 01:56:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9F0AC11F66 for ; Thu, 1 Jul 2021 01:56:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 97D8061420 for ; Thu, 1 Jul 2021 01:56:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 97D8061420 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1F53D8D0274; Wed, 30 Jun 2021 21:56:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 157408D024F; Wed, 30 Jun 2021 21:56:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 044988D0274; Wed, 30 Jun 2021 21:56:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0103.hostedemail.com [216.40.44.103]) by kanga.kvack.org (Postfix) with ESMTP id CFEF98D024F for ; Wed, 30 Jun 2021 21:56:08 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A08C582E2339 for ; Thu, 1 Jul 2021 01:56:08 +0000 (UTC) X-FDA: 78312353616.24.B962935 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 5BA40F0000AD for ; Thu, 1 Jul 2021 01:56:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 752F161418; Thu, 1 Jul 2021 01:56:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104567; bh=DybbZ8rsYQiFbecc+hyKN7dV0yKfOMcJMQZluXePPoQ=; h=Date:From:To:Subject:In-Reply-To:From; b=Osz/UgAPAOTUiAEE7s488pjtshnkmridjd7icR7wKLKF4V22o0FDyMQCWffxKcYM8 f2ax+qn3iL3AAJtRff+tdQeaHEybIFAcrlQ/b+bxB8Z3HI9NplU8MnrseZsBt7H5mT 7j3vZHuegk68Zvj++PtCFu9Jw4frHjOsDPXZ+yZU= Date: Wed, 30 Jun 2021 18:56:07 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mcroce@microsoft.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 169/192] lib/test_string.c: allow module removal Message-ID: <20210701015607.Irez2ivp1%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="Osz/UgAP"; spf=pass (imf11.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5BA40F0000AD X-Stat-Signature: x7qobaxrq1oo8okbwpx8bggzyw78utt7 X-HE-Tag: 1625104568-530459 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Matteo Croce Subject: lib/test_string.c: allow module removal The test_string module can't be removed because it lacks an exit hook. Since there is no reason for it to be permanent, add an empty one to allow module removal. Link: https://lkml.kernel.org/r/20210616234503.28678-1-mcroce@linux.microsoft.com Signed-off-by: Matteo Croce Signed-off-by: Andrew Morton --- lib/test_string.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/lib/test_string.c~lib-test_stringc-allow-mode-removal +++ a/lib/test_string.c @@ -179,6 +179,10 @@ static __init int strnchr_selftest(void) return 0; } +static __exit void string_selftest_remove(void) +{ +} + static __init int string_selftest_init(void) { int test, subtest; @@ -216,4 +220,5 @@ fail: } module_init(string_selftest_init); +module_exit(string_selftest_remove); MODULE_LICENSE("GPL v2"); From patchwork Thu Jul 1 01:56:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4189AC11F68 for ; Thu, 1 Jul 2021 01:56:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DFA3C61469 for ; Thu, 1 Jul 2021 01:56:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFA3C61469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6611D8D0275; Wed, 30 Jun 2021 21:56:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 637E28D024F; Wed, 30 Jun 2021 21:56:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D8A08D0275; Wed, 30 Jun 2021 21:56:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id 17D878D024F for ; Wed, 30 Jun 2021 21:56:12 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id E4EB5180ACF0E for ; Thu, 1 Jul 2021 01:56:11 +0000 (UTC) X-FDA: 78312353742.23.84D9EA9 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 7FFB3700009D for ; Thu, 1 Jul 2021 01:56:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6933C6141A; Thu, 1 Jul 2021 01:56:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104570; bh=EEPC75fzbW9XZ3F52rrT7rZHx3IVd/Tlj00WafeDCVw=; h=Date:From:To:Subject:In-Reply-To:From; b=Tgi0wucdDtNjhYnyyiri3aYYqqVV6YVNbjloZiMNnWlvu5PR47iWcL0Wu5g5nqTxZ 1nIjKQCAgkJUpmmmbnbBif1sddyaV58kTx+ZPIhTofPslOW5a3EgKj8KMiVnF+VUTZ fNGOIViZ/88xvnhQ0a0fQpinL9oLvFZ9uKzLA+Ps= Date: Wed, 30 Jun 2021 18:56:10 -0700 From: Andrew Morton To: akpm@linux-foundation.org, andriy.shevchenko@linux.intel.com, anna.schumaker@netapp.com, bfields@fieldses.org, chuck.lever@oracle.com, Jonathan.Cameron@huawei.com, kerneldev@karsmulder.nl, laniel_francis@privacyrequired.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, torvalds@linux-foundation.org, trond.myklebust@hammerspace.com Subject: [patch 170/192] kernel.h: split out kstrtox() and simple_strtox() to a separate header Message-ID: <20210701015610.LwOPRe5qf%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Tgi0wucd; dmarc=none; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: jrzfzne5sjd9tkipyyjs6cp8rigzd5f4 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7FFB3700009D X-HE-Tag: 1625104571-148770 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andy Shevchenko Subject: kernel.h: split out kstrtox() and simple_strtox() to a separate header kernel.h is being used as a dump for all kinds of stuff for a long time. Here is the attempt to start cleaning it up by splitting out kstrtox() and simple_strtox() helpers. At the same time convert users in header and lib folders to use new header. Though for time being include new header back to kernel.h to avoid twisted indirected includes for existing users. [andy.shevchenko@gmail.com: fix documentation references] Link: https://lkml.kernel.org/r/20210615220003.377901-1-andy.shevchenko@gmail.com Link: https://lkml.kernel.org/r/20210611185815.44103-1-andriy.shevchenko@linux.intel.com Signed-off-by: Andy Shevchenko Acked-by: Jonathan Cameron Cc: Francis Laniel Cc: Randy Dunlap Cc: Kars Mulder Cc: Trond Myklebust Cc: Anna Schumaker Cc: "J. Bruce Fields" Cc: Chuck Lever Signed-off-by: Andrew Morton --- Documentation/core-api/kernel-api.rst | 7 - include/linux/kernel.h | 143 ---------------------- include/linux/kstrtox.h | 155 ++++++++++++++++++++++++ include/linux/string.h | 7 - include/linux/sunrpc/cache.h | 1 lib/kstrtox.c | 5 lib/parser.c | 1 7 files changed, 163 insertions(+), 156 deletions(-) --- a/Documentation/core-api/kernel-api.rst~kernelh-split-out-kstrtox-and-simple_strtox-to-a-separate-header +++ a/Documentation/core-api/kernel-api.rst @@ -24,11 +24,8 @@ String Conversions .. kernel-doc:: lib/vsprintf.c :export: -.. kernel-doc:: include/linux/kernel.h - :functions: kstrtol - -.. kernel-doc:: include/linux/kernel.h - :functions: kstrtoul +.. kernel-doc:: include/linux/kstrtox.h + :functions: kstrtol kstrtoul .. kernel-doc:: lib/kstrtox.c :export: --- a/include/linux/kernel.h~kernelh-split-out-kstrtox-and-simple_strtox-to-a-separate-header +++ a/include/linux/kernel.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -180,148 +181,6 @@ static inline void might_fault(void) { } void do_exit(long error_code) __noreturn; void complete_and_exit(struct completion *, long) __noreturn; -/* Internal, do not use. */ -int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res); -int __must_check _kstrtol(const char *s, unsigned int base, long *res); - -int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res); -int __must_check kstrtoll(const char *s, unsigned int base, long long *res); - -/** - * kstrtoul - convert a string to an unsigned long - * @s: The start of the string. The string must be null-terminated, and may also - * include a single newline before its terminating null. The first character - * may also be a plus sign, but not a minus sign. - * @base: The number base to use. The maximum supported base is 16. If base is - * given as 0, then the base of the string is automatically detected with the - * conventional semantics - If it begins with 0x the number will be parsed as a - * hexadecimal (case insensitive), if it otherwise begins with 0, it will be - * parsed as an octal number. Otherwise it will be parsed as a decimal. - * @res: Where to write the result of the conversion on success. - * - * Returns 0 on success, -ERANGE on overflow and -EINVAL on parsing error. - * Preferred over simple_strtoul(). Return code must be checked. -*/ -static inline int __must_check kstrtoul(const char *s, unsigned int base, unsigned long *res) -{ - /* - * We want to shortcut function call, but - * __builtin_types_compatible_p(unsigned long, unsigned long long) = 0. - */ - if (sizeof(unsigned long) == sizeof(unsigned long long) && - __alignof__(unsigned long) == __alignof__(unsigned long long)) - return kstrtoull(s, base, (unsigned long long *)res); - else - return _kstrtoul(s, base, res); -} - -/** - * kstrtol - convert a string to a long - * @s: The start of the string. The string must be null-terminated, and may also - * include a single newline before its terminating null. The first character - * may also be a plus sign or a minus sign. - * @base: The number base to use. The maximum supported base is 16. If base is - * given as 0, then the base of the string is automatically detected with the - * conventional semantics - If it begins with 0x the number will be parsed as a - * hexadecimal (case insensitive), if it otherwise begins with 0, it will be - * parsed as an octal number. Otherwise it will be parsed as a decimal. - * @res: Where to write the result of the conversion on success. - * - * Returns 0 on success, -ERANGE on overflow and -EINVAL on parsing error. - * Preferred over simple_strtol(). Return code must be checked. - */ -static inline int __must_check kstrtol(const char *s, unsigned int base, long *res) -{ - /* - * We want to shortcut function call, but - * __builtin_types_compatible_p(long, long long) = 0. - */ - if (sizeof(long) == sizeof(long long) && - __alignof__(long) == __alignof__(long long)) - return kstrtoll(s, base, (long long *)res); - else - return _kstrtol(s, base, res); -} - -int __must_check kstrtouint(const char *s, unsigned int base, unsigned int *res); -int __must_check kstrtoint(const char *s, unsigned int base, int *res); - -static inline int __must_check kstrtou64(const char *s, unsigned int base, u64 *res) -{ - return kstrtoull(s, base, res); -} - -static inline int __must_check kstrtos64(const char *s, unsigned int base, s64 *res) -{ - return kstrtoll(s, base, res); -} - -static inline int __must_check kstrtou32(const char *s, unsigned int base, u32 *res) -{ - return kstrtouint(s, base, res); -} - -static inline int __must_check kstrtos32(const char *s, unsigned int base, s32 *res) -{ - return kstrtoint(s, base, res); -} - -int __must_check kstrtou16(const char *s, unsigned int base, u16 *res); -int __must_check kstrtos16(const char *s, unsigned int base, s16 *res); -int __must_check kstrtou8(const char *s, unsigned int base, u8 *res); -int __must_check kstrtos8(const char *s, unsigned int base, s8 *res); -int __must_check kstrtobool(const char *s, bool *res); - -int __must_check kstrtoull_from_user(const char __user *s, size_t count, unsigned int base, unsigned long long *res); -int __must_check kstrtoll_from_user(const char __user *s, size_t count, unsigned int base, long long *res); -int __must_check kstrtoul_from_user(const char __user *s, size_t count, unsigned int base, unsigned long *res); -int __must_check kstrtol_from_user(const char __user *s, size_t count, unsigned int base, long *res); -int __must_check kstrtouint_from_user(const char __user *s, size_t count, unsigned int base, unsigned int *res); -int __must_check kstrtoint_from_user(const char __user *s, size_t count, unsigned int base, int *res); -int __must_check kstrtou16_from_user(const char __user *s, size_t count, unsigned int base, u16 *res); -int __must_check kstrtos16_from_user(const char __user *s, size_t count, unsigned int base, s16 *res); -int __must_check kstrtou8_from_user(const char __user *s, size_t count, unsigned int base, u8 *res); -int __must_check kstrtos8_from_user(const char __user *s, size_t count, unsigned int base, s8 *res); -int __must_check kstrtobool_from_user(const char __user *s, size_t count, bool *res); - -static inline int __must_check kstrtou64_from_user(const char __user *s, size_t count, unsigned int base, u64 *res) -{ - return kstrtoull_from_user(s, count, base, res); -} - -static inline int __must_check kstrtos64_from_user(const char __user *s, size_t count, unsigned int base, s64 *res) -{ - return kstrtoll_from_user(s, count, base, res); -} - -static inline int __must_check kstrtou32_from_user(const char __user *s, size_t count, unsigned int base, u32 *res) -{ - return kstrtouint_from_user(s, count, base, res); -} - -static inline int __must_check kstrtos32_from_user(const char __user *s, size_t count, unsigned int base, s32 *res) -{ - return kstrtoint_from_user(s, count, base, res); -} - -/* - * Use kstrto instead. - * - * NOTE: simple_strto does not check for the range overflow and, - * depending on the input, may give interesting results. - * - * Use these functions if and only if you cannot use kstrto, because - * the conversion ends on the first non-digit character, which may be far - * beyond the supported range. It might be useful to parse the strings like - * 10x50 or 12:21 without altering original string or temporary buffer in use. - * Keep in mind above caveat. - */ - -extern unsigned long simple_strtoul(const char *,char **,unsigned int); -extern long simple_strtol(const char *,char **,unsigned int); -extern unsigned long long simple_strtoull(const char *,char **,unsigned int); -extern long long simple_strtoll(const char *,char **,unsigned int); - extern int num_to_str(char *buf, int size, unsigned long long num, unsigned int width); --- /dev/null +++ a/include/linux/kstrtox.h @@ -0,0 +1,155 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _LINUX_KSTRTOX_H +#define _LINUX_KSTRTOX_H + +#include +#include + +/* Internal, do not use. */ +int __must_check _kstrtoul(const char *s, unsigned int base, unsigned long *res); +int __must_check _kstrtol(const char *s, unsigned int base, long *res); + +int __must_check kstrtoull(const char *s, unsigned int base, unsigned long long *res); +int __must_check kstrtoll(const char *s, unsigned int base, long long *res); + +/** + * kstrtoul - convert a string to an unsigned long + * @s: The start of the string. The string must be null-terminated, and may also + * include a single newline before its terminating null. The first character + * may also be a plus sign, but not a minus sign. + * @base: The number base to use. The maximum supported base is 16. If base is + * given as 0, then the base of the string is automatically detected with the + * conventional semantics - If it begins with 0x the number will be parsed as a + * hexadecimal (case insensitive), if it otherwise begins with 0, it will be + * parsed as an octal number. Otherwise it will be parsed as a decimal. + * @res: Where to write the result of the conversion on success. + * + * Returns 0 on success, -ERANGE on overflow and -EINVAL on parsing error. + * Preferred over simple_strtoul(). Return code must be checked. +*/ +static inline int __must_check kstrtoul(const char *s, unsigned int base, unsigned long *res) +{ + /* + * We want to shortcut function call, but + * __builtin_types_compatible_p(unsigned long, unsigned long long) = 0. + */ + if (sizeof(unsigned long) == sizeof(unsigned long long) && + __alignof__(unsigned long) == __alignof__(unsigned long long)) + return kstrtoull(s, base, (unsigned long long *)res); + else + return _kstrtoul(s, base, res); +} + +/** + * kstrtol - convert a string to a long + * @s: The start of the string. The string must be null-terminated, and may also + * include a single newline before its terminating null. The first character + * may also be a plus sign or a minus sign. + * @base: The number base to use. The maximum supported base is 16. If base is + * given as 0, then the base of the string is automatically detected with the + * conventional semantics - If it begins with 0x the number will be parsed as a + * hexadecimal (case insensitive), if it otherwise begins with 0, it will be + * parsed as an octal number. Otherwise it will be parsed as a decimal. + * @res: Where to write the result of the conversion on success. + * + * Returns 0 on success, -ERANGE on overflow and -EINVAL on parsing error. + * Preferred over simple_strtol(). Return code must be checked. + */ +static inline int __must_check kstrtol(const char *s, unsigned int base, long *res) +{ + /* + * We want to shortcut function call, but + * __builtin_types_compatible_p(long, long long) = 0. + */ + if (sizeof(long) == sizeof(long long) && + __alignof__(long) == __alignof__(long long)) + return kstrtoll(s, base, (long long *)res); + else + return _kstrtol(s, base, res); +} + +int __must_check kstrtouint(const char *s, unsigned int base, unsigned int *res); +int __must_check kstrtoint(const char *s, unsigned int base, int *res); + +static inline int __must_check kstrtou64(const char *s, unsigned int base, u64 *res) +{ + return kstrtoull(s, base, res); +} + +static inline int __must_check kstrtos64(const char *s, unsigned int base, s64 *res) +{ + return kstrtoll(s, base, res); +} + +static inline int __must_check kstrtou32(const char *s, unsigned int base, u32 *res) +{ + return kstrtouint(s, base, res); +} + +static inline int __must_check kstrtos32(const char *s, unsigned int base, s32 *res) +{ + return kstrtoint(s, base, res); +} + +int __must_check kstrtou16(const char *s, unsigned int base, u16 *res); +int __must_check kstrtos16(const char *s, unsigned int base, s16 *res); +int __must_check kstrtou8(const char *s, unsigned int base, u8 *res); +int __must_check kstrtos8(const char *s, unsigned int base, s8 *res); +int __must_check kstrtobool(const char *s, bool *res); + +int __must_check kstrtoull_from_user(const char __user *s, size_t count, unsigned int base, unsigned long long *res); +int __must_check kstrtoll_from_user(const char __user *s, size_t count, unsigned int base, long long *res); +int __must_check kstrtoul_from_user(const char __user *s, size_t count, unsigned int base, unsigned long *res); +int __must_check kstrtol_from_user(const char __user *s, size_t count, unsigned int base, long *res); +int __must_check kstrtouint_from_user(const char __user *s, size_t count, unsigned int base, unsigned int *res); +int __must_check kstrtoint_from_user(const char __user *s, size_t count, unsigned int base, int *res); +int __must_check kstrtou16_from_user(const char __user *s, size_t count, unsigned int base, u16 *res); +int __must_check kstrtos16_from_user(const char __user *s, size_t count, unsigned int base, s16 *res); +int __must_check kstrtou8_from_user(const char __user *s, size_t count, unsigned int base, u8 *res); +int __must_check kstrtos8_from_user(const char __user *s, size_t count, unsigned int base, s8 *res); +int __must_check kstrtobool_from_user(const char __user *s, size_t count, bool *res); + +static inline int __must_check kstrtou64_from_user(const char __user *s, size_t count, unsigned int base, u64 *res) +{ + return kstrtoull_from_user(s, count, base, res); +} + +static inline int __must_check kstrtos64_from_user(const char __user *s, size_t count, unsigned int base, s64 *res) +{ + return kstrtoll_from_user(s, count, base, res); +} + +static inline int __must_check kstrtou32_from_user(const char __user *s, size_t count, unsigned int base, u32 *res) +{ + return kstrtouint_from_user(s, count, base, res); +} + +static inline int __must_check kstrtos32_from_user(const char __user *s, size_t count, unsigned int base, s32 *res) +{ + return kstrtoint_from_user(s, count, base, res); +} + +/* + * Use kstrto instead. + * + * NOTE: simple_strto does not check for the range overflow and, + * depending on the input, may give interesting results. + * + * Use these functions if and only if you cannot use kstrto, because + * the conversion ends on the first non-digit character, which may be far + * beyond the supported range. It might be useful to parse the strings like + * 10x50 or 12:21 without altering original string or temporary buffer in use. + * Keep in mind above caveat. + */ + +extern unsigned long simple_strtoul(const char *,char **,unsigned int); +extern long simple_strtol(const char *,char **,unsigned int); +extern unsigned long long simple_strtoull(const char *,char **,unsigned int); +extern long long simple_strtoll(const char *,char **,unsigned int); + +static inline int strtobool(const char *s, bool *res) +{ + return kstrtobool(s, res); +} + +#endif /* _LINUX_KSTRTOX_H */ --- a/include/linux/string.h~kernelh-split-out-kstrtox-and-simple_strtox-to-a-separate-header +++ a/include/linux/string.h @@ -2,7 +2,6 @@ #ifndef _LINUX_STRING_H_ #define _LINUX_STRING_H_ - #include /* for inline */ #include /* for size_t */ #include /* for NULL */ @@ -184,12 +183,6 @@ extern char **argv_split(gfp_t gfp, cons extern void argv_free(char **argv); extern bool sysfs_streq(const char *s1, const char *s2); -extern int kstrtobool(const char *s, bool *res); -static inline int strtobool(const char *s, bool *res) -{ - return kstrtobool(s, res); -} - int match_string(const char * const *array, size_t n, const char *string); int __sysfs_match_string(const char * const *array, size_t n, const char *s); --- a/include/linux/sunrpc/cache.h~kernelh-split-out-kstrtox-and-simple_strtox-to-a-separate-header +++ a/include/linux/sunrpc/cache.h @@ -14,6 +14,7 @@ #include #include #include +#include #include /* --- a/lib/kstrtox.c~kernelh-split-out-kstrtox-and-simple_strtox-to-a-separate-header +++ a/lib/kstrtox.c @@ -14,11 +14,12 @@ */ #include #include -#include -#include #include +#include +#include #include #include + #include "kstrtox.h" const char *_parse_integer_fixup_radix(const char *s, unsigned int *base) --- a/lib/parser.c~kernelh-split-out-kstrtox-and-simple_strtox-to-a-separate-header +++ a/lib/parser.c @@ -6,6 +6,7 @@ #include #include #include +#include #include #include #include From patchwork Thu Jul 1 01:56:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07372C11F66 for ; Thu, 1 Jul 2021 01:56:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AED9161468 for ; Thu, 1 Jul 2021 01:56:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AED9161468 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 32C4D8D0276; Wed, 30 Jun 2021 21:56:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3035D8D024F; Wed, 30 Jun 2021 21:56:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CB7A8D0276; Wed, 30 Jun 2021 21:56:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id EBF3C8D024F for ; Wed, 30 Jun 2021 21:56:14 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id C0C03181AF5C1 for ; Thu, 1 Jul 2021 01:56:14 +0000 (UTC) X-FDA: 78312353868.05.0872185 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 78BB8700009B for ; Thu, 1 Jul 2021 01:56:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7DFFE61418; Thu, 1 Jul 2021 01:56:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104573; bh=YFimTB8Trav/BodnvstdleU+peLP5UK6Zk8PfTKBsAE=; h=Date:From:To:Subject:In-Reply-To:From; b=cMHkhIwCEyxFFt/O0sCWF+GFblcKFbtSTwg7O9NPG552I7uJ6nPjQYxmNWnVtONLT RQdS+nnf4K6zudTf2BPqTt3ASIV8oSBkTKeLXOv9iBOGSOiVYH9prB9beiB+wIyz8L ZvE2xOfuOOs5EzP9yN/rUUjTD9jAvtbGiFNZCO3Q= Date: Wed, 30 Jun 2021 18:56:13 -0700 From: Andrew Morton To: akpm@linux-foundation.org, hsiangkao@redhat.com, joe@perches.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, terrelln@fb.com, thisisrast7@gmail.com, torvalds@linux-foundation.org Subject: [patch 171/192] lz4_decompress: declare LZ4_decompress_safe_withPrefix64k static Message-ID: <20210701015613.APX-MDiYp%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=cMHkhIwC; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: sy5p79nasxhwz9c7z49huxjehb85dia3 X-Rspamd-Queue-Id: 78BB8700009B X-Rspamd-Server: rspam06 X-HE-Tag: 1625104574-73824 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Rajat Asthana Subject: lz4_decompress: declare LZ4_decompress_safe_withPrefix64k static Declare LZ4_decompress_safe_withPrefix64k as static to fix sparse warning: > warning: symbol 'LZ4_decompress_safe_withPrefix64k' was not declared. > Should it be static? Link: https://lkml.kernel.org/r/20210511154345.610569-1-thisisrast7@gmail.com Signed-off-by: Rajat Asthana Reviewed-by: Nick Terrell Cc: Gao Xiang Cc: Joe Perches Signed-off-by: Andrew Morton --- lib/lz4/lz4_decompress.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/lib/lz4/lz4_decompress.c~lz4_decompress-declare-lz4_decompress_safe_withprefix64k-static +++ a/lib/lz4/lz4_decompress.c @@ -481,7 +481,7 @@ int LZ4_decompress_fast(const char *sour /* ===== Instantiate a few more decoding cases, used more than once. ===== */ -int LZ4_decompress_safe_withPrefix64k(const char *source, char *dest, +static int LZ4_decompress_safe_withPrefix64k(const char *source, char *dest, int compressedSize, int maxOutputSize) { return LZ4_decompress_generic(source, dest, From patchwork Thu Jul 1 01:56:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50F1FC11F69 for ; Thu, 1 Jul 2021 01:56:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0071061469 for ; Thu, 1 Jul 2021 01:56:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0071061469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C07D8D0277; Wed, 30 Jun 2021 21:56:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 797138D024F; Wed, 30 Jun 2021 21:56:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 637B28D0277; Wed, 30 Jun 2021 21:56:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0227.hostedemail.com [216.40.44.227]) by kanga.kvack.org (Postfix) with ESMTP id 388B08D024F for ; Wed, 30 Jun 2021 21:56:18 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0CFAD252B8 for ; Thu, 1 Jul 2021 01:56:18 +0000 (UTC) X-FDA: 78312354036.10.92A483E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id A46EB30000A6 for ; Thu, 1 Jul 2021 01:56:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9127F61468; Thu, 1 Jul 2021 01:56:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104577; bh=9MBrLmQ5O2ytNTSuFSNr1kI2RbdC7XVerY793xPqLYY=; h=Date:From:To:Subject:In-Reply-To:From; b=hPc6QzyO2kJp8njE6kfHHvi8yWfyx5vD1LwLhkgh+zXUn9IL5IHQ3NpW7WCqmA6Mz uFDjI3Uh9cH+wvUof4Zv5SUKYd1UgVV9RpShnDZkZ7bZwNGVcQVM8oEqpSWD/ytBA6 IAm4Yl/m4pb14oIov7auQgagbVixkjHO8QFFF1NM= Date: Wed, 30 Jun 2021 18:56:16 -0700 From: Andrew Morton To: 4sschmid@informatik.uni-hamburg.de, akpm@linux-foundation.org, bongkyu.kim@lge.com, dimitri.ledkov@canonical.com, hsiangkao@redhat.com, keescook@chromium.org, kyungsik.lee@lge.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, terrelln@fb.com, thisisrast7@gmail.com, torvalds@linux-foundation.org, yinghai@kernel.org Subject: [patch 172/192] lib/decompress_unlz4.c: correctly handle zero-padding around initrds. Message-ID: <20210701015616.3oaIBGjeo%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A46EB30000A6 X-Stat-Signature: f6diqgy35f4sn4p7kswqdfk4x9z464xp Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=hPc6QzyO; dmarc=none; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104577-722011 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dimitri John Ledkov Subject: lib/decompress_unlz4.c: correctly handle zero-padding around initrds. lz4 compatible decompressor is simple. The format is underspecified and relies on EOF notification to determine when to stop. Initramfs buffer format[1] explicitly states that it can have arbitrary number of zero padding. Thus when operating without a fill function, be extra careful to ensure that sizes less than 4, or apperantly empty chunksizes are treated as EOF. To test this I have created two cpio initrds, first a normal one, main.cpio. And second one with just a single /test-file with content "second" second.cpio. Then i compressed both of them with gzip, and with lz4 -l. Then I created a padding of 4 bytes (dd if=/dev/zero of=pad4 bs=1 count=4). To create four testcase initrds: 1) main.cpio.gzip + extra.cpio.gzip = pad0.gzip 2) main.cpio.lz4 + extra.cpio.lz4 = pad0.lz4 3) main.cpio.gzip + pad4 + extra.cpio.gzip = pad4.gzip 4) main.cpio.lz4 + pad4 + extra.cpio.lz4 = pad4.lz4 The pad4 test-cases replicate the initrd load by grub, as it pads and aligns every initrd it loads. All of the above boot, however /test-file was not accessible in the initrd for the testcase #4, as decoding in lz4 decompressor failed. Also an error message printed which usually is harmless. Whith a patched kernel, all of the above testcases now pass, and /test-file is accessible. This fixes lz4 initrd decompress warning on every boot with grub. And more importantly this fixes inability to load multiple lz4 compressed initrds with grub. This patch has been shipping in Ubuntu kernels since January 2021. [1] ./Documentation/driver-api/early-userspace/buffer-format.rst BugLink: https://bugs.launchpad.net/bugs/1835660 Link: https://lore.kernel.org/lkml/20210114200256.196589-1-xnox@ubuntu.com/ # v0 Link: https://lkml.kernel.org/r/20210513104831.432975-1-dimitri.ledkov@canonical.com Signed-off-by: Dimitri John Ledkov Cc: Kyungsik Lee Cc: Yinghai Lu Cc: Bongkyu Kim Cc: Kees Cook Cc: Sven Schmidt <4sschmid@informatik.uni-hamburg.de> Cc: Rajat Asthana Cc: Nick Terrell Cc: Gao Xiang Signed-off-by: Andrew Morton --- lib/decompress_unlz4.c | 8 ++++++++ 1 file changed, 8 insertions(+) --- a/lib/decompress_unlz4.c~lib-decompress_unlz4c-correctly-handle-zero-padding-around-initrds +++ a/lib/decompress_unlz4.c @@ -112,6 +112,9 @@ STATIC inline int INIT unlz4(u8 *input, error("data corrupted"); goto exit_2; } + } else if (size < 4) { + /* empty or end-of-file */ + goto exit_3; } chunksize = get_unaligned_le32(inp); @@ -125,6 +128,10 @@ STATIC inline int INIT unlz4(u8 *input, continue; } + if (!fill && chunksize == 0) { + /* empty or end-of-file */ + goto exit_3; + } if (posp) *posp += 4; @@ -184,6 +191,7 @@ STATIC inline int INIT unlz4(u8 *input, } } +exit_3: ret = 0; exit_2: if (!input) From patchwork Thu Jul 1 01:56:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4304AC11F66 for ; Thu, 1 Jul 2021 01:56:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EF4AF6141A for ; Thu, 1 Jul 2021 01:56:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF4AF6141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 771EE8D0278; Wed, 30 Jun 2021 21:56:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7228C8D024F; Wed, 30 Jun 2021 21:56:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E9FB8D0278; Wed, 30 Jun 2021 21:56:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0113.hostedemail.com [216.40.44.113]) by kanga.kvack.org (Postfix) with ESMTP id 36CCF8D024F for ; Wed, 30 Jun 2021 21:56:21 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1120218035425 for ; Thu, 1 Jul 2021 01:56:21 +0000 (UTC) X-FDA: 78312354162.26.E38F7B1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id BE44ED000156 for ; Thu, 1 Jul 2021 01:56:20 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B582D61418; Thu, 1 Jul 2021 01:56:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104580; bh=Nymilikzj/TnMBVrVhiFvZvmgWHcPO+Rl9MvzE+2fIo=; h=Date:From:To:Subject:In-Reply-To:From; b=g3rwObb+LFzA9X7OTSEqBYRZyg4IdF/teqtQjqSy06lnqxvvStf6rncV7dJYisv6U Dkpq+pQAHsXcQd2r5tQu4X41m0u/cUY2ULWgcFlmuqt1QvihsYwLhRCs8/tjOoMmh+ 0CDTQLa8T17IIF9t8VOV0GJ3Vt9nSjgHi9mwqZjE= Date: Wed, 30 Jun 2021 18:56:19 -0700 From: Andrew Morton To: akpm@linux-foundation.org, bert@biot.com, dwaipayanray1@gmail.com, joe@perches.com, linux-mm@kvack.org, linux@roeck-us.net, lukas.bulwahn@gmail.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 173/192] checkpatch: scripts/spdxcheck.py now requires python3 Message-ID: <20210701015619.3yumN4pdi%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: BE44ED000156 X-Stat-Signature: hfdq3tdz9x4mwueaigiujf68ztn8hn1f Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=g3rwObb+; dmarc=none; spf=pass (imf21.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104580-270791 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Guenter Roeck Subject: checkpatch: scripts/spdxcheck.py now requires python3 Since commit d0259c42abff ("spdxcheck.py: Use Python 3"), spdxcheck.py explicitly expects to run as python3 script. If "python" still points to python v2.7 and the script is executed with "python scripts/spdxcheck.py", the following error may be seen even if git-python is installed for python3. Traceback (most recent call last): File "scripts/spdxcheck.py", line 10, in import git ImportError: No module named git To fix the problem, check for the existence of python3, check if the script is executable and not just for its existence, and execute it directly. Link: https://lkml.kernel.org/r/20210505211720.447111-1-linux@roeck-us.net Signed-off-by: Guenter Roeck Cc: Joe Perches Cc: Bert Vermeulen Cc: Dwaipayan Ray Cc: Lukas Bulwahn Cc: Bert Vermeulen Signed-off-by: Andrew Morton --- scripts/checkpatch.pl | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/scripts/checkpatch.pl~checkpatch-scripts-spdxcheckpy-now-requires-python3 +++ a/scripts/checkpatch.pl @@ -1084,10 +1084,10 @@ sub is_maintained_obsolete { sub is_SPDX_License_valid { my ($license) = @_; - return 1 if (!$tree || which("python") eq "" || !(-e "$root/scripts/spdxcheck.py") || !(-e "$gitroot")); + return 1 if (!$tree || which("python3") eq "" || !(-x "$root/scripts/spdxcheck.py") || !(-e "$gitroot")); my $root_path = abs_path($root); - my $status = `cd "$root_path"; echo "$license" | python scripts/spdxcheck.py -`; + my $status = `cd "$root_path"; echo "$license" | scripts/spdxcheck.py -`; return 0 if ($status ne ""); return 1; } From patchwork Thu Jul 1 01:56:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 290CFC11F69 for ; Thu, 1 Jul 2021 01:56:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D4CE86141A for ; Thu, 1 Jul 2021 01:56:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4CE86141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5EBD08D024F; Wed, 30 Jun 2021 21:56:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 574348D0279; Wed, 30 Jun 2021 21:56:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FC278D024F; Wed, 30 Jun 2021 21:56:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id 17BA08D024F for ; Wed, 30 Jun 2021 21:56:24 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E999982E8192 for ; Thu, 1 Jul 2021 01:56:23 +0000 (UTC) X-FDA: 78312354246.23.4FD3FE7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 9FF1BD000155 for ; Thu, 1 Jul 2021 01:56:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B7C6061469; Thu, 1 Jul 2021 01:56:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104583; bh=ycD4VMGuhVPokmYYI/5gsUNLK1BzminCOCdV/oMVzdw=; h=Date:From:To:Subject:In-Reply-To:From; b=wXKLtvNOgWKpA2FaF+opTYsHpMuhdMrKUTV7Oy1ATlqf5OMRJnjhPYdAjGWVyxyhc imjOJmtwt2wpbEYIfMRhyrNGziWnOUjI/hv5u+CrgR1rfZsXKUc6UAwGPj7rRrkIG7 o4pk6+ou2dMolCubo5zt3anyHhe5C7358nDL+g74= Date: Wed, 30 Jun 2021 18:56:22 -0700 From: Andrew Morton To: akpm@linux-foundation.org, elder@ieee.org, gregkh@linuxfoundation.org, joe@perches.com, linux-mm@kvack.org, manikishanghantasala@gmail.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 174/192] checkpatch: improve the indented label test Message-ID: <20210701015622.z0Hi22fow%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=wXKLtvNO; dmarc=none; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: qty5xzro7pt1ns19uzgt6b4ewqn9fr9f X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9FF1BD000155 X-HE-Tag: 1625104583-8260 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joe Perches Subject: checkpatch: improve the indented label test checkpatch identifies a label only when a terminating colon immediately follows an identifier. Bitfield definitions can appear to be labels so ignore any spaces between the identifier terminating colon and any digit that may be used to define a bitfield length. Miscellanea: o Improve the initial checkpatch comment o Use the more typical '&&' instead of 'and' o Require the initial label character to be a non-digit (Can't use $Ident here because $Ident allows ## concatenation) o Use $sline instead of $line to ignore comments o Use '$sline !~ /.../' instead of '!($line =~ /.../)' Link: https://lkml.kernel.org/r/b54d673e7cde7de5de0c9ba4dd57dd0858580ca4.camel@perches.com Signed-off-by: Joe Perches Cc: Greg Kroah-Hartman Cc: Manikishan Ghantasala Cc: Alex Elder Signed-off-by: Andrew Morton --- scripts/checkpatch.pl | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) --- a/scripts/checkpatch.pl~checkpatch-improve-the-indented-label-test +++ a/scripts/checkpatch.pl @@ -5361,9 +5361,13 @@ sub process { } } -#goto labels aren't indented, allow a single space however - if ($line=~/^.\s+[A-Za-z\d_]+:(?![0-9]+)/ and - !($line=~/^. [A-Za-z\d_]+:/) and !($line=~/^.\s+default:/)) { +# check that goto labels aren't indented (allow a single space indentation) +# and ignore bitfield definitions like foo:1 +# Strictly, labels can have whitespace after the identifier and before the : +# but this is not allowed here as many ?: uses would appear to be labels + if ($sline =~ /^.\s+[A-Za-z_][A-Za-z\d_]*:(?!\s*\d+)/ && + $sline !~ /^. [A-Za-z\d_][A-Za-z\d_]*:/ && + $sline !~ /^.\s+default:/) { if (WARN("INDENTED_LABEL", "labels should not be indented\n" . $herecurr) && $fix) { From patchwork Thu Jul 1 01:56:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353407 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33043C11F66 for ; Thu, 1 Jul 2021 01:56:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E102A6146D for ; Thu, 1 Jul 2021 01:56:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E102A6146D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6AEE38D027A; Wed, 30 Jun 2021 21:56:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 685E68D0279; Wed, 30 Jun 2021 21:56:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 574F28D027A; Wed, 30 Jun 2021 21:56:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 2D8668D0279 for ; Wed, 30 Jun 2021 21:56:27 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 036BF181AF5C1 for ; Thu, 1 Jul 2021 01:56:26 +0000 (UTC) X-FDA: 78312354414.30.8BA83D0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id AA76990000B3 for ; Thu, 1 Jul 2021 01:56:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B357A61469; Thu, 1 Jul 2021 01:56:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104585; bh=bbvVy1qxmudmoewmDYOVkrblofvGNtoijRMesP4b3Ec=; h=Date:From:To:Subject:In-Reply-To:From; b=GOVt2u3495kAFpaCla5KplQe17BOWjXc2xt6Qew9/pu7/hlJCqRTjpN7WZ8/7rkv0 UyTFTAaMLCNjN150u8yOhs+gXn7OFEyyvPUbMmybbP9uYsxALbe59YbW7H/xK9D7qw qj5O0i5B7Bbvx7DVnLsirBBLG/DR/XrqB1kz1zF8= Date: Wed, 30 Jun 2021 18:56:25 -0700 From: Andrew Morton To: akpm@linux-foundation.org, joe@perches.com, linux-mm@kvack.org, linux@roeck-us.net, mm-commits@vger.kernel.org, ribalda@chromium.org, torvalds@linux-foundation.org Subject: [patch 175/192] checkpatch: do not complain about positive return values starting with EPOLL Message-ID: <20210701015625.hw3VerOqR%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=GOVt2u34; dmarc=none; spf=pass (imf23.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: n17x9xndrtuaq1ur8jpy5ps8t4xkqiju X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: AA76990000B3 X-HE-Tag: 1625104586-376143 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Guenter Roeck Subject: checkpatch: do not complain about positive return values starting with EPOLL checkpatch complains about positive return values of poll functions. Example: WARNING: return of an errno should typically be negative (ie: return -EPOLLIN) + return EPOLLIN; Poll functions return positive values. The defines for the return values of poll functions all start with EPOLL, resulting in a number of false positives. An often used workaround is to assign poll function return values to variables and returning that variable, but that is a less than perfect solution. There is no error definition which starts with EPOLL, so it is safe to omit the warning for return values starting with EPOLL. Link: https://lkml.kernel.org/r/20210622004334.638680-1-linux@roeck-us.net Signed-off-by: Guenter Roeck Acked-by: Joe Perches Cc: Ricardo Ribalda Signed-off-by: Andrew Morton --- scripts/checkpatch.pl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/scripts/checkpatch.pl~checkpatch-do-not-complain-about-positive-return-values-starting-with-epoll +++ a/scripts/checkpatch.pl @@ -5462,7 +5462,7 @@ sub process { # Return of what appears to be an errno should normally be negative if ($sline =~ /\breturn(?:\s*\(+\s*|\s+)(E[A-Z]+)(?:\s*\)+\s*|\s*)[;:,]/) { my $name = $1; - if ($name ne 'EOF' && $name ne 'ERROR') { + if ($name ne 'EOF' && $name ne 'ERROR' && $name !~ /^EPOLL/) { WARN("USE_NEGATIVE_ERRNO", "return of an errno should typically be negative (ie: return -$1)\n" . $herecurr); } From patchwork Thu Jul 1 01:56:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27D03C11F66 for ; Thu, 1 Jul 2021 01:56:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D22D761477 for ; Thu, 1 Jul 2021 01:56:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D22D761477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5C4628D027B; Wed, 30 Jun 2021 21:56:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 59B588D0279; Wed, 30 Jun 2021 21:56:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43D1F8D027B; Wed, 30 Jun 2021 21:56:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0068.hostedemail.com [216.40.44.68]) by kanga.kvack.org (Postfix) with ESMTP id 2108E8D0279 for ; Wed, 30 Jun 2021 21:56:30 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E56FE253C4 for ; Thu, 1 Jul 2021 01:56:29 +0000 (UTC) X-FDA: 78312354498.27.F41D4A5 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP id 94E13B00037C for ; Thu, 1 Jul 2021 01:56:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A1F126146D; Thu, 1 Jul 2021 01:56:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104588; bh=/ls2NnoC8Zx5fW4dsg+rZbYGyeQEYYiYj9cexnSdSKc=; h=Date:From:To:Subject:In-Reply-To:From; b=ZNBeHY1LKTJJYLJy5zKt5765d2VZdCG2u01LZep8P9e/GVBWTKYlnQOGFuDdn2DiN m2MZtnMrEIFC+33rlOMka5C8pDJk77fO4r5WbnR+cSs+3HKP6nBxW5i+C9XsbKfPJT Vd5i1o6E4OxZh+8f9hLYm3kALgV+tjrs+5XJwEik= Date: Wed, 30 Jun 2021 18:56:28 -0700 From: Andrew Morton To: ahalaney@redhat.com, akpm@linux-foundation.org, bp@suse.de, linux-mm@kvack.org, mm-commits@vger.kernel.org, rostedt@goodmis.org, torvalds@linux-foundation.org Subject: [patch 176/192] init: print out unknown kernel parameters Message-ID: <20210701015628.vk9Y1Fqho%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ZNBeHY1L; spf=pass (imf25.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: jyanmef5ssdd9zkcusg3r8q11xmhnhzh X-Rspamd-Queue-Id: 94E13B00037C X-Rspamd-Server: rspam06 X-HE-Tag: 1625104589-645984 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrew Halaney Subject: init: print out unknown kernel parameters It is easy to foobar setting a kernel parameter on the command line without realizing it, there's not much output that you can use to assess what the kernel did with that parameter by default. Make it a little more explicit which parameters on the command line _looked_ like a valid parameter for the kernel, but did not match anything and ultimately got tossed to init. This is very similar to the unknown parameter message received when loading a module. This assumes the parameters are processed in a normal fashion, some parameters (dyndbg= for example) don't register their parameter with the rest of the kernel's parameters, and therefore always show up in this list (and are also given to init - like the rest of this list). Another example is BOOT_IMAGE= is highlighted as an offender, which it technically is, but is passed by LILO and GRUB so most systems will see that complaint. An example output where "foobared" and "unrecognized" are intentionally invalid parameters: Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.12-dirty debug log_buf_len=4M foobared unrecognized=foo Unknown command line parameters: foobared BOOT_IMAGE=/boot/vmlinuz-5.12-dirty unrecognized=foo Link: https://lkml.kernel.org/r/20210511211009.42259-1-ahalaney@redhat.com Signed-off-by: Andrew Halaney Suggested-by: Steven Rostedt Suggested-by: Borislav Petkov Acked-by: Borislav Petkov Signed-off-by: Andrew Morton --- init/main.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) --- a/init/main.c~init-print-out-unknown-kernel-parameters +++ a/init/main.c @@ -872,6 +872,47 @@ void __init __weak arch_call_rest_init(v rest_init(); } +static void __init print_unknown_bootoptions(void) +{ + char *unknown_options; + char *end; + const char *const *p; + size_t len; + + if (panic_later || (!argv_init[1] && !envp_init[2])) + return; + + /* + * Determine how many options we have to print out, plus a space + * before each + */ + len = 1; /* null terminator */ + for (p = &argv_init[1]; *p; p++) { + len++; + len += strlen(*p); + } + for (p = &envp_init[2]; *p; p++) { + len++; + len += strlen(*p); + } + + unknown_options = memblock_alloc(len, SMP_CACHE_BYTES); + if (!unknown_options) { + pr_err("%s: Failed to allocate %zu bytes\n", + __func__, len); + return; + } + end = unknown_options; + + for (p = &argv_init[1]; *p; p++) + end += sprintf(end, " %s", *p); + for (p = &envp_init[2]; *p; p++) + end += sprintf(end, " %s", *p); + + pr_notice("Unknown command line parameters:%s\n", unknown_options); + memblock_free(__pa(unknown_options), len); +} + asmlinkage __visible void __init __no_sanitize_address start_kernel(void) { char *command_line; @@ -913,6 +954,7 @@ asmlinkage __visible void __init __no_sa static_command_line, __start___param, __stop___param - __start___param, -1, -1, NULL, &unknown_bootoption); + print_unknown_bootoptions(); if (!IS_ERR_OR_NULL(after_dashes)) parse_args("Setting init args", after_dashes, NULL, 0, -1, -1, NULL, set_init_arg); From patchwork Thu Jul 1 01:56:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99E5AC11F68 for ; Thu, 1 Jul 2021 01:56:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5117C61477 for ; Thu, 1 Jul 2021 01:56:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5117C61477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CC9878D027C; Wed, 30 Jun 2021 21:56:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA0C18D0279; Wed, 30 Jun 2021 21:56:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B68918D027C; Wed, 30 Jun 2021 21:56:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 8F2198D0279 for ; Wed, 30 Jun 2021 21:56:33 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 66008253B6 for ; Thu, 1 Jul 2021 01:56:33 +0000 (UTC) X-FDA: 78312354666.02.CCB60B2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 0DA715000BC5 for ; Thu, 1 Jul 2021 01:56:32 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B422A6141A; Thu, 1 Jul 2021 01:56:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104592; bh=401q/z+V0pDuTxo9El745SlNiAqNW+WdwxGt9/qV6P4=; h=Date:From:To:Subject:In-Reply-To:From; b=PYeKkW5kD0qcTQmMSduw5ZoDxury8ffej4N+GCoZlHJdmo8l5fQiYe7IAx679SsAn szVb6rlwi/dqgbofSdS4AyF+Go3Jew1ipTQSlTGUFebT1lcH76hPDA4N+bvwY+fnXx +z0cQno/hK84Zz5H1erKAYYzDhLlzA03PL0Q2kqA= Date: Wed, 30 Jun 2021 18:56:31 -0700 From: Andrew Morton To: akpm@linux-foundation.org, anil.s.keshavamurthy@intel.com, borntraeger@de.ibm.com, bp@alien8.de, davem@davemloft.net, gor@linux.ibm.com, hca@linux.ibm.com, hch@lst.de, hpa@zytor.com, linux-mm@kvack.org, liuqi115@huawei.com, mhiramat@kernel.org, mingo@redhat.com, mm-commits@vger.kernel.org, naveen.n.rao@linux.ibm.com, song.bao.hua@hisilicon.com, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 177/192] kprobes: remove duplicated strong free_insn_page in x86 and s390 Message-ID: <20210701015631.VKMj8Ffy0%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=PYeKkW5k; dmarc=none; spf=pass (imf05.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: x3zr7nsyc3umr9m1rixpnmbntfrc15hw X-Rspamd-Queue-Id: 0DA715000BC5 X-HE-Tag: 1625104592-580503 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Barry Song Subject: kprobes: remove duplicated strong free_insn_page in x86 and s390 free_insn_page() in x86 and s390 is same with the common weak function in kernel/kprobes.c. Plus, the comment "Recover page to RW mode before releasing it" in x86 seems insensible to be there since resetting mapping is done by common code in vfree() of module_memfree(). So drop these two duplicated strong functions and related comment, then mark the common one in kernel/kprobes.c strong. Link: https://lkml.kernel.org/r/20210608065736.32656-1-song.bao.hua@hisilicon.com Signed-off-by: Barry Song Acked-by: Masami Hiramatsu Acked-by: Heiko Carstens Reviewed-by: Christoph Hellwig Cc: "H. Peter Anvin" Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Borislav Petkov Cc: Vasily Gorbik Cc: Christian Borntraeger Cc: "Naveen N. Rao" Cc: Anil S Keshavamurthy Cc: David S. Miller Cc: Qi Liu Signed-off-by: Andrew Morton --- arch/s390/kernel/kprobes.c | 5 ----- arch/x86/kernel/kprobes/core.c | 6 ------ include/linux/kprobes.h | 1 - kernel/kprobes.c | 2 +- 4 files changed, 1 insertion(+), 13 deletions(-) --- a/arch/s390/kernel/kprobes.c~kprobes-remove-duplicated-strong-free_insn_page-in-x86-and-s390 +++ a/arch/s390/kernel/kprobes.c @@ -44,11 +44,6 @@ void *alloc_insn_page(void) return page; } -void free_insn_page(void *page) -{ - module_memfree(page); -} - static void *alloc_s390_insn_page(void) { if (xchg(&insn_page_in_use, 1) == 1) --- a/arch/x86/kernel/kprobes/core.c~kprobes-remove-duplicated-strong-free_insn_page-in-x86-and-s390 +++ a/arch/x86/kernel/kprobes/core.c @@ -422,12 +422,6 @@ void *alloc_insn_page(void) return page; } -/* Recover page to RW mode before releasing it */ -void free_insn_page(void *page) -{ - module_memfree(page); -} - /* Kprobe x86 instruction emulation - only regs->ip or IF flag modifiers */ static void kprobe_emulate_ifmodifiers(struct kprobe *p, struct pt_regs *regs) --- a/include/linux/kprobes.h~kprobes-remove-duplicated-strong-free_insn_page-in-x86-and-s390 +++ a/include/linux/kprobes.h @@ -407,7 +407,6 @@ int enable_kprobe(struct kprobe *kp); void dump_kprobe(struct kprobe *kp); void *alloc_insn_page(void); -void free_insn_page(void *page); int kprobe_get_kallsym(unsigned int symnum, unsigned long *value, char *type, char *sym); --- a/kernel/kprobes.c~kprobes-remove-duplicated-strong-free_insn_page-in-x86-and-s390 +++ a/kernel/kprobes.c @@ -106,7 +106,7 @@ void __weak *alloc_insn_page(void) return module_alloc(PAGE_SIZE); } -void __weak free_insn_page(void *page) +static void free_insn_page(void *page) { module_memfree(page); } From patchwork Thu Jul 1 01:56:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353413 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43928C11F66 for ; Thu, 1 Jul 2021 01:56:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0F8D861477 for ; Thu, 1 Jul 2021 01:56:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F8D861477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 883378D027D; Wed, 30 Jun 2021 21:56:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 859DA8D0279; Wed, 30 Jun 2021 21:56:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7220E8D027D; Wed, 30 Jun 2021 21:56:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0188.hostedemail.com [216.40.44.188]) by kanga.kvack.org (Postfix) with ESMTP id 501C18D0279 for ; Wed, 30 Jun 2021 21:56:36 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1F9A0181AF5C1 for ; Thu, 1 Jul 2021 01:56:36 +0000 (UTC) X-FDA: 78312354792.26.954DC31 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id CA83C100009D for ; Thu, 1 Jul 2021 01:56:35 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id ED7C26147D; Thu, 1 Jul 2021 01:56:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104595; bh=t4fUpIseMo9kGaMebfPbXWFx074uUhQC5SxSfDL5mz4=; h=Date:From:To:Subject:In-Reply-To:From; b=olspHHV5YGIn1SOEi0AxNYmBqAZewivSKpH5LS1XfMh2IAaz+hNWVF+uvdNrNA5ji RidZdrE7QzQbeb7dREfPmD/lVP6/Hkx3j6GlCyUx9sr7sLqUE0A9Vmk3jlx0OzaHR/ kQRg+7veXhxkk7oxYpERmghj4UrTH7/C3JKSyqgY= Date: Wed, 30 Jun 2021 18:56:34 -0700 From: Andrew Morton To: akpm@linux-foundation.org, colin.king@canonical.com, konishi.ryusuke@gmail.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 178/192] nilfs2: remove redundant continue statement in a while-loop Message-ID: <20210701015634.WGEgit9PU%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=olspHHV5; dmarc=none; spf=pass (imf12.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: p5gws8ssh76z3usnydmar17asuguwhke X-Rspamd-Queue-Id: CA83C100009D X-HE-Tag: 1625104595-567057 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Colin Ian King Subject: nilfs2: remove redundant continue statement in a while-loop The continue statement at the end of the while-loop is redundant, remove it. Addresses-Coverity: ("Continue has no effect") Link: https://lkml.kernel.org/r/20210621100519.10257-1-colin.king@canonical.com Link: https://lkml.kernel.org/r/1624557664-17159-1-git-send-email-konishi.ryusuke@gmail.com Signed-off-by: Colin Ian King Signed-off-by: Ryusuke Konishi Signed-off-by: Andrew Morton --- fs/nilfs2/btree.c | 1 - 1 file changed, 1 deletion(-) --- a/fs/nilfs2/btree.c~nilfs2-remove-redundant-continue-statement-in-a-while-loop +++ a/fs/nilfs2/btree.c @@ -738,7 +738,6 @@ static int nilfs_btree_lookup_contig(con if (ptr2 != ptr + cnt || ++cnt == maxblocks) goto end; index++; - continue; } if (level == maxlevel) break; From patchwork Thu Jul 1 01:56:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 344EFC11F66 for ; Thu, 1 Jul 2021 01:56:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F061061477 for ; Thu, 1 Jul 2021 01:56:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F061061477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7726B8D027E; Wed, 30 Jun 2021 21:56:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 748508D0279; Wed, 30 Jun 2021 21:56:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6114A8D027E; Wed, 30 Jun 2021 21:56:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0231.hostedemail.com [216.40.44.231]) by kanga.kvack.org (Postfix) with ESMTP id 3CF918D0279 for ; Wed, 30 Jun 2021 21:56:39 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1759818043646 for ; Thu, 1 Jul 2021 01:56:39 +0000 (UTC) X-FDA: 78312354918.30.86FA137 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id C860D6001A8C for ; Thu, 1 Jul 2021 01:56:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D929F61469; Thu, 1 Jul 2021 01:56:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104598; bh=oygytOw7VDumW93VzYSF4Gu6w91eRl9BB8m8m5UI6w0=; h=Date:From:To:Subject:In-Reply-To:From; b=wW99Dzkhy3gHaDSqymtYneCbzJBNPo4+fORql8ZGxpIia36NvemApbg0Cnuy7z+DA yPIKqsX/u7jY4FGuUOe9MHDTEMs+l5YiCm+PpBO95g9tRQdB/kXdMJCJxNiyEMqTGO aS9FhfIairdoqAO5DCbAXdSJqHDo0sLwmxeZ1DCE= Date: Wed, 30 Jun 2021 18:56:37 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, thunder.leizhen@huawei.com, torvalds@linux-foundation.org Subject: [patch 179/192] hfsplus: remove unnecessary oom message Message-ID: <20210701015637.LXYhTDzFj%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C860D6001A8C Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=wW99Dzkh; spf=pass (imf10.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: p8d8upz6zk34f85rirt39csnftta1rs8 X-HE-Tag: 1625104598-194251 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhen Lei Subject: hfsplus: remove unnecessary oom message Fixes scripts/checkpatch.pl warning: WARNING: Possible unnecessary 'out of memory' message Remove it can help us save a bit of memory. Link: https://lkml.kernel.org/r/20210617084944.1279-1-thunder.leizhen@huawei.com Signed-off-by: Zhen Lei Signed-off-by: Andrew Morton --- fs/hfsplus/xattr.c | 1 - 1 file changed, 1 deletion(-) --- a/fs/hfsplus/xattr.c~hfsplus-remove-unnecessary-oom-message +++ a/fs/hfsplus/xattr.c @@ -204,7 +204,6 @@ check_attr_tree_state_again: buf = kzalloc(node_size, GFP_NOFS); if (!buf) { - pr_err("failed to allocate memory for header node\n"); err = -ENOMEM; goto end_attr_file_creation; } From patchwork Thu Jul 1 01:56:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353417 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EC99C11F66 for ; Thu, 1 Jul 2021 01:56:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D9AFB6147D for ; Thu, 1 Jul 2021 01:56:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9AFB6147D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 611F68D027F; Wed, 30 Jun 2021 21:56:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C2248D0279; Wed, 30 Jun 2021 21:56:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 489A78D027F; Wed, 30 Jun 2021 21:56:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id 25B898D0279 for ; Wed, 30 Jun 2021 21:56:42 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id EDD72181AF5C1 for ; Thu, 1 Jul 2021 01:56:41 +0000 (UTC) X-FDA: 78312355002.07.0CDE332 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id A87EEF00008E for ; Thu, 1 Jul 2021 01:56:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BC94361469; Thu, 1 Jul 2021 01:56:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104601; bh=0AhemfqD8GVqBCuMQ5lCqSm+IlfuRkwZDWpdpQtehws=; h=Date:From:To:Subject:In-Reply-To:From; b=u9+0w7cvSAdHpkR1JfrVE9fEZLYRh3Y1lfN91Oam9jqeXskFbBMozpw4NdEVS24Us s6Vie9SuZ9xuQurbrbDTawsqV5sswjsBPPpQoIDiLP3KvczFbiszu0xXaOejYteEtX M7Tx9kxC6ig+VYv7J2qE/NH60XHDXnDW5I/HCuLc= Date: Wed, 30 Jun 2021 18:56:40 -0700 From: Andrew Morton To: akpm@linux-foundation.org, axboe@kernel.dk, cccheng@synology.com, christian.brauner@ubuntu.com, jamorris@linux.microsoft.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, shepjeng@gmail.com, slava@dubeyko.com, torvalds@linux-foundation.org Subject: [patch 180/192] hfsplus: report create_date to kstat.btime Message-ID: <20210701015640.stiIQSnDi%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A87EEF00008E X-Stat-Signature: cq11zh4yjd3jd39mga3x8yqbo1zbiypd Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=u9+0w7cv; dmarc=none; spf=pass (imf16.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104601-294262 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chung-Chiang Cheng Subject: hfsplus: report create_date to kstat.btime The create_date field of inode in hfsplus is corresponding to kstat.btime and could be reported in statx. Link: https://lkml.kernel.org/r/20210416172147.8736-1-cccheng@synology.com Signed-off-by: Chung-Chiang Cheng Reviewed-by: Viacheslav Dubeyko Cc: Christian Brauner Cc: James Morris Cc: Jens Axboe Signed-off-by: Andrew Morton --- fs/hfsplus/inode.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/fs/hfsplus/inode.c~hfsplus-report-create_date-to-kstatbtime +++ a/fs/hfsplus/inode.c @@ -281,6 +281,11 @@ int hfsplus_getattr(struct user_namespac struct inode *inode = d_inode(path->dentry); struct hfsplus_inode_info *hip = HFSPLUS_I(inode); + if (request_mask & STATX_BTIME) { + stat->result_mask |= STATX_BTIME; + stat->btime = hfsp_mt2ut(hip->create_date); + } + if (inode->i_flags & S_APPEND) stat->attributes |= STATX_ATTR_APPEND; if (inode->i_flags & S_IMMUTABLE) From patchwork Thu Jul 1 01:56:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353419 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C673C11F66 for ; Thu, 1 Jul 2021 01:56:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1F1336147D for ; Thu, 1 Jul 2021 01:56:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F1336147D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9BB738D0280; Wed, 30 Jun 2021 21:56:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9922C8D0279; Wed, 30 Jun 2021 21:56:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8332E8D0280; Wed, 30 Jun 2021 21:56:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id 5EDFA8D0279 for ; Wed, 30 Jun 2021 21:56:45 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 35EF582F486D for ; Thu, 1 Jul 2021 01:56:45 +0000 (UTC) X-FDA: 78312355170.16.95E60B3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id CA48C50000A0 for ; Thu, 1 Jul 2021 01:56:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C0B0061477; Thu, 1 Jul 2021 01:56:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104604; bh=ZcPVPi6wM0ULgNDWaOEOdswcX9YivQvbpxGN7AHIwvg=; h=Date:From:To:Subject:In-Reply-To:From; b=AIUEFoC70D+0oBvVHOl3qknDkLXmxIJcZYY6UANsjEw+Rk3GMEL9iWIgXDCLabP5X ARasylhJKMOiXyAQqDIvz1DADHZuOh9juvVDllMGPNLbGM/3WPIkFC9ytZwAixpDTB 60Tnj1XQnGT+Hp3ex1S4oFDanF3sVyvU056noWtc= Date: Wed, 30 Jun 2021 18:56:43 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, natechancellor@gmail.com, ndesaulniers@google.com, oleg@redhat.com, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 181/192] x86: signal: don't do sas_ss_reset() until we are certain that sigframe won't be abandoned Message-ID: <20210701015643.SBUjrzC2l%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CA48C50000A0 X-Stat-Signature: jhiuhuq3d716uuuykwgwu8err3b6e57e Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=AIUEFoC7; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104604-315477 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Al Viro Subject: x86: signal: don't do sas_ss_reset() until we are certain that sigframe won't be abandoned Currently we handle SS_AUTODISARM as soon as we have stored the altstack settings into sigframe - that's the point when we have set the things up for eventual sigreturn to restore the old settings. And if we manage to set the sigframe up (we are not done with that yet), everything's fine. However, in case of failure we end up with sigframe-to-be abandoned and SIGSEGV force-delivered. And in that case we end up with inconsistent rules - late failures have altstack reset, early ones do not. It's trivial to get consistent behaviour - just handle SS_AUTODISARM once we have set the sigframe up and are committed to entering the handler, i.e. in signal_delivered(). Link: https://lore.kernel.org/lkml/20200404170604.GN23230@ZenIV.linux.org.uk/ Link: https://github.com/ClangBuiltLinux/linux/issues/876 Link: https://lkml.kernel.org/r/20210422230846.1756380-1-ndesaulniers@google.com Signed-off-by: Al Viro Signed-off-by: Nick Desaulniers Acked-by: Oleg Nesterov Tested-by: Nathan Chancellor Signed-off-by: Andrew Morton --- include/linux/compat.h | 2 -- include/linux/signal.h | 2 -- kernel/signal.c | 14 ++++---------- 3 files changed, 4 insertions(+), 14 deletions(-) --- a/include/linux/compat.h~x86-signal-dont-do-sas_ss_reset-until-we-are-certain-that-sigframe-wont-be-abandoned +++ a/include/linux/compat.h @@ -532,8 +532,6 @@ int __compat_save_altstack(compat_stack_ &__uss->ss_sp, label); \ unsafe_put_user(t->sas_ss_flags, &__uss->ss_flags, label); \ unsafe_put_user(t->sas_ss_size, &__uss->ss_size, label); \ - if (t->sas_ss_flags & SS_AUTODISARM) \ - sas_ss_reset(t); \ } while (0); /* --- a/include/linux/signal.h~x86-signal-dont-do-sas_ss_reset-until-we-are-certain-that-sigframe-wont-be-abandoned +++ a/include/linux/signal.h @@ -462,8 +462,6 @@ int __save_altstack(stack_t __user *, un unsafe_put_user((void __user *)t->sas_ss_sp, &__uss->ss_sp, label); \ unsafe_put_user(t->sas_ss_flags, &__uss->ss_flags, label); \ unsafe_put_user(t->sas_ss_size, &__uss->ss_size, label); \ - if (t->sas_ss_flags & SS_AUTODISARM) \ - sas_ss_reset(t); \ } while (0); #ifdef CONFIG_PROC_FS --- a/kernel/signal.c~x86-signal-dont-do-sas_ss_reset-until-we-are-certain-that-sigframe-wont-be-abandoned +++ a/kernel/signal.c @@ -2829,6 +2829,8 @@ static void signal_delivered(struct ksig if (!(ksig->ka.sa.sa_flags & SA_NODEFER)) sigaddset(&blocked, ksig->sig); set_current_blocked(&blocked); + if (current->sas_ss_flags & SS_AUTODISARM) + sas_ss_reset(current); tracehook_signal_handler(stepping); } @@ -4147,11 +4149,7 @@ int __save_altstack(stack_t __user *uss, int err = __put_user((void __user *)t->sas_ss_sp, &uss->ss_sp) | __put_user(t->sas_ss_flags, &uss->ss_flags) | __put_user(t->sas_ss_size, &uss->ss_size); - if (err) - return err; - if (t->sas_ss_flags & SS_AUTODISARM) - sas_ss_reset(t); - return 0; + return err; } #ifdef CONFIG_COMPAT @@ -4206,11 +4204,7 @@ int __compat_save_altstack(compat_stack_ &uss->ss_sp) | __put_user(t->sas_ss_flags, &uss->ss_flags) | __put_user(t->sas_ss_size, &uss->ss_size); - if (err) - return err; - if (t->sas_ss_flags & SS_AUTODISARM) - sas_ss_reset(t); - return 0; + return err; } #endif From patchwork Thu Jul 1 01:56:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB6A5C11F69 for ; Thu, 1 Jul 2021 01:56:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AD21861480 for ; Thu, 1 Jul 2021 01:56:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD21861480 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 37ADB8D0281; Wed, 30 Jun 2021 21:56:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 329488D0279; Wed, 30 Jun 2021 21:56:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CAC98D0281; Wed, 30 Jun 2021 21:56:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0078.hostedemail.com [216.40.44.78]) by kanga.kvack.org (Postfix) with ESMTP id ED3A48D0279 for ; Wed, 30 Jun 2021 21:56:47 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id C5B9E256E1 for ; Thu, 1 Jul 2021 01:56:47 +0000 (UTC) X-FDA: 78312355254.22.04ED708 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 89AB7400208A for ; Thu, 1 Jul 2021 01:56:47 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AA6E86147E; Thu, 1 Jul 2021 01:56:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104606; bh=+PhBcqb+MsvVrRS+Aj75Y1qdvvqJBHiUbaDUyHX3OPA=; h=Date:From:To:Subject:In-Reply-To:From; b=eC5NDlbU085RVBmPJyVSJBAD3yWQOfm6umfP4sJPjTT5/6KVoPLAJ9gPn5+QbXmGj X0Vwe+wr2UIOPMWC6D+MMoj/FHt/BCPjpDk62czvUNVBVPkf82r/WRw2Nw69wtUN9B zi0VmmTN5dGHcygTUoraOJm0wxPhgHwQoake45o4= Date: Wed, 30 Jun 2021 18:56:46 -0700 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 182/192] exec: remove checks in __register_bimfmt() Message-ID: <20210701015646.oDCgWS-8y%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=eC5NDlbU; spf=pass (imf18.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: sd3q6rwbyscia5k1mwge1um7nm1kfjm5 X-Rspamd-Queue-Id: 89AB7400208A X-Rspamd-Server: rspam06 X-HE-Tag: 1625104607-222808 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexey Dobriyan Subject: exec: remove checks in __register_bimfmt() Delete NULL check, all callers pass valid pointer. Delete ->load_binary check -- failure to provide hook in a custom module will be very noticeable at the very first execve call. Link: https://lkml.kernel.org/r/YK1Gy1qXaLAR+tPl@localhost.localdomain Signed-off-by: Alexey Dobriyan Signed-off-by: Andrew Morton --- fs/exec.c | 3 --- 1 file changed, 3 deletions(-) --- a/fs/exec.c~exec-remove-checks-in-__register_bimfmt +++ a/fs/exec.c @@ -84,9 +84,6 @@ static DEFINE_RWLOCK(binfmt_lock); void __register_binfmt(struct linux_binfmt * fmt, int insert) { - BUG_ON(!fmt); - if (WARN_ON(!fmt->load_binary)) - return; write_lock(&binfmt_lock); insert ? list_add(&fmt->lh, &formats) : list_add_tail(&fmt->lh, &formats); From patchwork Thu Jul 1 01:56:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE1DFC11F66 for ; Thu, 1 Jul 2021 01:56:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 69F9C6147D for ; Thu, 1 Jul 2021 01:56:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 69F9C6147D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E76A08D0282; Wed, 30 Jun 2021 21:56:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4D388D0279; Wed, 30 Jun 2021 21:56:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D14DA8D0282; Wed, 30 Jun 2021 21:56:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id A96878D0279 for ; Wed, 30 Jun 2021 21:56:51 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 88A8C256E1 for ; Thu, 1 Jul 2021 01:56:51 +0000 (UTC) X-FDA: 78312355422.09.0FC8F0D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id 31A282001709 for ; Thu, 1 Jul 2021 01:56:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CF31761477; Thu, 1 Jul 2021 01:56:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104610; bh=tVj8h0/d1Wc7454HiRxxN7FOSqp9osNr9V232lwKMJQ=; h=Date:From:To:Subject:In-Reply-To:From; b=zO1FMPi0Vdq3JJJBlwob3qb52rps/N5wx27Wg0AZM8PkD1C9czcASqoxu0Zgg7Hp0 fBXxGRiyypD5YJ8Mx9zhz6dqg0aXwTFRoPZqmeeHJGWhdjZjIFOIySJ2lh+/VpaGzB 8EYJ5fa3cwty7YPjN5fP0dyayvr/jCBDcQDB7MNo= Date: Wed, 30 Jun 2021 18:56:49 -0700 From: Andrew Morton To: akpm@linux-foundation.org, ardb@kernel.org, arnd@arndb.de, dvyukov@google.com, elver@google.com, keescook@chromium.org, linux-mm@kvack.org, luc.vanoostenryck@gmail.com, mark.rutland@arm.com, masahiroy@kernel.org, mm-commits@vger.kernel.org, nathan@kernel.org, ndesaulniers@google.com, nivedita@alum.mit.edu, ojeda@kernel.org, peterz@infradead.org, samitolvanen@google.com, torvalds@linux-foundation.org, will@kernel.org Subject: [patch 183/192] kcov: add __no_sanitize_coverage to fix noinstr for all architectures Message-ID: <20210701015649.5yaBp3d6B%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=zO1FMPi0; spf=pass (imf26.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 31A282001709 X-Stat-Signature: 6bwuzkk8qjxz9ypwcxai8fq61drm7zqp X-HE-Tag: 1625104611-393019 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Marco Elver Subject: kcov: add __no_sanitize_coverage to fix noinstr for all architectures Until now no compiler supported an attribute to disable coverage instrumentation as used by KCOV. To work around this limitation on x86, noinstr functions have their coverage instrumentation turned into nops by objtool. However, this solution doesn't scale automatically to other architectures, such as arm64, which are migrating to use the generic entry code. Clang [1] and GCC [2] have added support for the attribute recently. [1] https://github.com/llvm/llvm-project/commit/280333021e9550d80f5c1152a34e33e81df1e178 [2] https://gcc.gnu.org/git/?p=gcc.git;a=commit;h=cec4d4a6782c9bd8d071839c50a239c49caca689 The changes will appear in Clang 13 and GCC 12. Add __no_sanitize_coverage for both compilers, and add it to noinstr. Note: In the Clang case, __has_feature(coverage_sanitizer) is only true if the feature is enabled, and therefore we do not require an additional defined(CONFIG_KCOV) (like in the GCC case where __has_attribute(..) is always true) to avoid adding redundant attributes to functions if KCOV is off. That being said, compilers that support the attribute will not generate errors/warnings if the attribute is redundantly used; however, where possible let's avoid it as it reduces preprocessed code size and associated compile-time overheads. [elver@google.com: Implement __has_feature(coverage_sanitizer) in Clang] Link: https://lkml.kernel.org/r/20210527162655.3246381-1-elver@google.com [elver@google.com: add comment explaining __has_feature() in Clang] Link: https://lkml.kernel.org/r/20210527194448.3470080-1-elver@google.com Link: https://lkml.kernel.org/r/20210525175819.699786-1-elver@google.com Signed-off-by: Marco Elver Acked-by: Peter Zijlstra (Intel) Reviewed-by: Miguel Ojeda Reviewed-by: Nathan Chancellor Cc: Nick Desaulniers Cc: Kees Cook Cc: Will Deacon Cc: Ard Biesheuvel Cc: Luc Van Oostenryck Cc: Arvind Sankar Cc: Masahiro Yamada Cc: Sami Tolvanen Cc: Arnd Bergmann Cc: Dmitry Vyukov Cc: Mark Rutland Signed-off-by: Andrew Morton --- include/linux/compiler-clang.h | 17 +++++++++++++++++ include/linux/compiler-gcc.h | 6 ++++++ include/linux/compiler_types.h | 2 +- 3 files changed, 24 insertions(+), 1 deletion(-) --- a/include/linux/compiler-clang.h~kcov-add-__no_sanitize_coverage-to-fix-noinstr-for-all-architectures +++ a/include/linux/compiler-clang.h @@ -13,6 +13,12 @@ /* all clang versions usable with the kernel support KASAN ABI version 5 */ #define KASAN_ABI_VERSION 5 +/* + * Note: Checking __has_feature(*_sanitizer) is only true if the feature is + * enabled. Therefore it is not required to additionally check defined(CONFIG_*) + * to avoid adding redundant attributes in other configurations. + */ + #if __has_feature(address_sanitizer) || __has_feature(hwaddress_sanitizer) /* Emulate GCC's __SANITIZE_ADDRESS__ flag */ #define __SANITIZE_ADDRESS__ @@ -46,6 +52,17 @@ #endif /* + * Support for __has_feature(coverage_sanitizer) was added in Clang 13 together + * with no_sanitize("coverage"). Prior versions of Clang support coverage + * instrumentation, but cannot be queried for support by the preprocessor. + */ +#if __has_feature(coverage_sanitizer) +#define __no_sanitize_coverage __attribute__((no_sanitize("coverage"))) +#else +#define __no_sanitize_coverage +#endif + +/* * Not all versions of clang implement the type-generic versions * of the builtin overflow checkers. Fortunately, clang implements * __has_builtin allowing us to avoid awkward version --- a/include/linux/compiler-gcc.h~kcov-add-__no_sanitize_coverage-to-fix-noinstr-for-all-architectures +++ a/include/linux/compiler-gcc.h @@ -122,6 +122,12 @@ #define __no_sanitize_undefined #endif +#if defined(CONFIG_KCOV) && __has_attribute(__no_sanitize_coverage__) +#define __no_sanitize_coverage __attribute__((no_sanitize_coverage)) +#else +#define __no_sanitize_coverage +#endif + #if GCC_VERSION >= 50100 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1 #endif --- a/include/linux/compiler_types.h~kcov-add-__no_sanitize_coverage-to-fix-noinstr-for-all-architectures +++ a/include/linux/compiler_types.h @@ -210,7 +210,7 @@ struct ftrace_likely_data { /* Section for code which can't be instrumented at all */ #define noinstr \ noinline notrace __attribute((__section__(".noinstr.text"))) \ - __no_kcsan __no_sanitize_address + __no_kcsan __no_sanitize_address __no_sanitize_coverage #endif /* __KERNEL__ */ From patchwork Thu Jul 1 01:56:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22660C11F69 for ; Thu, 1 Jul 2021 01:56:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CEBC861477 for ; Thu, 1 Jul 2021 01:56:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CEBC861477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 54E828D0283; Wed, 30 Jun 2021 21:56:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 525238D0279; Wed, 30 Jun 2021 21:56:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C6418D0283; Wed, 30 Jun 2021 21:56:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id 1A38B8D0279 for ; Wed, 30 Jun 2021 21:56:55 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DBD1682EA68A for ; Thu, 1 Jul 2021 01:56:54 +0000 (UTC) X-FDA: 78312355548.33.EFD2A0C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 879EB1000163 for ; Thu, 1 Jul 2021 01:56:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5D45461469; Thu, 1 Jul 2021 01:56:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104613; bh=cqxbYiErgZtVYUSkM4VYkRzC/jz12CYWhGmhSkVIzSg=; h=Date:From:To:Subject:In-Reply-To:From; b=lPuzgU2YzhOouFB+dD0aFuxEjfM03CATIhbqLkDqk+/+sk0xYHsZvSL/dRoox/WQN MehTT2Wmo+SsHeTEAxAoZwAmaDhr2BL9JpMSoR9i/qFFGpQb/Pfk038/W9TjG0KH+g hrFX1+4JcsAARp0n2cDxvOPRSgLGM3FQILXkg4QQ= Date: Wed, 30 Jun 2021 18:56:53 -0700 From: Andrew Morton To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, bauerman@linux.ibm.com, dave.hansen@linux.intel.com, desnesn@linux.vnet.ibm.com, fweimer@redhat.com, linux-mm@kvack.org, linuxram@us.ibm.com, mhocko@kernel.org, mingo@kernel.org, mm-commits@vger.kernel.org, mpe@ellerman.id.au, msuchanek@suse.de, sandipan@linux.ibm.com, shuah@kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 184/192] selftests/vm/pkeys: fix alloc_random_pkey() to make it really, really random Message-ID: <20210701015653.jvoI9mpfP%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 879EB1000163 X-Stat-Signature: 5oogdo7a3bomskkj76hfj6zqsqqtub3z Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lPuzgU2Y; dmarc=none; spf=pass (imf13.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104614-428360 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen Subject: selftests/vm/pkeys: fix alloc_random_pkey() to make it really, really random Patch series "selftests/vm/pkeys: Bug fixes and a new test". There has been a lot of activity on the x86 front around the XSAVE architecture which is used to context-switch processor state (among other things). In addition, AMD has recently joined the protection keys club by adding processor support for PKU. The AMD implementation helped uncover a kernel bug around the PKRU "init state", which actually applied to Intel's implementation but was just harder to hit. This series adds a test which is expected to help find this class of bug both on AMD and Intel. All the work around pkeys on x86 also uncovered a few bugs in the selftest. This patch (of 4): The "random" pkey allocation code currently does the good old: srand((unsigned int)time(NULL)); *But*, it unfortunately does this on every random pkey allocation. There may be thousands of these a second. time() has a one second resolution. So, each time alloc_random_pkey() is called, the PRNG is *RESET* to time(). This is nasty. Normally, if you do: srand(); foo = rand(); bar = rand(); You'll be quite guaranteed that 'foo' and 'bar' are different. But, if you do: srand(1); foo = rand(); srand(1); bar = rand(); You are quite guaranteed that 'foo' and 'bar' are the *SAME*. The recent "fix" effectively forced the test case to use the same "random" pkey for the whole test, unless the test run crossed a second boundary. Only run srand() once at program startup. This explains some very odd and persistent test failures I've been seeing. Link: https://lkml.kernel.org/r/20210611164153.91B76FB8@viggo.jf.intel.com Link: https://lkml.kernel.org/r/20210611164155.192D00FF@viggo.jf.intel.com Fixes: 6e373263ce07 ("selftests/vm/pkeys: fix alloc_random_pkey() to make it really random") Signed-off-by: Dave Hansen Signed-off-by: Thomas Gleixner Tested-by: Aneesh Kumar K.V Cc: Ram Pai Cc: Sandipan Das Cc: Florian Weimer Cc: "Desnes A. Nunes do Rosario" Cc: Ingo Molnar Cc: Thiago Jung Bauermann Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Suchanek Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/protection_keys.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/tools/testing/selftests/vm/protection_keys.c~selftests-vm-pkeys-fix-alloc_random_pkey-to-make-it-really-really-random +++ a/tools/testing/selftests/vm/protection_keys.c @@ -561,7 +561,6 @@ int alloc_random_pkey(void) int nr_alloced = 0; int random_index; memset(alloced_pkeys, 0, sizeof(alloced_pkeys)); - srand((unsigned int)time(NULL)); /* allocate every possible key and make a note of which ones we got */ max_nr_pkey_allocs = NR_PKEYS; @@ -1552,6 +1551,8 @@ int main(void) int nr_iterations = 22; int pkeys_supported = is_pkeys_supported(); + srand((unsigned int)time(NULL)); + setup_handlers(); printf("has pkeys: %d\n", pkeys_supported); From patchwork Thu Jul 1 01:56:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7785BC11F68 for ; Thu, 1 Jul 2021 01:56:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2E59661469 for ; Thu, 1 Jul 2021 01:56:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E59661469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AA5C28D0284; Wed, 30 Jun 2021 21:56:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A55148D0279; Wed, 30 Jun 2021 21:56:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F6988D0284; Wed, 30 Jun 2021 21:56:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0031.hostedemail.com [216.40.44.31]) by kanga.kvack.org (Postfix) with ESMTP id 6A9568D0279 for ; Wed, 30 Jun 2021 21:56:58 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 3C54182E86BE for ; Thu, 1 Jul 2021 01:56:58 +0000 (UTC) X-FDA: 78312355716.36.A88A55E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP id F41EFB000378 for ; Thu, 1 Jul 2021 01:56:57 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B6FD661469; Thu, 1 Jul 2021 01:56:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104617; bh=tBU1UmFCF+nvvGh9zqvUY0uSqlTwG6FcC0wPmXxNgJM=; h=Date:From:To:Subject:In-Reply-To:From; b=tGoiRx1rmhdgClcaMbADGP0IHyziBJrwcjJ2mia7LY1sCL4Yt/Twj7jdvdMGHyjgA kT0GsXnabU5Jcf2YKCdsPCw8yiOaSHQ83nJMp8PqoXyZPgx+JMn8aIe/ecv4Yp91zE 7Yfe8eRjG0QI5sg29XRIHhfO3oSZalqmdpDVWQko= Date: Wed, 30 Jun 2021 18:56:56 -0700 From: Andrew Morton To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, bauerman@linux.ibm.com, dave.hansen@linux.intel.com, desnesn@linux.vnet.ibm.com, fweimer@redhat.com, linux-mm@kvack.org, linuxram@us.ibm.com, mhocko@kernel.org, mingo@kernel.org, mm-commits@vger.kernel.org, mpe@ellerman.id.au, msuchanek@suse.de, sandipan@linux.ibm.com, shuah@kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 185/192] selftests/vm/pkeys: handle negative sys_pkey_alloc() return code Message-ID: <20210701015656.FzKVF4eCF%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=tGoiRx1r; dmarc=none; spf=pass (imf25.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: jhiep1zxko9b9e77kk5ojjathwmm3ei4 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F41EFB000378 X-HE-Tag: 1625104617-164068 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen Subject: selftests/vm/pkeys: handle negative sys_pkey_alloc() return code The alloc_pkey() sefltest function wraps the sys_pkey_alloc() system call. On success, it updates its "shadow" register value because sys_pkey_alloc() updates the real register. But, the success check is wrong. pkey_alloc() considers any non-zero return code to indicate success where the pkey register will be modified. This fails to take negative return codes into account. Consider only a positive return value as a successful call. Link: https://lkml.kernel.org/r/20210611164157.87AB4246@viggo.jf.intel.com Fixes: 5f23f6d082a9 ("x86/pkeys: Add self-tests") Reported-by: Thomas Gleixner Signed-off-by: Dave Hansen Tested-by: Aneesh Kumar K.V Cc: Ram Pai Cc: Sandipan Das Cc: Florian Weimer Cc: "Desnes A. Nunes do Rosario" Cc: Ingo Molnar Cc: Thiago Jung Bauermann Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Suchanek Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/protection_keys.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/tools/testing/selftests/vm/protection_keys.c~selftests-vm-pkeys-handle-negative-sys_pkey_alloc-return-code +++ a/tools/testing/selftests/vm/protection_keys.c @@ -510,7 +510,7 @@ int alloc_pkey(void) " shadow: 0x%016llx\n", __func__, __LINE__, ret, __read_pkey_reg(), shadow_pkey_reg); - if (ret) { + if (ret > 0) { /* clear both the bits: */ shadow_pkey_reg = set_pkey_bits(shadow_pkey_reg, ret, ~PKEY_MASK); From patchwork Thu Jul 1 01:56:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F196AC11F68 for ; Thu, 1 Jul 2021 01:57:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A442D61477 for ; Thu, 1 Jul 2021 01:57:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A442D61477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B7268D0285; Wed, 30 Jun 2021 21:57:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 28CDF8D0279; Wed, 30 Jun 2021 21:57:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 155658D0285; Wed, 30 Jun 2021 21:57:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0029.hostedemail.com [216.40.44.29]) by kanga.kvack.org (Postfix) with ESMTP id E61768D0279 for ; Wed, 30 Jun 2021 21:57:01 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id BFFBB250DE for ; Thu, 1 Jul 2021 01:57:01 +0000 (UTC) X-FDA: 78312355842.29.9A42088 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id 6829C2001708 for ; Thu, 1 Jul 2021 01:57:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2E79A6146D; Thu, 1 Jul 2021 01:57:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104620; bh=PGCoSFkJ8mh+vwAjxrskYiMiUmdXj0EnVYM979+NM3k=; h=Date:From:To:Subject:In-Reply-To:From; b=ONDVul+bNZiSStzeXDogeokxFldHKJUrU18QOT0a74H7plN3oZBACG8Sm7/oVriWs 9Z7oiAE2L5nMIPntliLCCwmZ9cUHt57DHwkThW01ULJZcLA2kCZ/yE/btMArweYLVu rjjJKXIKaj0Hjkzz4NbGYSBXZgEKspuo4wBSPJSw= Date: Wed, 30 Jun 2021 18:56:59 -0700 From: Andrew Morton To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, bauerman@linux.ibm.com, dave.hansen@linux.intel.com, desnesn@linux.vnet.ibm.com, fweimer@redhat.com, linux-mm@kvack.org, linuxram@us.ibm.com, mhocko@kernel.org, mingo@kernel.org, mm-commits@vger.kernel.org, mpe@ellerman.id.au, msuchanek@suse.de, sandipan@linux.ibm.com, shuah@kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 186/192] selftests/vm/pkeys: refill shadow register after implicit kernel write Message-ID: <20210701015659.gankA7b3z%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ONDVul+b; spf=pass (imf26.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: jt94u58eza8httmb4df7d9a7eu6jrnqx X-Rspamd-Queue-Id: 6829C2001708 X-Rspamd-Server: rspam06 X-HE-Tag: 1625104621-885809 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen Subject: selftests/vm/pkeys: refill shadow register after implicit kernel write The pkey test code keeps a "shadow" of the pkey register around. This ensures that any bugs which might write to the register can be caught more quickly. Generally, userspace has a good idea when the kernel is going to write to the register. For instance, alloc_pkey() is passed a permission mask. The caller of alloc_pkey() can update the shadow based on the return value and the mask. But, the kernel can also modify the pkey register in a more sneaky way. For mprotect(PROT_EXEC) mappings, the kernel will allocate a pkey and write the pkey register to create an execute-only mapping. The kernel never tells userspace what key it uses for this. This can cause the test to fail with messages like: protection_keys_64.2: pkey-helpers.h:132: _read_pkey_reg: Assertion `pkey_reg == shadow_pkey_reg' failed. because the shadow was not updated with the new kernel-set value. Forcibly update the shadow value immediately after an mprotect(). Link: https://lkml.kernel.org/r/20210611164200.EF76AB73@viggo.jf.intel.com Fixes: 6af17cf89e99 ("x86/pkeys/selftests: Add PROT_EXEC test") Signed-off-by: Dave Hansen Signed-off-by: Thomas Gleixner Tested-by: Aneesh Kumar K.V Cc: Ram Pai Cc: Sandipan Das Cc: Florian Weimer Cc: "Desnes A. Nunes do Rosario" Cc: Ingo Molnar Cc: Thiago Jung Bauermann Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Suchanek Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/protection_keys.c | 7 +++++++ 1 file changed, 7 insertions(+) --- a/tools/testing/selftests/vm/protection_keys.c~selftests-vm-pkeys-refill-shadow-register-after-implicit-kernel-write +++ a/tools/testing/selftests/vm/protection_keys.c @@ -1448,6 +1448,13 @@ void test_implicit_mprotect_exec_only_me ret = mprotect(p1, PAGE_SIZE, PROT_EXEC); pkey_assert(!ret); + /* + * Reset the shadow, assuming that the above mprotect() + * correctly changed PKRU, but to an unknown value since + * the actual alllocated pkey is unknown. + */ + shadow_pkey_reg = __read_pkey_reg(); + dprintf2("pkey_reg: %016llx\n", read_pkey_reg()); /* Make sure this is an *instruction* fault */ From patchwork Thu Jul 1 01:57:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BC24C11F68 for ; Thu, 1 Jul 2021 01:57:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 253DA6147D for ; Thu, 1 Jul 2021 01:57:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 253DA6147D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A30978D0286; Wed, 30 Jun 2021 21:57:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A06F88D0279; Wed, 30 Jun 2021 21:57:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 881078D0286; Wed, 30 Jun 2021 21:57:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 65CBB8D0279 for ; Wed, 30 Jun 2021 21:57:05 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 3ED98181AF5C1 for ; Thu, 1 Jul 2021 01:57:05 +0000 (UTC) X-FDA: 78312356010.11.ECB87B3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf25.hostedemail.com (Postfix) with ESMTP id D13A5B000376 for ; Thu, 1 Jul 2021 01:57:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9083B61469; Thu, 1 Jul 2021 01:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104624; bh=Qc5T4Uzo85QvDvT3Mdj6zFleyJ2Y1yRjwceMxt/sHOc=; h=Date:From:To:Subject:In-Reply-To:From; b=tOl/wRxkquvr0O9VD7ICeGeh1lFwVNv7fdXUagBRn0nibjYtC1NEBNBWwtT8WrYu5 novmZhAD/ZRff++TO4Icn1OSotuYq4+4fEOyKDpN817grn9NWf7J5E0kz2hse/3AOy mGe3zatjaz7Uhma8QcIHJrHlT1J1vz7XOAoKvFK8= Date: Wed, 30 Jun 2021 18:57:03 -0700 From: Andrew Morton To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, bauerman@linux.ibm.com, dave.hansen@linux.intel.com, desnesn@linux.vnet.ibm.com, fweimer@redhat.com, linux-mm@kvack.org, linuxram@us.ibm.com, mhocko@kernel.org, mingo@kernel.org, mm-commits@vger.kernel.org, mpe@ellerman.id.au, msuchanek@suse.de, sandipan@linux.ibm.com, shuah@kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 187/192] selftests/vm/pkeys: exercise x86 XSAVE init state Message-ID: <20210701015703.jlojc0hpo%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="tOl/wRxk"; dmarc=none; spf=pass (imf25.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspamd-Server: rspam02 X-Stat-Signature: yu1rrxf7oj3bnf9codgsyw7nkzg81qgq X-Rspamd-Queue-Id: D13A5B000376 X-HE-Tag: 1625104624-512641 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen Subject: selftests/vm/pkeys: exercise x86 XSAVE init state On x86, there is a set of instructions used to save and restore register state collectively known as the XSAVE architecture. There are about a dozen different features managed with XSAVE. The protection keys register, PKRU, is one of those features. The hardware optimizes XSAVE by tracking when the state has not changed from its initial (init) state. In this case, it can avoid the cost of writing state to memory (it would usually just be a bunch of 0's). When the pkey register is 0x0 the hardware optionally choose to track the register as being in the init state (optimize away the writes). AMD CPUs do this more aggressively compared to Intel. On x86, PKRU is rarely in its (very permissive) init state. Instead, the value defaults to something very restrictive. It is not surprising that bugs have popped up in the rare cases when PKRU reaches its init state. Add a protection key selftest which gets the protection keys register into its init state in a way that should work on Intel and AMD. Then, do a bunch of pkey register reads to watch for inadvertent changes. This adds "-mxsave" to CFLAGS for all the x86 vm selftests in order to allow use of the XSAVE instruction __builtin functions. This will make the builtins available on all of the vm selftests, but is expected to be harmless. Link: https://lkml.kernel.org/r/20210611164202.1849B712@viggo.jf.intel.com Signed-off-by: Dave Hansen Signed-off-by: Thomas Gleixner Tested-by: Aneesh Kumar K.V Cc: Ram Pai Cc: Sandipan Das Cc: Florian Weimer Cc: "Desnes A. Nunes do Rosario" Cc: Ingo Molnar Cc: Thiago Jung Bauermann Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Suchanek Cc: Shuah Khan Signed-off-by: Andrew Morton --- tools/testing/selftests/vm/Makefile | 4 tools/testing/selftests/vm/pkey-x86.h | 1 tools/testing/selftests/vm/protection_keys.c | 73 +++++++++++++++++ 3 files changed, 76 insertions(+), 2 deletions(-) --- a/tools/testing/selftests/vm/Makefile~selftests-vm-pkeys-exercise-x86-xsave-init-state +++ a/tools/testing/selftests/vm/Makefile @@ -101,7 +101,7 @@ $(1) $(1)_64: $(OUTPUT)/$(1)_64 endef ifeq ($(CAN_BUILD_I386),1) -$(BINARIES_32): CFLAGS += -m32 +$(BINARIES_32): CFLAGS += -m32 -mxsave $(BINARIES_32): LDLIBS += -lrt -ldl -lm $(BINARIES_32): $(OUTPUT)/%_32: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@ @@ -109,7 +109,7 @@ $(foreach t,$(TARGETS),$(eval $(call gen endif ifeq ($(CAN_BUILD_X86_64),1) -$(BINARIES_64): CFLAGS += -m64 +$(BINARIES_64): CFLAGS += -m64 -mxsave $(BINARIES_64): LDLIBS += -lrt -ldl $(BINARIES_64): $(OUTPUT)/%_64: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@ --- a/tools/testing/selftests/vm/pkey-x86.h~selftests-vm-pkeys-exercise-x86-xsave-init-state +++ a/tools/testing/selftests/vm/pkey-x86.h @@ -126,6 +126,7 @@ static inline u32 pkey_bit_position(int #define XSTATE_PKEY_BIT (9) #define XSTATE_PKEY 0x200 +#define XSTATE_BV_OFFSET 512 int pkey_reg_xstate_offset(void) { --- a/tools/testing/selftests/vm/protection_keys.c~selftests-vm-pkeys-exercise-x86-xsave-init-state +++ a/tools/testing/selftests/vm/protection_keys.c @@ -1277,6 +1277,78 @@ void test_pkey_alloc_exhaust(int *ptr, u } } +void arch_force_pkey_reg_init(void) +{ +#if defined(__i386__) || defined(__x86_64__) /* arch */ + u64 *buf; + + /* + * All keys should be allocated and set to allow reads and + * writes, so the register should be all 0. If not, just + * skip the test. + */ + if (read_pkey_reg()) + return; + + /* + * Just allocate an absurd about of memory rather than + * doing the XSAVE size enumeration dance. + */ + buf = mmap(NULL, 1*MB, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); + + /* These __builtins require compiling with -mxsave */ + + /* XSAVE to build a valid buffer: */ + __builtin_ia32_xsave(buf, XSTATE_PKEY); + /* Clear XSTATE_BV[PKRU]: */ + buf[XSTATE_BV_OFFSET/sizeof(u64)] &= ~XSTATE_PKEY; + /* XRSTOR will likely get PKRU back to the init state: */ + __builtin_ia32_xrstor(buf, XSTATE_PKEY); + + munmap(buf, 1*MB); +#endif +} + + +/* + * This is mostly useless on ppc for now. But it will not + * hurt anything and should give some better coverage as + * a long-running test that continually checks the pkey + * register. + */ +void test_pkey_init_state(int *ptr, u16 pkey) +{ + int err; + int allocated_pkeys[NR_PKEYS] = {0}; + int nr_allocated_pkeys = 0; + int i; + + for (i = 0; i < NR_PKEYS; i++) { + int new_pkey = alloc_pkey(); + + if (new_pkey < 0) + continue; + allocated_pkeys[nr_allocated_pkeys++] = new_pkey; + } + + dprintf3("%s()::%d\n", __func__, __LINE__); + + arch_force_pkey_reg_init(); + + /* + * Loop for a bit, hoping to get exercise the kernel + * context switch code. + */ + for (i = 0; i < 1000000; i++) + read_pkey_reg(); + + for (i = 0; i < nr_allocated_pkeys; i++) { + err = sys_pkey_free(allocated_pkeys[i]); + pkey_assert(!err); + read_pkey_reg(); /* for shadow checking */ + } +} + /* * pkey 0 is special. It is allocated by default, so you do not * have to call pkey_alloc() to use it first. Make sure that it @@ -1508,6 +1580,7 @@ void (*pkey_tests[])(int *ptr, u16 pkey) test_implicit_mprotect_exec_only_memory, test_mprotect_with_pkey_0, test_ptrace_of_child, + test_pkey_init_state, test_pkey_syscalls_on_non_allocated_pkey, test_pkey_syscalls_bad_args, test_pkey_alloc_exhaust, From patchwork Thu Jul 1 01:57:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3042C11F68 for ; Thu, 1 Jul 2021 01:57:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C0DA361477 for ; Thu, 1 Jul 2021 01:57:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C0DA361477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 48B418D0287; Wed, 30 Jun 2021 21:57:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 461BE8D0279; Wed, 30 Jun 2021 21:57:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 329B48D0287; Wed, 30 Jun 2021 21:57:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0161.hostedemail.com [216.40.44.161]) by kanga.kvack.org (Postfix) with ESMTP id 0AE4D8D0279 for ; Wed, 30 Jun 2021 21:57:08 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D2073250B9 for ; Thu, 1 Jul 2021 01:57:07 +0000 (UTC) X-FDA: 78312356094.15.CFC3B31 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 937C7D000156 for ; Thu, 1 Jul 2021 01:57:07 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C06726146D; Thu, 1 Jul 2021 01:57:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104626; bh=BOl99PpQx/wtTG7qlhr4tvHWsX3BMnAJ9IrTcNiZPSE=; h=Date:From:To:Subject:In-Reply-To:From; b=vq/Cl0fUx93X5xoM/Lo/mjQyTn3zviOwTlYNWQlYPW/Jd8y2fLZyLoI64/ri4oAYC qlIxOet04/A+i0Y0OVA0EgF2unUbPPyMAuB9RM5alb2PgnLDBu1LOB/a8SWBcpLqGR k2gv/djm5uVU8+HIIosWRCEmYU96Nz/E18Ayhzjg= Date: Wed, 30 Jun 2021 18:57:06 -0700 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yukuai3@huawei.com Subject: [patch 188/192] lib/decompressors: remove set but not used variabled 'level' Message-ID: <20210701015706.U6KPx2_gJ%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 937C7D000156 Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="vq/Cl0fU"; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: hwf8fxq8q6st6nc4kufqfrtszbzumx9n X-HE-Tag: 1625104627-201719 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Kuai Subject: lib/decompressors: remove set but not used variabled 'level' Fixes gcc '-Wunused-but-set-variable' warning: lib/decompress_unlzo.c:46:5: warning: variable `level' set but not used [-Wunused-but-set-variable] It is never used and so can be removed. [akpm@linux-foundation.org: warning: value computed is not used] Link: https://lkml.kernel.org/r/20210514062050.3532344-1-yukuai3@huawei.com Fixes: 7dd65feb6c60 ("lib: add support for LZO-compressed kernels") Signed-off-by: Yu Kuai Signed-off-by: Andrew Morton --- lib/decompress_unlzo.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/lib/decompress_unlzo.c~lib-decompressors-remove-set-but-not-used-variabled-level +++ a/lib/decompress_unlzo.c @@ -43,7 +43,6 @@ STATIC inline long INIT parse_header(u8 int l; u8 *parse = input; u8 *end = input + in_len; - u8 level = 0; u16 version; /* @@ -65,7 +64,7 @@ STATIC inline long INIT parse_header(u8 version = get_unaligned_be16(parse); parse += 7; if (version >= 0x0940) - level = *parse++; + parse++; if (get_unaligned_be32(parse) & HEADER_HAS_FILTER) parse += 8; /* flags + filter info */ else From patchwork Thu Jul 1 01:57:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67F6FC11F69 for ; Thu, 1 Jul 2021 01:57:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 20A1F61477 for ; Thu, 1 Jul 2021 01:57:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 20A1F61477 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9E2BC8D0288; Wed, 30 Jun 2021 21:57:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B8B48D0279; Wed, 30 Jun 2021 21:57:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 880EC8D0288; Wed, 30 Jun 2021 21:57:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id 64C788D0279 for ; Wed, 30 Jun 2021 21:57:11 -0400 (EDT) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3FB6D250B9 for ; Thu, 1 Jul 2021 01:57:11 +0000 (UTC) X-FDA: 78312356262.18.9796478 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id EF6332001708 for ; Thu, 1 Jul 2021 01:57:10 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BE4B96147D; Thu, 1 Jul 2021 01:57:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104630; bh=XGFDavzVD7Fn/Cellz8FUzr89lBVCuEqIFceijbBztw=; h=Date:From:To:Subject:In-Reply-To:From; b=lWqafO6Qp0ArxBBS0mXccD+1JrqOLxH1NvT4tlrV7EFBZcLFahD11ipzVbs/fOUfj wcpwxBYIIJAGfZnbVg/d0K7+7lXudYfE4oHJZDMcyAzfR9pgvI+X4d6Se2WIGGMeTH TyrtdSl1erD4GTIxsMj5k0EHBYYeWj801JLSci04= Date: Wed, 30 Jun 2021 18:57:09 -0700 From: Andrew Morton To: 0x7f454c46@gmail.com, adobriyan@gmail.com, akpm@linux-foundation.org, dave@stgolabs.net, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, manfred@colorfullife.com, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com, vvs@virtuozzo.com Subject: [patch 189/192] ipc sem: use kvmalloc for sem_undo allocation Message-ID: <20210701015709.svgVwpvBh%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lWqafO6Q; dmarc=none; spf=pass (imf26.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Stat-Signature: ipa1oxnes161t9pjfaiu6apeda4maz7r X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EF6332001708 X-HE-Tag: 1625104630-630023 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vasily Averin Subject: ipc sem: use kvmalloc for sem_undo allocation Patch series "ipc: allocations cleanup", v2. Some ipc objects use the wrong allocation functions: small objects can use kmalloc(), and vice versa, potentially large objects can use kmalloc(). This patch (of 2): Size of sem_undo can exceed one page and with the maximum possible nsems = 32000 it can grow up to 64Kb. Let's switch its allocation to kvmalloc to avoid user-triggered disruptive actions like OOM killer in case of high-order memory shortage. User triggerable high order allocations are quite a problem on heavily fragmented systems. They can be a DoS vector. Link: https://lkml.kernel.org/r/ebc3ac79-3190-520d-81ce-22ad194986ec@virtuozzo.com Link: https://lkml.kernel.org/r/a6354fd9-2d55-2e63-dd4d-fa7dc1d11134@virtuozzo.com Signed-off-by: Vasily Averin Acked-by: Michal Hocko Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Cc: Alexey Dobriyan Cc: Davidlohr Bueso Cc: Dmitry Safonov <0x7f454c46@gmail.com> Cc: Johannes Weiner Cc: Manfred Spraul Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- ipc/sem.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) --- a/ipc/sem.c~ipc-sem-use-kvmalloc-for-sem_undo-allocation +++ a/ipc/sem.c @@ -1154,7 +1154,7 @@ static void freeary(struct ipc_namespace un->semid = -1; list_del_rcu(&un->list_proc); spin_unlock(&un->ulp->lock); - kfree_rcu(un, rcu); + kvfree_rcu(un, rcu); } /* Wake up all pending processes and let them fail with EIDRM. */ @@ -1937,7 +1937,8 @@ static struct sem_undo *find_alloc_undo( rcu_read_unlock(); /* step 2: allocate new undo structure */ - new = kzalloc(sizeof(struct sem_undo) + sizeof(short)*nsems, GFP_KERNEL); + new = kvzalloc(sizeof(struct sem_undo) + sizeof(short)*nsems, + GFP_KERNEL); if (!new) { ipc_rcu_putref(&sma->sem_perm, sem_rcu_free); return ERR_PTR(-ENOMEM); @@ -1949,7 +1950,7 @@ static struct sem_undo *find_alloc_undo( if (!ipc_valid_object(&sma->sem_perm)) { sem_unlock(sma, -1); rcu_read_unlock(); - kfree(new); + kvfree(new); un = ERR_PTR(-EIDRM); goto out; } @@ -1960,7 +1961,7 @@ static struct sem_undo *find_alloc_undo( */ un = lookup_undo(ulp, semid); if (un) { - kfree(new); + kvfree(new); goto success; } /* step 5: initialize & link new undo structure */ @@ -2420,7 +2421,7 @@ void exit_sem(struct task_struct *tsk) rcu_read_unlock(); wake_up_q(&wake_q); - kfree_rcu(un, rcu); + kvfree_rcu(un, rcu); } kfree(ulp); } From patchwork Thu Jul 1 01:57:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A46EEC11F6B for ; Thu, 1 Jul 2021 01:57:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5D38161469 for ; Thu, 1 Jul 2021 01:57:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D38161469 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D64A68D0289; Wed, 30 Jun 2021 21:57:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D14DF8D0279; Wed, 30 Jun 2021 21:57:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB5888D0289; Wed, 30 Jun 2021 21:57:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0036.hostedemail.com [216.40.44.36]) by kanga.kvack.org (Postfix) with ESMTP id 980B48D0279 for ; Wed, 30 Jun 2021 21:57:14 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 716641815A82C for ; Thu, 1 Jul 2021 01:57:14 +0000 (UTC) X-FDA: 78312356388.26.E57AFCA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 29D875000097 for ; Thu, 1 Jul 2021 01:57:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0D1AA6146D; Thu, 1 Jul 2021 01:57:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104633; bh=RWMikqVNe9GFFLBjT8gPp+EoB8SsLkS8okjU115bTw4=; h=Date:From:To:Subject:In-Reply-To:From; b=JJARmUVilyqU4Npuvw+S5ouovK55jFEMbjSYnAzOMjgxPcXlO+IbLjwu6SpoA3I+T 11GsT4MViEzUeTe1q6LICxJ3VsJJQoAwmhKhpa8MVGq6xVu3vQ9qC/5uNl/yQ0AVsY BxG3HBa0NkuOdZ0RnOpdaCHbo3E0Mhaurn145XTI= Date: Wed, 30 Jun 2021 18:57:12 -0700 From: Andrew Morton To: 0x7f454c46@gmail.com, adobriyan@gmail.com, akpm@linux-foundation.org, dave@stgolabs.net, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, manfred@colorfullife.com, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com, vvs@virtuozzo.com Subject: [patch 190/192] ipc: use kmalloc for msg_queue and shmid_kernel Message-ID: <20210701015712.SmE6S0wIN%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 29D875000097 X-Stat-Signature: bjndggqw7cwde9a13ime8a4b3oo7fnh1 Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=JJARmUVi; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-HE-Tag: 1625104634-15267 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vasily Averin Subject: ipc: use kmalloc for msg_queue and shmid_kernel msg_queue and shmid_kernel are quite small objects, no need to use kvmalloc for them. mhocko@: "Both of them are 256B on most 64b systems." Previously these objects was allocated via ipc_alloc/ipc_rcu_alloc(), common function for several ipc objects. It had kvmalloc call inside(). Later, this function went away and was finally replaced by direct kvmalloc call, and now we can use more suitable kmalloc/kfree for them. Link: https://lkml.kernel.org/r/0d0b6c9b-8af3-29d8-34e2-a565c53780f3@virtuozzo.com Reported-by: Alexey Dobriyan Signed-off-by: Vasily Averin Acked-by: Michal Hocko Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Cc: Johannes Weiner Cc: Vladimir Davydov Cc: Dmitry Safonov <0x7f454c46@gmail.com> Cc: Davidlohr Bueso Cc: Manfred Spraul Signed-off-by: Andrew Morton --- ipc/msg.c | 6 +++--- ipc/shm.c | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) --- a/ipc/msg.c~ipc-use-kmalloc-for-msg_queue-and-shmid_kernel +++ a/ipc/msg.c @@ -130,7 +130,7 @@ static void msg_rcu_free(struct rcu_head struct msg_queue *msq = container_of(p, struct msg_queue, q_perm); security_msg_queue_free(&msq->q_perm); - kvfree(msq); + kfree(msq); } /** @@ -147,7 +147,7 @@ static int newque(struct ipc_namespace * key_t key = params->key; int msgflg = params->flg; - msq = kvmalloc(sizeof(*msq), GFP_KERNEL); + msq = kmalloc(sizeof(*msq), GFP_KERNEL); if (unlikely(!msq)) return -ENOMEM; @@ -157,7 +157,7 @@ static int newque(struct ipc_namespace * msq->q_perm.security = NULL; retval = security_msg_queue_alloc(&msq->q_perm); if (retval) { - kvfree(msq); + kfree(msq); return retval; } --- a/ipc/shm.c~ipc-use-kmalloc-for-msg_queue-and-shmid_kernel +++ a/ipc/shm.c @@ -222,7 +222,7 @@ static void shm_rcu_free(struct rcu_head struct shmid_kernel *shp = container_of(ptr, struct shmid_kernel, shm_perm); security_shm_free(&shp->shm_perm); - kvfree(shp); + kfree(shp); } static inline void shm_rmid(struct ipc_namespace *ns, struct shmid_kernel *s) @@ -619,7 +619,7 @@ static int newseg(struct ipc_namespace * ns->shm_tot + numpages > ns->shm_ctlall) return -ENOSPC; - shp = kvmalloc(sizeof(*shp), GFP_KERNEL); + shp = kmalloc(sizeof(*shp), GFP_KERNEL); if (unlikely(!shp)) return -ENOMEM; @@ -630,7 +630,7 @@ static int newseg(struct ipc_namespace * shp->shm_perm.security = NULL; error = security_shm_alloc(&shp->shm_perm); if (error) { - kvfree(shp); + kfree(shp); return error; } From patchwork Thu Jul 1 01:57:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0C4CC11F66 for ; Thu, 1 Jul 2021 01:57:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6BC606141A for ; Thu, 1 Jul 2021 01:57:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6BC606141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E9D228D028A; Wed, 30 Jun 2021 21:57:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E73988D0279; Wed, 30 Jun 2021 21:57:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3BE68D028A; Wed, 30 Jun 2021 21:57:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0231.hostedemail.com [216.40.44.231]) by kanga.kvack.org (Postfix) with ESMTP id B01748D0279 for ; Wed, 30 Jun 2021 21:57:17 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 8888F25F28 for ; Thu, 1 Jul 2021 01:57:17 +0000 (UTC) X-FDA: 78312356514.02.3971F7F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 35FE130000A6 for ; Thu, 1 Jul 2021 01:57:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3308A61477; Thu, 1 Jul 2021 01:57:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104636; bh=vtxtVvkPVJotapYAALkueXUn29ps2ZzIARVyW405+hk=; h=Date:From:To:Subject:In-Reply-To:From; b=V8c6JxduSahh9vyN6RGNDt7QRS+2RMpCHXiq5qB+iPvpKJj4t+pjy7sX9cDbY7QCO kJh9uV1YvFfUDhxdtgHz68SyZGLUTfPO2Tc33m6UAuEf4eo2jPJ8aoRCHOf2vAB4zw chEC5klLfdpoPMJMIS9PVwf5KbSugOETBaObOCpA= Date: Wed, 30 Jun 2021 18:57:15 -0700 From: Andrew Morton To: 1vier1@web.de, akpm@linux-foundation.org, dbueso@suse.de, linux-mm@kvack.org, manfred@colorfullife.com, mm-commits@vger.kernel.org, paulmck@kernel.org, torvalds@linux-foundation.org Subject: [patch 191/192] ipc/sem.c: use READ_ONCE()/WRITE_ONCE() for use_global_lock Message-ID: <20210701015715.AOCgzEHZo%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 35FE130000A6 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=V8c6Jxdu; spf=pass (imf03.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Stat-Signature: a4y1j7t4ese7j87do3nw1h4ywmy1q8zw X-HE-Tag: 1625104637-964066 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Manfred Spraul Subject: ipc/sem.c: use READ_ONCE()/WRITE_ONCE() for use_global_lock The patch solves three weaknesses in ipc/sem.c: 1) The initial read of use_global_lock in sem_lock() is an intentional race. KCSAN detects these accesses and prints a warning. 2) The code assumes that plain C read/writes are not mangled by the CPU or the compiler. 3) The comment it sysvipc_sem_proc_show() was hard to understand: The rest of the comments in ipc/sem.c speaks about sem_perm.lock, and suddenly this function speaks about ipc_lock_object(). To solve 1) and 2), use READ_ONCE()/WRITE_ONCE(). Plain C reads are used in code that owns sma->sem_perm.lock. The comment is updated to solve 3) [manfred@colorfullife.com: use READ_ONCE()/WRITE_ONCE() for use_global_lock] Link: https://lkml.kernel.org/r/20210627161919.3196-3-manfred@colorfullife.com Link: https://lkml.kernel.org/r/20210514175319.12195-1-manfred@colorfullife.com Signed-off-by: Manfred Spraul Reviewed-by: Paul E. McKenney Reviewed-by: Davidlohr Bueso Cc: <1vier1@web.de> Signed-off-by: Andrew Morton --- ipc/sem.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) --- a/ipc/sem.c~ipc-semc-use-read_once-write_once-for-use_global_lock +++ a/ipc/sem.c @@ -217,6 +217,8 @@ static int sysvipc_sem_proc_show(struct * this smp_load_acquire(), this is guaranteed because the smp_load_acquire() * is inside a spin_lock() and after a write from 0 to non-zero a * spin_lock()+spin_unlock() is done. + * To prevent the compiler/cpu temporarily writing 0 to use_global_lock, + * READ_ONCE()/WRITE_ONCE() is used. * * 2) queue.status: (SEM_BARRIER_2) * Initialization is done while holding sem_lock(), so no further barrier is @@ -342,10 +344,10 @@ static void complexmode_enter(struct sem * Nothing to do, just reset the * counter until we return to simple mode. */ - sma->use_global_lock = USE_GLOBAL_LOCK_HYSTERESIS; + WRITE_ONCE(sma->use_global_lock, USE_GLOBAL_LOCK_HYSTERESIS); return; } - sma->use_global_lock = USE_GLOBAL_LOCK_HYSTERESIS; + WRITE_ONCE(sma->use_global_lock, USE_GLOBAL_LOCK_HYSTERESIS); for (i = 0; i < sma->sem_nsems; i++) { sem = &sma->sems[i]; @@ -371,7 +373,8 @@ static void complexmode_tryleave(struct /* See SEM_BARRIER_1 for purpose/pairing */ smp_store_release(&sma->use_global_lock, 0); } else { - sma->use_global_lock--; + WRITE_ONCE(sma->use_global_lock, + sma->use_global_lock-1); } } @@ -412,7 +415,7 @@ static inline int sem_lock(struct sem_ar * Initial check for use_global_lock. Just an optimization, * no locking, no memory barrier. */ - if (!sma->use_global_lock) { + if (!READ_ONCE(sma->use_global_lock)) { /* * It appears that no complex operation is around. * Acquire the per-semaphore lock. @@ -2436,7 +2439,8 @@ static int sysvipc_sem_proc_show(struct /* * The proc interface isn't aware of sem_lock(), it calls - * ipc_lock_object() directly (in sysvipc_find_ipc). + * ipc_lock_object(), i.e. spin_lock(&sma->sem_perm.lock). + * (in sysvipc_find_ipc) * In order to stay compatible with sem_lock(), we must * enter / leave complex_mode. */ From patchwork Thu Jul 1 01:57:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12353441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AE16C11F66 for ; Thu, 1 Jul 2021 01:57:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 529256141A for ; Thu, 1 Jul 2021 01:57:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 529256141A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D16158D028B; Wed, 30 Jun 2021 21:57:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC5C18D0279; Wed, 30 Jun 2021 21:57:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B73618D028B; Wed, 30 Jun 2021 21:57:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id 919C88D0279 for ; Wed, 30 Jun 2021 21:57:20 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6341A82F4878 for ; Thu, 1 Jul 2021 01:57:20 +0000 (UTC) X-FDA: 78312356640.01.9DCBECB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 0D199700009E for ; Thu, 1 Jul 2021 01:57:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2743C61483; Thu, 1 Jul 2021 01:57:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1625104639; bh=38fiufiaI4VV8d67qzuB5qatPPBfMY/7Tvd0MK24Tac=; h=Date:From:To:Subject:In-Reply-To:From; b=ADVUHgp2A7P+2CDoykIjkDvvuNw7cGR5QDFtCyugTcuzDZArBRZBcUTdt4hFMS6Xh lCwQob9rjWevjGjHjs+l+gPAzWrtQDJBBAwtcsVD6LmjPYyWeZ5wyLILJczSpgmUM3 O1GmCFy+rW81zTgJX4OyUrhd0OgbAgb3S20Gm4CQ= Date: Wed, 30 Jun 2021 18:57:18 -0700 From: Andrew Morton To: 1vier1@web.de, akpm@linux-foundation.org, dbueso@suse.de, linux-mm@kvack.org, manfred@colorfullife.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 192/192] ipc/util.c: use binary search for max_idx Message-ID: <20210701015718.0DUvPUD_l%akpm@linux-foundation.org> In-Reply-To: <20210630184624.9ca1937310b0dd5ce66b30e7@linux-foundation.org> User-Agent: s-nail v14.8.16 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=ADVUHgp2; spf=pass (imf27.hostedemail.com: domain of akpm@linux-foundation.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 0D199700009E X-Stat-Signature: tzeefihmaremmejttujxyomzo7i8xjit X-HE-Tag: 1625104639-501718 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Manfred Spraul Subject: ipc/util.c: use binary search for max_idx If semctl(), msgctl() and shmctl() are called with IPC_INFO, SEM_INFO, MSG_INFO or SHM_INFO, then the return value is the index of the highest used index in the kernel's internal array recording information about all SysV objects of the requested type for the current namespace. (This information can be used with repeated ..._STAT or ..._STAT_ANY operations to obtain information about all SysV objects on the system.) There is a cache for this value. But when the cache needs up be updated, then the highest used index is determined by looping over all possible values. With the introduction of IPCMNI_EXTEND_SHIFT, this could be a loop over 16 million entries. And due to /proc/sys/kernel/*next_id, the index values do not need to be consecutive. With , msgget(), msgctl(,IPC_RMID) in a loop, I have observed a performance increase of around factor 13000. As there is no get_last() function for idr structures: Implement a "get_last()" using a binary search. As far as I see, ipc is the only user that needs get_last(), thus implement it in ipc/util.c and not in a central location. [akpm@linux-foundation.org: tweak comment, fix typo] Link: https://lkml.kernel.org/r/20210425075208.11777-2-manfred@colorfullife.com Signed-off-by: Manfred Spraul Acked-by: Davidlohr Bueso Cc: <1vier1@web.de> Signed-off-by: Andrew Morton --- ipc/util.c | 44 +++++++++++++++++++++++++++++++++++++++----- ipc/util.h | 3 +++ 2 files changed, 42 insertions(+), 5 deletions(-) --- a/ipc/util.c~ipc-utilc-use-binary-search-for-max_idx +++ a/ipc/util.c @@ -64,6 +64,7 @@ #include #include #include +#include #include @@ -451,6 +452,41 @@ static void ipc_kht_remove(struct ipc_id } /** + * ipc_search_maxidx - search for the highest assigned index + * @ids: ipc identifier set + * @limit: known upper limit for highest assigned index + * + * The function determines the highest assigned index in @ids. It is intended + * to be called when ids->max_idx needs to be updated. + * Updating ids->max_idx is necessary when the current highest index ipc + * object is deleted. + * If no ipc object is allocated, then -1 is returned. + * + * ipc_ids.rwsem needs to be held by the caller. + */ +static int ipc_search_maxidx(struct ipc_ids *ids, int limit) +{ + int tmpidx; + int i; + int retval; + + i = ilog2(limit+1); + + retval = 0; + for (; i >= 0; i--) { + tmpidx = retval | (1<ipcs_idr, &tmpidx)) + retval |= (1<deleted = true; if (unlikely(idx == ids->max_idx)) { - do { - idx--; - if (idx == -1) - break; - } while (!idr_find(&ids->ipcs_idr, idx)); + idx = ids->max_idx-1; + if (idx >= 0) + idx = ipc_search_maxidx(ids, idx); ids->max_idx = idx; } } --- a/ipc/util.h~ipc-utilc-use-binary-search-for-max_idx +++ a/ipc/util.h @@ -145,6 +145,9 @@ int ipcperms(struct ipc_namespace *ns, s * ipc_get_maxidx - get the highest assigned index * @ids: ipc identifier set * + * The function returns the highest assigned index for @ids. The function + * doesn't scan the idr tree, it uses a cached value. + * * Called with ipc_ids.rwsem held for reading. */ static inline int ipc_get_maxidx(struct ipc_ids *ids)