[165/165] mm/gup: remove task_struct pointer for all gup code

Message ID	20200812013901.GOAJWGS3d%akpm@linux-foundation.org (mailing list archive)
State	New, archived
Headers	show Return-Path: <SRS0=d60w=BW=kvack.org=owner-linux-mm@kernel.org> DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2AF582054F Date: Tue, 11 Aug 2020 18:39:01 -0700 From: Andrew Morton <akpm@linux-foundation.org> To: akpm@linux-foundation.org, jhubbard@nvidia.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, peterx@redhat.com, torvalds@linux-foundation.org Subject: [patch 165/165] mm/gup: remove task_struct pointer for all gup code Message-ID: <20200812013901.GOAJWGS3d%akpm@linux-foundation.org> In-Reply-To: <20200811182949.e12ae9a472e3b5e27e16ad6c@linux-foundation.org> User-Agent: s-nail v14.8.16 Sender: owner-linux-mm@kvack.org Precedence: bulk
Series	[001/165] percpu: return number of released bytes from pcpu_free_area() \| expand [001/165] percpu: return number of released bytes from pcpu_free_area() [002/165] mm: memcg/percpu: account percpu memory to memory cgroups [003/165] mm: memcg/percpu: per-memcg percpu memory statistics [004/165] mm: memcg: charge memcg percpu memory to the parent cgroup [006/165] mm/hugetlb: add mempolicy check in the reservation routine [007/165] mm/vmscan: make active/inactive ratio as 1:1 for anon lru [008/165] mm/vmscan: protect the workingset on anonymous LRU [009/165] mm/workingset: prepare the workingset detection infrastructure for anon LRU [010/165] mm/swapcache: support to handle the shadow entries [011/165] mm/swap: implement workingset detection for anonymous LRU [012/165] mm/vmscan: restore active/inactive ratio for anonymous LRU [013/165] /proc/PID/smaps: consistent whitespace output format [014/165] mm: proactive compaction [015/165] mm: fix compile error due to COMPACTION_HPAGE_ORDER [016/165] mm: use unsigned types for fragmentation score [017/165] mm/compaction: correct the comments of compact_defer_shift [018/165] mm: mempolicy: fix kerneldoc of numa_map_to_online_node() [019/165] mm/mempolicy.c: check parameters first in kernel_get_mempolicy [020/165] include/linux/mempolicy.h: fix typo [021/165] mm, oom: make the calculation of oom badness more accurate [022/165] doc, mm: sync up oom_score_adj documentation [023/165] doc, mm: clarify /proc/<pid>/oom_score value range [024/165] mm, oom: show process exiting information in __oom_kill_process() [025/165] hugetlbfs: prevent filesystem stacking of hugetlbfs [026/165] hugetlbfs: remove call to huge_pte_alloc without i_mmap_rwsem [027/165] mm/migrate: optimize migrate_vma_setup() for holes [028/165] mm/migrate: add migrate-shared test for migrate_vma_() [029/165] mm: thp: remove debug_cow switch [030/165] mm/vmstat: add events for THP migration without split [031/165] mm/cma.c: fix NULL pointer dereference when cma could not be activated [032/165] mm: cma: fix the name of CMA areas [033/165] mm: hugetlb: fix the name of hugetlb CMA [034/165] cma: don't quit at first error when activating reserved areas [035/165] include/linux/sched/mm.h: optimize current_gfp_context() [036/165] mm: mmu_notifier: fix and extend kerneldoc [037/165] x86/mm: use max memory block size on bare metal [038/165] mm/memory_hotplug: introduce default dummy memory_add_physaddr_to_nid() [039/165] mm/memory_hotplug: fix unpaired mem_hotplug_begin/done [040/165] mm, memory_hotplug: update pcp lists everytime onlining a memory block [041/165] mm: drop duplicated words in <linux/pgtable.h> [042/165] mm: drop duplicated words in <linux/mm.h> [043/165] include/linux/highmem.h: fix duplicated words in a comment [044/165] include/linux/frontswap.h: drop duplicated word in a comment [045/165] include/linux/memcontrol.h: drop duplicate word and fix spello [046/165] sh/mm: drop unused MAX_PHYSADDR_BITS [047/165] sparc: drop unused MAX_PHYSADDR_BITS [048/165] mm/compaction.c: delete duplicated word [049/165] mm/filemap.c: delete duplicated word [050/165] mm/hmm.c: delete duplicated word [051/165] mm/hugetlb.c: delete duplicated words [052/165] mm/memcontrol.c: delete duplicated words [053/165] mm/memory.c: delete duplicated words [054/165] mm/migrate.c: delete duplicated word [055/165] mm/nommu.c: delete duplicated words [056/165] mm/page_alloc.c: delete or fix duplicated words [057/165] mm/shmem.c: delete duplicated word [058/165] mm/slab_common.c: delete duplicated word [059/165] mm/usercopy.c: delete duplicated word [060/165] mm/vmscan.c: delete or fix duplicated words [061/165] mm/zpool.c: delete duplicated word and fix grammar [062/165] mm/zsmalloc.c: fix duplicated words [063/165] syscalls: use uaccess_kernel in addr_limit_user_check [064/165] nds32: use uaccess_kernel in show_regs [065/165] riscv: include <asm/pgtable.h> in <asm/uaccess.h> [066/165] uaccess: remove segment_eq [067/165] uaccess: add force_uaccess_{begin,end} helpers [068/165] exec: use force_uaccess_begin during exec and exit [069/165] alpha: fix annotation of io{read,write}{16,32}be() [070/165] include/linux/compiler-clang.h: drop duplicated word in a comment [071/165] include/linux/exportfs.h: drop duplicated word in a comment [072/165] include/linux/async_tx.h: drop duplicated word in a comment [073/165] include/linux/xz.h: drop duplicated word [074/165] kernel: add a kernel_wait helper [075/165] ./Makefile: add debug option to enable function aligned on 32 bytes [076/165] kernel.h: remove duplicate include of asm/div64.h [077/165] include/: replace HTTP links with HTTPS ones [078/165] include/linux/poison.h: remove obsolete comment [079/165] sparse: group the defines by functionality [080/165] lib/bitmap.c: fix bitmap_cut() for partial overlapping case [081/165] lib/test_bitmap.c: add test for bitmap_cut() [082/165] lib/generic-radix-tree.c: remove unneeded __rcu [083/165] lib/test_bitops: do the full test during module init [084/165] lib/test_lockup.c: make symbol 'test_works' static [085/165] lib/Kconfig.debug: make TEST_LOCKUP depend on module [086/165] lib/test_lockup.c: fix return value of test_lockup_init() [087/165] lib/: replace HTTP links with HTTPS ones [088/165] kstrto: correct documentation references to simple_strto() [089/165] kstrto: do not describe simple_strto*() as obsolete/replaced [090/165] lz4: fix kernel decompression speed [091/165] lib/test_bits.c: add tests of GENMASK [092/165] checkpatch: add test for possible misuse of IS_ENABLED() without CONFIG_ [093/165] checkpatch: add --fix option for ASSIGN_IN_IF [094/165] checkpatch: fix CONST_STRUCT when const_structs.checkpatch is missing [095/165] checkpatch: add test for repeated words [096/165] checkpatch: remove missing switch/case break test [097/165] autofs: fix doubled word [098/165] fs/minix: check return value of sb_getblk() [099/165] fs/minix: don't allow getting deleted inodes [100/165] fs/minix: reject too-large maximum file size [101/165] fs/minix: set s_maxbytes correctly [102/165] fs/minix: fix block limit check for V1 filesystems [103/165] fs/minix: remove expected error message in block_to_path() [104/165] nilfs2: only call unlock_new_inode() if I_NEW [105/165] nilfs2: convert __nilfs_msg to integrate the level and format [106/165] nilfs2: use a more common logging style [107/165] fs/ufs: avoid potential u32 multiplication overflow [108/165] fatfs: switch write_lock to read_lock in fat_ioctl_get_attributes [109/165] VFAT/FAT/MSDOS FILESYSTEM: replace HTTP links with HTTPS ones [110/165] fat: fix fat_ra_init() for data clusters == 0 [111/165] fs/signalfd.c: fix inconsistent return codes for signalfd4 [112/165] selftests: kmod: use variable NAME in kmod_test_0001() [113/165] kmod: remove redundant "be an" in the comment [114/165] test_kmod: avoid potential double free in trigger_config_run_type() [115/165] coredump: add %f for executable filename [116/165] exec: change uselib(2) IS_SREG() failure to EACCES [117/165] exec: move S_ISREG() check earlier [118/165] exec: move path_noexec() check earlier [119/165] kdump: append kernel build-id string to VMCOREINFO [120/165] drivers/rapidio/devices/rio_mport_cdev.c: use struct_size() helper [121/165] drivers/rapidio/rio-scan.c: use struct_size() helper [122/165] rapidio/rio_mport_cdev: use array_size() helper in copy_{from,to}_user() [123/165] kernel/panic.c: make oops_may_print() return bool [124/165] lib/Kconfig.debug: fix typo in the help text of CONFIG_PANIC_TIMEOUT [125/165] panic: make print_oops_end_marker() static [126/165] kcov: unconditionally add -fno-stack-protector to compiler options [127/165] kcov: make some symbols static [128/165] scripts/gdb: fix python 3.8 SyntaxWarning [129/165] ipc: uninline functions [130/165] ipc/shm.c: remove the superfluous break [131/165] mm/page_isolation: prefer the node of the source page [132/165] mm/migrate: move migration helper from .h to .c [133/165] mm/hugetlb: unify migration callbacks [134/165] mm/migrate: clear __GFP_RECLAIM to make the migration callback consistent with regular TH… [135/165] mm/migrate: introduce a standard migration target allocation function [136/165] mm/mempolicy: use a standard migration target allocation callback [137/165] mm/page_alloc: remove a wrapper for alloc_migration_target() [138/165] mm/gup: restrict CMA region by using allocation scope API [139/165] mm/hugetlb: make hugetlb migration callback CMA aware [140/165] mm/gup: use a standard migration target allocation callback [141/165] mm: do page fault accounting in handle_mm_fault [142/165] mm/alpha: use general page fault accounting [143/165] mm/arc: use general page fault accounting [144/165] mm/arm: use general page fault accounting [145/165] mm/arm64: use general page fault accounting [146/165] mm/csky: use general page fault accounting [147/165] mm/hexagon: use general page fault accounting [148/165] mm/ia64: use general page fault accounting [149/165] mm/m68k: use general page fault accounting [150/165] mm/microblaze: use general page fault accounting [151/165] mm/mips: use general page fault accounting [152/165] mm/nds32: use general page fault accounting [153/165] mm/nios2: use general page fault accounting [154/165] mm/openrisc: use general page fault accounting [155/165] mm/parisc: use general page fault accounting [156/165] mm/powerpc: use general page fault accounting [157/165] mm/riscv: use general page fault accounting [158/165] mm/s390: use general page fault accounting [159/165] mm/sh: use general page fault accounting [160/165] mm/sparc32: use general page fault accounting [161/165] mm/sparc64: use general page fault accounting [162/165] mm/x86: use general page fault accounting [163/165] mm/xtensa: use general page fault accounting [164/165] mm: clean up the last pieces of page fault accountings [165/165] mm/gup: remove task_struct pointer for all gup code

--- a/arch/arc/kernel/process.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/arch/arc/kernel/process.c @@ -91,7 +91,7 @@ fault: goto fail; mmap_read_lock(current->mm); - ret = fixup_user_fault(current, current->mm, (unsigned long) uaddr, + ret = fixup_user_fault(current->mm, (unsigned long) uaddr, FAULT_FLAG_WRITE, NULL); mmap_read_unlock(current->mm); --- a/arch/s390/kvm/interrupt.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/arch/s390/kvm/interrupt.c @@ -2768,7 +2768,7 @@ static struct page *get_map_page(struct struct page *page = NULL; mmap_read_lock(kvm->mm); - get_user_pages_remote(NULL, kvm->mm, uaddr, 1, FOLL_WRITE, + get_user_pages_remote(kvm->mm, uaddr, 1, FOLL_WRITE, &page, NULL, NULL); mmap_read_unlock(kvm->mm); return page; --- a/arch/s390/kvm/kvm-s390.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/arch/s390/kvm/kvm-s390.c @@ -1892,7 +1892,7 @@ static long kvm_s390_set_skeys(struct kv r = set_guest_storage_key(current->mm, hva, keys[i], 0); if (r) { - r = fixup_user_fault(current, current->mm, hva, + r = fixup_user_fault(current->mm, hva, FAULT_FLAG_WRITE, &unlocked); if (r) break; --- a/arch/s390/kvm/priv.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/arch/s390/kvm/priv.c @@ -273,7 +273,7 @@ retry: rc = get_guest_storage_key(current->mm, vmaddr, &key); if (rc) { - rc = fixup_user_fault(current, current->mm, vmaddr, + rc = fixup_user_fault(current->mm, vmaddr, FAULT_FLAG_WRITE, &unlocked); if (!rc) { mmap_read_unlock(current->mm); @@ -319,7 +319,7 @@ retry: mmap_read_lock(current->mm); rc = reset_guest_reference_bit(current->mm, vmaddr); if (rc < 0) { - rc = fixup_user_fault(current, current->mm, vmaddr, + rc = fixup_user_fault(current->mm, vmaddr, FAULT_FLAG_WRITE, &unlocked); if (!rc) { mmap_read_unlock(current->mm); @@ -390,7 +390,7 @@ static int handle_sske(struct kvm_vcpu * m3 & SSKE_MC); if (rc < 0) { - rc = fixup_user_fault(current, current->mm, vmaddr, + rc = fixup_user_fault(current->mm, vmaddr, FAULT_FLAG_WRITE, &unlocked); rc = !rc ? -EAGAIN : rc; } @@ -1094,7 +1094,7 @@ static int handle_pfmf(struct kvm_vcpu * rc = cond_set_guest_storage_key(current->mm, vmaddr, key, NULL, nq, mr, mc); if (rc < 0) { - rc = fixup_user_fault(current, current->mm, vmaddr, + rc = fixup_user_fault(current->mm, vmaddr, FAULT_FLAG_WRITE, &unlocked); rc = !rc ? -EAGAIN : rc; } --- a/arch/s390/mm/gmap.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/arch/s390/mm/gmap.c @@ -649,7 +649,7 @@ retry: rc = vmaddr; goto out_up; } - if (fixup_user_fault(current, gmap->mm, vmaddr, fault_flags, + if (fixup_user_fault(gmap->mm, vmaddr, fault_flags, &unlocked)) { rc = -EFAULT; goto out_up; @@ -879,7 +879,7 @@ static int gmap_pte_op_fixup(struct gmap BUG_ON(gmap_is_shadow(gmap)); fault_flags = (prot == PROT_WRITE) ? FAULT_FLAG_WRITE : 0; - if (fixup_user_fault(current, mm, vmaddr, fault_flags, &unlocked)) + if (fixup_user_fault(mm, vmaddr, fault_flags, &unlocked)) return -EFAULT; if (unlocked) /* lost mmap_lock, caller has to retry __gmap_translate */ --- a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/drivers/gpu/drm/i915/gem/i915_gem_userptr.c @@ -469,7 +469,7 @@ __i915_gem_userptr_get_pages_worker(stru locked = 1; } ret = pin_user_pages_remote - (work->task, mm, + (mm, obj->userptr.ptr + pinned * PAGE_SIZE, npages - pinned, flags, --- a/drivers/infiniband/core/umem_odp.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/drivers/infiniband/core/umem_odp.c @@ -439,7 +439,7 @@ int ib_umem_odp_map_dma_pages(struct ib_ * complex (and doesn't gain us much performance in most use * cases). */ - npages = get_user_pages_remote(owning_process, owning_mm, + npages = get_user_pages_remote(owning_mm, user_virt, gup_num_pages, flags, local_page_list, NULL, NULL); mmap_read_unlock(owning_mm); --- a/drivers/vfio/vfio_iommu_type1.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/drivers/vfio/vfio_iommu_type1.c @@ -425,7 +425,7 @@ static int follow_fault_pfn(struct vm_ar if (ret) { bool unlocked = false; - ret = fixup_user_fault(NULL, mm, vaddr, + ret = fixup_user_fault(mm, vaddr, FAULT_FLAG_REMOTE | (write_fault ? FAULT_FLAG_WRITE : 0), &unlocked); @@ -453,7 +453,7 @@ static int vaddr_get_pfn(struct mm_struc flags |= FOLL_WRITE; mmap_read_lock(mm); - ret = pin_user_pages_remote(NULL, mm, vaddr, 1, flags | FOLL_LONGTERM, + ret = pin_user_pages_remote(mm, vaddr, 1, flags | FOLL_LONGTERM, page, NULL, NULL); if (ret == 1) { *pfn = page_to_pfn(page[0]); --- a/fs/exec.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/fs/exec.c @@ -217,7 +217,7 @@ static struct page *get_arg_page(struct * We are doing an exec(). 'current' is the process * doing the exec and bprm->mm is the new process's mm. */ - ret = get_user_pages_remote(current, bprm->mm, pos, 1, gup_flags, + ret = get_user_pages_remote(bprm->mm, pos, 1, gup_flags, &page, NULL, NULL); if (ret <= 0) return NULL; --- a/include/linux/mm.h~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/include/linux/mm.h @@ -1661,7 +1661,7 @@ int invalidate_inode_page(struct page *p extern vm_fault_t handle_mm_fault(struct vm_area_struct *vma, unsigned long address, unsigned int flags, struct pt_regs *regs); -extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, +extern int fixup_user_fault(struct mm_struct *mm, unsigned long address, unsigned int fault_flags, bool *unlocked); void unmap_mapping_pages(struct address_space *mapping, @@ -1677,8 +1677,7 @@ static inline vm_fault_t handle_mm_fault BUG(); return VM_FAULT_SIGBUS; } -static inline int fixup_user_fault(struct task_struct *tsk, - struct mm_struct *mm, unsigned long address, +static inline int fixup_user_fault(struct mm_struct *mm, unsigned long address, unsigned int fault_flags, bool *unlocked) { /* should never happen if there's no MMU */ @@ -1704,11 +1703,11 @@ extern int access_remote_vm(struct mm_st extern int __access_remote_vm(struct task_struct *tsk, struct mm_struct *mm, unsigned long addr, void *buf, int len, unsigned int gup_flags); -long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, +long get_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked); -long pin_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, +long pin_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked); --- a/kernel/events/uprobes.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/kernel/events/uprobes.c @@ -376,7 +376,7 @@ __update_ref_ctr(struct mm_struct *mm, u if (!vaddr || !d) return -EINVAL; - ret = get_user_pages_remote(NULL, mm, vaddr, 1, + ret = get_user_pages_remote(mm, vaddr, 1, FOLL_WRITE, &page, &vma, NULL); if (unlikely(ret <= 0)) { /* @@ -477,7 +477,7 @@ retry: if (is_register) gup_flags |= FOLL_SPLIT_PMD; /* Read the page with vaddr into memory */ - ret = get_user_pages_remote(NULL, mm, vaddr, 1, gup_flags, + ret = get_user_pages_remote(mm, vaddr, 1, gup_flags, &old_page, &vma, NULL); if (ret <= 0) return ret; @@ -2029,7 +2029,7 @@ static int is_trap_at_addr(struct mm_str * but we treat this as a 'remote' access since it is * essentially a kernel access to the memory. */ - result = get_user_pages_remote(NULL, mm, vaddr, 1, FOLL_FORCE, &page, + result = get_user_pages_remote(mm, vaddr, 1, FOLL_FORCE, &page, NULL, NULL); if (result < 0) return result; --- a/kernel/futex.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/kernel/futex.c @@ -678,7 +678,7 @@ static int fault_in_user_writeable(u32 _ int ret; mmap_read_lock(mm); - ret = fixup_user_fault(current, mm, (unsigned long)uaddr, + ret = fixup_user_fault(mm, (unsigned long)uaddr, FAULT_FLAG_WRITE, NULL); mmap_read_unlock(mm); --- a/mm/gup.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/mm/gup.c @@ -859,7 +859,7 @@ unmap: * does not include FOLL_NOWAIT, the mmap_lock may be released. If it * is, *@locked will be set to 0 and -EBUSY returned. */ -static int faultin_page(struct task_struct *tsk, struct vm_area_struct *vma, +static int faultin_page(struct vm_area_struct *vma, unsigned long address, unsigned int *flags, int *locked) { unsigned int fault_flags = 0; @@ -962,7 +962,6 @@ static int check_vma_flags(struct vm_are /** * __get_user_pages() - pin user pages in memory - * @tsk: task_struct of target task * @mm: mm_struct of target mm * @start: starting user address * @nr_pages: number of pages from start to pin @@ -1021,7 +1020,7 @@ static int check_vma_flags(struct vm_are * instead of __get_user_pages. __get_user_pages should be used only if * you need some special @gup_flags. */ -static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm, +static long __get_user_pages(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) @@ -1103,8 +1102,7 @@ retry: page = follow_page_mask(vma, start, foll_flags, &ctx); if (!page) { - ret = faultin_page(tsk, vma, start, &foll_flags, - locked); + ret = faultin_page(vma, start, &foll_flags, locked); switch (ret) { case 0: goto retry; @@ -1178,8 +1176,6 @@ static bool vma_permits_fault(struct vm_ /** * fixup_user_fault() - manually resolve a user page fault - * @tsk: the task_struct to use for page fault accounting, or - * NULL if faults are not to be recorded. * @mm: mm_struct of target mm * @address: user address * @fault_flags:flags to pass down to handle_mm_fault() @@ -1207,7 +1203,7 @@ static bool vma_permits_fault(struct vm_ * This function will not return with an unlocked mmap_lock. So it has not the * same semantics wrt the @mm->mmap_lock as does filemap_fault(). */ -int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm, +int fixup_user_fault(struct mm_struct *mm, unsigned long address, unsigned int fault_flags, bool *unlocked) { @@ -1256,8 +1252,7 @@ EXPORT_SYMBOL_GPL(fixup_user_fault); * Please note that this function, unlike __get_user_pages will not * return 0 for nr_pages > 0 without FOLL_NOWAIT */ -static __always_inline long __get_user_pages_locked(struct task_struct *tsk, - struct mm_struct *mm, +static __always_inline long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, struct page **pages, @@ -1290,7 +1285,7 @@ static __always_inline long __get_user_p pages_done = 0; lock_dropped = false; for (;;) { - ret = __get_user_pages(tsk, mm, start, nr_pages, flags, pages, + ret = __get_user_pages(mm, start, nr_pages, flags, pages, vmas, locked); if (!locked) /* VM_FAULT_RETRY couldn't trigger, bypass */ @@ -1350,7 +1345,7 @@ retry: } *locked = 1; - ret = __get_user_pages(tsk, mm, start, 1, flags | FOLL_TRIED, + ret = __get_user_pages(mm, start, 1, flags | FOLL_TRIED, pages, NULL, locked); if (!*locked) { /* Continue to retry until we succeeded */ @@ -1437,7 +1432,7 @@ long populate_vma_page_range(struct vm_a * We made sure addr is within a VMA, so the following will * not result in a stack expansion that recurses back here. */ - return __get_user_pages(current, mm, start, nr_pages, gup_flags, + return __get_user_pages(mm, start, nr_pages, gup_flags, NULL, NULL, locked); } @@ -1521,7 +1516,7 @@ struct page *get_dump_page(unsigned long struct vm_area_struct *vma; struct page *page; - if (__get_user_pages(current, current->mm, addr, 1, + if (__get_user_pages(current->mm, addr, 1, FOLL_FORCE | FOLL_DUMP | FOLL_GET, &page, &vma, NULL) < 1) return NULL; @@ -1530,8 +1525,7 @@ struct page *get_dump_page(unsigned long } #endif /* CONFIG_ELF_CORE */ #else /* CONFIG_MMU */ -static long __get_user_pages_locked(struct task_struct *tsk, - struct mm_struct *mm, unsigned long start, +static long __get_user_pages_locked(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, struct page **pages, struct vm_area_struct **vmas, int *locked, unsigned int foll_flags) @@ -1596,8 +1590,7 @@ static bool check_dax_vmas(struct vm_are } #ifdef CONFIG_CMA -static long check_and_migrate_cma_pages(struct task_struct *tsk, - struct mm_struct *mm, +static long check_and_migrate_cma_pages(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, struct page **pages, @@ -1675,7 +1668,7 @@ check_again: * again migrating any new CMA pages which we failed to isolate * earlier. */ - ret = __get_user_pages_locked(tsk, mm, start, nr_pages, + ret = __get_user_pages_locked(mm, start, nr_pages, pages, vmas, NULL, gup_flags); @@ -1689,8 +1682,7 @@ check_again: return ret; } #else -static long check_and_migrate_cma_pages(struct task_struct *tsk, - struct mm_struct *mm, +static long check_and_migrate_cma_pages(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, struct page **pages, @@ -1705,8 +1697,7 @@ static long check_and_migrate_cma_pages( * __gup_longterm_locked() is a wrapper for __get_user_pages_locked which * allows us to process the FOLL_LONGTERM flag. */ -static long __gup_longterm_locked(struct task_struct *tsk, - struct mm_struct *mm, +static long __gup_longterm_locked(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, struct page **pages, @@ -1731,7 +1722,7 @@ static long __gup_longterm_locked(struct flags = memalloc_nocma_save(); } - rc = __get_user_pages_locked(tsk, mm, start, nr_pages, pages, + rc = __get_user_pages_locked(mm, start, nr_pages, pages, vmas_tmp, NULL, gup_flags); if (gup_flags & FOLL_LONGTERM) { @@ -1745,7 +1736,7 @@ static long __gup_longterm_locked(struct goto out; } - rc = check_and_migrate_cma_pages(tsk, mm, start, rc, pages, + rc = check_and_migrate_cma_pages(mm, start, rc, pages, vmas_tmp, gup_flags); out: memalloc_nocma_restore(flags); @@ -1756,22 +1747,20 @@ out: return rc; } #else /* !CONFIG_FS_DAX && !CONFIG_CMA */ -static __always_inline long __gup_longterm_locked(struct task_struct *tsk, - struct mm_struct *mm, +static __always_inline long __gup_longterm_locked(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, struct page **pages, struct vm_area_struct **vmas, unsigned int flags) { - return __get_user_pages_locked(tsk, mm, start, nr_pages, pages, vmas, + return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, NULL, flags); } #endif /* CONFIG_FS_DAX || CONFIG_CMA */ #ifdef CONFIG_MMU -static long __get_user_pages_remote(struct task_struct *tsk, - struct mm_struct *mm, +static long __get_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) @@ -1790,20 +1779,18 @@ static long __get_user_pages_remote(stru * This will check the vmas (even if our vmas arg is NULL) * and return -ENOTSUPP if DAX isn't allowed in this case: */ - return __gup_longterm_locked(tsk, mm, start, nr_pages, pages, + return __gup_longterm_locked(mm, start, nr_pages, pages, vmas, gup_flags | FOLL_TOUCH | FOLL_REMOTE); } - return __get_user_pages_locked(tsk, mm, start, nr_pages, pages, vmas, + return __get_user_pages_locked(mm, start, nr_pages, pages, vmas, locked, gup_flags | FOLL_TOUCH | FOLL_REMOTE); } /** * get_user_pages_remote() - pin user pages in memory - * @tsk: the task_struct to use for page fault accounting, or - * NULL if faults are not to be recorded. * @mm: mm_struct of target mm * @start: starting user address * @nr_pages: number of pages from start to pin @@ -1862,7 +1849,7 @@ static long __get_user_pages_remote(stru * should use get_user_pages_remote because it cannot pass * FAULT_FLAG_ALLOW_RETRY to handle_mm_fault. */ -long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, +long get_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) @@ -1874,13 +1861,13 @@ long get_user_pages_remote(struct task_s if (WARN_ON_ONCE(gup_flags & FOLL_PIN)) return -EINVAL; - return __get_user_pages_remote(tsk, mm, start, nr_pages, gup_flags, + return __get_user_pages_remote(mm, start, nr_pages, gup_flags, pages, vmas, locked); } EXPORT_SYMBOL(get_user_pages_remote); #else /* CONFIG_MMU */ -long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, +long get_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) @@ -1888,8 +1875,7 @@ long get_user_pages_remote(struct task_s return 0; } -static long __get_user_pages_remote(struct task_struct *tsk, - struct mm_struct *mm, +static long __get_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) @@ -1909,11 +1895,10 @@ static long __get_user_pages_remote(stru * @vmas: array of pointers to vmas corresponding to each page. * Or NULL if the caller does not require them. * - * This is the same as get_user_pages_remote(), just with a - * less-flexible calling convention where we assume that the task - * and mm being operated on are the current task's and don't allow - * passing of a locked parameter. We also obviously don't pass - * FOLL_REMOTE in here. + * This is the same as get_user_pages_remote(), just with a less-flexible + * calling convention where we assume that the mm being operated on belongs to + * the current task, and doesn't allow passing of a locked parameter. We also + * obviously don't pass FOLL_REMOTE in here. */ long get_user_pages(unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, @@ -1926,7 +1911,7 @@ long get_user_pages(unsigned long start, if (WARN_ON_ONCE(gup_flags & FOLL_PIN)) return -EINVAL; - return __gup_longterm_locked(current, current->mm, start, nr_pages, + return __gup_longterm_locked(current->mm, start, nr_pages, pages, vmas, gup_flags | FOLL_TOUCH); } EXPORT_SYMBOL(get_user_pages); @@ -1936,7 +1921,7 @@ EXPORT_SYMBOL(get_user_pages); * * mmap_read_lock(mm); * do_something() - * get_user_pages(tsk, mm, ..., pages, NULL); + * get_user_pages(mm, ..., pages, NULL); * mmap_read_unlock(mm); * * to: @@ -1944,7 +1929,7 @@ EXPORT_SYMBOL(get_user_pages); * int locked = 1; * mmap_read_lock(mm); * do_something() - * get_user_pages_locked(tsk, mm, ..., pages, &locked); + * get_user_pages_locked(mm, ..., pages, &locked); * if (locked) * mmap_read_unlock(mm); * @@ -1982,7 +1967,7 @@ long get_user_pages_locked(unsigned long if (WARN_ON_ONCE(gup_flags & FOLL_PIN)) return -EINVAL; - return __get_user_pages_locked(current, current->mm, start, nr_pages, + return __get_user_pages_locked(current->mm, start, nr_pages, pages, NULL, locked, gup_flags | FOLL_TOUCH); } @@ -1992,12 +1977,12 @@ EXPORT_SYMBOL(get_user_pages_locked); * get_user_pages_unlocked() is suitable to replace the form: * * mmap_read_lock(mm); - * get_user_pages(tsk, mm, ..., pages, NULL); + * get_user_pages(mm, ..., pages, NULL); * mmap_read_unlock(mm); * * with: * - * get_user_pages_unlocked(tsk, mm, ..., pages); + * get_user_pages_unlocked(mm, ..., pages); * * It is functionally equivalent to get_user_pages_fast so * get_user_pages_fast should be used instead if specific gup_flags @@ -2020,7 +2005,7 @@ long get_user_pages_unlocked(unsigned lo return -EINVAL; mmap_read_lock(mm); - ret = __get_user_pages_locked(current, mm, start, nr_pages, pages, NULL, + ret = __get_user_pages_locked(mm, start, nr_pages, pages, NULL, &locked, gup_flags | FOLL_TOUCH); if (locked) mmap_read_unlock(mm); @@ -2665,7 +2650,7 @@ static int __gup_longterm_unlocked(unsig */ if (gup_flags & FOLL_LONGTERM) { mmap_read_lock(current->mm); - ret = __gup_longterm_locked(current, current->mm, + ret = __gup_longterm_locked(current->mm, start, nr_pages, pages, NULL, gup_flags); mmap_read_unlock(current->mm); @@ -2908,10 +2893,8 @@ int pin_user_pages_fast_only(unsigned lo EXPORT_SYMBOL_GPL(pin_user_pages_fast_only); /** - * pin_user_pages_remote() - pin pages of a remote process (task != current) + * pin_user_pages_remote() - pin pages of a remote process * - * @tsk: the task_struct to use for page fault accounting, or - * NULL if faults are not to be recorded. * @mm: mm_struct of target mm * @start: starting user address * @nr_pages: number of pages from start to pin @@ -2932,7 +2915,7 @@ EXPORT_SYMBOL_GPL(pin_user_pages_fast_on * FOLL_PIN means that the pages must be released via unpin_user_page(). Please * see Documentation/core-api/pin_user_pages.rst for details. */ -long pin_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, +long pin_user_pages_remote(struct mm_struct *mm, unsigned long start, unsigned long nr_pages, unsigned int gup_flags, struct page **pages, struct vm_area_struct **vmas, int *locked) @@ -2942,7 +2925,7 @@ long pin_user_pages_remote(struct task_s return -EINVAL; gup_flags |= FOLL_PIN; - return __get_user_pages_remote(tsk, mm, start, nr_pages, gup_flags, + return __get_user_pages_remote(mm, start, nr_pages, gup_flags, pages, vmas, locked); } EXPORT_SYMBOL(pin_user_pages_remote); @@ -2974,7 +2957,7 @@ long pin_user_pages(unsigned long start, return -EINVAL; gup_flags |= FOLL_PIN; - return __gup_longterm_locked(current, current->mm, start, nr_pages, + return __gup_longterm_locked(current->mm, start, nr_pages, pages, vmas, gup_flags); } EXPORT_SYMBOL(pin_user_pages); @@ -3019,7 +3002,7 @@ long pin_user_pages_locked(unsigned long return -EINVAL; gup_flags |= FOLL_PIN; - return __get_user_pages_locked(current, current->mm, start, nr_pages, + return __get_user_pages_locked(current->mm, start, nr_pages, pages, NULL, locked, gup_flags | FOLL_TOUCH); } --- a/mm/memory.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/mm/memory.c @@ -4742,7 +4742,7 @@ int __access_remote_vm(struct task_struc void *maddr; struct page *page = NULL; - ret = get_user_pages_remote(tsk, mm, addr, 1, + ret = get_user_pages_remote(mm, addr, 1, gup_flags, &page, &vma, NULL); if (ret <= 0) { #ifndef CONFIG_HAVE_IOREMAP_PROT --- a/mm/process_vm_access.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/mm/process_vm_access.c @@ -105,7 +105,7 @@ static int process_vm_rw_single_vec(unsi * current/current->mm */ mmap_read_lock(mm); - pinned_pages = pin_user_pages_remote(task, mm, pa, pinned_pages, + pinned_pages = pin_user_pages_remote(mm, pa, pinned_pages, flags, process_pages, NULL, &locked); if (locked) --- a/security/tomoyo/domain.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/security/tomoyo/domain.c @@ -914,7 +914,7 @@ bool tomoyo_dump_page(struct linux_binpr * (represented by bprm). 'current' is the process doing * the execve(). */ - if (get_user_pages_remote(current, bprm->mm, pos, 1, + if (get_user_pages_remote(bprm->mm, pos, 1, FOLL_FORCE, &page, NULL, NULL) <= 0) return false; #else --- a/virt/kvm/async_pf.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/virt/kvm/async_pf.c @@ -61,7 +61,7 @@ static void async_pf_execute(struct work * access remotely. */ mmap_read_lock(mm); - get_user_pages_remote(NULL, mm, addr, 1, FOLL_WRITE, NULL, NULL, + get_user_pages_remote(mm, addr, 1, FOLL_WRITE, NULL, NULL, &locked); if (locked) mmap_read_unlock(mm); --- a/virt/kvm/kvm_main.c~mm-gup-remove-task_struct-pointer-for-all-gup-code +++ a/virt/kvm/kvm_main.c @@ -1893,7 +1893,7 @@ static int hva_to_pfn_remapped(struct vm * not call the fault handler, so do it here. */ bool unlocked = false; - r = fixup_user_fault(current, current->mm, addr, + r = fixup_user_fault(current->mm, addr, (write_fault ? FAULT_FLAG_WRITE : 0), &unlocked); if (unlocked)

[165/165] mm/gup: remove task_struct pointer for all gup code

Commit Message

Patch