From patchwork Tue Mar 14 22:12:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13175060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0231C7618B for ; Tue, 14 Mar 2023 22:13:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 507766B0075; Tue, 14 Mar 2023 18:13:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4683F6B0078; Tue, 14 Mar 2023 18:13:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 30A1B6B007B; Tue, 14 Mar 2023 18:13:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 224106B0075 for ; Tue, 14 Mar 2023 18:13:10 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EB54440DA2 for ; Tue, 14 Mar 2023 22:13:09 +0000 (UTC) X-FDA: 80568905298.08.66DFF58 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf24.hostedemail.com (Postfix) with ESMTP id 2A5A918001B for ; Tue, 14 Mar 2023 22:13:07 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=MwYaay51; spf=pass (imf24.hostedemail.com: domain of 3c_EQZA0KCMAg3krxgys0yyktmuumrk.iusrot03-ssq1giq.uxm@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3c_EQZA0KCMAg3krxgys0yyktmuumrk.iusrot03-ssq1giq.uxm@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678831988; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kFDuYHkaOlvK+enkas00sVrIsC76CTfX34FshAQHzlg=; b=GLcEtXSOwA87E+L7rwkqTFCq3DlXl8AWQFWkeKyG13hFXoD/AN+oUNEzZL+ptpyBbFxSFL 9T3Clf6pbxRIhBUqosRR0kMIVI0gurtN3xrAs9vthBeUKCYdXXBH12ls5uHvlzxJVJGy5y 5v60Rt2T6g8PYQIfbMG70KNie30Vhc0= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=MwYaay51; spf=pass (imf24.hostedemail.com: domain of 3c_EQZA0KCMAg3krxgys0yyktmuumrk.iusrot03-ssq1giq.uxm@flex--axelrasmussen.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3c_EQZA0KCMAg3krxgys0yyktmuumrk.iusrot03-ssq1giq.uxm@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678831988; a=rsa-sha256; cv=none; b=XxQKDSthXFngytOQr6FLqhjQJBUhPRMM9+zBQO61LRIWtkzD37RH9+Wkmm9EkeZWY/cIX7 x9zuP9chI9HoIiCqxCXO3Y4Mq6iUrBHhh39Dw9l1hOE25K0q7FeDwws/TNOHzXp5NM1GSa 5NcuuIdgBqa0/j4X75P2qn5Iize5Ilw= Received: by mail-yb1-f201.google.com with SMTP id j125-20020a25d283000000b008f257b16d71so18382176ybg.15 for ; Tue, 14 Mar 2023 15:13:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1678831987; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kFDuYHkaOlvK+enkas00sVrIsC76CTfX34FshAQHzlg=; b=MwYaay51HRjaVNMna4SxO6mPWibtfJSnbxlCmTcqWdj/aMcNYci/2o/HXRev/PP/a/ N79JfEvLkW5mLC9XkXesBhQysJMJ4srDBJ4vL87QskYQK3mKuwwv/4taYZfGVJ7Jq80N 2/i6yWkt51L55RV3O0tGRGFKfDQ5Z+s2EYC7Mmdglk6XEyQvyunhJxyQ4SRwg2x4jTeD p181/GaDdx+Kz19kWgJLQFsvZvk2Cfi2ZVnytsLPoCoLnA0U1Z637qH2Bmo+80TX4w8A F5bhUn5Rg97TAfnCl4dd+4pVLbS4vYI1Bjj6ZE69VDbu70C/3HfVUy6wtL4eViJ7fnW8 i2Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678831987; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kFDuYHkaOlvK+enkas00sVrIsC76CTfX34FshAQHzlg=; b=tT7Xblm4b87TT1E8j9+lB+tVDqISZP5aDZ/szepC9NNSm6XQkJczt45vgQoVSyzbF1 V+x4lVfhxNKFgUlue5zTcMpPlmFfq5Fwf3p99lPoFubM3yeXu8GxQT4eNfbDW650y1o5 fSS6wYKfj46AbFbUHFQebZSJM9vffbZfFCtB95TbtWcCyZZ3hLX29D+pmZa7CVAiyKAh FKTzRd3H95LQr+WnlDAZEuOUWENZLD2r6/FLLk3YMcSOLaTvWWDdYs94aK80UY0L3dwO smn8OOVplQhS9lu4IJN0R2s9E0WvnqLpgNE/KGfOVWZRoaackibQIIbHN/d+ZAV3KUjm kBxg== X-Gm-Message-State: AO0yUKVzkcgsLVkkiHKw2rZF1kVB9gMkr6SzLjSU4kNUX4l3nKzWMaYA vx0eplqzG0dVM1dhJBGrRtkDmyXbNWVlG9lSbKQt X-Google-Smtp-Source: AK7set/wGv0xruYIZBBoEGIzpt2s96oHOq53uLqNN1Mq5bzIulMhX3WvmMWGiE43w4x90Uj10QrUMaAjjTrWV2ivLCbz X-Received: from axel.svl.corp.google.com ([2620:15c:2d4:203:21ce:bab3:17ec:2276]) (user=axelrasmussen job=sendgmr) by 2002:a81:4305:0:b0:544:6455:e023 with SMTP id q5-20020a814305000000b005446455e023mr2198836ywa.10.1678831987198; Tue, 14 Mar 2023 15:13:07 -0700 (PDT) Date: Tue, 14 Mar 2023 15:12:47 -0700 In-Reply-To: <20230314221250.682452-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230314221250.682452-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.40.0.rc1.284.g88254d51c5-goog Message-ID: <20230314221250.682452-2-axelrasmussen@google.com> Subject: [PATCH v5 1/4] mm: userfaultfd: rename functions for clarity + consistency From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Hugh Dickins , Jan Kara , "Liam R. Howlett" , Matthew Wilcox , Mike Kravetz , Mike Rapoport , Muchun Song , Nadav Amit , Peter Xu , Shuah Khan Cc: James Houghton , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2A5A918001B X-Stat-Signature: e9xxcxjkenb7rq7kcyzw16cwhdkpfegg X-HE-Tag: 1678831987-438246 X-HE-Meta: U2FsdGVkX19DmHJSZy/MmL4hiExoaZGkhCGpG77cZfi7Ar0tS3ZDKzPaA7DE7Ak/dWoizHaHEm5xRH4eAgOwj+zVMgi6d/q/xDi9LBb7OBK2uxM8JlA9SgwjDOK/S4dze1WSGWCEUm2F6O6cv3MPXxLw9PZk4bwxUqDY5V2I7BmubGhuIXc9UsLCNB/ZMijPvWDO1QuImNocET/iO5R8LUdLmA4xZAgA8rt7c7t2sTVfD31zeDVpSpR6qeTkknkwkG2UPirhtiBYvmJZ5c9qiwLvIDR+cT0c19qZ38+Rs2GasMAW51vBlKaLUxxDRoZIxRQEVQB9Y7Xf4RgqWShb3nsvFASGv2jtxj1Nb9sX36ZFxz/Bfj9FNfDeDZrRuwtaLq1IL4RpEjQY85rycxgWUrxua8AREohjyky3zrcojABsohj1Tns4GGgXdPGRWOg9pa2GdbvTwVh4r21jC07NvCKKGSo+YQ2wePiGjE8e1PwW0WqQ1VVoHD0DcfcQv7Mmqmhg320YhZL7dOMnl7HUWRorE9/3BPtzE2ruNZl8ChSa7FAxH+3o4OK4jjUClNDCuxD90DOm9UznfNSoPxcmury1/qATtGv7gScxTnTlENTWzn77vKPSxkDLaGEV42SoT4Lr1ZH5La8M26y1tCdQpBuQt4Fu+WDhE5l/EZ4iqdm1Ed+uRSBP1ckedvTkjvUm9feZO+1vCALUKO8u2w9sz5IKHKkmUR0Y8Jqdphch3AFg/uB4JYCrpixsUJhbWw5IO2S8TfgOHP/b8qSE/4nL0SnMnt6e7tOgGtWx6Adc9eBoh7zBltPHgXKkbhihP9XxjaZuORrgKneOdFEIvl4TrH5992VauuR0CCpP05C3fhkuuLYmIYIFDHmp2JOD83OcecSHx0k8JccGFSFT8tJk9CpOt0wCDTCxRWDBq5z14Y5RiM8gRJ1d150pfadby4dkgOBVnq9yslQL9+q8+gp 6ltHAAej 2W4gLcmhOs7GKEovKyNMyxC3btQ5kv1BGdnjnHyYSK9JgIVO2xx3Xvx1JCiSbaMQu9xcoioInXfxlIUREyks4gsurQnVjrZIo0diwC7jJVCTtSycyQgMrt8UUIwHIIthKjtRCXrKNkKo6s4NpWV836JOdWY6sUp1PAkb/t4aE93g3bC2xbCrOwXeWL4biT1puoTpDazbJ2QBz8ZjNiDMQ9/jyH4ZyYK88DU5R9jM+cheP/UdhP5M0/vsParbJHOC25ed1/noLjhdh5u3ApdGu4NvsKW8WvhbXCnmKTET3LQAeuNsqJnfoDnzRTAzuq0O5Zdm+qFH0kL0nR/LCgoVfTHjk/UCVwvLogPLy6wJMYOLRXENTOgmEM/PHPNcDlxGaW6dTEXsLGl+V0500Zp7FbcTZbPv98OvdgQkQL0SoZFmqT4M5GX8wFzuaAQO1yiybTFVWX99lkvBY95RrYAyMImidzV2tjZFqZ5Bncd+g0ubbdZ602zgqJ5SzPVNu8JAGqadqlid8jQvhstRz23d16roXU7B3v1mnvvB4X6quWtNflCIO2Gn6pmTJLWkajyyjaQ1C/MKifO+NbPslgcpPqmC1Q9c8J/uhjtCjU5l/OgV/reDaRldshKHYoodUfT97cW+oScg4q1L9jOBvFMG+NwTCmoKOUsUspExZGIGUaCwAs89m9q+cbpFpgjt1skB7gsvC2b7HWB6x7B2Nv3Z3w6IZXBs0cJCu9CLr8LrO1eIRxEcG1rDKfXmkTf0i+68omWjOIMWvusyQ1EY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The basic problem is, over time we've added new userfaultfd ioctls, and we've refactored the code so functions which used to handle only one case are now re-used to deal with several cases. While this happened, we didn't bother to rename the functions. Similarly, as we added new functions, we cargo-culted pieces of the now-inconsistent naming scheme, so those functions too ended up with names that don't make a lot of sense. A key point here is, "copy" in most userfaultfd code refers specifically to UFFDIO_COPY, where we allocate a new page and copy its contents from userspace. There are many functions with "copy" in the name that don't actually do this (at least in some cases). So, rename things into a consistent scheme. The high level idea is that the call stack for userfaultfd ioctls becomes: userfaultfd_ioctl -> userfaultfd_(particular ioctl) -> mfill_atomic_(particular kind of fill operation) -> mfill_atomic /* loops over pages in range */ -> mfill_atomic_pte /* deals with single pages */ -> mfill_atomic_pte_(particular kind of fill operation) -> mfill_atomic_install_pte There are of course some special cases (shmem, hugetlb), but this is the general structure which all function names now adhere to. Acked-by: Peter Xu Acked-by: Mike Rapoport (IBM) Signed-off-by: Axel Rasmussen --- fs/userfaultfd.c | 18 +++---- include/linux/hugetlb.h | 30 +++++------ include/linux/userfaultfd_k.h | 18 +++---- mm/hugetlb.c | 20 +++---- mm/userfaultfd.c | 98 +++++++++++++++++------------------ 5 files changed, 92 insertions(+), 92 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 44d1ee429eb0..365bf00dd8dd 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1741,9 +1741,9 @@ static int userfaultfd_copy(struct userfaultfd_ctx *ctx, if (uffdio_copy.mode & ~(UFFDIO_COPY_MODE_DONTWAKE|UFFDIO_COPY_MODE_WP)) goto out; if (mmget_not_zero(ctx->mm)) { - ret = mcopy_atomic(ctx->mm, uffdio_copy.dst, uffdio_copy.src, - uffdio_copy.len, &ctx->mmap_changing, - uffdio_copy.mode); + ret = mfill_atomic_copy(ctx->mm, uffdio_copy.dst, uffdio_copy.src, + uffdio_copy.len, &ctx->mmap_changing, + uffdio_copy.mode); mmput(ctx->mm); } else { return -ESRCH; @@ -1793,9 +1793,9 @@ static int userfaultfd_zeropage(struct userfaultfd_ctx *ctx, goto out; if (mmget_not_zero(ctx->mm)) { - ret = mfill_zeropage(ctx->mm, uffdio_zeropage.range.start, - uffdio_zeropage.range.len, - &ctx->mmap_changing); + ret = mfill_atomic_zeropage(ctx->mm, uffdio_zeropage.range.start, + uffdio_zeropage.range.len, + &ctx->mmap_changing); mmput(ctx->mm); } else { return -ESRCH; @@ -1903,9 +1903,9 @@ static int userfaultfd_continue(struct userfaultfd_ctx *ctx, unsigned long arg) goto out; if (mmget_not_zero(ctx->mm)) { - ret = mcopy_continue(ctx->mm, uffdio_continue.range.start, - uffdio_continue.range.len, - &ctx->mmap_changing); + ret = mfill_atomic_continue(ctx->mm, uffdio_continue.range.start, + uffdio_continue.range.len, + &ctx->mmap_changing); mmput(ctx->mm); } else { return -ESRCH; diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 7c977d234aba..8f0467bf1cbd 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -158,13 +158,13 @@ unsigned long hugetlb_total_pages(void); vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long address, unsigned int flags); #ifdef CONFIG_USERFAULTFD -int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, pte_t *dst_pte, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - enum mcopy_atomic_mode mode, - struct page **pagep, - bool wp_copy); +int hugetlb_mfill_atomic_pte(struct mm_struct *dst_mm, pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + enum mcopy_atomic_mode mode, + struct page **pagep, + bool wp_copy); #endif /* CONFIG_USERFAULTFD */ bool hugetlb_reserve_pages(struct inode *inode, long from, long to, struct vm_area_struct *vma, @@ -393,14 +393,14 @@ static inline void hugetlb_free_pgd_range(struct mmu_gather *tlb, } #ifdef CONFIG_USERFAULTFD -static inline int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, - pte_t *dst_pte, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - enum mcopy_atomic_mode mode, - struct page **pagep, - bool wp_copy) +static inline int hugetlb_mfill_atomic_pte(struct mm_struct *dst_mm, + pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + enum mcopy_atomic_mode mode, + struct page **pagep, + bool wp_copy) { BUG(); return 0; diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 3767f18114ef..468080125612 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -61,15 +61,15 @@ extern int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, unsigned long dst_addr, struct page *page, bool newly_allocated, bool wp_copy); -extern ssize_t mcopy_atomic(struct mm_struct *dst_mm, unsigned long dst_start, - unsigned long src_start, unsigned long len, - atomic_t *mmap_changing, __u64 mode); -extern ssize_t mfill_zeropage(struct mm_struct *dst_mm, - unsigned long dst_start, - unsigned long len, - atomic_t *mmap_changing); -extern ssize_t mcopy_continue(struct mm_struct *dst_mm, unsigned long dst_start, - unsigned long len, atomic_t *mmap_changing); +extern ssize_t mfill_atomic_copy(struct mm_struct *dst_mm, unsigned long dst_start, + unsigned long src_start, unsigned long len, + atomic_t *mmap_changing, __u64 mode); +extern ssize_t mfill_atomic_zeropage(struct mm_struct *dst_mm, + unsigned long dst_start, + unsigned long len, + atomic_t *mmap_changing); +extern ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long dst_start, + unsigned long len, atomic_t *mmap_changing); extern int mwriteprotect_range(struct mm_struct *dst_mm, unsigned long start, unsigned long len, bool enable_wp, atomic_t *mmap_changing); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 07abcb6eb203..4c9276549394 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6154,17 +6154,17 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, #ifdef CONFIG_USERFAULTFD /* - * Used by userfaultfd UFFDIO_COPY. Based on mcopy_atomic_pte with - * modifications for huge pages. + * Used by userfaultfd UFFDIO_* ioctls. Based on userfaultfd's mfill_atomic_pte + * with modifications for hugetlb pages. */ -int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, - pte_t *dst_pte, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - enum mcopy_atomic_mode mode, - struct page **pagep, - bool wp_copy) +int hugetlb_mfill_atomic_pte(struct mm_struct *dst_mm, + pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + enum mcopy_atomic_mode mode, + struct page **pagep, + bool wp_copy) { bool is_continue = (mode == MCOPY_ATOMIC_CONTINUE); struct hstate *h = hstate_vma(dst_vma); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 53c3d916ff66..84db5b2fad3a 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -127,13 +127,13 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, return ret; } -static int mcopy_atomic_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - struct page **pagep, - bool wp_copy) +static int mfill_atomic_pte_copy(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + struct page **pagep, + bool wp_copy) { void *page_kaddr; int ret; @@ -204,10 +204,10 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm, goto out; } -static int mfill_zeropage_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr) +static int mfill_atomic_pte_zeropage(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr) { pte_t _dst_pte, *dst_pte; spinlock_t *ptl; @@ -240,11 +240,11 @@ static int mfill_zeropage_pte(struct mm_struct *dst_mm, } /* Handles UFFDIO_CONTINUE for all shmem VMAs (shared or private). */ -static int mcontinue_atomic_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - bool wp_copy) +static int mfill_atomic_pte_continue(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + bool wp_copy) { struct inode *inode = file_inode(dst_vma->vm_file); pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); @@ -307,10 +307,10 @@ static pmd_t *mm_alloc_pmd(struct mm_struct *mm, unsigned long address) #ifdef CONFIG_HUGETLB_PAGE /* - * __mcopy_atomic processing for HUGETLB vmas. Note that this routine is + * mfill_atomic processing for HUGETLB vmas. Note that this routine is * called with mmap_lock held, it will release mmap_lock before returning. */ -static __always_inline ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, +static __always_inline ssize_t mfill_atomic_hugetlb(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma, unsigned long dst_start, unsigned long src_start, @@ -411,7 +411,7 @@ static __always_inline ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, goto out_unlock; } - err = hugetlb_mcopy_atomic_pte(dst_mm, dst_pte, dst_vma, + err = hugetlb_mfill_atomic_pte(dst_mm, dst_pte, dst_vma, dst_addr, src_addr, mode, &page, wp_copy); @@ -463,7 +463,7 @@ static __always_inline ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, } #else /* !CONFIG_HUGETLB_PAGE */ /* fail at build time if gcc attempts to use this */ -extern ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, +extern ssize_t mfill_atomic_hugetlb(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma, unsigned long dst_start, unsigned long src_start, @@ -484,8 +484,8 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, ssize_t err; if (mode == MCOPY_ATOMIC_CONTINUE) { - return mcontinue_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - wp_copy); + return mfill_atomic_pte_continue(dst_mm, dst_pmd, dst_vma, + dst_addr, wp_copy); } /* @@ -500,11 +500,11 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, */ if (!(dst_vma->vm_flags & VM_SHARED)) { if (mode == MCOPY_ATOMIC_NORMAL) - err = mcopy_atomic_pte(dst_mm, dst_pmd, dst_vma, - dst_addr, src_addr, page, - wp_copy); + err = mfill_atomic_pte_copy(dst_mm, dst_pmd, dst_vma, + dst_addr, src_addr, page, + wp_copy); else - err = mfill_zeropage_pte(dst_mm, dst_pmd, + err = mfill_atomic_pte_zeropage(dst_mm, dst_pmd, dst_vma, dst_addr); } else { err = shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, @@ -516,13 +516,13 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, return err; } -static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, - unsigned long dst_start, - unsigned long src_start, - unsigned long len, - enum mcopy_atomic_mode mcopy_mode, - atomic_t *mmap_changing, - __u64 mode) +static __always_inline ssize_t mfill_atomic(struct mm_struct *dst_mm, + unsigned long dst_start, + unsigned long src_start, + unsigned long len, + enum mcopy_atomic_mode mcopy_mode, + atomic_t *mmap_changing, + __u64 mode) { struct vm_area_struct *dst_vma; ssize_t err; @@ -588,9 +588,9 @@ static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, * If this is a HUGETLB vma, pass off to appropriate routine */ if (is_vm_hugetlb_page(dst_vma)) - return __mcopy_atomic_hugetlb(dst_mm, dst_vma, dst_start, - src_start, len, mcopy_mode, - wp_copy); + return mfill_atomic_hugetlb(dst_mm, dst_vma, dst_start, + src_start, len, mcopy_mode, + wp_copy); if (!vma_is_anonymous(dst_vma) && !vma_is_shmem(dst_vma)) goto out_unlock; @@ -688,26 +688,26 @@ static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, return copied ? copied : err; } -ssize_t mcopy_atomic(struct mm_struct *dst_mm, unsigned long dst_start, - unsigned long src_start, unsigned long len, - atomic_t *mmap_changing, __u64 mode) +ssize_t mfill_atomic_copy(struct mm_struct *dst_mm, unsigned long dst_start, + unsigned long src_start, unsigned long len, + atomic_t *mmap_changing, __u64 mode) { - return __mcopy_atomic(dst_mm, dst_start, src_start, len, - MCOPY_ATOMIC_NORMAL, mmap_changing, mode); + return mfill_atomic(dst_mm, dst_start, src_start, len, + MCOPY_ATOMIC_NORMAL, mmap_changing, mode); } -ssize_t mfill_zeropage(struct mm_struct *dst_mm, unsigned long start, - unsigned long len, atomic_t *mmap_changing) +ssize_t mfill_atomic_zeropage(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing) { - return __mcopy_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_ZEROPAGE, - mmap_changing, 0); + return mfill_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_ZEROPAGE, + mmap_changing, 0); } -ssize_t mcopy_continue(struct mm_struct *dst_mm, unsigned long start, - unsigned long len, atomic_t *mmap_changing) +ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing) { - return __mcopy_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_CONTINUE, - mmap_changing, 0); + return mfill_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_CONTINUE, + mmap_changing, 0); } long uffd_wp_range(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma,