From patchwork Tue Jun 18 09:12:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13701998 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29A7EC27C4F for ; Tue, 18 Jun 2024 09:13:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C0068D001E; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 670528D0002; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55E408D001E; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 39A478D0002 for ; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D4C0F14062D for ; Tue, 18 Jun 2024 09:13:13 +0000 (UTC) X-FDA: 82243445466.02.391A82C Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf06.hostedemail.com (Postfix) with ESMTP id 61D2218000B for ; Tue, 18 Jun 2024 09:13:11 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718701985; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DSFxSI3ZaXf25Jj4q7KMBQBPWCzMpg63k/mpFfoVNSQ=; b=f9Leevx2j9FOo9p1xZYj5o6Ksm6F0xfM07DFUseH+HDl4ZO1CiTGArIqjdaK6huNktLjvA TveupA8CFXDLAGN4I0IDHkI4VCYk5XcPeLvC/twPXgOytf2+2qDYJaZyWWHMEsS2XLbiPi TBEf0HrZXofi6DBZTISZfnCHjO4KFw0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718701985; a=rsa-sha256; cv=none; b=oiEzfsZ8H7MKdUmor6J0VL+Rz/ZwwiYWgrrL1jwDksBSfFX9bcebz7I8Pmj3zGCNl0Qc5f MxSsl3xv+J2fgFnBZgIvX7516H2ABM1WjB3mTvOkK3LtYsQaoW6Wffc9dt2foWSdJB3jIs dlnQZkCPv5eTIEN3DzQJONZSLE4ud2c= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; spf=pass (imf06.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.163.48]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4W3LbY0PZDzdbwn; Tue, 18 Jun 2024 17:11:37 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 32BEF18007E; Tue, 18 Jun 2024 17:13:07 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 18 Jun 2024 17:13:06 +0800 From: Kefeng Wang To: Andrew Morton CC: Muchun Song , "Matthew Wilcox (Oracle)" , David Hildenbrand , , Huang Ying , Kefeng Wang Subject: [PATCH v2 1/4] mm: memory: convert clear_huge_page() to folio_zero_user() Date: Tue, 18 Jun 2024 17:12:39 +0800 Message-ID: <20240618091242.2140164-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> References: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 61D2218000B X-Stat-Signature: kayj6g4y6bnp9a1kdeo9rju1eax7jow7 X-HE-Tag: 1718701991-544177 X-HE-Meta: U2FsdGVkX18tMe6lC3VoNfjHISZnU0klzV8lyrG/DIdPB6ScWIcfGaUKNX/KzbfjluY39u/EUnsnOFChPRRhS2O4MHdA3RpMFyvRiretYzAqmsjY5E3zLtgr64a5mYfayWBbdqE8T2rEtThVoxAsmwc5YJ+j9GmN1t5fe8bPHYBPb/egWtVja1+X1ymP0uj+eZ5R/4O7hA7/w/KTZ3ZYF2mmTlhlQOLOpE7blINzcoy4w9iFGT0XEubgxk7i08J/RSjaU2jav8PDDyOR7+TMfIlFWukvcSYbcl1PzrGpMXqibzWwbsUrMbZlSdoeoNmgeRA50HKN8gMdaXa7muc4YiDhc9j3FHUY/AIMKsYV8INLeWIOIhIyo2zuS0IexoM7bY+JwW1btyXIvYWeTEmadeT+eaaXCsUOUkUlKDxXmANRjyYc3123KtZQN52wyKOeQlGJtMpVtE3Z6wKjGYtBarAbZNqZSig2vHbZ+UbOXy9EiNG1p94FZiiq7Wc5atcOG73pW0dZmMGO3oJPB0T3HXiJkn5+eE0qMNfSuJwQVJjPnuJBDy45wKK4acLktfK/lB97PvNk34SzIbGA051y4Pf6xPxPdLe3W6OJlueWSJJhanXIAi1ziP7j+123WKDtEontjPZIAVJTBd/4woagD7rConr7Y6Xx+VMBpNgBZ8JQ2J4YiGKiBQVFU9TsCPEHQffqlOajzQtQjwNFgJWKhtLoiuGZUOYZ8l7gjI/KZsCNzG3D4pDEZKHAEIc5FyTm4T90zAegUR8cfOJhEyPi00VA3qKO8QkYEv+/5zgz4PjNRZ1lUVRjXufli4yqtCMrZ9z+vfjuhcAIiDFt5NGUYS6UaLlUdgC2ViN9d9gkYxfsrQtFna10hue7PCsRWisC/g8KSt7LmMrjpF2bvO8GXyEveMNJsmh8+1qaNN4Bwgqha6CgStz7vPNRWvrwWrFtgKzkRur2cDOKNGOnBmQ zMvfLmbw qNCS3zsT1vbFGhm2NFJFvkZBuyjTQKuPVhjz/Z5vI2Rxj6hmUNnDH/PVh+WPLYUbfumQ6zZn0UWc2HY2TA7tyOgJRG2/A5y8j8lz7SODlgo0KpfjjQtFD4TcUCBgk3JEfIn7EsJCcNHD7NDTdZrZUDNvlULlp1llrLFnzK2E+3vPKZejvsJ26TjAhDm8r+O5PwEF7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Replace clear_huge_page() with folio_zero_user(), and take a folio instead of a page. Directly get number of pages by folio_nr_pages() to remove pages_per_huge_page argument, furthermore, move the address alignment from folio_zero_user() to the callers since the alignment is only needed when we don't know which address will be accessed. Signed-off-by: Kefeng Wang --- fs/hugetlbfs/inode.c | 2 +- include/linux/mm.h | 4 +--- mm/huge_memory.c | 4 ++-- mm/hugetlb.c | 3 +-- mm/memory.c | 34 ++++++++++++++++------------------ 5 files changed, 21 insertions(+), 26 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 1107e5aa8343..ecad73a4f713 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -892,7 +892,7 @@ static long hugetlbfs_fallocate(struct file *file, int mode, loff_t offset, error = PTR_ERR(folio); goto out; } - clear_huge_page(&folio->page, addr, pages_per_huge_page(h)); + folio_zero_user(folio, ALIGN_DOWN(addr, hpage_size)); __folio_mark_uptodate(folio); error = hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { diff --git a/include/linux/mm.h b/include/linux/mm.h index 106bb0310352..ecbca6281a4e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4071,9 +4071,7 @@ enum mf_action_page_type { }; #if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLBFS) -extern void clear_huge_page(struct page *page, - unsigned long addr_hint, - unsigned int pages_per_huge_page); +void folio_zero_user(struct folio *folio, unsigned long addr_hint); int copy_user_large_folio(struct folio *dst, struct folio *src, unsigned long addr_hint, struct vm_area_struct *vma); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f409ea9fcc18..85b852861610 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -943,10 +943,10 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, goto release; } - clear_huge_page(page, vmf->address, HPAGE_PMD_NR); + folio_zero_user(folio, vmf->address); /* * The memory barrier inside __folio_mark_uptodate makes sure that - * clear_huge_page writes become visible before the set_pmd_at() + * folio_zero_user writes become visible before the set_pmd_at() * write. */ __folio_mark_uptodate(folio); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3518321f6598..58d8703a1065 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6296,8 +6296,7 @@ static vm_fault_t hugetlb_no_page(struct address_space *mapping, ret = 0; goto out; } - clear_huge_page(&folio->page, vmf->real_address, - pages_per_huge_page(h)); + folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio = true; diff --git a/mm/memory.c b/mm/memory.c index 54d7d2acdf39..bc5d446d9174 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4477,7 +4477,7 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) goto next; } folio_throttle_swaprate(folio, gfp); - clear_huge_page(&folio->page, vmf->address, 1 << order); + folio_zero_user(folio, vmf->address); return folio; } next: @@ -6420,41 +6420,39 @@ static inline int process_huge_page( return 0; } -static void clear_gigantic_page(struct page *page, - unsigned long addr, +static void clear_gigantic_page(struct folio *folio, unsigned long addr, unsigned int pages_per_huge_page) { int i; - struct page *p; might_sleep(); for (i = 0; i < pages_per_huge_page; i++) { - p = nth_page(page, i); cond_resched(); - clear_user_highpage(p, addr + i * PAGE_SIZE); + clear_user_highpage(folio_page(folio, i), addr + i * PAGE_SIZE); } } static int clear_subpage(unsigned long addr, int idx, void *arg) { - struct page *page = arg; + struct folio *folio = arg; - clear_user_highpage(nth_page(page, idx), addr); + clear_user_highpage(folio_page(folio, idx), addr); return 0; } -void clear_huge_page(struct page *page, - unsigned long addr_hint, unsigned int pages_per_huge_page) +/** + * folio_zero_user - Zero a folio which will be mapped to userspace. + * @folio: The folio to zero. + * @addr_hint: The address will be accessed or the base address if uncelar. + */ +void folio_zero_user(struct folio *folio, unsigned long addr_hint) { - unsigned long addr = addr_hint & - ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + unsigned int nr_pages = folio_nr_pages(folio); - if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) { - clear_gigantic_page(page, addr, pages_per_huge_page); - return; - } - - process_huge_page(addr_hint, pages_per_huge_page, clear_subpage, page); + if (unlikely(nr_pages > MAX_ORDER_NR_PAGES)) + clear_gigantic_page(folio, addr_hint, nr_pages); + else + process_huge_page(addr_hint, nr_pages, clear_subpage, folio); } static int copy_user_gigantic_page(struct folio *dst, struct folio *src, From patchwork Tue Jun 18 09:12:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13702001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0CB6C2BA15 for ; Tue, 18 Jun 2024 09:13:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B80B8D0022; Tue, 18 Jun 2024 05:13:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A4A68D001F; Tue, 18 Jun 2024 05:13:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C89B08D0020; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8B7948D0022 for ; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id E2564140627 for ; Tue, 18 Jun 2024 09:13:13 +0000 (UTC) X-FDA: 82243445466.27.4C3E66D Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf10.hostedemail.com (Postfix) with ESMTP id 773C8C000B for ; Tue, 18 Jun 2024 09:13:11 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718701989; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=47pB7SRGf5FzTbu0P5M0rb8nOT0uxMIvqdM4ZBeP3fU=; b=IOIajcWB8gNUGSy7aZdXpZJ/0lIRRbkN+QgrE7xaTD6XzApq5qVD5VHq69ZnkL9StCQnuf fNDFuQYQ7tnlTD/aRb4CWoRf4YJow3ZKrnJNIRLgFARj2QrY+zDOhs1Y6rJSwwHpunYzSk G3E08Qz0YK7TQST3XL3DRiws5WnQWMw= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718701989; a=rsa-sha256; cv=none; b=gnsBPuoBTcQUqy0aAoCI+5UbkYTS0SP4EEIGoAynJeejhdXKI/wulZyD1Z6Nf27fiDp2kI z/V7R6VhDiKbgr/lOKHbjxNwMytJ56U0+W11xyALK1G3Y/5DomYu632FtlzezAWh0DJame zpaGnROPRZbTOiKuAWphc2/JJtYrLYI= Received: from mail.maildlp.com (unknown [172.19.88.214]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4W3LXp6pk7z1j5SP; Tue, 18 Jun 2024 17:09:14 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 9ADF21A016C; Tue, 18 Jun 2024 17:13:07 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 18 Jun 2024 17:13:07 +0800 From: Kefeng Wang To: Andrew Morton CC: Muchun Song , "Matthew Wilcox (Oracle)" , David Hildenbrand , , Huang Ying , Kefeng Wang Subject: [PATCH v2 2/4] mm: memory: use folio in struct copy_subpage_arg Date: Tue, 18 Jun 2024 17:12:40 +0800 Message-ID: <20240618091242.2140164-3-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> References: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 773C8C000B X-Stat-Signature: 9yna3ajqipdh5auhxix941n7fck6du61 X-HE-Tag: 1718701991-71839 X-HE-Meta: U2FsdGVkX1+S0oZtRldP37rl5A4Jh+vzoK3hNWPf7wTO1u451XgeOTXKtxppieb7Iish4hFt01eSIvCwBoM/Aq4kS3E5/ro+pSIx+RSZd/TklDSyCq3ceJ8U2gJtcOZRVGQvlwk0U+VrLGYkR1Mc7EJItDMmUOUUFQzgg16l7tv9Dk+gB5aU4/10Z3xmrSKJNkPzmbPk3VhFd4wT+ZZcCsKNFbuYPTuoSm3fwJsFlMH7LDn3yWRXzQ2saaRdwdp1yqkXLbDDxy7GPG2aM4EQH5NW0hurWdMvF7tM18NTj+2QUyqb/lakw6HZ4fCx47Og7qmkpbFOr2ZzoDDavwiJMZYDbMJIAkjIR+QfIBUr8pwpVcQIhoeYJiBLDl9UBa4tcpb9QtG35aVlWgJ2VV1lf2jSKtXZWqBkRvKYPVPsYElgla3DnJ2SGkricHwnCf8wULuebtvZx+VzusPK0xeFQEE5Ss7CiIuztwF7t0y9aEvFVN7uPzASzN7xKiNZEzDFQC8ZnAOhTDnFgzmaEG96p4f4adpVCKxgMcaKeQ34Vui5g878JB0dT2/6ZkXy67pBGn7wEQFB06XNdQGX4yyAmcKSivt3copo5Ho+VmfaefomMCp3agmH7Zf4WnDS/RzE2/Wp4ZEFgRMfzNpnQHiMNcSo4FaRnx8uo9lYPcN2FXGfkEt76+xpRvBQemDJEwf2+7RyLTtvNSSti9ldp5tmOzU8LPINLiD7/2DAewCOqjpHJo8Aa9pvIcdjmZ2fAvOYXHdTHqkLL1zJJaozgMmU7BS8w8nr4tZO7gvsxqLuGqlw+7ZJnFbOAAm7vsO7h//cHVaVc+2GOjoMoTKt9l/BdoKBxMHD6Z1FG12Of2ImU4FDnTAoUAy1lFJO/30X1ZnGVKcfDY3ERU+hqGO+u3FCZQnQg1d3rlnM/mS1d71TGh87O7c5wQW4NwyoCSsM0JpMXwBWObBQMc8S6GKLJOR goYZrWXh Z3NzkAf7fpET0CCeWnCqi/6IcLm8HTXaHafD4GffN2mMQiWLU/LEmmFCiNiAQ28uK1ak5IxhT0X60R6it1B3rhZzQtqCY5drl7nzTCMeBAMw6dkqmB+Cpe4qw8j4zYRaDWdUfWWyfTQp83NMlRNzGdP5hffVcH2I2Gc0dVe0z4z2yRaaGh+kTeNyg2+CkxZmW5M0dztCCJQGe7vp3Uz2rUPGseQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Directly use folio in struct copy_subpage_arg. Acked-by: David Hildenbrand Signed-off-by: Kefeng Wang --- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index bc5d446d9174..a48a790a2b5b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6477,16 +6477,16 @@ static int copy_user_gigantic_page(struct folio *dst, struct folio *src, } struct copy_subpage_arg { - struct page *dst; - struct page *src; + struct folio *dst; + struct folio *src; struct vm_area_struct *vma; }; static int copy_subpage(unsigned long addr, int idx, void *arg) { struct copy_subpage_arg *copy_arg = arg; - struct page *dst = nth_page(copy_arg->dst, idx); - struct page *src = nth_page(copy_arg->src, idx); + struct page *dst = folio_page(copy_arg->dst, idx); + struct page *src = folio_page(copy_arg->src, idx); if (copy_mc_user_highpage(dst, src, addr, copy_arg->vma)) return -EHWPOISON; @@ -6500,8 +6500,8 @@ int copy_user_large_folio(struct folio *dst, struct folio *src, unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); struct copy_subpage_arg arg = { - .dst = &dst->page, - .src = &src->page, + .dst = dst, + .src = src, .vma = vma, }; From patchwork Tue Jun 18 09:12:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13701999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3180AC2BA1A for ; Tue, 18 Jun 2024 09:13:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2CC68D0021; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD9FD8D0002; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ADC5E8D0002; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8889E8D001F for ; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2EBCF1C1275 for ; Tue, 18 Jun 2024 09:13:14 +0000 (UTC) X-FDA: 82243445508.26.EE8694C Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf09.hostedemail.com (Postfix) with ESMTP id 8983614000B for ; Tue, 18 Jun 2024 09:13:11 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf09.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718701987; a=rsa-sha256; cv=none; b=hBX/UPBpA44wAEVyS93eth8MufFFyiIwz3GgKFShQBAddJLe59c46dSXrXG7N2Qq3JULB+ Xrdt5OPPLqTHKmDNpCmHl0yROxu6FeSiZfT87wSGDvsn5/2GQ+rhT0Ss6iZ0FTdruOGMmI FWOrmAcJtG7PjK86Wa45z5nZA65H8E4= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf09.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718701987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y7T/ckPRHgJSoX8YPnK9KqfkKG43qW1lzBH+tUwBwRY=; b=4KnBG0Ffp0rQaE2wF1R8M83NxFw6kY67cLzpGzYNNLxB+AvCKgXBXpc7ycjq9wEzhX/tUx oLyPq0lpadu9lE0RtHpe8ey16kWYgc5b8WPsByRgwDwBIYCyb3qUL4VNWUOfuYvhfcoJUX zMgkqnsciiXXpLBj2bY+NrtwjolEW50= Received: from mail.maildlp.com (unknown [172.19.88.194]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4W3LXQ2rzdzwTCT; Tue, 18 Jun 2024 17:08:54 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 11D1E14041A; Tue, 18 Jun 2024 17:13:08 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 18 Jun 2024 17:13:07 +0800 From: Kefeng Wang To: Andrew Morton CC: Muchun Song , "Matthew Wilcox (Oracle)" , David Hildenbrand , , Huang Ying , Kefeng Wang Subject: [PATCH v2 3/4] mm: memory: improve copy_user_large_folio() Date: Tue, 18 Jun 2024 17:12:41 +0800 Message-ID: <20240618091242.2140164-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> References: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspamd-Queue-Id: 8983614000B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: i1wqdoqyqwf9sknp8ofn9h8hptweeukq X-HE-Tag: 1718701991-997283 X-HE-Meta: U2FsdGVkX1+ehHINSrB72v8+S+ldDS+Ensp7cR9k/4zTwFtcmEc3GL41hykVuGCCvKxusR5WR6VZguYksaeAlJN0oV9kdIO1qHt4c0mE6u68I2OK5vKVs6sWV0xqyxktuOmnrnVKspW0CEc16RW0pzbs2ylzzpbLU/nzLQSjSy/zIBqe34poLmS/r4Ts3TuzcewUq1u4X/5/WqKtEPF0QCjcm9zhcZWTQ2ehDEAkWIYPydkyoxOPkxEsE/8LXEFy6J9aIgmTaYdR8FSXQRNGlfSxASpJ6CTAlVgUEg2kI0oVu6xw9t32b8SevD+bLLFRALbGA3ijBmfxvNi6K709DpvxNB5DZaCM7MHajP/KlCeMYRtsre0+sPkuIinEM0gulX2vSOwY3iR0Lo4tr1bl1pcpbx/m2gJznIZ0oYaTB3ThRFjzzl0rJZQsErpG37H8zgetzfCyedR0RwkEnuclsHQgSuNXoRNJzCnMEgWXIwFySVPuKS5zMoaCgEuTTsg9ENZ/skOw/AdgfJFA5iDbx1M1HMibHlmh5gdobsOe3y9t/ZKJqujcs6KvY9nJAVUJzsLbggMFpAs0Rxj7eDVxrHVWFgdNNo0oPtjTeCNTAiAY1eaLU7nZO6d175so1MebrDL03ZPQf4HrC0Hzk4hI85rdjbdGKqVMDvQnTKNNm+mDtc5h4ZfwqKpNDciY9IUATA37Ctf9mwIBC0+jrSPaGXq5JlDDOQp4vO2/abGXzzaE5ZtMftWv9haOoz1y4+tRhvdMn8p9P3ydnMRnLeY0SaFI/D3pUzdzO2lvtAv6cnPl6qQDjkPkcH7JisPLZuiHOJwuXdhRyKp4Ivnbb4FLdIxp9KakWjy1lFRhN3cIb4Zi3SyjF1kKTgUagN6eeteu+zXxqEAVqpG1xmlwAxHBSJVdcxWdKKlbjjX0k1dq1TCpi1Q3dhPIyZLagCyiCsDfw/PNNabxdFnmtNVEQd1 Cx7PcMAH 1J483OrfJ8hkTG8C4zkZMlWMbIcFoSh7qwV+lAghMbkIy5FZhNkE5hHRnfBwZpFnSQsGgZcbHiHQ97hQg58MWXBxNMuMRrr06tBnsBsxZZ7w4+LbmPdRMblIvdAuegrgHUHFQSSCqZ+45bpfIw8mZUJNkdYq5XH46GgkuaRiI7ePKeDHC6fX3Savw+ZAobxXg3Sto X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Use nr_pages instead of pages_per_huge_page and move the address alignment from copy_user_large_folio() into the callers since it is only needed when we don't know which address will be accessed. Signed-off-by: Kefeng Wang --- mm/hugetlb.c | 18 ++++++++---------- mm/memory.c | 11 ++++------- 2 files changed, 12 insertions(+), 17 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 58d8703a1065..a41afeeb2188 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5488,9 +5488,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, ret = PTR_ERR(new_folio); break; } - ret = copy_user_large_folio(new_folio, - pte_folio, - addr, dst_vma); + ret = copy_user_large_folio(new_folio, pte_folio, + ALIGN_DOWN(addr, sz), dst_vma); folio_put(pte_folio); if (ret) { folio_put(new_folio); @@ -6680,7 +6679,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, struct hstate *h = hstate_vma(dst_vma); struct address_space *mapping = dst_vma->vm_file->f_mapping; pgoff_t idx = vma_hugecache_offset(h, dst_vma, dst_addr); - unsigned long size; + unsigned long size = huge_page_size(h); int vm_shared = dst_vma->vm_flags & VM_SHARED; pte_t _dst_pte; spinlock_t *ptl; @@ -6699,8 +6698,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, } _dst_pte = make_pte_marker(PTE_MARKER_POISONED); - set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, - huge_page_size(h)); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, size); /* No need to invalidate - it was non-present before */ update_mmu_cache(dst_vma, dst_addr, dst_pte); @@ -6774,7 +6772,8 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, *foliop = NULL; goto out; } - ret = copy_user_large_folio(folio, *foliop, dst_addr, dst_vma); + ret = copy_user_large_folio(folio, *foliop, + ALIGN_DOWN(dst_addr, size), dst_vma); folio_put(*foliop); *foliop = NULL; if (ret) { @@ -6801,9 +6800,8 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, /* Add shared, newly allocated pages to the page cache. */ if (vm_shared && !is_continue) { - size = i_size_read(mapping->host) >> huge_page_shift(h); ret = -EFAULT; - if (idx >= size) + if (idx >= (i_size_read(mapping->host) >> huge_page_shift(h))) goto out_release_nounlock; /* @@ -6860,7 +6858,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, if (wp_enabled) _dst_pte = huge_pte_mkuffd_wp(_dst_pte); - set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, huge_page_size(h)); + set_huge_pte_at(dst_mm, dst_addr, dst_pte, _dst_pte, size); hugetlb_count_add(pages_per_huge_page(h), dst_mm); diff --git a/mm/memory.c b/mm/memory.c index a48a790a2b5b..12115e45dc24 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6496,20 +6496,17 @@ static int copy_subpage(unsigned long addr, int idx, void *arg) int copy_user_large_folio(struct folio *dst, struct folio *src, unsigned long addr_hint, struct vm_area_struct *vma) { - unsigned int pages_per_huge_page = folio_nr_pages(dst); - unsigned long addr = addr_hint & - ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + unsigned int nr_pages = folio_nr_pages(dst); struct copy_subpage_arg arg = { .dst = dst, .src = src, .vma = vma, }; - if (unlikely(pages_per_huge_page > MAX_ORDER_NR_PAGES)) - return copy_user_gigantic_page(dst, src, addr, vma, - pages_per_huge_page); + if (unlikely(nr_pages > MAX_ORDER_NR_PAGES)) + return copy_user_gigantic_page(dst, src, addr_hint, vma, nr_pages); - return process_huge_page(addr_hint, pages_per_huge_page, copy_subpage, &arg); + return process_huge_page(addr_hint, nr_pages, copy_subpage, &arg); } long copy_folio_from_user(struct folio *dst_folio, From patchwork Tue Jun 18 09:12:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13702002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F288C27C4F for ; Tue, 18 Jun 2024 09:13:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 779268D001F; Tue, 18 Jun 2024 05:13:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 205B38D0020; Tue, 18 Jun 2024 05:13:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D27638D0022; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8B0418D0021 for ; Tue, 18 Jun 2024 05:13:14 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0C869C061C for ; Tue, 18 Jun 2024 09:13:14 +0000 (UTC) X-FDA: 82243445508.09.A0FDED7 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf07.hostedemail.com (Postfix) with ESMTP id 8E84A4000B for ; Tue, 18 Jun 2024 09:13:11 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718701987; a=rsa-sha256; cv=none; b=wAMaw6V4M/g9oHeY5FDVnnbA8da487KfO2TEB/344pQcOVkqfG3MLAGHcLWnd/QEPhx5GQ i3gP02uxXPdUvJoVIun7ThnXZ+EaR0Ag5FpjMP/c7sWzIK4qMBrPtpJJHBRNVcLEVMZ943 uu3sqANhxeo3pJu4xWEe4JoxtrggjXI= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718701987; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nRIjZedw+wW8oKTmRo6eO8q+Cr+8TTKpUyEuUONj+UI=; b=mMFzJ+CJ6UHkXanQfS1LVBqYX1WwDMA+7F0AfqYD3AiJ5xwqCwJtFUtnTjX3w8t0/u1p9Y j2HFSQKSmwzezsZIg9tDJkwXxC7ex7IDye0RJ8/ClCo2RRUfGoI8NWaisdER/GqtByCy5B 5pTqzdl+N11/wy7jWR+LHjHwL0T+Xfg= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4W3LYF2WZwz1ytHL; Tue, 18 Jun 2024 17:09:37 +0800 (CST) Received: from dggpemf100008.china.huawei.com (unknown [7.185.36.138]) by mail.maildlp.com (Postfix) with ESMTPS id 7D66B1402D0; Tue, 18 Jun 2024 17:13:08 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemf100008.china.huawei.com (7.185.36.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Tue, 18 Jun 2024 17:13:08 +0800 From: Kefeng Wang To: Andrew Morton CC: Muchun Song , "Matthew Wilcox (Oracle)" , David Hildenbrand , , Huang Ying , Kefeng Wang Subject: [PATCH v2 4/4] mm: memory: rename pages_per_huge_page to nr_pages Date: Tue, 18 Jun 2024 17:12:42 +0800 Message-ID: <20240618091242.2140164-5-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> References: <20240618091242.2140164-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemf100008.china.huawei.com (7.185.36.138) X-Rspamd-Queue-Id: 8E84A4000B X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: fwxkbnearpmqcw46btoxojxawn5joi5s X-HE-Tag: 1718701991-428206 X-HE-Meta: U2FsdGVkX18xc8Xbqey5ePa+h+0no9+YADL3N9txkU+KJYt+6BVyd9ldljsHYr3dAuUtmODoy+eZKM8VSHoBqEVagxWk/Eu8RFxx39g3SkPg4utcU+EYelaF2E0t06jFyt/qViUo6S5Ny3wwnIOtzfK+oGGgkIl7Pfz8Shy90si3h8U5QX7dzW2lw+TNgFsSQDEVOHOg6eCOo8Ps1IKQdRvl/A73/05mljDvCvQNZ2PZGnC66A4fKo7KuRbbJp6nbetEuq2dG6OvVC/5hiLimJvHB3mH1dQFtqb5cd+1/4a+XI1qJFVweF1a4B/G0XMfTDvELJPIdAPcaZg9wzTxW/LiWjOv6gQ2fNfy55a2ztVuaCQsoa3Q6D1q7JlcKsUegah9ybydLdElWR3ky1BieK52PceHCDXRCLvgNedYyhofMHPrnYt0otk4G8dUcWlQ6V1F7XYfGgBKtRGT0nG4uyl7RGs33+SoehE3hRa0p9NjCmMyQCdZbLfHuwT4ZsGjJGU4pPp6w7JdkIykfi1tFlJE+Td6FnwcKsLMueXcyFUqFz3SB6qC+U4279GGD3SNpr0mkD/ETQxteusKvgdNdopVTi2thIs9VlcvEDRSPvb5osdHAmRkOuPampgA4E39pf5AfIP3sDp2V1dotdU+qxF/4Imh4ALjO9zoQCvY/BlYauCDX5N0LEzIF+xMwx0x3M2AWxvMzeNX18ZgritFHDD5acEBocOTm3vrF6IXUowieQFuLrzBnEmNrW9Q2hcRnTFn5OhdE8773sGEXJZ6ebEobR5KfkEwbePyUznNz1jvqzV4DEbm6gJs/nBKySq0XCb/wFjvNsEBjCj27qun4E03EVcjp2gKfMbH6rtq5VSmtnHb5dr0uzru4+pTFVWtlY4Q3WzMlADaoevIRG8VunKI+R1h/9YJlEwph21j+yaqfhFxuUsJ0waU7Hn8TraQks4myynH9r6LSwNakyr 173QN4fD BdcLCLDmEU7Oi0eX0OLOvWRsk+uuSEQNHuag39I5DRPIk8lzbW3ZI6QwvfJ+EjkmojOJRoGs+s4gv3byqYmE+G/hbGRKmn0qd4gr4xgtMDZkr/81m8W721gBh6JsuoLIaCQ5MNe4o3SYG6XQMJCwY6Sr/+7BhpGdNqYPalfCTXXyXGnk8ujF/M6C6ueLZhs07amfc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Since the callers are converted to use nr_pages naming, use it inside too. Signed-off-by: Kefeng Wang --- mm/memory.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 12115e45dc24..d48879e317b3 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -6366,23 +6366,23 @@ EXPORT_SYMBOL(__might_fault); * cache lines hot. */ static inline int process_huge_page( - unsigned long addr_hint, unsigned int pages_per_huge_page, + unsigned long addr_hint, unsigned int nr_pages, int (*process_subpage)(unsigned long addr, int idx, void *arg), void *arg) { int i, n, base, l, ret; unsigned long addr = addr_hint & - ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); + ~(((unsigned long)nr_pages << PAGE_SHIFT) - 1); /* Process target subpage last to keep its cache lines hot */ might_sleep(); n = (addr_hint - addr) / PAGE_SIZE; - if (2 * n <= pages_per_huge_page) { + if (2 * n <= nr_pages) { /* If target subpage in first half of huge page */ base = 0; l = n; /* Process subpages at the end of huge page */ - for (i = pages_per_huge_page - 1; i >= 2 * n; i--) { + for (i = nr_pages - 1; i >= 2 * n; i--) { cond_resched(); ret = process_subpage(addr + i * PAGE_SIZE, i, arg); if (ret) @@ -6390,8 +6390,8 @@ static inline int process_huge_page( } } else { /* If target subpage in second half of huge page */ - base = pages_per_huge_page - 2 * (pages_per_huge_page - n); - l = pages_per_huge_page - n; + base = nr_pages - 2 * (nr_pages - n); + l = nr_pages - n; /* Process subpages at the begin of huge page */ for (i = 0; i < base; i++) { cond_resched(); @@ -6421,12 +6421,12 @@ static inline int process_huge_page( } static void clear_gigantic_page(struct folio *folio, unsigned long addr, - unsigned int pages_per_huge_page) + unsigned int nr_pages) { int i; might_sleep(); - for (i = 0; i < pages_per_huge_page; i++) { + for (i = 0; i < nr_pages; i++) { cond_resched(); clear_user_highpage(folio_page(folio, i), addr + i * PAGE_SIZE); } @@ -6456,15 +6456,15 @@ void folio_zero_user(struct folio *folio, unsigned long addr_hint) } static int copy_user_gigantic_page(struct folio *dst, struct folio *src, - unsigned long addr, - struct vm_area_struct *vma, - unsigned int pages_per_huge_page) + unsigned long addr, + struct vm_area_struct *vma, + unsigned int nr_pages) { int i; struct page *dst_page; struct page *src_page; - for (i = 0; i < pages_per_huge_page; i++) { + for (i = 0; i < nr_pages; i++) { dst_page = folio_page(dst, i); src_page = folio_page(src, i);