From patchwork Mon Apr 4 19:30:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12800659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1158C433F5 for ; Mon, 4 Apr 2022 19:32:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4B7C66B0078; Mon, 4 Apr 2022 15:30:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 466B66B007B; Mon, 4 Apr 2022 15:30:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 331BB6B007D; Mon, 4 Apr 2022 15:30:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0144.hostedemail.com [216.40.44.144]) by kanga.kvack.org (Postfix) with ESMTP id 2605B6B0078 for ; Mon, 4 Apr 2022 15:30:35 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CBF9A824C424 for ; Mon, 4 Apr 2022 19:30:24 +0000 (UTC) X-FDA: 79320187968.22.F6B2520 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id 5B4E9180021 for ; Mon, 4 Apr 2022 19:30:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=aaht8wjestcuAz9uUmgOlm+Z6jPOdwv91UY9zwH12ng=; b=j9dXNAVZKFuyzfso72UeyH6GQI NrRnfmMEnBMtzhYp0v59UfMFQgeJMP5euuxVEPEuXoNtFvUfgqfYaqW3vxd6QtH0j6/eLRDGqT6Kt 2WZ/02DJYDNaQQ1lJ+8Lh/2UtQ5XUiFQ5bIDvpy+uY+9Q1P8K8ieTMeD/Jol/GkWtPzpRQ6/J88sA u5SSehhtRCdQLBCbKFUKyC/riArNrkF1Hrzlw6JQv8xFyOodiWBhgIaDF2m++oA0kyLLYApEdhtuQ 5Qr7N5QoySTZFo8QUA6Anw8VOH04fNqurO7GfbaY6+vNZyks7s/hVvmrq8RN0hpY/5D5QyWEjOE7x j8yxiJTg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbSOv-005zqJ-6M; Mon, 04 Apr 2022 19:30:09 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton , Zi Yan Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 1/4] mm/migrate: Use a folio in alloc_migration_target() Date: Mon, 4 Apr 2022 20:30:03 +0100 Message-Id: <20220404193006.1429250-2-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220404193006.1429250-1-willy@infradead.org> References: <20220404193006.1429250-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: cbjh7k435j9tijar19panig7kduarwdx Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=j9dXNAVZ; dmarc=none; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 5B4E9180021 X-HE-Tag: 1649100624-411325 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This removes an assumption that a large folio is HPAGE_PMD_ORDER as well as letting us remove the call to prep_transhuge_page() and a few hidden calls to compound_head(). Signed-off-by: Matthew Wilcox (Oracle) --- mm/migrate.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index de175e2fdba5..9894e90db006 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1520,10 +1520,11 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, struct page *alloc_migration_target(struct page *page, unsigned long private) { + struct folio *folio = page_folio(page); struct migration_target_control *mtc; gfp_t gfp_mask; unsigned int order = 0; - struct page *new_page = NULL; + struct folio *new_folio = NULL; int nid; int zidx; @@ -1531,34 +1532,31 @@ struct page *alloc_migration_target(struct page *page, unsigned long private) gfp_mask = mtc->gfp_mask; nid = mtc->nid; if (nid == NUMA_NO_NODE) - nid = page_to_nid(page); + nid = folio_nid(folio); - if (PageHuge(page)) { - struct hstate *h = page_hstate(compound_head(page)); + if (folio_test_hugetlb(folio)) { + struct hstate *h = page_hstate(&folio->page); gfp_mask = htlb_modify_alloc_mask(h, gfp_mask); return alloc_huge_page_nodemask(h, nid, mtc->nmask, gfp_mask); } - if (PageTransHuge(page)) { + if (folio_test_large(folio)) { /* * clear __GFP_RECLAIM to make the migration callback * consistent with regular THP allocations. */ gfp_mask &= ~__GFP_RECLAIM; gfp_mask |= GFP_TRANSHUGE; - order = HPAGE_PMD_ORDER; + order = folio_order(folio); } - zidx = zone_idx(page_zone(page)); + zidx = zone_idx(folio_zone(folio)); if (is_highmem_idx(zidx) || zidx == ZONE_MOVABLE) gfp_mask |= __GFP_HIGHMEM; - new_page = __alloc_pages(gfp_mask, order, nid, mtc->nmask); - - if (new_page && PageTransHuge(new_page)) - prep_transhuge_page(new_page); + new_folio = __folio_alloc(gfp_mask, order, nid, mtc->nmask); - return new_page; + return &new_folio->page; } #ifdef CONFIG_NUMA From patchwork Mon Apr 4 19:30:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12800657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2550C433EF for ; Mon, 4 Apr 2022 19:31:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1AA756B0074; Mon, 4 Apr 2022 15:30:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 159E86B0075; Mon, 4 Apr 2022 15:30:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 022D16B0078; Mon, 4 Apr 2022 15:30:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id E7F896B0074 for ; Mon, 4 Apr 2022 15:30:28 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A71F7ACF73 for ; Mon, 4 Apr 2022 19:30:18 +0000 (UTC) X-FDA: 79320187716.16.1948FAA Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf06.hostedemail.com (Postfix) with ESMTP id 41413180023 for ; Mon, 4 Apr 2022 19:30:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=zGKU3GX4K8cLnPSJeQb0clMP2eDTdoqf6QidgEjwTGg=; b=fbUPRaeA8zq1jA/TU/IGcDhe4f sSQIiha3tsQwkm9KOa1LMqar+iCoeZscOv3ZvKjkHxwqILelhwxprSDIYruf6xwM3OW3lzLaMvUTZ IaFywqd4u/5u0Zb1Kxr6SXR2QSp4UUb1rSQREeuwU33SGS+RI1gEK5fASUK5Sj2HUMEMrRYjaO5iR g76KOdxQocEp5dGcuqw1OC3+CqykbRDIzwAnYpBWvebB8qVO5aHBY9dcRMFmsoxGmV8VIVso+oTq6 9S/qncGCMadJ+gg76mAdXxJy903A0BaUCBAvJTATW8YZPeVWmbuxkv3FYH+gZ9IAgovTbUSMSrL6g VeDOMrdw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbSOv-005zqL-9d; Mon, 04 Apr 2022 19:30:09 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton , Zi Yan Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 2/4] mm/migrate: Use a folio in migrate_misplaced_transhuge_page() Date: Mon, 4 Apr 2022 20:30:04 +0100 Message-Id: <20220404193006.1429250-3-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220404193006.1429250-1-willy@infradead.org> References: <20220404193006.1429250-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: b9cjzcmms3ukkemhpmytmfgn5r686od3 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=fbUPRaeA; spf=none (imf06.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 41413180023 X-HE-Tag: 1649100618-774796 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Unify alloc_misplaced_dst_page() and alloc_misplaced_dst_page_thp(). Removes an assumption that compound pages are either order 0 or HPAGE_PMD_ORDER. Signed-off-by: Matthew Wilcox (Oracle) --- mm/migrate.c | 58 +++++++++++++++------------------------------------- 1 file changed, 16 insertions(+), 42 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 9894e90db006..6c31ee1e1c9b 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1997,32 +1997,20 @@ static struct page *alloc_misplaced_dst_page(struct page *page, unsigned long data) { int nid = (int) data; - struct page *newpage; - - newpage = __alloc_pages_node(nid, - (GFP_HIGHUSER_MOVABLE | - __GFP_THISNODE | __GFP_NOMEMALLOC | - __GFP_NORETRY | __GFP_NOWARN) & - ~__GFP_RECLAIM, 0); - - return newpage; -} - -static struct page *alloc_misplaced_dst_page_thp(struct page *page, - unsigned long data) -{ - int nid = (int) data; - struct page *newpage; - - newpage = alloc_pages_node(nid, (GFP_TRANSHUGE_LIGHT | __GFP_THISNODE), - HPAGE_PMD_ORDER); - if (!newpage) - goto out; - - prep_transhuge_page(newpage); + int order = compound_order(page); + gfp_t gfp = __GFP_THISNODE; + struct folio *new; + + if (order > 0) + gfp |= GFP_TRANSHUGE_LIGHT; + else { + gfp |= GFP_HIGHUSER_MOVABLE | __GFP_NOMEMALLOC | __GFP_NORETRY | + __GFP_NOWARN; + gfp &= ~__GFP_RECLAIM; + } + new = __folio_alloc_node(gfp, order, nid); -out: - return newpage; + return &new->page; } static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) @@ -2080,22 +2068,8 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, int nr_remaining; unsigned int nr_succeeded; LIST_HEAD(migratepages); - new_page_t *new; - bool compound; int nr_pages = thp_nr_pages(page); - /* - * PTE mapped THP or HugeTLB page can't reach here so the page could - * be either base page or THP. And it must be head page if it is - * THP. - */ - compound = PageTransHuge(page); - - if (compound) - new = alloc_misplaced_dst_page_thp; - else - new = alloc_misplaced_dst_page; - /* * Don't migrate file pages that are mapped in multiple processes * with execute permissions as they are probably shared libraries. @@ -2116,9 +2090,9 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, goto out; list_add(&page->lru, &migratepages); - nr_remaining = migrate_pages(&migratepages, *new, NULL, node, - MIGRATE_ASYNC, MR_NUMA_MISPLACED, - &nr_succeeded); + nr_remaining = migrate_pages(&migratepages, alloc_misplaced_dst_page, + NULL, node, MIGRATE_ASYNC, + MR_NUMA_MISPLACED, &nr_succeeded); if (nr_remaining) { if (!list_empty(&migratepages)) { list_del(&page->lru); From patchwork Mon Apr 4 19:30:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12800655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC02AC433F5 for ; Mon, 4 Apr 2022 19:30:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BB366B0071; Mon, 4 Apr 2022 15:30:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 26B0A6B0073; Mon, 4 Apr 2022 15:30:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10C486B0074; Mon, 4 Apr 2022 15:30:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 01BB96B0071 for ; Mon, 4 Apr 2022 15:30:24 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D52FD60D22 for ; Mon, 4 Apr 2022 19:30:14 +0000 (UTC) X-FDA: 79320187548.01.DEEFFE8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id 19FB114000C for ; Mon, 4 Apr 2022 19:30:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=g5EuA3sUOB9xc2yyUxVMVulM7hFVZ05exS99l8H4tIs=; b=m9WE9605BZ1hXWvnHZQT06QR25 QD6EabJtrDmuW6pYF3ALUP2lRv6Vd3RDsWghPhwyNzcVRrF9SNm/4kd4kVdq2ZAZRtEFllChwi0UH qpq0Pc/3i48YzGGusqB1aeMEPA8bknPoEOVSgObtJgB1pjr2sWkUr+sV7fbNoIiC9VX8Qrxqp+nfW AwX07/R0Wedyx228CU0fiH57nwk0t8PGWUSLBJcuOJyy7PnZRP6HpzugR1AmoyKc3NQUY6c/IucBB YHCrL2Y57X6N3ucBlZVQiHTdDxds/5bUM5NqvKkiVmZUZBDtn+TU4QcJor/Xm7WYXH2qBrl2pto6S RyICoQaA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbSOv-005zqN-C2; Mon, 04 Apr 2022 19:30:09 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton , Zi Yan Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 3/4] mm: Add vma_alloc_folio() Date: Mon, 4 Apr 2022 20:30:05 +0100 Message-Id: <20220404193006.1429250-4-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220404193006.1429250-1-willy@infradead.org> References: <20220404193006.1429250-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: wjbyjwqjaz33ie89zmcrghxrwihiqfki Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=m9WE9605; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 19FB114000C X-HE-Tag: 1649100612-130157 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This wrapper around alloc_pages_vma() calls prep_transhuge_page(), removing the obligation from the caller. This is in the same spirit as __folio_alloc(). Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/gfp.h | 8 ++++++-- mm/mempolicy.c | 13 +++++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 761f8f1885c7..3e3d36fc2109 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -613,9 +613,11 @@ static inline struct page *alloc_pages_node(int nid, gfp_t gfp_mask, #ifdef CONFIG_NUMA struct page *alloc_pages(gfp_t gfp, unsigned int order); struct folio *folio_alloc(gfp_t gfp, unsigned order); -extern struct page *alloc_pages_vma(gfp_t gfp_mask, int order, +struct page *alloc_pages_vma(gfp_t gfp_mask, int order, struct vm_area_struct *vma, unsigned long addr, bool hugepage); +struct folio *vma_alloc_folio(gfp_t gfp, int order, struct vm_area_struct *vma, + unsigned long addr, bool hugepage); #define alloc_hugepage_vma(gfp_mask, vma, addr, order) \ alloc_pages_vma(gfp_mask, order, vma, addr, true) #else @@ -627,8 +629,10 @@ static inline struct folio *folio_alloc(gfp_t gfp, unsigned int order) { return __folio_alloc_node(gfp, order, numa_node_id()); } -#define alloc_pages_vma(gfp_mask, order, vma, addr, false)\ +#define alloc_pages_vma(gfp_mask, order, vma, addr, hugepage) \ alloc_pages(gfp_mask, order) +#define vma_alloc_folio(gfp, order, vma, addr, hugepage) \ + folio_alloc(gfp, order) #define alloc_hugepage_vma(gfp_mask, vma, addr, order) \ alloc_pages(gfp_mask, order) #endif diff --git a/mm/mempolicy.c b/mm/mempolicy.c index a2516d31db6c..ec15f4f4b714 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2227,6 +2227,19 @@ struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, } EXPORT_SYMBOL(alloc_pages_vma); +struct folio *vma_alloc_folio(gfp_t gfp, int order, struct vm_area_struct *vma, + unsigned long addr, bool hugepage) +{ + struct folio *folio; + + folio = (struct folio *)alloc_pages_vma(gfp, order, vma, addr, + hugepage); + if (folio && order > 1) + prep_transhuge_page(&folio->page); + + return folio; +} + /** * alloc_pages - Allocate pages. * @gfp: GFP flags. From patchwork Mon Apr 4 19:30:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12800658 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6F42C433EF for ; Mon, 4 Apr 2022 19:31:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 358F66B0075; Mon, 4 Apr 2022 15:30:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 307376B0078; Mon, 4 Apr 2022 15:30:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1CFA26B007B; Mon, 4 Apr 2022 15:30:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id 0E9946B0075 for ; Mon, 4 Apr 2022 15:30:32 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B07F3AD6C0 for ; Mon, 4 Apr 2022 19:30:21 +0000 (UTC) X-FDA: 79320187842.24.32F9CD8 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf29.hostedemail.com (Postfix) with ESMTP id 5502012002A for ; Mon, 4 Apr 2022 19:30:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=uKUep2HlJ8UfbnV1THdQ1lp4KELm5uKT2Ceq2OpuL64=; b=ugdSQE+V5s7/a10CzLczHaUUdC e4mQ4OMUj9CfB+7TKiRhjZRHiXdMXguLBAsZ8WIpQa9Px1wfYf+PFchDyZcm+1ulH/t4Jl4K3zCCK hZwwUwPd97obXV+Q6NUj8/0rDA0Un9uzESsT5Oddimo9j1J9mjEJ7bhYZuFDzXur48mm7unqF5V1h D2AHz3zItU4VInOeFV5z/QvJlQt/b/Z5ar8FWbvKKrkTBH/i/Phep97ezEWGoLKcqvBdcYV6MTHfc wR5itOsztrj6qlNKuJbFiwBfOvRsL8Wcp8i//l6Q6Uf5WUrNaFya5UecAkmqGuAQ3+aY0iPJw0Tjt JR2RZYGg==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nbSOv-005zqP-Ec; Mon, 04 Apr 2022 19:30:09 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton , Zi Yan Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 4/4] mm/mempolicy: Use vma_alloc_folio() in new_page() Date: Mon, 4 Apr 2022 20:30:06 +0100 Message-Id: <20220404193006.1429250-5-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220404193006.1429250-1-willy@infradead.org> References: <20220404193006.1429250-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: ofq9bnxra4fdbaqe73nbwj9nacftsd7c Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=ugdSQE+V; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Queue-Id: 5502012002A X-HE-Tag: 1649100621-369097 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Simplify new_page() by unifying the THP and base page cases, and handle orders other than 0 and HPAGE_PMD_ORDER correctly. Signed-off-by: Matthew Wilcox (Oracle) --- mm/mempolicy.c | 25 +++++++++++-------------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index ec15f4f4b714..d6d7a95b0cce 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1191,8 +1191,10 @@ int do_migrate_pages(struct mm_struct *mm, const nodemask_t *from, */ static struct page *new_page(struct page *page, unsigned long start) { + struct folio *dst, *src = page_folio(page); struct vm_area_struct *vma; unsigned long address; + gfp_t gfp = GFP_HIGHUSER_MOVABLE | __GFP_RETRY_MAYFAIL; vma = find_vma(current->mm, start); while (vma) { @@ -1202,24 +1204,19 @@ static struct page *new_page(struct page *page, unsigned long start) vma = vma->vm_next; } - if (PageHuge(page)) { - return alloc_huge_page_vma(page_hstate(compound_head(page)), + if (folio_test_hugetlb(src)) { + return alloc_huge_page_vma(page_hstate(&src->page), vma, address); - } else if (PageTransHuge(page)) { - struct page *thp; - - thp = alloc_hugepage_vma(GFP_TRANSHUGE, vma, address, - HPAGE_PMD_ORDER); - if (!thp) - return NULL; - prep_transhuge_page(thp); - return thp; + } else if (folio_test_large(src)) { + gfp = GFP_TRANSHUGE; } + /* - * if !vma, alloc_page_vma() will use task or system default policy + * if !vma, vma_alloc_folio() will use task or system default policy */ - return alloc_page_vma(GFP_HIGHUSER_MOVABLE | __GFP_RETRY_MAYFAIL, - vma, address); + dst = vma_alloc_folio(gfp, folio_order(src), vma, address, + folio_test_large(src)); + return &dst->page; } #else