From patchwork Fri Mar 1 21:47:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13579162 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6760CC54E4A for ; Fri, 1 Mar 2024 21:47:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DD896B009B; Fri, 1 Mar 2024 16:47:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 66BB46B009F; Fri, 1 Mar 2024 16:47:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D84F6B009D; Fri, 1 Mar 2024 16:47:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3F2456B009B for ; Fri, 1 Mar 2024 16:47:17 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 119BD8012C for ; Fri, 1 Mar 2024 21:47:17 +0000 (UTC) X-FDA: 81849806514.05.DA9B004 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf25.hostedemail.com (Postfix) with ESMTP id 92E15A000E for ; Fri, 1 Mar 2024 21:47:15 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Bi6Jm4kc; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709329635; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F8cxoSYHCxUiA/zgEJyY7ynTvnpkxtkAiF+Pei7dmM4=; b=PLeUVlI+CdMq2AhnsgNSL0VFuYcp8FBUNSGs4T8KJKYo8+THxqHzLssKwBkXHhe9+zczrd Bb//n0D6Ii9SdHFYZ8huas6psjpzEFdr7C9Bgew7YLzgJyFfGg/nHdmkrSIIK5zecVtA0j QqPOWieyMwO5IkeErXp8gRF+NBGqYAI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Bi6Jm4kc; dmarc=none; spf=none (imf25.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709329635; a=rsa-sha256; cv=none; b=fcW3B+4lx3aFBCd3bVLWK1PQoajTKYjvX66LIZzV54d0kRX0eKDoff0veqOc7fVDeRKcQu iySlJqeq7Q8tZPKjA8JgMos6UKPmiCv+s2SAQJ14iOFAjhZvmYLYHwWdsGTUFPpF8KiMe8 JwHLpLrNc0TOejl8INikfH+T/2NSFfA= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=F8cxoSYHCxUiA/zgEJyY7ynTvnpkxtkAiF+Pei7dmM4=; b=Bi6Jm4kc8KkQNzDOW3clachwrc nMJCuqTG2pV86TVGZFVfaQ9H99Dc7fpYVvZgKNFiN1rTmeKNXvpQV0+vjifUGzD32ZcTjWtNvoopc wggsbh/bo7qxOQf12VB7CeWebC+mZQs7LYIsliUe5HLzgNQHgM/ZE49ZtRAjLeLwbgmD5XFZ7+eOW 0+lC/PDJu/+iFYHCxInZ7nuLcKxJhtGsMLA4KdNxSegag7Vj6y3Pmc4LAf8hKqV9Gq/Hsq+fR1J8k TJER4b9NUiOqNrXLwC4g5tVk7HCLUJ0B1FUKj2Q+dbV3GRn4w9xguuAtKmuAHdK20+AeNIGJJ+wAT BwqT7VCQ==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rgAin-0000000ByEw-3rRH; Fri, 01 Mar 2024 21:47:13 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org Cc: "Matthew Wilcox (Oracle)" , Oscar Salvador Subject: [PATCH 5/5] compaction: Use hugetlb_pfn_folio in isolate_migratepages_block Date: Fri, 1 Mar 2024 21:47:10 +0000 Message-ID: <20240301214712.2853147-6-willy@infradead.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240301214712.2853147-1-willy@infradead.org> References: <20240301214712.2853147-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 92E15A000E X-Stat-Signature: 1cyubfzmdb6w88yjhm4uj998h1msmcy8 X-HE-Tag: 1709329635-831103 X-HE-Meta: U2FsdGVkX1/hN/2sWhV+fyPG+NEeZ6TfqfIrUG4CFDZgy+A0faHuI5b5gPJkwh+uX7d8zV0YRySongLtEQacIGmpamM233ZCLFk0wvokQUFDlL8HI+qqDH2yO+Pzh+Jd1PPlokKd3rkujzzDKm3DUoqoSE02ppMNBHBHhzkRtgKWHOfqGTfHyNhIp8PFiEgo2V4ztiuYdO/vI8oznuUJh2W4fEc1Q6I6pEbyuDhxLJ1tVj5n4ifV5D602ccgMxH9mA3m5OS6es+gJ6Dz2m1suhDGPQS+Sno111uOeXxFOoYjUtlXudlnq1QhOPkXw2ueRYdBFKnRTEEnmwanEQE9FPhvQV/DBJlFQfSL2amwnmc+ZjpRJCxtJuymu/+qX2cFlP66ZKQlIeTeghDiJQ6X5A0q7jVUymRW50bcnwtpJTwXdhdm7xDWXZJDPYhYtLAgFASMY3KDNyaA2hv5hAwFPKqGgk5uo/RdLfu5ljW2Hvz6KAfItd/FVFbklnLy6D2MQLSo+NkdxhIvH155HYmQaBK9cgMOfVRcscM+zOWgI5Eansnf2vxlvc3EDvd8N7KURAGmy1kWIu+iAD89luxGDYDs9FZe+usrLq2WtZxLZRaDpnse+d0hLsmxwlr1eXX9oOOrdTHgd0rCc98DES+tt7d7BATZP+8m1jzfRHYKXM/99vyfXukrT9lzfBo462DncyJ4QNfalJbg5XFSpCbE01kNFZiBvbnSUQe8aOSZky2Prq7ZCg2LycQ1kEwamPj0AHVU2xm9yoroxc5ZjiwbU+N1TmH0HjglT2zv/P43TXIVlUsUy7mfSsa//1yHgL+fpJ9NHax1wkBpSA3/yw0sPafR0WvEkur/7JndLM1tR4Ehdl9yTNzaITK0VclkgXCDfrfnMCu5yjkh6B8XDCuWSLBfAYIWWuhl/rCwnbISj7Q= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is more efficient than calling page_folio() ourselves, and removes a call to PageHuge() which we're trying to get rid of. We do not hold the hugetlb_lock here, so the folio may be dissolved under us, but we check for that in hugetlb_isolate_or_dissolve() (which now takes a folio argument instead of converting to a folio itself). Signed-off-by: Matthew Wilcox (Oracle) --- include/linux/hugetlb.h | 4 ++-- mm/compaction.c | 16 ++++++++-------- mm/hugetlb.c | 10 +++++----- 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 89f4b90eec68..436086a300f5 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -736,7 +736,7 @@ struct huge_bootmem_page { struct hstate *hstate; }; -int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list); +int hugetlb_isolate_or_dissolve(struct folio *folio, struct list_head *list); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, int avoid_reserve); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred_nid, @@ -1043,7 +1043,7 @@ static inline struct folio *filemap_lock_hugetlb_folio(struct hstate *h, return NULL; } -static inline int isolate_or_dissolve_huge_page(struct page *page, +static inline int hugetlb_isolate_or_dissolve(struct folio *folio, struct list_head *list) { return -ENOMEM; diff --git a/mm/compaction.c b/mm/compaction.c index 807b58e6eb68..ad3430c22ede 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1001,17 +1001,18 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, valid_page = page; } - if (PageHuge(page)) { + folio = hugetlb_pfn_folio(low_pfn); + if (folio) { /* * skip hugetlbfs if we are not compacting for pages * bigger than its order. THPs and other compound pages * are handled below. */ if (!cc->alloc_contig) { - const unsigned int order = compound_order(page); + const unsigned int order = folio_order(folio); if (order <= MAX_PAGE_ORDER) { - low_pfn += (1UL << order) - 1; + low_pfn |= (1UL << order) - 1; nr_scanned += (1UL << order) - 1; } goto isolate_fail; @@ -1022,7 +1023,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, locked = NULL; } - ret = isolate_or_dissolve_huge_page(page, &cc->migratepages); + ret = hugetlb_isolate_or_dissolve(folio, &cc->migratepages); /* * Fail isolation in case isolate_or_dissolve_huge_page() @@ -1032,17 +1033,16 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, /* Do not report -EBUSY down the chain */ if (ret == -EBUSY) ret = 0; - low_pfn += compound_nr(page) - 1; - nr_scanned += compound_nr(page) - 1; + low_pfn += folio_nr_pages(folio) - 1; + nr_scanned += folio_nr_pages(folio) - 1; goto isolate_fail; } - if (PageHuge(page)) { + if (folio_test_hugetlb(folio)) { /* * Hugepage was successfully isolated and placed * on the cc->migratepages list. */ - folio = page_folio(page); low_pfn += folio_nr_pages(folio) - 1; goto isolate_success_no_list; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 2e6ebedb75a8..7832b0730e80 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3160,16 +3160,16 @@ static int alloc_and_dissolve_hugetlb_folio(struct hstate *h, return ret; } -int isolate_or_dissolve_huge_page(struct page *page, struct list_head *list) +int hugetlb_isolate_or_dissolve(struct folio *folio, struct list_head *list) { struct hstate *h; - struct folio *folio = page_folio(page); int ret = -EBUSY; /* - * The page might have been dissolved from under our feet, so make sure - * to carefully check the state under the lock. - * Return success when racing as if we dissolved the page ourselves. + * The folio might have been dissolved from under our feet, + * so make sure to carefully check the state under the lock. + * Return success when racing as if we dissolved the folio + * ourselves. */ spin_lock_irq(&hugetlb_lock); if (folio_test_hugetlb(folio)) {