From patchwork Thu Aug 24 14:13:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13364247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F649C27C40 for ; Thu, 24 Aug 2023 14:13:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC0132800B1; Thu, 24 Aug 2023 10:13:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D6EE42800AB; Thu, 24 Aug 2023 10:13:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC1952800B1; Thu, 24 Aug 2023 10:13:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AF7842800AB for ; Thu, 24 Aug 2023 10:13:42 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 85FBD16038A for ; Thu, 24 Aug 2023 14:13:42 +0000 (UTC) X-FDA: 81159191484.19.771B629 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id D27B3100017 for ; Thu, 24 Aug 2023 14:13:40 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Op18IUhA; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692886420; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=70ENJ+e43W+JEfZAYbhdnR1EAncIpy4QJD8Z04bKBDo=; b=hZM8EETPdqaig41VAkBfxBsZbGRAyWmu4SQ/Im+Y610OAV4oPCn3N/984YtM48jdc7CbsJ 1SwDCNQ0hlV0y02KLLtd4m4DSieBz8yi2HCka5H0o3uXw4i0hu0W/OvQonojPQCYqYBpTi mI5sb+/fMPcF8WCcrdxiGxabE02lRjA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Op18IUhA; dmarc=none; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692886420; a=rsa-sha256; cv=none; b=2N3xE8BLbb/TZqp1G7maOaefj4o6QNfjxVeWPFw+AMBR4NaFCrHquEbCZckOnqLIm4Q1/+ +Uss62sGT1B5PrecksyLLWmTPgBQvFzQg0XALmvMNAkRlcY+fjIAZ/XEPvufpbUCofV5Jt 7vhCndBUyM8mQ5Zpq9dth3O49DMzMGU= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=70ENJ+e43W+JEfZAYbhdnR1EAncIpy4QJD8Z04bKBDo=; b=Op18IUhAzFaxGRMhbBPMPn0Ozo W6le7n2ERB4vtEPBO1rvvfEG1IFs0QTJlx26h72wO7nlwhLP15jF7q7UYAXVMSgcoqBjYpDj5cJOZ o9jsaBtjrmh3cyj1qU7UX+JIxu8cNG5yn8dU2KKVgSBNNHOEnpzjzIBROHrfeqmyItz7x7RDGOXpy TgjZQr+GgNK5oIXS675v7QRr6ScsJNbcbfGL8g06/PirwoPAcVoRHgTNoD8cE4sWXrw4hBd77OzX+ ib408JPLpUZEYSnl26s1bOEQE/RslPw4viwkVkS9uo6RWLkTSdAq7IAkWSOlSsWe89v7zQk+Y/W74 NC61e9tw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qZB5U-00BLac-OO; Thu, 24 Aug 2023 14:13:28 +0000 From: "Matthew Wilcox (Oracle)" To: Mike Kravetz Cc: "Matthew Wilcox (Oracle)" , Muchun Song , linux-mm@kvack.org, Muchun Song , Sidhartha Kumar Subject: [PATCH v2 2/3] hugetlb: Remove a few calls to page_folio() Date: Thu, 24 Aug 2023 15:13:24 +0100 Message-Id: <20230824141325.2704553-3-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230824141325.2704553-1-willy@infradead.org> References: <20230824141325.2704553-1-willy@infradead.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: eja6q4n3yw97wtt8sq3ruxtjcos77wdm X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D27B3100017 X-HE-Tag: 1692886420-523606 X-HE-Meta: U2FsdGVkX1/AyF3PWhgb54vr/3xhyD74s7Z1xqQ2kzWPsXzSB8w1cUKcHpogNzgQotHvul4HiJ6xiUFvDJanTkA3jnaiFGondAJ0wg3IHCmZbpeNbQj1y2ZBAts6u3Ryh/1SjTSnKMhMxDpOYrkhMOTCO1QCZ9hheteUDhF1VeJxkCq836DMoctrDvWPsBsdZNgZ9Z7Dcp00Yr8e90Etzimt84SlJfJGgtOWOZDFGApGaq2R3INQpFlWl0uN7kEJhe+g8K5UiJrHn4yc/XuYEI64VYNDhH49JEFkf621bIgsqOrilU0hw/MRaFcCWuf3fCrtS70x0JLVT9vVMNdFZuBJkmy13F1JaIuxD0/diIBleTctdhEmOylYehR2ynCyF5uvclSjP+mxGkdwybERUky/ZvaSRFrUGKlh3rLL7q6Jh55n2uJ3r3Y+HCCHiLy5dOvv/g1ul2pLMaw2OFVzsHP0MPbx6KFTiEfJj4z8ayMK4ThUMNNkIv/r+A97SyR7E5dpsYwSWSyuEVswMm93QQ4fIuXU5xLiicnP2czWXvC5TQENDpa6ZA84lxR3oEbySZG1R9HNHmGqBgXrwXOE3qI1dXtjBLURQSJ4tpZ2Z+xMxxtavKVWaoOWRbKT8kfltSXALe3HKmXDjB5OsaMdE/SH+N8O/RZiBk8v17hphlYDC4roqYSWC5GsBDw08vPuVNTZpmR7yIz5xL1sxmvxoj5k3IHYETMDo4VgKata5a/4iAJEZ6sHQLcCJ2Nfwez1DXlBaSgMSR7nToRvWGk/DSkbZRlvhZc4CcMJ3NPpDOSgeHQcnfZA+idOjr3BQV7YzwAGevjXQRz4Ly9wBdhNR4BjcVbynl1k8ek2YZ1H0T4m3BOolXBj5CEeum6fnrsP/+7msptqJ48fKuZzax61F1dHw8pH9A2NePFYnr4y2RBYixw3DbrU+FZ4dpN2lHNKuvKSNr+XXPvUL8O8++5 Zu+bfRl4 PpIDFAGv4+DqgbYjdB/un7HTJNEVstsc51/yFsZQ8ZBazMQZZbFlYse0rvmG2Nik9/CASSS4dEc+S2hjRVUosrTwgAvZn6y25uHPCnaYtWG4tDBMPqKtFZD3DbZvuXRP2VAimSZNOQxpcDH4RjneKrz+QApOKKlcgkl0TutE5ufCzm1q9lDd79VOD7VhjaYyQ8LiHqDFAZ5HQkEX62cEc/Zs+SeuJdfUP0M9D+ic1o7W0rOaCWHpKodDicMtW+xG6fetkjTQhwJcV0TcZ/D+RihV2YU9ABo19ELm/DQ568jdLZYiZFvJYvze+Cw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Anything found on a linked list threaded through ->lru is guaranteed to be a folio as the compound_head found in a tail page overlaps the ->lru member of struct page. So we can pull folios directly off these lists no matter whether pages or folios were added to the list. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Mike Kravetz Reviewed-by: Muchun Song Cc: Sidhartha Kumar --- mm/hugetlb.c | 26 +++++++++++--------------- 1 file changed, 11 insertions(+), 15 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index d1c856628bac..8f548a274e3c 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1835,11 +1835,9 @@ static void update_and_free_hugetlb_folio(struct hstate *h, struct folio *folio, static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) { - struct page *page, *t_page; - struct folio *folio; + struct folio *folio, *t_folio; - list_for_each_entry_safe(page, t_page, list, lru) { - folio = page_folio(page); + list_for_each_entry_safe(folio, t_folio, list, lru) { update_and_free_hugetlb_folio(h, folio, false); cond_resched(); } @@ -2228,8 +2226,7 @@ static struct page *remove_pool_huge_page(struct hstate *h, bool acct_surplus) { int nr_nodes, node; - struct page *page = NULL; - struct folio *folio; + struct folio *folio = NULL; lockdep_assert_held(&hugetlb_lock); for_each_node_mask_to_free(h, nr_nodes, node, nodes_allowed) { @@ -2239,15 +2236,14 @@ static struct page *remove_pool_huge_page(struct hstate *h, */ if ((!acct_surplus || h->surplus_huge_pages_node[node]) && !list_empty(&h->hugepage_freelists[node])) { - page = list_entry(h->hugepage_freelists[node].next, - struct page, lru); - folio = page_folio(page); + folio = list_entry(h->hugepage_freelists[node].next, + struct folio, lru); remove_hugetlb_folio(h, folio, acct_surplus); break; } } - return page; + return &folio->page; } /* @@ -3363,15 +3359,15 @@ static void try_to_free_low(struct hstate *h, unsigned long count, * Collect pages to be freed on a list, and free after dropping lock */ for_each_node_mask(i, *nodes_allowed) { - struct page *page, *next; + struct folio *folio, *next; struct list_head *freel = &h->hugepage_freelists[i]; - list_for_each_entry_safe(page, next, freel, lru) { + list_for_each_entry_safe(folio, next, freel, lru) { if (count >= h->nr_huge_pages) goto out; - if (PageHighMem(page)) + if (folio_test_highmem(folio)) continue; - remove_hugetlb_folio(h, page_folio(page), false); - list_add(&page->lru, &page_list); + remove_hugetlb_folio(h, folio, false); + list_add(&folio->lru, &page_list); } }