From patchwork Fri Jun 17 15:42:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12885780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C31EC43334 for ; Fri, 17 Jun 2022 15:42:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF9156B0093; Fri, 17 Jun 2022 11:42:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA9056B0095; Fri, 17 Jun 2022 11:42:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A49066B0096; Fri, 17 Jun 2022 11:42:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 945EB6B0093 for ; Fri, 17 Jun 2022 11:42:58 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 6F8D135081 for ; Fri, 17 Jun 2022 15:42:58 +0000 (UTC) X-FDA: 79588146036.25.D6B1BDB Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf12.hostedemail.com (Postfix) with ESMTP id 13B7C4009C for ; Fri, 17 Jun 2022 15:42:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=rDbjAESLnHCRiSLPfWEbzhDiwvftzaCRXzKfAi1/IOE=; b=nej3F/zUcxPD9/PSWwuE8N0RlD ra4ZQTIdQiO5GJO62vj7PJVXORBFAw/W9DFz3PgZFy0nky7kaKE9RJpUk3cBCk8/anWuLh//j32Qe UX8kXIIB5RyFa7hRq1GIoxStSa1+ipuUCM7gGVc7fN83XsebJC0Osa+oTzSG1Ay9hRBFPN3P3CK5g rGYODE+m8UsZr8URxy1oVmU4l8+5aFfw6feqyx8a8C8gjVno7sc2ad6z81fJ4MNixWoT7SlcMYzCn ggx+spJK/RG3at2JSb87nPMC0dvv9bLKGVxJE/VJkcTcqbKgs6jDjZnaymod6azv56YSnoOzpys15 55XqY8WA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1o2E7W-002wEl-FB; Fri, 17 Jun 2022 15:42:50 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 3/5] mm/vmscan: Convert move_pages_to_lru() to use a folio Date: Fri, 17 Jun 2022 16:42:46 +0100 Message-Id: <20220617154248.700416-4-willy@infradead.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220617154248.700416-1-willy@infradead.org> References: <20220617154248.700416-1-willy@infradead.org> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="nej3F/zU"; dmarc=none; spf=none (imf12.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655480578; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rDbjAESLnHCRiSLPfWEbzhDiwvftzaCRXzKfAi1/IOE=; b=AJz5wSRmr80Kwr1aCY3Jehke8imR5lFlf9zKjDkn4rGQMzyJTP6a7+p9cahMyJNcerIqYf QVS7B3WUrKP1aIVv4EbFv+8hqUHsh/zm96h8mPpq7+z0tqvBjgS3Q7pXaFSTfUHjQELinS uxSslPK1/F4oMiU+wly+rcZNQZkWqUE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655480578; a=rsa-sha256; cv=none; b=lcofOmtgjNKfwURkc+eLGDqLsHVcVjaywsnc/M20szLgwUEJMMfI5CaX9NoC1yT7ODLuz+ UgIGrbSrB4zA+AuWyHy9qWi/LILmWVznaJDgfnnz6RpS+oOWZlfpGcYbYcUXcSK7oSiDIY iauLKSyZ5giJt9QX+0nUyTzQJIssf8s= X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 13B7C4009C X-Rspam-User: Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b="nej3F/zU"; dmarc=none; spf=none (imf12.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Stat-Signature: 58iucnsn1uhw4b4ds47t8idhey3chpzq X-HE-Tag: 1655480577-305860 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Remove a few hidden calls to compound_head, saving 387 bytes of text on my test configuration. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig --- mm/vmscan.c | 54 ++++++++++++++++++++++++++--------------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 81d61139bbb1..f8ec446041c3 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2254,8 +2254,8 @@ static int too_many_isolated(struct pglist_data *pgdat, int file, } /* - * move_pages_to_lru() moves pages from private @list to appropriate LRU list. - * On return, @list is reused as a list of pages to be freed by the caller. + * move_pages_to_lru() moves folios from private @list to appropriate LRU list. + * On return, @list is reused as a list of folios to be freed by the caller. * * Returns the number of pages moved to the given lruvec. */ @@ -2263,42 +2263,42 @@ static unsigned int move_pages_to_lru(struct lruvec *lruvec, struct list_head *list) { int nr_pages, nr_moved = 0; - LIST_HEAD(pages_to_free); - struct page *page; + LIST_HEAD(folios_to_free); while (!list_empty(list)) { - page = lru_to_page(list); - VM_BUG_ON_PAGE(PageLRU(page), page); - list_del(&page->lru); - if (unlikely(!page_evictable(page))) { + struct folio *folio = lru_to_folio(list); + + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + list_del(&folio->lru); + if (unlikely(!folio_evictable(folio))) { spin_unlock_irq(&lruvec->lru_lock); - putback_lru_page(page); + folio_putback_lru(folio); spin_lock_irq(&lruvec->lru_lock); continue; } /* - * The SetPageLRU needs to be kept here for list integrity. + * The folio_set_lru needs to be kept here for list integrity. * Otherwise: * #0 move_pages_to_lru #1 release_pages - * if !put_page_testzero - * if (put_page_testzero()) - * !PageLRU //skip lru_lock - * SetPageLRU() - * list_add(&page->lru,) - * list_add(&page->lru,) + * if (!folio_put_testzero()) + * if (folio_put_testzero()) + * !lru //skip lru_lock + * folio_set_lru() + * list_add(&folio->lru,) + * list_add(&folio->lru,) */ - SetPageLRU(page); + folio_set_lru(folio); - if (unlikely(put_page_testzero(page))) { - __clear_page_lru_flags(page); + if (unlikely(folio_put_testzero(folio))) { + __folio_clear_lru_flags(folio); - if (unlikely(PageCompound(page))) { + if (unlikely(folio_test_large(folio))) { spin_unlock_irq(&lruvec->lru_lock); - destroy_compound_page(page); + destroy_compound_page(&folio->page); spin_lock_irq(&lruvec->lru_lock); } else - list_add(&page->lru, &pages_to_free); + list_add(&folio->lru, &folios_to_free); continue; } @@ -2307,18 +2307,18 @@ static unsigned int move_pages_to_lru(struct lruvec *lruvec, * All pages were isolated from the same lruvec (and isolation * inhibits memcg migration). */ - VM_BUG_ON_PAGE(!folio_matches_lruvec(page_folio(page), lruvec), page); - add_page_to_lru_list(page, lruvec); - nr_pages = thp_nr_pages(page); + VM_BUG_ON_FOLIO(!folio_matches_lruvec(folio, lruvec), folio); + lruvec_add_folio(lruvec, folio); + nr_pages = folio_nr_pages(folio); nr_moved += nr_pages; - if (PageActive(page)) + if (folio_test_active(folio)) workingset_age_nonresident(lruvec, nr_pages); } /* * To save our caller's stack, now use input list for pages to free. */ - list_splice(&pages_to_free, list); + list_splice(&folios_to_free, list); return nr_moved; }