From patchwork Wed Oct 2 15:25:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 13819964 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46ED8CF6D3A for ; Wed, 2 Oct 2024 15:25:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89FD54401B7; Wed, 2 Oct 2024 11:25:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 82A144401B5; Wed, 2 Oct 2024 11:25:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CBC24401B7; Wed, 2 Oct 2024 11:25:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3BEF44401B5 for ; Wed, 2 Oct 2024 11:25:51 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A729C120998 for ; Wed, 2 Oct 2024 15:25:50 +0000 (UTC) X-FDA: 82629037260.04.19B14A1 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 1436020002 for ; Wed, 2 Oct 2024 15:25:48 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=n2UuYYWT; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727882620; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TLcqyXn9lEXIkVbnutuzpQdCuib6QuSqazRct+A+ufM=; b=xzNptz41mQtbB9NI5AElF+sNsaeF/DQMsQOyltfAg+iy6ieBealKyuowVEuGNOPJdMTnJC DybtlHPOkodGKn3NRw+4o7fy+uuucJ8Hi1gJAFmB03stR6kwTCASrOXy9l/kTOb29TTUB7 f4o4HBj+/YghUIZgn9MlVpWx/E0dm80= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727882620; a=rsa-sha256; cv=none; b=LJmlZsJh8qCdc3Y3FXEDEjwvS7GQS5HRuvQF94MxpNKQOHC7XwbDKBdZK+bdDUtMzPXn4+ trLjw3KhDJLKCqwT+ZO24psNRYhwk1Lm0vgFHDLgG2hogImglhyCNqrQwIxDbIfNj5Qh8k sAj8gWlrAe55qVvuDqwQUAzk8YkNZzM= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=n2UuYYWT; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TLcqyXn9lEXIkVbnutuzpQdCuib6QuSqazRct+A+ufM=; b=n2UuYYWTLV6rslbk6yPeqNpoXE SmuoNaLgah+4nxdeSBnkIkz4yQDvusctbmzbtCtExS+bU9ZJQ5e1sBg5PLGcYHGDVJvNQtUDplqZN eOdU7d/mnsGkuiNsx2TJtIPJacKAqcVGpfWRK2gvgKC8PChMUNykQ/10turONZ4tQ7YyAtqLzFu4T hSEdlOiPpkf7NZp/uGOYKBAPmzz9x/+Hps9OCSJIwL8nNvMOGKU+oOGEp9RviEZxKuvKG1iU+EWw9 +a8iyjN003enG8apgV39yJnZ1juWvU8nXBuAp+oBbA0kZ/VBrcb3UUQ3k2tIuvaFb7dWdYzjlXTkD c9vbgPJg==; Received: from willy by casper.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1sw1EN-00000005fMo-14FT; Wed, 02 Oct 2024 15:25:35 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Alex Shi Subject: [PATCH 2/5] ksm: Convert cmp_and_merge_page() to use a folio Date: Wed, 2 Oct 2024 16:25:28 +0100 Message-ID: <20241002152533.1350629-3-willy@infradead.org> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20241002152533.1350629-1-willy@infradead.org> References: <20241002152533.1350629-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: waa1em9tbzggn797g4k7rxjk7dymkynh X-Rspamd-Queue-Id: 1436020002 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1727882748-777933 X-HE-Meta: U2FsdGVkX18GpYUHkVVHS/INX4SZ9oK1fkR+rPdhSxkrOpNf7HGHjlTIYPCDHnesNAPAcdRiNk3Uajv03qDo30vyUq1LmhWxBofcr3LAm4AkNvM8nNBqKaoRwmI8szoZypOnB0reYSoFX05VV8lCZ+uqEV79y/2arzeP3LEeh5uLKl/JR710/cIRicXQQCp6egBFbfUJjhxeVpJ2eGC/UoYDTSN1rH3uCTqeJiX0iKAok6JT4/NIRZcyMLbQIiK+d4fYUnOhRdPpnzw+AWkzID+pF37fONX/z3bb5DOW67hgnFuFIgCveD57rUzOjcxi30n/MqOXGaYiTf1T4vLE+z7BhxHRe2RpHkZlYq3pLICACo2sjM1Xb/K2QeK6bP6sP32dxqnxvaWy3JkSqgTF1HsMkzs04b/VkrWbKN/2GISFLQeJ8Pl27S7fWwVaLKGeDCSh9/hexiP2bqvq/s4fMlKDc8HIF8qMrK0XLYqA3lq63vBDTkLWl0PwJZApIqTSxUGmBkGeSXpuedrgKc1ii3soJWj7oPb2xptHOPNvFylIcBy92mOxsAGwN+GQujQRxgotzw765N1RzLcqII1WzS6pGWgnLaPdmHQM3Gniwl6m0zkn1kJsY1Mr5xeBrU3pF5zghKzqe/b/Rk97lY60/kNGIjGDIEhUsQF01gnCnjU2xVQwO2lxMzrwMQksAXlZznb4XYt1PMHiyIjtZ5vygMrBfaSmpB9Kn2hkHbUzMlRlj4u9gXIw8KoO4oVJP9L1jy2LbdI0ObNl6tauYd5gDxyYCoB6FYB5KtxKNSOx1EgNHZjBoggi9Y3Gfa3AC15VvDzBxfkt17gzFrkskfnPZIhhDbza3iO57NrK2kQ6oq9rBKhAK50ZEEarz6HwJos1mKNVDcL8Wf8Pd9LIZMDUt6NDq2RcpwinwihvebmN/w023q24VS9IV11jd5wtu0m1VB1NBYnNVflSolex3Xe wIa4VC4C 1guQxcF/zSonDYH4eia7x5yl0ca5MptMYd09uLcXBk616wFiG1rK0o6TvfSgqu4uyK3y53b0Cb5laFCN43g5cu2oDlVtjmcs4FCK5HeW0i60OoqG5j9DiU+QOpZgWFyZF1H4ai8qjdXd/CnC49bXCgNs8yeNY8m5f3S/51Lk3jzlB1RqLUE57fbMd2NGSoKT98kJPAIPtCzSX1IBkfo0vDIEa9G7u2uYJLFdxve5ZeLPjItcMo2y2BWfgwPDlWGYa8e1/xmlCBnoJazrKgCVZKJOBP8By3TOdYfGL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: By making try_to_merge_two_pages() and stable_tree_search() return a folio, we can replace kpage with kfolio. This replaces 7 calls to compound_head() with one. Signed-off-by: Matthew Wilcox (Oracle) Acked-by: David Hildenbrand --- mm/ksm.c | 48 ++++++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 57f998b172e6..19e17b228ae1 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1583,7 +1583,7 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item, * Note that this function upgrades page to ksm page: if one of the pages * is already a ksm page, try_to_merge_with_ksm_page should be used. */ -static struct page *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item, +static struct folio *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item, struct page *page, struct ksm_rmap_item *tree_rmap_item, struct page *tree_page) @@ -1601,7 +1601,7 @@ static struct page *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item, if (err) break_cow(rmap_item); } - return err ? NULL : page; + return err ? NULL : page_folio(page); } static __always_inline @@ -1790,7 +1790,7 @@ static __always_inline struct folio *chain(struct ksm_stable_node **s_n_d, * This function returns the stable tree node of identical content if found, * NULL otherwise. */ -static struct page *stable_tree_search(struct page *page) +static struct folio *stable_tree_search(struct page *page) { int nid; struct rb_root *root; @@ -1805,7 +1805,7 @@ static struct page *stable_tree_search(struct page *page) if (page_node && page_node->head != &migrate_nodes) { /* ksm page forked */ folio_get(folio); - return &folio->page; + return folio; } nid = get_kpfn_nid(folio_pfn(folio)); @@ -1900,7 +1900,7 @@ static struct page *stable_tree_search(struct page *page) folio_put(tree_folio); goto replace; } - return &tree_folio->page; + return tree_folio; } } @@ -1914,7 +1914,7 @@ static struct page *stable_tree_search(struct page *page) out: if (is_page_sharing_candidate(page_node)) { folio_get(folio); - return &folio->page; + return folio; } else return NULL; @@ -1964,7 +1964,7 @@ static struct page *stable_tree_search(struct page *page) } stable_node_dup->head = &migrate_nodes; list_add(&stable_node_dup->list, stable_node_dup->head); - return &folio->page; + return folio; chain_append: /* @@ -2218,7 +2218,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite struct ksm_rmap_item *tree_rmap_item; struct page *tree_page = NULL; struct ksm_stable_node *stable_node; - struct page *kpage; + struct folio *kfolio; unsigned int checksum; int err; bool max_page_sharing_bypass = false; @@ -2260,31 +2260,31 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite return; } - /* We first start with searching the page inside the stable tree */ - kpage = stable_tree_search(page); - if (kpage == page && rmap_item->head == stable_node) { - put_page(kpage); + /* Start by searching for the folio in the stable tree */ + kfolio = stable_tree_search(page); + if (&kfolio->page == page && rmap_item->head == stable_node) { + folio_put(kfolio); return; } remove_rmap_item_from_tree(rmap_item); - if (kpage) { - if (PTR_ERR(kpage) == -EBUSY) + if (kfolio) { + if (kfolio == ERR_PTR(-EBUSY)) return; - err = try_to_merge_with_ksm_page(rmap_item, page, kpage); + err = try_to_merge_with_ksm_page(rmap_item, page, &kfolio->page); if (!err) { /* * The page was successfully merged: * add its rmap_item to the stable tree. */ - lock_page(kpage); - stable_tree_append(rmap_item, page_stable_node(kpage), + folio_lock(kfolio); + stable_tree_append(rmap_item, folio_stable_node(kfolio), max_page_sharing_bypass); - unlock_page(kpage); + folio_unlock(kfolio); } - put_page(kpage); + folio_put(kfolio); return; } @@ -2293,7 +2293,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite if (tree_rmap_item) { bool split; - kpage = try_to_merge_two_pages(rmap_item, page, + kfolio = try_to_merge_two_pages(rmap_item, page, tree_rmap_item, tree_page); /* * If both pages we tried to merge belong to the same compound @@ -2308,20 +2308,20 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite split = PageTransCompound(page) && compound_head(page) == compound_head(tree_page); put_page(tree_page); - if (kpage) { + if (kfolio) { /* * The pages were successfully merged: insert new * node in the stable tree and add both rmap_items. */ - lock_page(kpage); - stable_node = stable_tree_insert(page_folio(kpage)); + folio_lock(kfolio); + stable_node = stable_tree_insert(kfolio); if (stable_node) { stable_tree_append(tree_rmap_item, stable_node, false); stable_tree_append(rmap_item, stable_node, false); } - unlock_page(kpage); + folio_unlock(kfolio); /* * If we fail to insert the page into the stable tree,