From patchwork Tue Jun 4 04:24:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13684613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2657AC25B74 for ; Tue, 4 Jun 2024 04:22:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B49D56B009F; Tue, 4 Jun 2024 00:21:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF79D6B00A2; Tue, 4 Jun 2024 00:21:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79DBD6B009F; Tue, 4 Jun 2024 00:21:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 549806B00A0 for ; Tue, 4 Jun 2024 00:21:58 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 044FA80FBF for ; Tue, 4 Jun 2024 04:21:58 +0000 (UTC) X-FDA: 82191908316.19.62ACAB0 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id 6A29F80009 for ; Tue, 4 Jun 2024 04:21:56 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Snb8+csf; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717474916; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ab8yOUA35E8hZghbTFRcZg/Dm26N2XAaXftBThSJ4wU=; b=S1SXNErwm/F8T2wKZoae0yLs+FhOWddLosR1XrIssIQ91qIeHdZGPVaHJJxl502aDWXrO7 CVrxj1kWy0vWUL/3O5nILFPBa/lt0r4YcEGQj224wA0XtX+xWE14IQd+5CVP4VsrV6OxxV vJpB6q8H9gULJN9glSNtDlbaJfNsdbM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Snb8+csf; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf30.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717474916; a=rsa-sha256; cv=none; b=c/5px9cR2+ET7sVmCX7n4FxP3Go3trXcpnTJHnXw2B40nE/K7nKgANs7zDPd0pcLvC1LlZ Wbf4T9yaTie/DNkXpNcYgg8Wamt6ETLGNqS0kvU0K8pJhLMGmDnUYey1AcSoKk7Oiij+oX ENFOcOoAh7As+ai0IJ1umvV7NQ5JF14= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id B99746120B; Tue, 4 Jun 2024 04:21:55 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 455FCC4AF08; Tue, 4 Jun 2024 04:21:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717474915; bh=+wxYOCQhGAvuabF7WUDH5FG6NDiLukYmLnN5Qc7mdHw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Snb8+csfuq27jG6c8lfDxSgt/4j/QIAj9TQhYFFMh9FFT0v2DENV7qrcE36FqNJ2r 1bE47raxT2o35HbYAezf6ONYaO4XuHDqGEBVYscppHzq4Y7XMwccQyYP1ERXNIrTbT RFvF8+Z84L8XaaPM3RcfZ9HCjdVaCoECnBZDoBxEviTQ8tzrh/6ZTdVKmaJIYNmerq 7L2s0BkeVPHs7RoKsrQCcFw+ddIyu/Gs4l1Arj+k8Kg+S5tT25HXNzu6XTI3wu26hh 0Dih/9LM2Mv/2NLu2Kmwz63CkFfrgCJGwJL8B2MxkYUPExNcSMpPJLyWiT/6uCJ+ea aW7RreqKYOa0Q== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, izik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, chrisw@sous-sol.org, hughd@google.com, david@redhat.com Cc: "Alex Shi (tencent)" Subject: [PATCH 08/10] mm/ksm: use folio in try_to_merge_xx serie funcs Date: Tue, 4 Jun 2024 12:24:50 +0800 Message-ID: <20240604042454.2012091-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240604042454.2012091-1-alexs@kernel.org> References: <20240604042454.2012091-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6A29F80009 X-Stat-Signature: zqzt8xtr9pwp6sngpitheh81wpjbrgea X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1717474916-641269 X-HE-Meta: U2FsdGVkX18BJodYJ/GnAlrv/mjUoG/LmPNvMIOab0YOsl+lXlSsUYP66nT7Y+2ror29SxO6EnLa0357L7sg8/wZ98AyKJXPgR2EBfcij1+fhgZsvmiThtil0C6opFuhgvmzVtd3lI1om/D2hctWVqzq8HnOjbJnTlhD7mcRbnT0LAakkwkQ25DQNKv7Vk13U0MThlSxkXiO5XF++wArA3YkFcqP4c+nuYr9ZE3Z4H8/v4ne6ILCEOllg4bW9OWFLiqh6p0MvpRXRypd72pQANlz5VoFnwYjp4xzobsufBeCdR4NcasmCZi0joVMpc9nHpTWZ7BCWcDqa5HACEH2B+nBLY9G+QzF/uIYFakA4M+BVFOZ7hkNrsmgbSHu7qAUDLX6Cq5IQZ0gjWLoudtHJiczh4LIfgApmztH3MwLxNIXlJYdabq0EKTFWY+GpjgtDA/XLStkiAubtS0kDiuLYbO63YnM4y4HCD6H+njWiCcC58iolnD0FljAOwgvzeKLEXJi5nD7+DhTK5Kgx7Z+1JISPemW9EX+xPxAmvXRfhclq2V3GulPUidOzBNjwB62Eqqk/+m3LYH1tdHPYUaGOzUq10SWiBKEfFRJn3jMG5M+OrNLWo9YtcLXQjYUlL17UZZc6pQcUfUHSus3o7AT5NIRX38lmkcU6FZGtn6pxGla4p4OlS2DWYjgnSxyc9081TZnHXfSV4a+isYDt2sunY59fkNBpyqRXgSi3seIPa9+aIWZRDvA/oSsmD7kLwF0jKVkADUD2h6/G1KZQa7SfMpoJC33L8xnqrB6/9CXyvcvmCoKY5dSd5welNuMNXLy89doOgxHUclVTMgry//G+2KCQYsqNpqsUDx2nNhdgbhvir1v+mFiLblboTaHjiwZuu2CfYe6U35Tw71gszRufU15GYVWm2K8OdwTM2fp/jRhpj6IFqgB3GF+E5YEpIAvuPoC6gXSRVc2HqvDYrs T2WXdzaL bvHes+rw7Q/KTdB5CdF2Eh3dZs8QK5+hnRrygCPJqEtcXCqekosbaI4C2fZxFmHttOMf8xVrsiFyDd1fU7nT43nD/sWsfaScBEGulD2O3NYOxUKeOjr68dxg0el6gVN9Q7nfR1pWG45jVC2+1oNONf2a1UIMZlMGY3TdPMbO4/rLXGjSdA9fonhRUycN4YSV6NHhvkFxjq24rh/tYj1FS5zvvUs7Bw9UtSTVEwXTQqAZi8Siq+OS1+I33ILT+8T67l5nwpzCeDE8PMJttfwbFmxj7u8rHhTRNKU22 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" The try_to_merge_ serie funcs are using folios, so change the parameter and variable 'page' to folio to save compound checks. Signed-off-by: Alex Shi (tencent) --- mm/ksm.c | 54 ++++++++++++++++++++++++------------------------------ 1 file changed, 24 insertions(+), 30 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 14a7ca53fc91..b9c04ce677b9 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1540,22 +1540,18 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, struct folio *folio * This function returns 0 if the pages were merged, -EFAULT otherwise. */ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item, - struct page *page, struct page *kpage) + struct folio *folio, struct folio *kfolio) { struct mm_struct *mm = rmap_item->mm; struct vm_area_struct *vma; int err = -EFAULT; - struct folio *kfolio; mmap_read_lock(mm); vma = find_mergeable_vma(mm, rmap_item->address); if (!vma) goto out; - if (kpage) - kfolio = page_folio(kpage); - - err = try_to_merge_one_page(vma, page_folio(page), rmap_item, kfolio); + err = try_to_merge_one_page(vma, folio, rmap_item, kfolio); if (err) goto out; @@ -1567,8 +1563,8 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item, get_anon_vma(vma->anon_vma); out: mmap_read_unlock(mm); - trace_ksm_merge_with_ksm_page(kpage, page_to_pfn(kpage ? kpage : page), - rmap_item, mm, err); + trace_ksm_merge_with_ksm_page(kfolio, folio_pfn(kfolio ? kfolio : folio), + rmap_item, mm, err); return err; } @@ -1582,17 +1578,17 @@ static int try_to_merge_with_ksm_page(struct ksm_rmap_item *rmap_item, * Note that this function upgrades page to ksm page: if one of the pages * is already a ksm page, try_to_merge_with_ksm_page should be used. */ -static struct page *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item, - struct page *page, +static struct folio *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item, + struct folio *folio, struct ksm_rmap_item *tree_rmap_item, - struct page *tree_page) + struct folio *tree_folio) { int err; - err = try_to_merge_with_ksm_page(rmap_item, page, NULL); + err = try_to_merge_with_ksm_page(rmap_item, folio, NULL); if (!err) { err = try_to_merge_with_ksm_page(tree_rmap_item, - tree_page, page); + tree_folio, folio); /* * If that fails, we have a ksm page with only one pte * pointing to it: so break it. @@ -1600,7 +1596,7 @@ static struct page *try_to_merge_two_pages(struct ksm_rmap_item *rmap_item, if (err) break_cow(rmap_item); } - return err ? NULL : page; + return err ? NULL : folio; } static __always_inline @@ -2310,14 +2306,13 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite { struct mm_struct *mm = rmap_item->mm; struct ksm_rmap_item *tree_rmap_item; - struct page *tree_page = NULL; struct folio *tree_folio = NULL; struct ksm_stable_node *stable_node; - struct page *kpage; + struct folio *kfolio; unsigned int checksum; int err; bool max_page_sharing_bypass = false; - struct folio *folio, *kfolio; + struct folio *folio; folio = page_folio(page); stable_node = folio_stable_node(folio); @@ -2353,7 +2348,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite if (kfolio == ERR_PTR(-EBUSY)) return; - err = try_to_merge_with_ksm_page(rmap_item, page, folio_page(kfolio, 0)); + err = try_to_merge_with_ksm_page(rmap_item, folio, kfolio); if (!err) { /* * The page was successfully merged: @@ -2415,8 +2410,8 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite if (tree_rmap_item) { bool split; - kpage = try_to_merge_two_pages(rmap_item, page, - tree_rmap_item, tree_page); + kfolio = try_to_merge_two_pages(rmap_item, folio, + tree_rmap_item, tree_folio); /* * If both pages we tried to merge belong to the same compound * page, then we actually ended up increasing the reference @@ -2427,23 +2422,22 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * afterwards, the reference count will be correct and * split_huge_page should succeed. */ - split = PageTransCompound(page) - && compound_head(page) == compound_head(tree_page); - put_page(tree_page); - if (kpage) { + split = folio_test_large(folio) && folio == kfolio; + folio_put(tree_folio); + if (kfolio) { /* * The pages were successfully merged: insert new * node in the stable tree and add both rmap_items. */ - lock_page(kpage); - stable_node = stable_tree_insert(page_folio(kpage)); + folio_lock(kfolio); + stable_node = stable_tree_insert(kfolio); if (stable_node) { stable_tree_append(tree_rmap_item, stable_node, false); stable_tree_append(rmap_item, stable_node, false); } - unlock_page(kpage); + folio_unlock(kfolio); /* * If we fail to insert the page into the stable tree, @@ -2465,10 +2459,10 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * the page is locked, it is better to skip it and * perhaps try again later. */ - if (!trylock_page(page)) + if (!folio_trylock(folio)) return; - split_huge_page(page); - unlock_page(page); + split_folio(folio); + folio_unlock(folio); } } }