From patchwork Thu Apr 11 06:17:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9ED4CD1292 for ; Thu, 11 Apr 2024 06:14:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 228266B0087; Thu, 11 Apr 2024 02:14:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D7D36B0088; Thu, 11 Apr 2024 02:14:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09F966B0089; Thu, 11 Apr 2024 02:14:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id DEE7F6B0087 for ; Thu, 11 Apr 2024 02:14:23 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 783F6140A7B for ; Thu, 11 Apr 2024 06:14:23 +0000 (UTC) X-FDA: 81996236406.29.7FB7F06 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf21.hostedemail.com (Postfix) with ESMTP id 380BF1C0005 for ; Thu, 11 Apr 2024 06:14:20 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XgJlk6qU; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816061; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ziOYdijR/WepKC3J3licszNW8MOaRGCBasaGIcou9Eg=; b=SOLbGJpVTAG/RDajrDt7B6N5DjZqj9kK2OAMNTzZawkYxoCFEzarOVMIbmqAObnBZqnC1o 2Ih82OAVF6ie3BEBdelzGTpAuniMbrT6UPdCavk/KccGx8TpJs2zA3wLA9AblSVQdYjkfC rgYoH6M3fo7IPKII6jXXeMfjhUM+IcE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816061; a=rsa-sha256; cv=none; b=UrHT9J9D31jWACvfndLZTY3UsqzXIPkOjHszVIFQr+qIshIOPM+u7V6NHZm7yh9uBjFO2T jPo3Zvl7nH9OYF3f0mRLct5RBVYfkLfwRG0rMS4y6U+2dZ2wvu2AevfegoWEGY0/hBSIe8 B8yClsaqTZZVrj2saxJJN/fH5vjsVcc= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XgJlk6qU; spf=pass (imf21.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 97978CE2F04; Thu, 11 Apr 2024 06:14:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E5A9EC433C7; Thu, 11 Apr 2024 06:14:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816057; bh=oezv8n190iRAYJbqG1rSb+HEO3aakytr9BMxPO0jqOc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XgJlk6qUo71If1Hxiz9eQsY0pFcpm4jNK7/rtAzAt/q4vCYQU03Bl1aZ5ZiY8T50U 3SNtQ75PjlVB+Y0VNKaGnr0UqLF1Phb9fwrNgXo18gnLvsJT7FChrz2OuTpUa9k6+A 7CLGzC5F3kZwMwngHWwW42Ddk91bRjqfI4zoGb//eWcMRxIqkPPFrsOWo8vv28pzB9 k5BOFJGgkRmvIxtiA3fPAT1wRfDn8OJXuOrXGutaJNGVLD6BdVs4HxQmCaTC15xVPX PyauH5xarZ2NzpCIuJlDGmEQXiWk3VzrrCtxAlc/AQ7XFAIZNY/yS3T5v/PSPTCZOl Ld+oDpm/7WNhg== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 01/10] mm/ksm: add ksm_get_folio Date: Thu, 11 Apr 2024 14:17:02 +0800 Message-ID: <20240411061713.1847574-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 380BF1C0005 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: xccqgif5dytgj8anjz6dxdtg59a7dae6 X-HE-Tag: 1712816060-374621 X-HE-Meta: U2FsdGVkX194BqqU2fjD9KxPdUP/N2PUepTDFQ+iqVnbQPGgUyJDR8bMcoxulqHoaYLmn1Jv9kpJxwEwUlKb2f79Z1SsYEKIZJtPdhLt3Jt51r4z4UyGCAGFGimZ9r+TJmgK4Vl6OIRBSwvzK0NF3TcYu0ddGYPBipgQd8jcIgCUiH3DSLDqXAVCKX6cC0VA3KlD7mwSxyIZMDg+plExzU/bbCmDDcm2OIjMurPuA7i2K40Bi0fcuyoDoIRfBkasA7eV+n1pY/wQgL5CpKm6U8N5uR9Wcpc5Mt4hOBIPx+Ive0Alx/tUxt/C9p5ECkGnKfiDB8wAPX5vJRhdiIgPgO/Kunfdpa/jLo39xu5M7FP4XMz7MXgtLwVWz8EBEDmHKTD4s1NhXYg1mkopj/DudIpCYV/t/krKGOlQE8eJwzIrB54hJgPKrsVjCZK54eXH29jN3hENiaL3DbzJAJ/J5mEJVqg4SKDY08AXyiDE3hpfctfCocphNDx5pnx9C7iEhbOuTZh0uBPR4Sdzr62acj1+g+FIHmX8L3guEdnPnZtKK7+z7t5kkXsD4GEFrqflVR9NFqNrE18Kkgum8RexfKt1YHmlCOIWR9bKNWEk6EaTZQ2pX6RdbGj+Twoy+iwAPj8cjr0T82HlpwClcYhWc/6HK88zWpzM8MyYSQubS4f2wafXdZVCReGjUoGe5pxb9l990WWISOm5jl+YPs88T9qeUhLonKkxUIFitxAyw4T7Kp7q9QghY7V8g9uag8koixY6RUiFy3kBTQGr6oYojNbVvPUoecwRkCMQbdRq0yfFPDQ9yzhEDvb95I/h86QpAuaUGqui3aSJlCvaWco2KFfsbUFUrIvk12uVArLz9Ggr/vwQayicJWICCQvnux/kQnCrNbzcibpRV57Vur6HMNqCN8ZEzF5j2FmpwcIM5IEngCh0wfhAuG+q7jSR3iLGYSlmjG02DlcGpZ5JpDd bV+okUSH G1NUSulX45TREKuBQ75gwrajvdhMLu9aq/FxssIvUr9urECoYSBtFVrvPMasVOkkKC9doE3NuZXel4rae+LwcAkcL9Y5hPqXm1DKC9McOVFWDivwKaX+fbko89u9egr9GX3zR/RQeb0EWFYgw7uBpeGL3MF+EiJMbeqlBFXvttZWF0UKVWKEv/eRXMyNWVAlXz25XcmPXLl4a/FV7Cvs/jhftC91zglcqxhKDeHgR7UQuqvXaTyVs9qHzErhNom3M9aXsgyhduELGY9Lfc35aAyykzD027w1Jzad2EZ74f4mpIJz2n1SdkBXsAOI2eoOo5UlC2Nue+B4Fypzr4rYer9ivjv9UF6AwUMJhCd4Q5/aHGMHbDnNwykLBeicJbC6GW8ZBb8/WNpTHY7y5zC+ZoscPiZIxVo36ibvByoPvb+qTulVk/MB10Chxwk+96GN7EeWni6YfpQF/63A57jftDEouY+piPE4tROU2301nwMOTC2IKfHg+dIQCvGWN3tumZkG0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" The ksm only contains single pages, so we could add a new func ksm_get_folio for get_ksm_page to use folio instead of pages to save a couple of compound_head calls. After all caller replaced, get_ksm_page will be removed. Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) Reviewed-by: David Hildenbrand --- mm/ksm.c | 42 +++++++++++++++++++++++++----------------- 1 file changed, 25 insertions(+), 17 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 8c001819cf10..ac126a4c245c 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -897,7 +897,7 @@ enum get_ksm_page_flags { }; /* - * get_ksm_page: checks if the page indicated by the stable node + * ksm_get_folio: checks if the page indicated by the stable node * is still its ksm page, despite having held no reference to it. * In which case we can trust the content of the page, and it * returns the gotten page; but if the page has now been zapped, @@ -915,10 +915,10 @@ enum get_ksm_page_flags { * a page to put something that might look like our key in page->mapping. * is on its way to being freed; but it is an anomaly to bear in mind. */ -static struct page *get_ksm_page(struct ksm_stable_node *stable_node, +static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, enum get_ksm_page_flags flags) { - struct page *page; + struct folio *folio; void *expected_mapping; unsigned long kpfn; @@ -926,8 +926,8 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, PAGE_MAPPING_KSM); again: kpfn = READ_ONCE(stable_node->kpfn); /* Address dependency. */ - page = pfn_to_page(kpfn); - if (READ_ONCE(page->mapping) != expected_mapping) + folio = pfn_folio(kpfn); + if (READ_ONCE(folio->mapping) != expected_mapping) goto stale; /* @@ -940,41 +940,41 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, * in folio_migrate_mapping(), it might still be our page, * in which case it's essential to keep the node. */ - while (!get_page_unless_zero(page)) { + while (!folio_try_get(folio)) { /* * Another check for page->mapping != expected_mapping would * work here too. We have chosen the !PageSwapCache test to * optimize the common case, when the page is or is about to * be freed: PageSwapCache is cleared (under spin_lock_irq) * in the ref_freeze section of __remove_mapping(); but Anon - * page->mapping reset to NULL later, in free_pages_prepare(). + * folio->mapping reset to NULL later, in free_pages_prepare(). */ - if (!PageSwapCache(page)) + if (!folio_test_swapcache(folio)) goto stale; cpu_relax(); } - if (READ_ONCE(page->mapping) != expected_mapping) { - put_page(page); + if (READ_ONCE(folio->mapping) != expected_mapping) { + folio_put(folio); goto stale; } if (flags == GET_KSM_PAGE_TRYLOCK) { - if (!trylock_page(page)) { - put_page(page); + if (!folio_trylock(folio)) { + folio_put(folio); return ERR_PTR(-EBUSY); } } else if (flags == GET_KSM_PAGE_LOCK) - lock_page(page); + folio_lock(folio); if (flags != GET_KSM_PAGE_NOLOCK) { - if (READ_ONCE(page->mapping) != expected_mapping) { - unlock_page(page); - put_page(page); + if (READ_ONCE(folio->mapping) != expected_mapping) { + folio_unlock(folio); + folio_put(folio); goto stale; } } - return page; + return folio; stale: /* @@ -990,6 +990,14 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, return NULL; } +static struct page *get_ksm_page(struct ksm_stable_node *stable_node, + enum get_ksm_page_flags flags) +{ + struct folio *folio = ksm_get_folio(stable_node, flags); + + return &folio->page; +} + /* * Removing rmap_item from stable or unstable tree. * This function will clean the information from the stable/unstable tree. From patchwork Thu Apr 11 06:17:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8789ECD128A for ; Thu, 11 Apr 2024 06:14:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 054ED6B0088; Thu, 11 Apr 2024 02:14:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EF9F16B0089; Thu, 11 Apr 2024 02:14:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4A766B008A; Thu, 11 Apr 2024 02:14:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AD88B6B0088 for ; Thu, 11 Apr 2024 02:14:25 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 719111C0EEF for ; Thu, 11 Apr 2024 06:14:25 +0000 (UTC) X-FDA: 81996236490.06.6F21531 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf26.hostedemail.com (Postfix) with ESMTP id 961F4140011 for ; Thu, 11 Apr 2024 06:14:23 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=RG3iMi2h; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816064; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IjYVnJ5Hdf+yMGGai65+HBktlmyEg7+Suk3YKRYz0tQ=; b=eLIIyYaJhaIe4GGTdEBjVQRJdshpmCwRx46wycDW8P8drpsn/RlMbX/y06OORLyZ3O4jPJ s/LJ/S1kISI3K1H2dKG9vzhPSdQnKkLl/RlxOWQHPzb+ARAYPyFCDh2jqth86oi/1SWacs Bk+ODf2U8qEVAnuM81fplGKD1F1Fc9I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816064; a=rsa-sha256; cv=none; b=xN7cSWVdMjWpwixr+YMJs57VYK5YwjvGyWybxSxk8bZ7cH1GFPmW6HdX+WmHaHSVLH81gF V03RiPA6Jk7oHpNrDQG2UzbTElJrYTDExnLCGRYLDKY4vUjwdMMsu3HBtMdkmQjEeKJ9mx 5lu6xDjnG74SUbEoeSsFs6vyFpkp8ac= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=RG3iMi2h; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 7D50BCE2F0B; Thu, 11 Apr 2024 06:14:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CA75DC43399; Thu, 11 Apr 2024 06:14:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816060; bh=Ti8omFOFkWyaGFVyRP2niT/Lsh/B9sgK71P36pfGxwA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RG3iMi2h+4rJwzgmeoyIiZZmtvBYfoMXgDfJJgFS9lv62v+NcgJ4l1yZGNF37CwLt GLu7qoK1rTHy87WRBFLJeFfla6DXdLbnWVXmoHo+vutb25QVE7gpOvhH6sy4suZF7c fx9Xgk/iQscsOVBNRzbjxgLRYzjBEOdqoJfSq8ukSDTLbzmYjo3LRgVcgpPrJWJWPQ 4NwI6Vo5VI+85LFXnMpvl8FG8Ku/0SSrkpA9NTN1372DphlJLHBWeaEp+JxGd2Ml/J DYmWnmWPezuUFHIF1lxEuOA9TTvygMuWSBPT373PdQ/xEctBKLJkB1euNR1IV9NiWV UCWIpZCc1917A== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 02/10] mm/ksm: use folio in remove_rmap_item_from_tree Date: Thu, 11 Apr 2024 14:17:03 +0800 Message-ID: <20240411061713.1847574-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 961F4140011 X-Rspam-User: X-Stat-Signature: cztsoudcfhoxqruqub8qcpghfst4jiuu X-Rspamd-Server: rspam03 X-HE-Tag: 1712816063-696906 X-HE-Meta: U2FsdGVkX198dVFZM8Hl5Nqwk3yvgHK1mXi7bZzLnzKtduB6RuK9lhkwZkJuuxBZuBJlTSqbUTH5pJzX0M6i0VmjhBA/9icXOT1FhgcoVqCcXfs7OAb+WtlhiFS59v/wNXJhhePye4bnc+9Sf+JMPDf0e8S/4KZWjGHUD3sSV6CC0COsP1K+AJkBt3j6yypKfosYlXzwhjNHHagFFwylTOrOetJs5rAbPFrJpiMwNLMPRO5GGCCnW2ERkK+UaZ7jEfqWuk7W9Au/4hIBNilHzhoNE1pcIzbOLFsLl0odGd+RWIk2Eq2zOjIWhsj2WmAK6rMXyC2i7675EIyGQb9aptnzx9LYoW//32W5KWLcCTmJ5rvEUtzPukSZJ7qSA8wu4jOnsQmnrHOx8IFdYuV4OzMfLjTxBHPIk8PpJyUf3uhV43JI3Q6Fv7ArmzqD0DepJovOpMG8ro09/Vr13wJDI/wWWmhwiF/dbM52u/OkgdQOhTfz62L4tFSTZUQ7bb5DRyppaKmQPstt7AwL7sMXWHdctLXCmb9KWIqTXjjYVZjE1wNhqbGA/0UcZnbA4IeoY3OArImHhj0RkpMjpF1XoD/50eIZalQUEtJazR6bfI63B4exCNJdff/G0NaBoEkSddUGIQPlvXKVqlfRKQJzVXvjvl8DZ6AIu4DV4n8DhjfYRi+6JD2Ih2JYpNsnOeMCtSy3BJq8arAY2abuGxRXjoxa4JUealtU45LUD0aguNwtil9pIak5xh9eiQRiAThqCu1yyVv/7IgzRlEoAYpofVWtieWZUxfWO7huHXRkfVMGIFDsM1udRsb5RstNU98Ok11r7O/q/bk4yNR+n03yFbDYHHeaftj1VguMlJLGK8b7ezHmRRZoM/1VduqG9q7kL51628MbVMIoGp0Sk5Oroe1U0lz1lnNq3VcSUp/15wKuuNtjPNNDX5ShobWmZiVjdgRgj0L1+A0s1LCOIsH vZHbaOwS QIIPf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" To save 2 compound_head calls. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index ac126a4c245c..ef5c4b6d377c 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1006,16 +1006,16 @@ static void remove_rmap_item_from_tree(struct ksm_rmap_item *rmap_item) { if (rmap_item->address & STABLE_FLAG) { struct ksm_stable_node *stable_node; - struct page *page; + struct folio *folio; stable_node = rmap_item->head; - page = get_ksm_page(stable_node, GET_KSM_PAGE_LOCK); - if (!page) + folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); + if (!folio) goto out; hlist_del(&rmap_item->hlist); - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); if (!hlist_empty(&stable_node->hlist)) ksm_pages_sharing--; From patchwork Thu Apr 11 06:17:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C38DCD1297 for ; Thu, 11 Apr 2024 06:14:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC5FA6B008A; Thu, 11 Apr 2024 02:14:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B75AB6B008C; Thu, 11 Apr 2024 02:14:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9EE656B0092; Thu, 11 Apr 2024 02:14:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7EBF96B008A for ; Thu, 11 Apr 2024 02:14:28 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 135A1120A2D for ; Thu, 11 Apr 2024 06:14:28 +0000 (UTC) X-FDA: 81996236616.30.416C55F Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id 38E3A180002 for ; Thu, 11 Apr 2024 06:14:25 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=IeCaLcCJ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816066; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bB4WH2pM8jvLy/cxq8hHZnYa/Dl93OmWdGl0309FAUY=; b=GBzS65xkT8nJ67xtwe3ihJLR68N1R41csf23sjedAspI2cVc+Qc+DNak6cCA3LIXHhP1u8 KYUBJv8aLMdcPR2Jphfzjv46dzVT9mG2AinGiJ/TQ4oJ8coWtkJ+UscgXlIHNw6Kymf3sp TQxutb/SHRp5xJVhk88PiQNP5kHdVyo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=IeCaLcCJ; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816066; a=rsa-sha256; cv=none; b=3Hh2e5xJDvg6mHBE4FzmCvOat2azxIuuZ8kK/PnclggOFJYR/0DO4sA78Q0KpYxAGsVdRZ 1BTqmUoLE4VyUhLq4aYnuuewnDlLKjajAHhgE5Nq/Yo57a61Ncl8CQ6aBR9xEZENST5IVV ocOQ7PSehfHkmLmtroh+HeDCTvKdhXw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 43A7DCE2F10; Thu, 11 Apr 2024 06:14:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF6C0C433C7; Thu, 11 Apr 2024 06:14:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816062; bh=qC1mthl25nwOp6y3rOKx92fTxAINpkUFXHoU2+YaTrc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IeCaLcCJY1GT60Q7AK9seqx+UVCi+HEDRS5ESPdYFwWmaoRyhC+uf8a/tp1U0dwjT wt6uBhs92tZ7dBvQAr+9+kTUJhGOXqlsZ006oR64EZ1MC85axd3cARsi2cW3k23YRX uLnA3kT7VS+kVAk04VJJBr4LxERj6mfwFZ+2LoOMpc8wlCYJgEKWVGTgQVENb+Mems C3vD9bBLeWtulpk4I7Wa+g01SjslvX8DWmJvfnMgz8WhRM9emVqFyfZSEQpbnQzwTX lMZCl8JaL46hQAO17OlxhJS3dshevOSZB0qc9ifR5HdY5RwAu5x3ZEfH+wU6dzA+xI +s6u98GnpUpOw== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 03/10] mm/ksm: add folio_set_stable_node Date: Thu, 11 Apr 2024 14:17:04 +0800 Message-ID: <20240411061713.1847574-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 38E3A180002 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 8mwuzjmcc5nfqa8wkk965dc4xxjspn15 X-HE-Tag: 1712816065-322878 X-HE-Meta: U2FsdGVkX1+RH8uv4ipMrzlntoAEFXWmKFuN1W0pbq4ghwRoudpieXQJBUzSVn9YgyILb5u7sGf4GgvAn2dSeNJ6Ve62zlMMHAlXzN2b1VIaeeje7JFyLC7T6UCpkWLU6iyZiKLdax75roZ+b0rFMSbGl18hifFK73XBqiLh+PhTYfvGEk+N3N8KjfogoL0ABwQsTvA21yOJ1sMVFUIbWjcKzRtTJIw9MiZxdw4SvK48Kg+MtJQ62H5yZx9MwHYOrl0rV2i9aKsjA0lzaPLV8SUjfvUxBWcrjQ43Tq+4wYr4twgupp2y9tyvctKtU75FFSvSPsgWcFaBKkL/UjqMlbBjoG8OEgUODGSlFhwJu713tGVs6SKJjTz0xNH96YkY34bYCnhTLNkOiEk+YMtsWM7ADMCQPyf8mGJPOVeUz45jtfng20njOlTW0xzt6fyKrmzLUn6DSJVlDw+VJ0qJh9FJk95TbTmaSkmXY5YH5HKQIfK93KKH9wHndBuFT+2/XXq/PJz2ocTnQnacqr/WWjohgGkIEz2ry57YptsqVxMrDIF9u2FKdw+z8NGZExhi9Ne2eSnKpymlfXceNiV2od60MC7WwDi9tFmzbecqNVIxbvZsMXaMXqCcnXTMB+R1Lw37iqa9+LXlCeArVl2VmJiJxsiguhTFk7XxTKOZjiJznRfC7syd/jWt6Y0cyuJbAK+R73h6U/pC+ZRuuZpzUkIj70GKLTLBpthC9v2ESnxAi1ScG45u1fR2A9PFH0OnVsRuv/B/ia9W/Qv7s5pXUXY6lwVMLgKn9IIwhlIVOMDjIbWZeXOxKxw1dsE3eU43royZRZRk4wzgNQAtGNa+uBu/xfUFinN6nzMQ3J44lBOxqmdQ20R35LibNiz+Ylmpe+19Fi8xJlBdhka3wd8txDjvFsQpyAnVfibJMqLgOyhNoXETL2xy+9n0leiC0RpyIDl6toDJcZkWFe5E9wL Ry5gyvfH df2tu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Turn set_page_stable_node() into a wrapper folio_set_stable_node, and then use it to replace the former. we will merge them together after all place converted to folio. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/ksm.c b/mm/ksm.c index ef5c4b6d377c..3c52bf9df84c 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1109,6 +1109,12 @@ static inline void set_page_stable_node(struct page *page, page->mapping = (void *)((unsigned long)stable_node | PAGE_MAPPING_KSM); } +static inline void folio_set_stable_node(struct folio *folio, + struct ksm_stable_node *stable_node) +{ + set_page_stable_node(&folio->page, stable_node); +} + #ifdef CONFIG_SYSFS /* * Only called through the sysfs control interface: @@ -3241,7 +3247,7 @@ void folio_migrate_ksm(struct folio *newfolio, struct folio *folio) * has gone stale (or that folio_test_swapcache has been cleared). */ smp_wmb(); - set_page_stable_node(&folio->page, NULL); + folio_set_stable_node(folio, NULL); } } #endif /* CONFIG_MIGRATION */ From patchwork Thu Apr 11 06:17:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55E87CD1292 for ; Thu, 11 Apr 2024 06:14:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9B596B0092; Thu, 11 Apr 2024 02:14:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFBE96B0095; Thu, 11 Apr 2024 02:14:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B756B6B0096; Thu, 11 Apr 2024 02:14:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 993E66B0092 for ; Thu, 11 Apr 2024 02:14:31 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2225CA0A93 for ; Thu, 11 Apr 2024 06:14:31 +0000 (UTC) X-FDA: 81996236742.25.5D88066 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id 13E6114001A for ; Thu, 11 Apr 2024 06:14:28 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=AtcW68Rk; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NV2OESMwGKMflyoTHmbwFP2rE14w1PWpBAxrQJN3/8E=; b=g5Y097DRHM0Yj5XPpq6vh+bt0BnoBavuDJwkIi4l9FanlxWTObqTcBnLinvHZCqRO/950t anBsQWROpiizvRtHD8VMXaYKiabeeYwrF2HDmNtXp+vtWGehaAUFC97nJPgJw6zv6GXpMy kPn51yYJnipncYHDXvfQoy27xNyurn0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816069; a=rsa-sha256; cv=none; b=ZjpdvYZEcDudzcmmYPDg5uWBhBLJoTqeii95jyCudbD9R9pCzsenH5/aSQh0NF8/q5tKYL z30ZCxfC/GNUh45HnEWFmfJJ94Ri5UCcm/c4usLfiZNtvo9wGrgnPsxExUI+HBkmxqBAQ8 vGEj2rp9Dv5GKcyKqYOlpEbOz7n0lYI= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=AtcW68Rk; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 04938CE2F14; Thu, 11 Apr 2024 06:14:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 740FCC43394; Thu, 11 Apr 2024 06:14:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816065; bh=Psjp2Yi0v76d8BNfvxYBx/0pfRjAkX6bKnyisDETHPQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AtcW68Rk2WAP6ax3CbHdN4r9JSIXDouQiaVp9hG8moEE+5MKD9ffWBUO5Tn/KJyGa NfMqOSI1Kg9SLWaCAt4mINhEpg8uA7V7AKNHrWSJYnO/ytFtlTDCyhvBN+CTNRELWd 0W7pOuXj7+4Qz1Yrt9MI3dBZ++m5FTGRNZwXgaBy7O6ENdkgvPdaLIokjOEX0Y1vEg sbXCZW8tv+k9RUzc3/1RDiZxV2ZLzjVu1im7P69RmdYadEdObhiNkPKUKPxbdSRiXH GTO+PjJ/hF0piGutbC7coJxsH5pq0KB1O1RE/qRF9ayCTCpxB/q5W/pf8Klp/KRFgE rF3mDjkj6WvAg== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 04/10] mm/ksm: use folio in remove_stable_node Date: Thu, 11 Apr 2024 14:17:05 +0800 Message-ID: <20240411061713.1847574-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 13E6114001A X-Rspam-User: X-Stat-Signature: u3ipbbu56o568n9jcgfut8qbwoggfdzw X-Rspamd-Server: rspam03 X-HE-Tag: 1712816068-704950 X-HE-Meta: U2FsdGVkX1+RbdH0Pn3ZHgDuaKyd+nz/So9VUkIdS2TGQxDXeKXl8m7Sm+GMYa59vJEM0aEyOh4nBSUJGrx8lC+p02oQYCM+MCrZyeWIIFQFXolrG73uRSAllNe16NHpEqElATxh26+VI/5o5ChkMLbdVJRMJ+DHnzenuH+rBUOwI56LJWbaJ2DocxMdBi8PEieq+P9DdZ3Vwb8aG47s/m5Nb+uvZn9MiNe96C7dnRv289LLNlwXoytOpfEAZJ6EmetZU0VSvpj7ArTKbdxkp6iPQxO8WPOTTM8KvtN95zXXayE7b0EwgMh9ej8PugwkKm1Yk1mZxwKcex303hSHTNWXoAKsV+X8nIPiG/JNp1Qjm3lV0Wp+aUp9a/nJMnTAL1+ip/jAxbfllhxpS78mcK7bCrwv5xCj6789UvKlE2oncGeczyqY6EHWQdCumNhbD9O5fa0x7Il+Nxluaes/biX/G+pFexUWQdH04q6Q0tbgkfUEOdqQwZbfncmL3GyGWu5NcgiGNl9KHw++1QeBP/m6LzDadUFZE1dDL7pNzM2kd6FNjLfFv8+80oXOWpC39VJoxKdTjmJXqB4MoYPBFkxQVtXT9c0Xe8PLD0FtvSepCgo+tpVjoKp9N8+PQQX8UOm8GEVSvVDnGXcfUn9ttp9V8dcBSMlvPkFBjSHwzwDdMk8LvI9sp6Tba0K1m6CpmauyHemZSMQuB0+RoLK1gPCU92jkDIJ+ExlrTkVeAUoG8Ya6tg7jy+k0pZBk4NEt5OHB2Pv9uvJTtAttB0IyvtF6nxcomHILVOpK5v4PjoP8THXjsOOEa+wJ3NZWdPST5ffMHdkjogpiVaWOHcaWRzDMTYNS31FbHY1tnwyUn2oZCyx97HuIk1STZgrwOPowGgmn/qXYLUDonqpDdl+Emmc6jE7a+VJ1uWbtn76f3E7h52BBMj/Hu3b8tKMy0ca9h1Sn5jHwY5dcTFrcso7 HuTv3Mdp b09vIFs98rhy8sxLRJiseDk+aGpBCggxeNzPE1vKd04mzadZbGwkhttpCvvhvM2lmc8OV+eXFTHUJ62sxVnn8j5TA4Io55xp3ZwcOcUgf9CxAgc8nWVTq7kUJVKTZ7gOOnFtpTLfHa3VU4pyBLETj8GyNzGHumq5VcsjJCwVVfOohxn/5gr0NP2P0HxDW9zDyXS/MlFNCFs1Vo16CPhHLsgi7CsDP6G+aXdEK0NK2QP+k/Ktw+BO2/tAy4KblcZEPya6eMG9jvYelV4rSDdqauA2ILHaqNP7k904/9K5OgaGe15iTEiVOOWAAeu0xjZBvA7BjRJXsGp/wPta/ciigPEUFrABLrJcRc59ScLg0UmHwJR/IH9FIGlc99fGrjOaJNWaYcvSSsIMM5lKahn0D0aqx9cfRMrYbZyQCCHDJowAbKmoE+Bfxi/NXUAibONfGvdcrzI30+GKhfNzCKWPD1svPhg15Sps9AR1G5THAwEcI8YoBM/9Q3G/J4C8WuG7sssm6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" pages in stable tree are all single normal page, so uses ksm_get_folio() and folio_set_stable_node(), also saves 3 calls to compound_head(). Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) Reviewed-by: David Hildenbrand --- mm/ksm.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 3c52bf9df84c..1a7b13004589 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1121,13 +1121,13 @@ static inline void folio_set_stable_node(struct folio *folio, */ static int remove_stable_node(struct ksm_stable_node *stable_node) { - struct page *page; + struct folio *folio; int err; - page = get_ksm_page(stable_node, GET_KSM_PAGE_LOCK); - if (!page) { + folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); + if (!folio) { /* - * get_ksm_page did remove_node_from_stable_tree itself. + * ksm_get_folio did remove_node_from_stable_tree itself. */ return 0; } @@ -1138,22 +1138,22 @@ static int remove_stable_node(struct ksm_stable_node *stable_node) * merge_across_nodes/max_page_sharing be switched. */ err = -EBUSY; - if (!page_mapped(page)) { + if (!folio_mapped(folio)) { /* - * The stable node did not yet appear stale to get_ksm_page(), - * since that allows for an unmapped ksm page to be recognized + * The stable node did not yet appear stale to ksm_get_folio(), + * since that allows for an unmapped ksm folio to be recognized * right up until it is freed; but the node is safe to remove. - * This page might be in an LRU cache waiting to be freed, - * or it might be PageSwapCache (perhaps under writeback), + * This folio might be in an LRU cache waiting to be freed, + * or it might be in the swapcache (perhaps under writeback), * or it might have been removed from swapcache a moment ago. */ - set_page_stable_node(page, NULL); + folio_set_stable_node(folio, NULL); remove_node_from_stable_tree(stable_node); err = 0; } - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); return err; } From patchwork Thu Apr 11 06:17:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 619DDCD128A for ; Thu, 11 Apr 2024 06:14:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E5D2A6B0098; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E0DCE6B0096; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB0016B009A; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9EA246B0096 for ; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2E172140A8C for ; Thu, 11 Apr 2024 06:14:34 +0000 (UTC) X-FDA: 81996236868.19.1141723 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf13.hostedemail.com (Postfix) with ESMTP id D81C92000D for ; Thu, 11 Apr 2024 06:14:31 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ohkAORx1; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816072; a=rsa-sha256; cv=none; b=e5pV6yxBtJUyWjh5oVuAis9u5vknqpu82NTcM8YFdg+ddupEHyfFD4w9QMIHHHThkl2sN/ RSLIgxHjzlCHSgHW8XTccWy4g3KZuW/dIofXAPJBB1wsm+D1tMAVX1tgjwJep6Gy7LuMFN AZy2WOseSe7+5nlap/9gV3d8IDFZJ24= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ohkAORx1; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816072; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=e96MJPMoqURApb6QyU8wh3HhTh59e9/jJOv1trFgZ/c=; b=AcZAJorjZ1lOtOI01Y1jlVnyM93YhbPT9sHIQJfOsGCLTbxBbeB6bHhoCpAD8wtXe+k9Kl ZxnsUPIv5kimFDVw1wTdDQRmeiLtDZrYgeZ+u7t4mAzNNjku1qP8DkwATTCDKrk4FyoHvM MKoACsEZZoNb8YNpBWXRLtGeYAIVMek= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id BBA84CE2F16; Thu, 11 Apr 2024 06:14:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37F82C433B2; Thu, 11 Apr 2024 06:14:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816068; bh=6Rigut9B0xei0o3RlU/2pvsNtNjz8VS58APxTOkaTzQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ohkAORx1XDD/r3zGvplob2RLchLHYpJtenMlCbrLa21NNHXYHsSFleenl2QJvPwDO BmJobQMM8/tiaJ2dc6Xqc1+pTXwNBj4/zO06Eeu4LRUemG65+A9n/DP1SoCXG6914z vmmFfjvM6AxZfKRbJh6c/IeVBz7S8/GXndViQaqA1DdKxNizoD+AfChC+db9N5Bnp9 7vRT2PZBknD6b550bTqQ9oSG/8iTN94euob/LzgnwW4VPDEHbwNrSjRjnWDXIqu14t sx0a9yK+5/uUZG1uXV0Si3q76gnQPcXFHtNdkX+DnqoIGAZzLazQIpwpJpOYxe4ryU EYYNzK4nJSBNA== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 05/10] mm/ksm: use folio in stable_node_dup Date: Thu, 11 Apr 2024 14:17:06 +0800 Message-ID: <20240411061713.1847574-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: D81C92000D X-Stat-Signature: j58xkhcea3dp57wcmw9hq45zqfsngmhn X-HE-Tag: 1712816071-591517 X-HE-Meta: U2FsdGVkX1/WmqcrnmVNsvA4FdXGdbxjAWq7yTAb/Jq5SJ48QV/w26hD22aKBhcbX8kIN5ZMlZqxLlRPv9gfnND4DlpRvKXmSEZ2tab7W3i2K5IlfKsrTFVo88Ih/2DxldypZqwQ1MLKMkGtEGm0usUqfnpj7FBdnEaKqqO1t5YOo6zKvcpNhszF4/DgYvn3JpmVj/qZaLScEAbyfuWeunnMPaflqwU6pch4xFEaslpOncl7mLwlDKauPKo+jXmCZFRWpB2MguDRAHPHL1o43ggRHilDxLCYHFcErzQ+8xtSrPV5ems2d88is8Zx53xA/msgX0eB8hsmJbuydWwNfiwI/SEemuwh4EgjtEr9iLjR5l0qCxrGIzVksI28wItF99xwuNw6e0wLJeTEx32OFpQDUFC12wG0S9ZcABFv+OtGuyCrfgmk++c5e/65w0AzhcQQtEE9A1d8TIB5JUoLj9JhPePPoQTOVHuuYua44VkiDl4a2imih3MrHtA3Dsvr6GAaFvNAJQUlMTiuD4UvAbQu65jnB3/Ku81szxV0CLKpkzBYxlSCOc2V4kDq7YDzzAKiYeJKxoY/hIT79YvB6MIHk3YgFjbikFZAp9OVD0yWNoTLrjhBGzB6fsplDpQ7OZrKKZJJSlM0EHuJ5lRR0tJfB7zO43+5RnyGNeFv5ramml+PPJE6qC75vw6/xkTqSHwpkg8DQcFhsHWTs+KGL6ckgqH8hcw7sXyqL0Aq17pbi2ZhGb7/R2vqxauwQRYkMGqZn11+N0hnS5iLw4N7syvftJregDOKWUqyFbq5JbRRm1cc3j1/p+2kLrjEIauqx6YaM39RCPVYi1iV9vz0DvFxUQ1woam2ZQd9I00LtJNEAEaC/ITQE6wJeDUrwLEleW1Ja1pPXJ5yG892BmKBXjFnc8VFwwi0dspIPz2QTIIU8xPmnAl1A1iHVncVQEnUPA+ZPv8yPjsQY/8Iyyo /GpLSCpY RxtMFpFNMiPc7ETLKPC+9jqP0eQSYlFw2pbB4Kwd7UEbUoo8a+al+qYsgk2ojNA65K5FX6X39u0XLQo36MlweTQbs8BIQxGoklCXA8Ywujo0uL/uvXYAnFlSFv64ynxmBetYjQR5hs0xwkuJmVQyLfJXHZcsLzQBzBmOQjQ3TS3Yssk89n2wi82QWSUAhiVPCrum5YAKqpp2gAeNMRJrPN6CoIENhcBstG0A1Py8SMiRWKQxcWWAmS17h7r/ggtuzQe//6oH2zdWN2YGQM4WkdMgJwpqXR/oTx/4/zIRxmu3VPxt1+hhPPzcBJLjhXAoqXyVcXhIC5iaWPnry5NnZ/lhDMVy1h445PelXxiVhxHlsz/kPaoHpplEKwRF5OLTFEqsIgiZdOe9hXVCFluDAkuPTcVfE+MbZfH79R2aSCqA5K6WYnFHa37vF8T0cxZs3yK+XCMVT9F7lAAZ1qQnIy9Wkf2mMBSjLSYecKR3mnlCsu+ONs/i9aDzAHZHY+PFB/Tez X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Use ksm_get_folio() and save 2 compound_head calls. Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) Reviewed-by: David Hildenbrand --- mm/ksm.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 1a7b13004589..654400f993fc 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1638,7 +1638,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, { struct ksm_stable_node *dup, *found = NULL, *stable_node = *_stable_node; struct hlist_node *hlist_safe; - struct page *_tree_page, *tree_page = NULL; + struct folio *folio, *tree_folio = NULL; int nr = 0; int found_rmap_hlist_len; @@ -1657,24 +1657,24 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, * We must walk all stable_node_dup to prune the stale * stable nodes during lookup. * - * get_ksm_page can drop the nodes from the + * ksm_get_folio can drop the nodes from the * stable_node->hlist if they point to freed pages * (that's why we do a _safe walk). The "dup" * stable_node parameter itself will be freed from * under us if it returns NULL. */ - _tree_page = get_ksm_page(dup, GET_KSM_PAGE_NOLOCK); - if (!_tree_page) + folio = ksm_get_folio(dup, GET_KSM_PAGE_NOLOCK); + if (!folio) continue; nr += 1; if (is_page_sharing_candidate(dup)) { if (!found || dup->rmap_hlist_len > found_rmap_hlist_len) { if (found) - put_page(tree_page); + folio_put(tree_folio); found = dup; found_rmap_hlist_len = found->rmap_hlist_len; - tree_page = _tree_page; + tree_folio = folio; /* skip put_page for found dup */ if (!prune_stale_stable_nodes) @@ -1682,7 +1682,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, continue; } } - put_page(_tree_page); + folio_put(folio); } if (found) { @@ -1747,7 +1747,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, } *_stable_node_dup = found; - return tree_page; + return &tree_folio->page; } static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stable_node, From patchwork Thu Apr 11 06:17:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77F74CD1292 for ; Thu, 11 Apr 2024 06:14:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 137D36B0096; Thu, 11 Apr 2024 02:14:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 013D06B009B; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D71426B0099; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B4F9F6B0098 for ; Thu, 11 Apr 2024 02:14:34 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 75052A0A5C for ; Thu, 11 Apr 2024 06:14:34 +0000 (UTC) X-FDA: 81996236868.06.4F3E4F5 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id 70710140018 for ; Thu, 11 Apr 2024 06:14:32 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=qrF2rdtE; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816072; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MsTQMsgVxf0dLcXEf4xqqAMX2k3L86zSBL7Zs51WxIM=; b=cko36JoMxP5P8mLXx7XaOtq2ePi7Ylkp+vkow06ycfmyP3x8FQRpA1NoVXMvjzMf0wa5Q6 cJhjmVhnwlc9oZTcJhwyGdkG3oYxuZkZZzemmPF7mn3yHcx9YqXWEO84LS8ShEYNcAk93T YWyOg4tR9cCCCIDNT35v7JES2KGnRBU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816072; a=rsa-sha256; cv=none; b=tRWnuSas9rgj7lsL1/f7Y9Z8c76ZEde+Q+pNPABdb1DslPMRAV5PyTdexbLMLCTlqdNLmo kYxakhpFoBtpt5VnwD5T4XRutvfWbxieciNfe7nGefYqS99UtSaoE4gzsciJzYbkyQA40c /39N6zrqoBurgOzlE6SQFu2kZ7R4oio= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=qrF2rdtE; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 7F6ECCE2EF7; Thu, 11 Apr 2024 06:14:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ED760C4166A; Thu, 11 Apr 2024 06:14:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816071; bh=AGS71Z17Hu5djhgaE9FvKpI+lMFnYwLUHMLt7uG9UW0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qrF2rdtEXCaBbJxjV8S9drLU01OLf5ihIeBBhb8Xl/1R3u2e1+jNFOQHsgdzdnX/c /9dnAtjbA06xzJ7rdG4i6UHkCzg4bXFPhC5813GaUDWCrEDUDslsnKoU5MDzlxVJhe bQCIBihwPd2Yf6WqwLk7a/krz164Iyqhl7cguWbc+gN2CSIdqycaiZ3uvV4syIwWR+ OXLmzBAXdeJ7ihn/uRbrBCamUfBPTBOKUJhPH4u9x3jsuvLltT9DtBOtQqh6LWg+y1 SRxVpNTshHI8KEdoIV06pJfSrWMH/GZw+buDUlIu4gRdDDu7bDH4uAd9mjSAUmXkgq x2DLn4gkEPyOg== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 06/10] mm/ksm: use ksm_get_folio in scan_get_next_rmap_item Date: Thu, 11 Apr 2024 14:17:07 +0800 Message-ID: <20240411061713.1847574-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 70710140018 X-Rspam-User: X-Stat-Signature: uqztzshacb8838cmtme7ncc41my6jr1t X-Rspamd-Server: rspam03 X-HE-Tag: 1712816072-3571 X-HE-Meta: U2FsdGVkX1/YEMtolmicz1T36Q5V0KdbTNgCKHLgfQV0oDwA06XGchvVRk8r1ZKHsTYGgFMYUu20E608h9SFFFSr8lFFx8yoCaQNMsQBPkUb06EkatO40Lwy6M7mJQMWOpGJYzhejM+5ch/i3fYw5wrVKgtFS6mDLEsz9OstxX+T9/g3LA9iH+uHHPqkc757dshOvB0ODZ4p5PIY/xTZVvwOrL0svrtyWLHsRrlDJRdkFVMSqSQ8OXwD4PVHppfz+dWW+QOoT7ksk/Ikc8y6hXV00zpDBfD3og9TQrV3aGeL9I/Fa3ZfBvdMTvWJc5MYAnqkDuLX50fYyt1awqtsFjsQfgHI0twU7eBt2NDNwYUv8yZJmltoD1hh+AArkt7uF2a5huLW7DT/v3UdwFvJFD+LZtJYMMizs8OSgcsNV2vU9GeKWrk8ogthoigClv4VoL1o/0ROfw0HlhXRODAHJWNB6UyGfuCUPM5hhxR92dRBL1fmT6qecs7A49mo5M59VEzvh7Xdn2ntgArn+H1F7js2jb8M5gSKT4EJMK0alruZ0icKK8qzNFmUNoW5UFLOHBocluTUwNdtA/yK3f8YPwc6AZT9YiXG++2QqZl6voIiD7/V0sDGcB7NeFPe6i97fuZ1GHiQznjEGSzgJelMj5Oj0bE1gE2OK/reKaTV40HrYBgyFsfxOswhWghlnBkgZ3YyHXjT9L6iw4OnCWKhU/r8HqSTrxLQ6OVw4aQMazEedqdxmkUPIc6H1HhXr0MtLS1irppYvBO4Axd45JxdKZPWTPj2c+ABsEFFSBJLfS+xBC7bsbF8s0AWnwtTfTunwro3wg4fe8ezxYJNrdMthqjBgfKTkuBIxN30JFF2uY24mMSNkir4wJqkYdwqtiVsp3XDXQLHI0juBC2yFZTvwcO5y2lreyAghyp4om2e3wzCJdoLZvaN9/xs07lz/xoHYqtc5h3XRDhgrJuUq3+ KfdnER2N rmPF5EKmSOPwJudfRdidfzZ/6n0ydiYG6nBolmM3Md/GslwtIoZLYdNuQEj/n/FdIWme3r2Jll1CIjBJ9mwAKXP7KKgOrZQILRrvNhKG7rAlhlh5qK3woXW1uMj1UZ5ZTMqPhFNuSvHSaxXs5XCdAGU9nIouwjdmkfzTxjsj+O+LYNcFdwPtNrA+g5ORXGj8fX3FAzWZtYEk8k04mquNU3cVOszR1lxduwXHFCZ6xVjCJdvZjvBwSMhwIe4BBA35twcwFYL6YnDpCHl6iOsqVB7k+52EWIQa11JcS13M8/NEGaeZP+ZxK6+NDV2+t1TWGauvP4xgzbTngkUh92dNZq9RPCHtA4uI/NRZrtw6ZzjdRaS/9mbgSR1+eizX0rh5Ut0DxG9sy6rFakhZgrkISNmOYoajzI6pMLY3CRJ/3GvJhf/bFtKf8+Ca/T0zrOjci/Rkb2+0XAd6t/v7Hyk1aOo31gp1QDS2P1DPCC80TaAdm1kmS+/Gzu4Y/nfC9jBDBCQS4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Save a compound calls. Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) Reviewed-by: David Hildenbrand --- mm/ksm.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 654400f993fc..b127d39c9af0 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2611,14 +2611,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) */ if (!ksm_merge_across_nodes) { struct ksm_stable_node *stable_node, *next; - struct page *page; + struct folio *folio; list_for_each_entry_safe(stable_node, next, &migrate_nodes, list) { - page = get_ksm_page(stable_node, - GET_KSM_PAGE_NOLOCK); - if (page) - put_page(page); + folio = ksm_get_folio(stable_node, + GET_KSM_PAGE_NOLOCK); + if (folio) + folio_put(folio); cond_resched(); } } From patchwork Thu Apr 11 06:17:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ED89CD1292 for ; Thu, 11 Apr 2024 06:14:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE9006B009B; Thu, 11 Apr 2024 02:14:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D72216B009C; Thu, 11 Apr 2024 02:14:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C12A56B009D; Thu, 11 Apr 2024 02:14:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9C0F56B009B for ; Thu, 11 Apr 2024 02:14:39 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6A77E140A86 for ; Thu, 11 Apr 2024 06:14:39 +0000 (UTC) X-FDA: 81996237078.25.8034809 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf23.hostedemail.com (Postfix) with ESMTP id 230D5140006 for ; Thu, 11 Apr 2024 06:14:36 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KC283FNT; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816077; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a8fdVXwZIC2+0jfsc23LnNEgvdTm8MtJqB0W/u7VV/I=; b=3O9nvq68hHk8xg69LHQzGgeyTzSHnboIl34bWCG6wSEsXbnzQdDxvqZh7NBPA+Oy4A4Fuy A23FT3xCDH2Nai5HIFaCIHy+wsII4cRIAQlLAixBxlT/5zu0t/xOu0eZ2tUMiTVgDwccvD iaGmhBR8MsAMQ6WvULmq2hzamFydjNM= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=KC283FNT; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf23.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816077; a=rsa-sha256; cv=none; b=mFRBtlwdVt6gaDB3EnAdRiXHzLhhRnI9UO9LJq+GHYOF+GmLV4eUDYAKzGdWAwwFkjkGSP bAaanYC36lf243uQgFI0e9OLWTSC1dr074LGmsrpkcf/KvqtQ1eN3I6jKcFBK8MUVGVmU4 ZDI6p+P9fa3DITWrFNXMB0GiJZl39yU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 421F6CE2F15; Thu, 11 Apr 2024 06:14:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B169CC43390; Thu, 11 Apr 2024 06:14:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816073; bh=sFkZO7Zaxh50oOXAu6R13zvDS+luqAu687nTeql3yIQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KC283FNT4ao2MpxFeaTvbWB0emRIG60QNJYtYHqPfmeIsjOwDnje007my1mGCHbr7 nfRzlHgiEt+8nqSnI2Rzz9caW/QhUDbKMurWAV3KSZVTF7Bu0a7LfeQ+o50k7IS9Tu G5siZcOLRKQm0js4dUzp+np8Rd7N+17FhMh/N+FoMO2LSUZn2uFA0Lqweak+78LTb3 NZjPajid42oiT2MINzFrl9LaG0exQ4Ynawq4rEopwlt/4GHR3kwEg6kImr4HFqwzMm Gq3jQqw9sQkIsGqZmUvKCqzJRkfX+JmRuRT797is7pY5H46bZ+ES7NHm55oNuHWOfq JzVnxlijt2ekw== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 07/10] mm/ksm: use folio in write_protect_page Date: Thu, 11 Apr 2024 14:17:08 +0800 Message-ID: <20240411061713.1847574-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 230D5140006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: yjnp68xzxmdc6wwqpgc58mu6xhd5syf8 X-HE-Tag: 1712816076-474828 X-HE-Meta: U2FsdGVkX18/V9Kncb4rsj9VONjZCnF89D56Td2URGGJn33uRgTwtISU/WbuIM/SP3DMchl3c1RlX9oXE6/NQM2y9QTmKVuH/n/Av2Ax3uT3VUEbubAS+p3nan7NCz1jm2YCF476G8++K8P3JpCl0h8WTy/3tRi2KFc0rIvJAthva6xM0Kfq2IYXxkUeTohedr8cKSHx1PHpLH7rQW1z+azJ+jAhX0gogGh9VDcAI2sWW+1/VwB9Zmob3IjN0dUxxdFAJTe8CLk8mFSll+quRm8RApMMgANtI3E4sf5+bt7gB2vv62y3UsORzjCXJ73BTSwP/A9nBL9uYFlMTPpQX4ZvUQnBm9z2YmUZIT18BHN1uACVPp8Uhfe+jmI9Wr8vHceTqdb35D1Kd9UZRMFb04Buhmb027JMLzTaM9BSYWcIMNV2sYgbs2Jq0GjaT2shb4/M2kSNJzEH6oFgGiXA99LKeWdHPlaqzTTZ6Ay2LE3kmjY31VQvHdWV8BGVbXJRAlcBGdY2WIxMRLQL39oQKDfKhgwNhvT9olgUhrfQmzuIIQo8dhbdG9FvvjtIFO7XnKXxhotx4EWPBryVREaWQd2PURtUuCXYJLw7NRaocwCf4YkPuue+DMyMOb6+Ctrmy0FRBnbYvu2Fu/enxz1+yksLPS8MlwyxQ2pSMqQHlvB0WUkzG4sRqqx6PAbbpNbiuJFrSTealvXDS1JS0d+zZcMZe366LR/aYQV2zOAOjqxToNRqu9GuuIQapmmLfm+FEMlIkX9nA/NOAfYZKYcOjsqiBdxWtn7AUl9ITgWqu0+3RsgJfdq4DGOpGioBlm8KGNlQ8oz3YLojPokKOP6IQkqOHf3PaPnC75/ltc2LslbRppA4e8MHJSxCNQPzbu5TYqEFBavNlUsdB4fqrdaW9wlo7EhotjSMCkG5oSf4kPVhAomWbpuphlWMAZNS4ChnEr2g1WJa69/w2arudHK 31jG5LKA /rtChYkjZZtPuToCL5uGioNs9xjppbut8b6Bk04NGoTz2vPMqmXJceagRqBzx7bILpe2y01UAzMQgQkdnUt3Srpo9TsKu1gdyveBlCVhSuPpsPVEj4t6nZwcOikni3adpUk83rxcE2tDUO/5samDShSxYlmhPtBd7bvC47Suj6EMYWK4LW6KMYFfv70xSi+3GT8PGo81hRgZSiM8OeXTVyS33DZJmc6xeSl5E+BIAX43a9mbk5wDGa3Ibvw6Nd+4apm/88Kl2t70yMoQXw6RhihJSI3rBpo9otQZ173NisOWZXvz9TkjiR4uakLmPaGEjRWJ/7BFtpJjTf8l7zchDIWHOwtWLUo9gtZhkEuFHvbF8JbWjQTDbs2kYRhUg8ptqSMLyvxxN2+vYS6BBDPmqlMvkbwhsoQ4PwN15aw4y+rGO0im83mX2nf3Kzk9TtagjG1WpgKsTIMd4SBUgmpTogs5dJOjpCI15YtMJvPVdK0nkKUAzW1KUY2ePPnhOiLsenJj8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Compound page is checked and skipped before write_protect_page() called, use folio to save a few compound_head checking. Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) Reviewed-by: David Hildenbrand --- mm/ksm.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index b127d39c9af0..2fdd6586a3a7 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1289,23 +1289,24 @@ static u32 calc_checksum(struct page *page) return checksum; } -static int write_protect_page(struct vm_area_struct *vma, struct page *page, +static int write_protect_page(struct vm_area_struct *vma, struct folio *folio, pte_t *orig_pte) { struct mm_struct *mm = vma->vm_mm; - DEFINE_PAGE_VMA_WALK(pvmw, page, vma, 0, 0); + DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, 0, 0); int swapped; int err = -EFAULT; struct mmu_notifier_range range; bool anon_exclusive; pte_t entry; - pvmw.address = page_address_in_vma(page, vma); + if (WARN_ON_ONCE(folio_test_large(folio))) + return err; + + pvmw.address = page_address_in_vma(&folio->page, vma); if (pvmw.address == -EFAULT) goto out; - BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm, pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1315,12 +1316,12 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, if (WARN_ONCE(!pvmw.pte, "Unexpected PMD mapping?")) goto out_unlock; - anon_exclusive = PageAnonExclusive(page); + anon_exclusive = PageAnonExclusive(&folio->page); entry = ptep_get(pvmw.pte); if (pte_write(entry) || pte_dirty(entry) || anon_exclusive || mm_tlb_flush_pending(mm)) { - swapped = PageSwapCache(page); - flush_cache_page(vma, pvmw.address, page_to_pfn(page)); + swapped = folio_test_swapcache(folio); + flush_cache_page(vma, pvmw.address, folio_pfn(folio)); /* * Ok this is tricky, when get_user_pages_fast() run it doesn't * take any lock, therefore the check that we are going to make @@ -1340,20 +1341,20 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, * Check that no O_DIRECT or similar I/O is in progress on the * page */ - if (page_mapcount(page) + 1 + swapped != page_count(page)) { + if (folio_mapcount(folio) + 1 + swapped != folio_ref_count(folio)) { set_pte_at(mm, pvmw.address, pvmw.pte, entry); goto out_unlock; } /* See folio_try_share_anon_rmap_pte(): clear PTE first. */ if (anon_exclusive && - folio_try_share_anon_rmap_pte(page_folio(page), page)) { + folio_try_share_anon_rmap_pte(folio, &folio->page)) { set_pte_at(mm, pvmw.address, pvmw.pte, entry); goto out_unlock; } if (pte_dirty(entry)) - set_page_dirty(page); + folio_mark_dirty(folio); entry = pte_mkclean(entry); if (pte_write(entry)) @@ -1519,7 +1520,7 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, * ptes are necessarily already write-protected. But in either * case, we need to lock and check page_count is not raised. */ - if (write_protect_page(vma, page, &orig_pte) == 0) { + if (write_protect_page(vma, page_folio(page), &orig_pte) == 0) { if (!kpage) { /* * While we hold page lock, upgrade page from From patchwork Thu Apr 11 06:17:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D185CD128A for ; Thu, 11 Apr 2024 06:14:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E867E6B009D; Thu, 11 Apr 2024 02:14:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E10256B009E; Thu, 11 Apr 2024 02:14:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEAC36B009F; Thu, 11 Apr 2024 02:14:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A0E2B6B009D for ; Thu, 11 Apr 2024 02:14:42 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6FF42A1380 for ; Thu, 11 Apr 2024 06:14:42 +0000 (UTC) X-FDA: 81996237204.01.905F9AA Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf22.hostedemail.com (Postfix) with ESMTP id 53E5AC000A for ; Thu, 11 Apr 2024 06:14:39 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tZbDdNNv; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816081; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=52pgxFwTl1nu/A9SyjlIt8sIQa2voklAnXRKCGj4JOM=; b=AOw+S5hA3W3LCU2x3OFF3JvozTUxDYrjoHdOAX8hqOMcHmJcPKt+f1gsktBZ58CFM9XQ9f UEsa2qER5VCR80CowJdW+LBKAgGJ8qc1lTjOdXcRd/Gmvp4V+KEyfw7ugbCuaE4Lmmvq/R oRGmB8UIhXyt+CogygG4iZalRKssZKc= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tZbDdNNv; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816081; a=rsa-sha256; cv=none; b=w7WARXbUVcYs5AbBECZzvtF8HL+ru9d3pmVuqn5U15uP6ALpXvPdP2fjdiuh1OG8mNeWs4 VVgrvD47k2ERnvyCgBpd4rJo9sKBWo5nk+8fV8vRAy40DjyRpvc4lFBzO0/lboT5s3M49+ aBqghJzEr6eICYfIC1W6PyUcQxXEdeE= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 2857ECE2F0A; Thu, 11 Apr 2024 06:14:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7467CC433A6; Thu, 11 Apr 2024 06:14:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816076; bh=2GVVNpZMi5SoTTNTvQBXF2hh1UdkIACaQFTD3ARx8EY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tZbDdNNv+PdK5e/4usAYNkLbBkxlnYZtRLhqtuKjdrjfruV8UtzX0jKTK/9QF9C7T 2izxyzQb3XkkUTQfYGWSJltVH7hJNqdRbI5ufVdeGrZ4eE8iRVg92gfk37kcgSam8R sNdH5DbwlRmSHHz70fyi4kA3ZzSGdp7Qer9Neo6/q1mEippdSBAO/6Nc4AnMd59Y1z /K31XoL8y/oSn3Urp0w8+MzRhKYkAkyREehCPKr6uavwQpqcYJB5SCkFK2Aw6tjZCo lxhClL9gOgqu39nQ+tQo+6z847FWXusNLdZ+I0PKDyyd8j9g485Cdoyg7ukJtuVfZ0 nGMvqhqYmaXdg== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 08/10] mm/ksm: Convert chain series funcs and replace get_ksm_page Date: Thu, 11 Apr 2024 14:17:09 +0800 Message-ID: <20240411061713.1847574-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 53E5AC000A X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 77taum3hkjimqrgtnijaz315z6zcr17y X-HE-Tag: 1712816079-424054 X-HE-Meta: U2FsdGVkX1+7F9djTzoxv7ufG0rR8yIA0mt4IJImZnTRzQvC0Nv1cNapEM2h2brotG/+VnraCNQV0jpEAwYA0F6Ha9FmTZqdI56qnUA306koyD4MdPt+zNybAORE7TPozfcWWo0gk3D6S69iU496sTfhNFhWbSnZrQSmBWGQBRcfW/rGOKEFH5auODDu+xGGlYyQ0oa8850E7I0x/SYlPBDfgkkyL3H1OmS1JZVdZwkf4IDbtziTTtZ5eoyuzfzIi6UYt+qtIT+ii/x5ds4ZWxXZTc5xBCxd0s+Tcet3wTWMjjA1R765izZDJEVwLK+YJ7BcUTicZpAGRJUX016EUz1imVU7nym0R8vNoeWvEFsGGqYdfPAjK4Ix2bVmA30xAUzKk9PVvhGhyAyxqGSa8R8HwCFTzA9UN7r5IJCZ7lTTLR6kk3I+n+XbOFblqmVcMiqRVK963h1F0udr3diCyOVJs+7Tqet296zTUPX0lFZE+wR45DvDiRizof9MlzfHbfSdfzYsZwJ0ntz5IUpieRK4yBZefd1QXPlZrPWGAiMfZ4TekeniEO1XcqLEoh4ZrMZ+VcPVGVaA37KysH14J8v7eOdfR5igiaewJ+HmqiNMTDrdnGSquC/uwSgFrAfC2Ja6rgCQCiJ0kVgAbcVF/PsyyueN/WqXDv4fYRK00X2VhlAYUOKpPJqTI28hUpWxpNoT6YR/QkQSqV8hrxuIOPXvA8VAKABpuKF1SWh8kUQ7pAktklvrrSzdswTdeUbL2IeMmdd3RPjx7Lwj416pJvNkJDqUPneGIiVJMe0M47D2PuFDP2XPtlj20k2g6JIikKF8GucbmAA8z7M5HP4Py5bkEdzeizEr4Lt3U48xpmgxPLT03tk4K+pUBG1vhmax7qEtDRUY6TZEpgoWgOJpP1Eis/0lhru0ORyvp1QKNdN9XhaqOWzoWZRY+fmMvTD3PB/t/jwlwPjiPFDEb9y u6hGU08s ASFxB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" In ksm stable tree all page are single, let's convert them to use and folios as well as stable_tree_insert/stable_tree_search funcs. And replace get_ksm_page() by ksm_get_folio() since there is no more needs. It could save a few compound_head calls. Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) --- mm/ksm.c | 136 ++++++++++++++++++++++++--------------------------- mm/migrate.c | 2 +- 2 files changed, 66 insertions(+), 72 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 2fdd6586a3a7..61a7b5b037a6 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -990,14 +990,6 @@ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, return NULL; } -static struct page *get_ksm_page(struct ksm_stable_node *stable_node, - enum get_ksm_page_flags flags) -{ - struct folio *folio = ksm_get_folio(stable_node, flags); - - return &folio->page; -} - /* * Removing rmap_item from stable or unstable tree. * This function will clean the information from the stable/unstable tree. @@ -1632,10 +1624,10 @@ bool is_page_sharing_candidate(struct ksm_stable_node *stable_node) return __is_page_sharing_candidate(stable_node, 0); } -static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, - struct ksm_stable_node **_stable_node, - struct rb_root *root, - bool prune_stale_stable_nodes) +static struct folio *stable_node_dup(struct ksm_stable_node **_stable_node_dup, + struct ksm_stable_node **_stable_node, + struct rb_root *root, + bool prune_stale_stable_nodes) { struct ksm_stable_node *dup, *found = NULL, *stable_node = *_stable_node; struct hlist_node *hlist_safe; @@ -1748,7 +1740,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, } *_stable_node_dup = found; - return &tree_folio->page; + return tree_folio; } static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stable_node, @@ -1765,7 +1757,7 @@ static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stabl } /* - * Like for get_ksm_page, this function can free the *_stable_node and + * Like for ksm_get_folio, this function can free the *_stable_node and * *_stable_node_dup if the returned tree_page is NULL. * * It can also free and overwrite *_stable_node with the found @@ -1778,16 +1770,16 @@ static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stabl * function and will be overwritten in all cases, the caller doesn't * need to initialize it. */ -static struct page *__stable_node_chain(struct ksm_stable_node **_stable_node_dup, - struct ksm_stable_node **_stable_node, - struct rb_root *root, - bool prune_stale_stable_nodes) +static struct folio *__stable_node_chain(struct ksm_stable_node **_stable_node_dup, + struct ksm_stable_node **_stable_node, + struct rb_root *root, + bool prune_stale_stable_nodes) { struct ksm_stable_node *stable_node = *_stable_node; if (!is_stable_node_chain(stable_node)) { if (is_page_sharing_candidate(stable_node)) { *_stable_node_dup = stable_node; - return get_ksm_page(stable_node, GET_KSM_PAGE_NOLOCK); + return ksm_get_folio(stable_node, GET_KSM_PAGE_NOLOCK); } /* * _stable_node_dup set to NULL means the stable_node @@ -1800,24 +1792,24 @@ static struct page *__stable_node_chain(struct ksm_stable_node **_stable_node_du prune_stale_stable_nodes); } -static __always_inline struct page *chain_prune(struct ksm_stable_node **s_n_d, - struct ksm_stable_node **s_n, - struct rb_root *root) +static __always_inline struct folio *chain_prune(struct ksm_stable_node **s_n_d, + struct ksm_stable_node **s_n, + struct rb_root *root) { return __stable_node_chain(s_n_d, s_n, root, true); } -static __always_inline struct page *chain(struct ksm_stable_node **s_n_d, - struct ksm_stable_node *s_n, - struct rb_root *root) +static __always_inline struct folio *chain(struct ksm_stable_node **s_n_d, + struct ksm_stable_node *s_n, + struct rb_root *root) { struct ksm_stable_node *old_stable_node = s_n; - struct page *tree_page; + struct folio *tree_folio; - tree_page = __stable_node_chain(s_n_d, &s_n, root, false); + tree_folio = __stable_node_chain(s_n_d, &s_n, root, false); /* not pruning dups so s_n cannot have changed */ VM_BUG_ON(s_n != old_stable_node); - return tree_page; + return tree_folio; } /* @@ -1837,28 +1829,30 @@ static struct page *stable_tree_search(struct page *page) struct rb_node *parent; struct ksm_stable_node *stable_node, *stable_node_dup, *stable_node_any; struct ksm_stable_node *page_node; + struct folio *folio; - page_node = page_stable_node(page); + folio = page_folio(page); + page_node = folio_stable_node(folio); if (page_node && page_node->head != &migrate_nodes) { /* ksm page forked */ - get_page(page); - return page; + folio_get(folio); + return &folio->page; } - nid = get_kpfn_nid(page_to_pfn(page)); + nid = get_kpfn_nid(folio_pfn(folio)); root = root_stable_tree + nid; again: new = &root->rb_node; parent = NULL; while (*new) { - struct page *tree_page; + struct folio *tree_folio; int ret; cond_resched(); stable_node = rb_entry(*new, struct ksm_stable_node, node); stable_node_any = NULL; - tree_page = chain_prune(&stable_node_dup, &stable_node, root); + tree_folio = chain_prune(&stable_node_dup, &stable_node, root); /* * NOTE: stable_node may have been freed by * chain_prune() if the returned stable_node_dup is @@ -1892,14 +1886,14 @@ static struct page *stable_tree_search(struct page *page) * write protected at all times. Any will work * fine to continue the walk. */ - tree_page = get_ksm_page(stable_node_any, - GET_KSM_PAGE_NOLOCK); + tree_folio = ksm_get_folio(stable_node_any, + GET_KSM_PAGE_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); - if (!tree_page) { + if (!tree_folio) { /* * If we walked over a stale stable_node, - * get_ksm_page() will call rb_erase() and it + * ksm_get_folio() will call rb_erase() and it * may rebalance the tree from under us. So * restart the search from scratch. Returning * NULL would be safe too, but we'd generate @@ -1909,8 +1903,8 @@ static struct page *stable_tree_search(struct page *page) goto again; } - ret = memcmp_pages(page, tree_page); - put_page(tree_page); + ret = memcmp_pages(page, &tree_folio->page); + folio_put(tree_folio); parent = *new; if (ret < 0) @@ -1953,26 +1947,26 @@ static struct page *stable_tree_search(struct page *page) * It would be more elegant to return stable_node * than kpage, but that involves more changes. */ - tree_page = get_ksm_page(stable_node_dup, - GET_KSM_PAGE_TRYLOCK); + tree_folio = ksm_get_folio(stable_node_dup, + GET_KSM_PAGE_TRYLOCK); - if (PTR_ERR(tree_page) == -EBUSY) + if (PTR_ERR(tree_folio) == -EBUSY) return ERR_PTR(-EBUSY); - if (unlikely(!tree_page)) + if (unlikely(!tree_folio)) /* * The tree may have been rebalanced, * so re-evaluate parent and new. */ goto again; - unlock_page(tree_page); + folio_unlock(tree_folio); if (get_kpfn_nid(stable_node_dup->kpfn) != NUMA(stable_node_dup->nid)) { - put_page(tree_page); + folio_put(tree_folio); goto replace; } - return tree_page; + return &tree_folio->page; } } @@ -1985,8 +1979,8 @@ static struct page *stable_tree_search(struct page *page) rb_insert_color(&page_node->node, root); out: if (is_page_sharing_candidate(page_node)) { - get_page(page); - return page; + folio_get(folio); + return &folio->page; } else return NULL; @@ -2011,12 +2005,12 @@ static struct page *stable_tree_search(struct page *page) &page_node->node, root); if (is_page_sharing_candidate(page_node)) - get_page(page); + folio_get(folio); else - page = NULL; + folio = NULL; } else { rb_erase(&stable_node_dup->node, root); - page = NULL; + folio = NULL; } } else { VM_BUG_ON(!is_stable_node_chain(stable_node)); @@ -2027,16 +2021,16 @@ static struct page *stable_tree_search(struct page *page) DO_NUMA(page_node->nid = nid); stable_node_chain_add_dup(page_node, stable_node); if (is_page_sharing_candidate(page_node)) - get_page(page); + folio_get(folio); else - page = NULL; + folio = NULL; } else { - page = NULL; + folio = NULL; } } stable_node_dup->head = &migrate_nodes; list_add(&stable_node_dup->list, stable_node_dup->head); - return page; + return &folio->page; chain_append: /* stable_node_dup could be null if it reached the limit */ @@ -2079,7 +2073,7 @@ static struct page *stable_tree_search(struct page *page) * This function returns the stable tree node just allocated on success, * NULL otherwise. */ -static struct ksm_stable_node *stable_tree_insert(struct page *kpage) +static struct ksm_stable_node *stable_tree_insert(struct folio *kfolio) { int nid; unsigned long kpfn; @@ -2089,7 +2083,7 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) struct ksm_stable_node *stable_node, *stable_node_dup, *stable_node_any; bool need_chain = false; - kpfn = page_to_pfn(kpage); + kpfn = folio_pfn(kfolio); nid = get_kpfn_nid(kpfn); root = root_stable_tree + nid; again: @@ -2097,13 +2091,13 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) new = &root->rb_node; while (*new) { - struct page *tree_page; + struct folio *tree_folio; int ret; cond_resched(); stable_node = rb_entry(*new, struct ksm_stable_node, node); stable_node_any = NULL; - tree_page = chain(&stable_node_dup, stable_node, root); + tree_folio = chain(&stable_node_dup, stable_node, root); if (!stable_node_dup) { /* * Either all stable_node dups were full in @@ -2125,14 +2119,14 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) * write protected at all times. Any will work * fine to continue the walk. */ - tree_page = get_ksm_page(stable_node_any, - GET_KSM_PAGE_NOLOCK); + tree_folio = ksm_get_folio(stable_node_any, + GET_KSM_PAGE_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); - if (!tree_page) { + if (!tree_folio) { /* * If we walked over a stale stable_node, - * get_ksm_page() will call rb_erase() and it + * ksm_get_folio() will call rb_erase() and it * may rebalance the tree from under us. So * restart the search from scratch. Returning * NULL would be safe too, but we'd generate @@ -2142,8 +2136,8 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) goto again; } - ret = memcmp_pages(kpage, tree_page); - put_page(tree_page); + ret = memcmp_pages(&kfolio->page, &tree_folio->page); + folio_put(tree_folio); parent = *new; if (ret < 0) @@ -2162,7 +2156,7 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) INIT_HLIST_HEAD(&stable_node_dup->hlist); stable_node_dup->kpfn = kpfn; - set_page_stable_node(kpage, stable_node_dup); + folio_set_stable_node(kfolio, stable_node_dup); stable_node_dup->rmap_hlist_len = 0; DO_NUMA(stable_node_dup->nid = nid); if (!need_chain) { @@ -2440,7 +2434,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * node in the stable tree and add both rmap_items. */ lock_page(kpage); - stable_node = stable_tree_insert(kpage); + stable_node = stable_tree_insert(page_folio(kpage)); if (stable_node) { stable_tree_append(tree_rmap_item, stable_node, false); @@ -3244,7 +3238,7 @@ void folio_migrate_ksm(struct folio *newfolio, struct folio *folio) /* * newfolio->mapping was set in advance; now we need smp_wmb() * to make sure that the new stable_node->kpfn is visible - * to get_ksm_page() before it can see that folio->mapping + * to ksm_get_folio() before it can see that folio->mapping * has gone stale (or that folio_test_swapcache has been cleared). */ smp_wmb(); @@ -3271,7 +3265,7 @@ static bool stable_node_dup_remove_range(struct ksm_stable_node *stable_node, if (stable_node->kpfn >= start_pfn && stable_node->kpfn < end_pfn) { /* - * Don't get_ksm_page, page has already gone: + * Don't ksm_get_folio, page has already gone: * which is why we keep kpfn instead of page* */ remove_node_from_stable_tree(stable_node); @@ -3359,7 +3353,7 @@ static int ksm_memory_callback(struct notifier_block *self, * Most of the work is done by page migration; but there might * be a few stable_nodes left over, still pointing to struct * pages which have been offlined: prune those from the tree, - * otherwise get_ksm_page() might later try to access a + * otherwise ksm_get_folio() might later try to access a * non-existent struct page. */ ksm_check_stable_tree(mn->start_pfn, diff --git a/mm/migrate.c b/mm/migrate.c index 73a052a382f1..9f0494fd902c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -616,7 +616,7 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) folio_migrate_ksm(newfolio, folio); /* * Please do not reorder this without considering how mm/ksm.c's - * get_ksm_page() depends upon ksm_migrate_page() and PageSwapCache(). + * ksm_get_folio() depends upon ksm_migrate_page() and PageSwapCache(). */ if (folio_test_swapcache(folio)) folio_clear_swapcache(folio); From patchwork Thu Apr 11 06:17:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92939CD128A for ; Thu, 11 Apr 2024 06:14:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3B3C6B009F; Thu, 11 Apr 2024 02:14:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BEEC56B00A0; Thu, 11 Apr 2024 02:14:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A64E96B00A1; Thu, 11 Apr 2024 02:14:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 816216B009F for ; Thu, 11 Apr 2024 02:14:45 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 0E0F040A33 for ; Thu, 11 Apr 2024 06:14:45 +0000 (UTC) X-FDA: 81996237330.02.E5365C3 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf11.hostedemail.com (Postfix) with ESMTP id B780740009 for ; Thu, 11 Apr 2024 06:14:42 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dDQK7Bqf; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816083; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=zRaE8XB9x3krm/w8wdPb38N4ZFM39swOoK9EEP9rGJE=; b=NO+Y6Shu+bM3Mgf3WGemW46OmWPZvhB07PP6Xpxi9mXLSgzrFjCPuyPIO4oWwP+ma5d4Ht ESU95duX0NVDHRyxJo+QiUJpDiPKKR7xUrD+wfABkkV3XTylzNsVwnY+NTB020+B+ILOBg FHmmQbLbXnZYKqbzhzeVBRL7JekcxXI= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=dDQK7Bqf; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf11.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816083; a=rsa-sha256; cv=none; b=VnXrPcgV3pHFOP/1wxfnKHPw4wHDx26lUGiwG8iPNbdU5/yZ+2FiRce8faVyu3zXfrUu8w gjKfijaKD2JNwkFkeT8lB6blUGIPc6bL0e3aFl9LVFZYYqBhcDGFGKNVFgNaMInOLnoeA6 63DT8Y6SvWoI4fgCFTxZsRX1nVs3sV4= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 1678BCE2F01; Thu, 11 Apr 2024 06:14:40 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 59CA9C43142; Thu, 11 Apr 2024 06:14:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816079; bh=wwxxrSyMrjaoURhO6OXX+a2E0cDwbJxoGFhUDcJuPic=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dDQK7BqfFTyKR3n/R+YvyO4I55kJp28xKTt8gOMJzVuB5hAcmSDUzIwLWBVsgvyfX Z6RjymuU0IhJe/n62nYuMNFQ5PMVSYhI5XxpLhIThOrYM4D92pBZP3z58FxKyiU3Qq 1kVWLlrq++6LvXjzYqEIHQEA9vJYF8cfrr5QHj1c1Eiu03aUiYbNOUdA4VGxgbLgXE /TPzMg7BdEdke3czh9Uxlrs/bG+nCAlWaMRO436c7KOe8uKYBOEGgAJECYwXFIdbxH i5gYTYmC76/TLxM5nbCeHVcTdn6lIK53zFtmshPmYl6SjDCvWHl/Ox68b39PndsJyM NAhveKN2mU7yw== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" Subject: [PATCH v5 09/10] mm/ksm: rename get_ksm_page_flags() to ksm_get_folio_flags Date: Thu, 11 Apr 2024 14:17:10 +0800 Message-ID: <20240411061713.1847574-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: B780740009 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: wk7qn35dy1nwrpx11yy5zee5gtkusdji X-HE-Tag: 1712816082-288250 X-HE-Meta: U2FsdGVkX1/EhUntZpQ5kMCnHGpBsOeV3TVrZ8eETJXCNRlueovIv1bmTRztQtpaMcqnm2zBh6nHwkfklGEB7Wulb9vktEi9g5c/rBtb3Jl7c+Gzqch98FNiVaVihu3UavXelF9Dn4QQhL74xOSSXxD+voo+g+zsvzDnpJqeB7ByAKz4rwG9fzRo9YeZBFiKma7nLNOBxcWTVP2HYhNQo7Fw1t8A7HMRkeNm74uAHj/OWGWyrUNWZS4q64Rvj8HeDVj9uagKjxKxBH5vaaQYgHXhPFMyfwpXfsVZba5IgfzNzr8PrwL/1eyNiKXSuVg6p4tTwVC6lVoZAGjRvhXZQbsO3IfHxqqoCX6P3aQ8uMzKlScw5MQDK01u0fCroWkxYpIYsGZD3Bt+WIBiFB4PltULpndaQnDxUkg1GulaXucIsAN6lqNCuj3gMb38IYijsoVF1KmiBHtFi2JUJcPExUXKWuFhZtn1pSW8cy9YNxrHUzBgIO9OxCOZRBWEd6SjBUV5HGWqfMyUVsuIqDDhfzl5KeSs6qT3xiGZj6pWBJ2UQ1IziJUdyl0KwGfTg2WxvwGAoghL3X7LRBou8GPTqOTL9wwXDBt14xtmTDD1LHtWuER2Kyx3wgFotA5mZ3pq0XPQkq0shQlfnexOt854N6pGDUzwmsLBcqmY8T0UBvIeIdoRGSXzBIfym773OWhJXPlT8Wi8eVU693FFan4XYRh0QctmdcjBVRZZZQKjjPsP9MEPb0CnlnFns7O05fdHYaYy3G2OjkDT5N/j56tjDohtaBMZdEGeh1+KDCYub1TO5aYuTgRsrUafTUs+jCCEBSGpQktil87pWLi0MQhm708dGbhfbBdRjw2N2L1J4gqZ73AgiP3AwnP6ZoST/uNHN5HDc58H2rS4qMmqTVZJtCXSMD0FIKUVps3P7ztN0W/hk0Lg10AkVQX0vwu/QzmyJpcSN8eiH8nEQZy6PX2 ssAFrVMz 3Cc9uqPn9Wh4el22ugIYZCg9cqAAALg2UC0u7T8s7ccbcHwqYuFepvrUj02MS//6+r4YAYFDq6JGd/JKAJGJZ+4u0vEkgGju+iUlK2ox4FlciLXQz+ih39CB9bJ9FhqrnsTYKfOVAT7YmGYmVT6nV7Brq8s2SKIUvHMr+cYZL70le8lMJk6hARn5mr4fpKcStNmTF5XE6zI/UvtGvp2lC7+9TDeX04JIV7iJytkUX0xbDs70zR86VMnj3Y33jvcfKpx9VMmmKR17rNw7QcrCgcHjL4al+80p2FdY5rBcPsnQcey5uzblPpCSFKw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: David Hildenbrand As we are removing get_ksm_page_flags(), make the flags match the new function name. Signed-off-by: David Hildenbrand Reviewed-by: Alex Shi --- mm/ksm.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 61a7b5b037a6..662fdaaf3ea3 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -890,10 +890,10 @@ static void remove_node_from_stable_tree(struct ksm_stable_node *stable_node) free_stable_node(stable_node); } -enum get_ksm_page_flags { - GET_KSM_PAGE_NOLOCK, - GET_KSM_PAGE_LOCK, - GET_KSM_PAGE_TRYLOCK +enum ksm_get_folio_flags { + KSM_GET_FOLIO_NOLOCK, + KSM_GET_FOLIO_LOCK, + KSM_GET_FOLIO_TRYLOCK }; /* @@ -916,7 +916,7 @@ enum get_ksm_page_flags { * is on its way to being freed; but it is an anomaly to bear in mind. */ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, - enum get_ksm_page_flags flags) + enum ksm_get_folio_flags flags) { struct folio *folio; void *expected_mapping; @@ -959,15 +959,15 @@ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, goto stale; } - if (flags == GET_KSM_PAGE_TRYLOCK) { + if (flags == KSM_GET_FOLIO_TRYLOCK) { if (!folio_trylock(folio)) { folio_put(folio); return ERR_PTR(-EBUSY); } - } else if (flags == GET_KSM_PAGE_LOCK) + } else if (flags == KSM_GET_FOLIO_LOCK) folio_lock(folio); - if (flags != GET_KSM_PAGE_NOLOCK) { + if (flags != KSM_GET_FOLIO_NOLOCK) { if (READ_ONCE(folio->mapping) != expected_mapping) { folio_unlock(folio); folio_put(folio); @@ -1001,7 +1001,7 @@ static void remove_rmap_item_from_tree(struct ksm_rmap_item *rmap_item) struct folio *folio; stable_node = rmap_item->head; - folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); + folio = ksm_get_folio(stable_node, KSM_GET_FOLIO_LOCK); if (!folio) goto out; @@ -1116,7 +1116,7 @@ static int remove_stable_node(struct ksm_stable_node *stable_node) struct folio *folio; int err; - folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); + folio = ksm_get_folio(stable_node, KSM_GET_FOLIO_LOCK); if (!folio) { /* * ksm_get_folio did remove_node_from_stable_tree itself. @@ -1656,7 +1656,7 @@ static struct folio *stable_node_dup(struct ksm_stable_node **_stable_node_dup, * stable_node parameter itself will be freed from * under us if it returns NULL. */ - folio = ksm_get_folio(dup, GET_KSM_PAGE_NOLOCK); + folio = ksm_get_folio(dup, KSM_GET_FOLIO_NOLOCK); if (!folio) continue; nr += 1; @@ -1779,7 +1779,7 @@ static struct folio *__stable_node_chain(struct ksm_stable_node **_stable_node_d if (!is_stable_node_chain(stable_node)) { if (is_page_sharing_candidate(stable_node)) { *_stable_node_dup = stable_node; - return ksm_get_folio(stable_node, GET_KSM_PAGE_NOLOCK); + return ksm_get_folio(stable_node, KSM_GET_FOLIO_NOLOCK); } /* * _stable_node_dup set to NULL means the stable_node @@ -1887,7 +1887,7 @@ static struct page *stable_tree_search(struct page *page) * fine to continue the walk. */ tree_folio = ksm_get_folio(stable_node_any, - GET_KSM_PAGE_NOLOCK); + KSM_GET_FOLIO_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); if (!tree_folio) { @@ -1948,7 +1948,7 @@ static struct page *stable_tree_search(struct page *page) * than kpage, but that involves more changes. */ tree_folio = ksm_get_folio(stable_node_dup, - GET_KSM_PAGE_TRYLOCK); + KSM_GET_FOLIO_TRYLOCK); if (PTR_ERR(tree_folio) == -EBUSY) return ERR_PTR(-EBUSY); @@ -2120,7 +2120,7 @@ static struct ksm_stable_node *stable_tree_insert(struct folio *kfolio) * fine to continue the walk. */ tree_folio = ksm_get_folio(stable_node_any, - GET_KSM_PAGE_NOLOCK); + KSM_GET_FOLIO_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); if (!tree_folio) { @@ -2611,7 +2611,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) list_for_each_entry_safe(stable_node, next, &migrate_nodes, list) { folio = ksm_get_folio(stable_node, - GET_KSM_PAGE_NOLOCK); + KSM_GET_FOLIO_NOLOCK); if (folio) folio_put(folio); cond_resched(); From patchwork Thu Apr 11 06:17:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13625448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A29F5CD128A for ; Thu, 11 Apr 2024 06:14:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1B57C6B00A0; Thu, 11 Apr 2024 02:14:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 165BB6B00A1; Thu, 11 Apr 2024 02:14:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 006216B00A2; Thu, 11 Apr 2024 02:14:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CA9D46B00A0 for ; Thu, 11 Apr 2024 02:14:47 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8B04F80A64 for ; Thu, 11 Apr 2024 06:14:47 +0000 (UTC) X-FDA: 81996237414.09.6201AD0 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf10.hostedemail.com (Postfix) with ESMTP id 87394C0008 for ; Thu, 11 Apr 2024 06:14:45 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=p8rhinow; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf10.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712816086; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MFcG4Yr2Hq5moJirQsn9mRkvbvfjJMIc5VQrDCoHTkM=; b=c04NEixCp5rZvmBjcV4FbEcGHs1EHL0PklKvqS/5UTXAhelAebqf0LKbcmEnbxsB0CNjYT L/7LbzKotyrA+Ao8rZlU4LN5Sz8YVSdcYx6MeMCxefuEwhc2oMNfpSCIY7P86uuDfy+R5Y 4cfaMIPg+Hv20Bu6ch6vPrGb5MT3OR4= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=p8rhinow; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf10.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712816086; a=rsa-sha256; cv=none; b=t40fyGa7mIgw48YeiW/dEcGuyVp8aKVBpMu5mf5Dhi1oz/khFQDqGVGkYxBIwefwiMvltv VstzOED47qQzrhivp/Sdc3fQInot/2HbgsSWE4pZrjavGCFzgDKRENhPq3F6LCTRZYIc0i YHWcAT7nHN2eQi4Qtx0sb1e3+UN9lHA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id D4D9DCE2EFB; Thu, 11 Apr 2024 06:14:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E3C97C433C7; Thu, 11 Apr 2024 06:14:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712816082; bh=2BbBT91rO5EKEsTUfQ1HnSSvv2jhr/Cd5b48tiAufmA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p8rhinowpXykMIv8xrZeYQ4/sM/rBmUtzroByI24TOcabbu/AkuHzwOo+wLvLyBpS ei0r0IEsaUt6MiLYL78P4HxgT4D0qn7aVE0oPYB+vS8d+2JgmWR7RC6mVVCbCYkxJS d96sO+9dyG3l6GtE7RAf2pH1a4qCulOXq6jesyBIpnKlqbyvXaujgi2UkUOtJ9tsME 3vjChWTYULjRrUtk/iQvsOXiOwgV8vZobvEoLodOJdwvRydog0NJLVNYYyq2vcvoi2 U4GARspuxshWeFuJK3WhJGdkBjbfZ05BoReePvWFmPG2XqxozAr9Nz/Z39EAWC3ew+ x5SoVAJSv6oPA== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, zik.eidus@ravellosystems.com, willy@infradead.org, aarcange@redhat.com, hughd@google.com, chrisw@sous-sol.org, david@redhat.com Cc: "Alex Shi (tencent)" , Izik Eidus Subject: [PATCH v5 10/10] mm/ksm: replace set_page_stable_node by folio_set_stable_node Date: Thu, 11 Apr 2024 14:17:11 +0800 Message-ID: <20240411061713.1847574-11-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411061713.1847574-1-alexs@kernel.org> References: <20240411061713.1847574-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: y3ofwwbsh46ioffyhqycu1ugbgkbmfzp X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 87394C0008 X-HE-Tag: 1712816085-276326 X-HE-Meta: U2FsdGVkX19Bo0gKpDfcS95TnulUBanxNTwLnSgROUkWectmRZFF4IhpX1KqKEojnYy2wUBvrEmKEaquqq4YdA6O+nJNK2rhM9M6pN/K7Kl8jYaUj1k8Oaedj1WYbFnciYySgi09MC7PKAOjDxrGVKMq7Wj0gYJdd6X1ZHAlCEAn8kBS7lgHcTIt2Dqabskg6wq3I7s4ldiuivqg0Lb1BIKETQQZPr8W9jCpl/tTyNjgM32lcKyN3A7s8hM+NA3raV0bjKt2Y7sytE+Cc2rc75XqrMSXYm6S3uGtV7YsgLXAeUJdWafzZ26XEOQ8+vXzPMVPSoue2asYvtH4LvBfp6KLxlN/Cx7qnfUqjpZWXBxXX/Vm//94RT9IDeIkrlElSlmQwBArRQirBeDNe6yFqMD3Vg9b/YAhlEuXzaIOW0RL/qdDe5HO4tOhARRX9mh5gy+Lt6UYSma9zPn54lWCanGZrgf9KUI8+LklHPRWH/nusaubUFxhZ4yRXAXmUIdl6yUdVCRT84tbXw5GTMqTj10JIFKJWDq65Fznk5MJ4JeqVqejn9e8cLPEplXGwclfc5ewetvDW77OZDLnUiW+s2IpdBsUyWF9NnFC1hQJjqhLfdJDhLuv9NAPWZ368dPfsFW4dZhBQqJMGm2CMCgNGsqoflFCOOjDGX6yvBYKTJL0HIEgHqTVfyiYXczigCd6ndgwyjzuoBO9uuSb8pAeSxR1GHOLOTVwW373FLFlURTOgZ1KYTLAtPdU+TqKrMiY4ZQUZLrQleNbqPCAZJ7zfF4tpjDke3gTsnWOikjksR4/cMwH23yRJn0xf4fxq6sHnnNkMbHBOqkXGfCF2pngV/DdDz+UQzUk+QNYfMudJ9FDqTrXpIugkB++S8qungmQmgsDJcdbDgcqjKqEBH+DXZ+z/GDvooltQ4Da41YqP1TfY+1C6tb85iGEoiOO5uHDhC/NF6K9a8NYmjQLmUv RGzTgYZ6 BpDRo4ad1v2LPKE3seuC3ze1ruNHS3dhb/fLlkHtpPzIpR5lxmJTMFPQ6J9AArHEkG5kE/BGdaCnpst5KWrM8/+B7L4Tl7to7pP46QFT1SHnI5aT23+zor5xXljfziuQ67ttASS/QU4tyT3P2Q+OeAhlEuQ8tdAjDl9jJ9u50ft5QBkdEMkzl/g5DavfQNO/FEMRNH+J0dIrKZfFohSkdRfQvxUoWq6h/tXh56IcrDYMbfX3TGuRusedY6jNQRJ7GJTtsc+nuYdPrTPrMeRusLpBeFPhWeDg8L51mIu0jwQuMRHyINtHnDJLeV+6JI30xFaVaPyjaDH/tH05+unDgLH++H4pZ5GelWTh4qHEF1dReUeJTcnzmXZuqKrmTc+7zkJvY5WDFj+KIPWERkOWaiQlALVoPgf3ZuExM+k7awXIFAMPIA1n0zpGK+Mzd2d8Z9dyzIAyBB68zV2cXXRXmMtu1ltMbgmNJ7U7MZ19JiTWeBPtXUG6Al6PuTr4m0/SD1fGI X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Only single page could be reached where we set stable node after write protect, so use folio converted func to replace page's. And remove the unused func set_page_stable_node(). Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Signed-off-by: Alex Shi (tencent) Reviewed-by: David Hildenbrand --- mm/ksm.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 662fdaaf3ea3..486c9974f8e2 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1094,17 +1094,11 @@ static inline struct ksm_stable_node *page_stable_node(struct page *page) return folio_stable_node(page_folio(page)); } -static inline void set_page_stable_node(struct page *page, - struct ksm_stable_node *stable_node) -{ - VM_BUG_ON_PAGE(PageAnon(page) && PageAnonExclusive(page), page); - page->mapping = (void *)((unsigned long)stable_node | PAGE_MAPPING_KSM); -} - static inline void folio_set_stable_node(struct folio *folio, struct ksm_stable_node *stable_node) { - set_page_stable_node(&folio->page, stable_node); + VM_WARN_ON_FOLIO(folio_test_anon(folio) && PageAnonExclusive(&folio->page), folio); + folio->mapping = (void *)((unsigned long)stable_node | PAGE_MAPPING_KSM); } #ifdef CONFIG_SYSFS @@ -1519,7 +1513,7 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, * PageAnon+anon_vma to PageKsm+NULL stable_node: * stable_tree_insert() will update stable_node. */ - set_page_stable_node(page, NULL); + folio_set_stable_node(page_folio(page), NULL); mark_page_accessed(page); /* * Page reclaim just frees a clean page with no dirty