From patchwork Fri Jun 21 07:54:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chengming Zhou X-Patchwork-Id: 13706966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31D68C27C4F for ; Fri, 21 Jun 2024 07:55:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F20306B0195; Fri, 21 Jun 2024 03:55:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFCDD6B0190; Fri, 21 Jun 2024 03:55:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A173D6B018B; Fri, 21 Jun 2024 03:55:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6E8096B0179 for ; Fri, 21 Jun 2024 03:55:16 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 22E9D140A5A for ; Fri, 21 Jun 2024 07:55:16 +0000 (UTC) X-FDA: 82254135432.20.77DD57A Received: from out-170.mta0.migadu.com (out-170.mta0.migadu.com [91.218.175.170]) by imf03.hostedemail.com (Postfix) with ESMTP id DE8C82000C for ; Fri, 21 Jun 2024 07:55:13 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=UsFO+6jv; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf03.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.170 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718956506; a=rsa-sha256; cv=none; b=p0sAYI+XWEviq/2O787Vu05Z0ocQACkGbGcW+nxBNJDpz7t270wgdjMZD4L4ytRyBL7B00 Dd2HgknGaHyjmaKP1OK+JNZDQS5u9B1o8oYmlpnajH/chvge3n34jbmtVCUElzTJhNH2gt LCz4YXy1D9am2vvUB8HRQwXo2l/71N8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=UsFO+6jv; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf03.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.170 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718956506; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3Sw5yRtWHNEncebzoxSE8G3qpCoVdCgLAzOgxBxk0ok=; b=JGAG89fRJkL2z2fymFN2FXIOc4ODToUpm8AYnqN8FUWi5hTwpCmdvHkE2XfLK/swICQE4p zYMSbXmFhB2U79bBMHrRJir1WGl4JCol7TJ6dY1+RSrg8vdoBgAwzQPa5S9WoNXWwwtk2U yO01dbfmy23mUL3G2D06jtCDmoh1gYQ= X-Envelope-To: david@redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1718956512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3Sw5yRtWHNEncebzoxSE8G3qpCoVdCgLAzOgxBxk0ok=; b=UsFO+6jvoTUIca3cM8LE6Q/FLwwPh2O7GrjHMbrKFS456cQzZL2wll0WrqklJKIbpOcaSh Od4d/qgiEbIfG4KtxQUenDjLLejLnjX7OCF5iw6KSalGLSin40c+pRTS6KtuLwPcNe4IRB MOoc6qc2MR7WB2d7s1utSWzdz6ivOHU= X-Envelope-To: linux-kernel@vger.kernel.org X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: hughd@google.com X-Envelope-To: chengming.zhou@linux.dev X-Envelope-To: zhouchengming@bytedance.com X-Envelope-To: linux-mm@kvack.org X-Envelope-To: aarcange@redhat.com X-Envelope-To: shr@devkernel.io X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou Date: Fri, 21 Jun 2024 15:54:30 +0800 Subject: [PATCH v2 2/3] mm/ksm: don't waste time searching stable tree for fast changing page MIME-Version: 1.0 Message-Id: <20240621-b4-ksm-scan-optimize-v2-2-1c328aa9e30b@linux.dev> References: <20240621-b4-ksm-scan-optimize-v2-0-1c328aa9e30b@linux.dev> In-Reply-To: <20240621-b4-ksm-scan-optimize-v2-0-1c328aa9e30b@linux.dev> To: Andrew Morton , david@redhat.com, aarcange@redhat.com, hughd@google.com, shr@devkernel.io Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, zhouchengming@bytedance.com, Chengming Zhou X-Developer-Signature: v=1; a=ed25519-sha256; t=1718956503; l=2616; i=chengming.zhou@linux.dev; s=20240617; h=from:subject:message-id; bh=CNJoKmOJG8f4fgj5rPkIJlw0KkteLIjSIgyHyFopqbI=; b=pKzOj1vYEVS8cLcJdoc/uLdiP+G/dkr/63SxIyokg2j6ABqeJkQHe1BneY9VkMaHaMJbezD2Y UKiUVY+NP24AbVV8kQrFJvtlLG8q10a1K7l3UEu74G5jaVQWrvAxiQR X-Developer-Key: i=chengming.zhou@linux.dev; a=ed25519; pk=/XPhIutBo+zyUeQyf4Ni5JYk/PEIWxIeUQqy2DYjmhI= X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: DE8C82000C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 5rrk4mmzetn8hu9rffjnaqbj8hquxopy X-HE-Tag: 1718956513-291102 X-HE-Meta: U2FsdGVkX1/akw2akBJ3e8GxtEkl7ooa08NzVIXYaVS9WYnEaB0r/MMpkc/kCCpPp7LYT+Y1JBX5glYCzxIt/guUiUMIqrqbV1SYO+TdNeU0yi0DYUXZp3m1jqJcp81aXHtB6ipMFN7V+GMxt5xrawFxgjvO/J6oupm7ak7GQH5x7s5QxXi671lI2vLkeIPYioAIQo4LtUns4Cez39i3VeHuSUy3mUz2wIwv72UUSr3ZOU825andW0LIGB+EfFiu1cpa22O24KaNhD01O9SPP/I0K/bSt9FDWD0A9JkIGon2xs1h6yd6M1Sf96XQ6bsPJbP9V/yCOzVlCArXldLowdRAwTUQnL7fiR2/7LvLzYe8qF+niL6GBlK1EWBGY7g5noISA7SSgjw5UnCs4bY0SlvN/N0T7js9kmCQeihhR7D3+SEbPp9U1uiO+u/QQJJ9glGSIhQCc9dsIcIFiYH7CE0yFsnjT+40SbEETaEUEPbCS0GFZBeCQFvb5swMmGZwoktIjn3PXPm/0uZD6ZFGodJAHvc62M89Ns6Q2bVdBRRBJ/IXi8OBHus/XWMQkomJ2RIYWmcSpwYH2q8fWiuwnXpsVP/EORJomkIaVQCiku8ippRwkozB7UAbztqvNq9HTBKBJL5AssHg+2bLjj3LtLK1xQ3tOzYQntFRhh/dmsCNh24kWoZC8we8rF1VM3h5pNyLVyiO3kdsHAJ8LEqU1hwME9QHanHzK4nDrZz8K8MgwCUg38DV9t3XuLI90aT9Dy1hXnvTxazejoXS4Vrh5gmCUNQd32JymRV/KfS5HDJtr86ST06DjTLKDiFIuAFYJ6fg9lvtN7f1qm2vXCv+4MGeeQOZ+o4erkeoXE7at5Fl+d8FyjI+HZK7BL5MDO2nuOCEBWGf3uGcf3AocxENNgRgUj6Cb5RXQ4RyEMYHPS0RWAKsRtbPWaM+d3Zhv1MI8cGWOtBPZ9g2bo+trOu 6qTGn0qD kJMp9O4w7S4rkXbnetzyGwywOvB+SNMjnVqmnmWMS5/BCINNAF6N8300oBPQ5FHjZS9IOU4325KajAfQe4LcAZZfTGt/f18W2gOCmFF5Tzse4lzaKosBvTVZiv1oGNDi+Ywd1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000067, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The code flow in cmp_and_merge_page() is suboptimal for handling the ksm page and non-ksm page at the same time. For example: - ksm page 1. Mostly just return if this ksm page is not migrated and this rmap_item has been on the rmap hlist. Or we have to fix this rmap_item mapping. 2. But we absolutely don't need to checksum for this ksm page, since it can't change. - non-ksm page 1. First don't need to waste time searching stable tree if fast changing. 2. Should try to merge with zero page before search the stable tree. 3. Then search stable tree to find mergeable ksm page. This patch optimizes the code flow so the handling differences between ksm page and non-ksm page become clearer and more efficient too. Signed-off-by: Chengming Zhou --- mm/ksm.c | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 1427abd18627..2cf836fb1367 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2370,6 +2370,23 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite */ if (!is_page_sharing_candidate(stable_node)) max_page_sharing_bypass = true; + } else { + remove_rmap_item_from_tree(rmap_item); + + /* + * If the hash value of the page has changed from the last time + * we calculated it, this page is changing frequently: therefore we + * don't want to insert it in the unstable tree, and we don't want + * to waste our time searching for something identical to it there. + */ + checksum = calc_checksum(page); + if (rmap_item->oldchecksum != checksum) { + rmap_item->oldchecksum = checksum; + return; + } + + if (!try_to_merge_with_zero_page(rmap_item, page)) + return; } /* We first start with searching the page inside the stable tree */ @@ -2400,21 +2417,6 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite return; } - /* - * If the hash value of the page has changed from the last time - * we calculated it, this page is changing frequently: therefore we - * don't want to insert it in the unstable tree, and we don't want - * to waste our time searching for something identical to it there. - */ - checksum = calc_checksum(page); - if (rmap_item->oldchecksum != checksum) { - rmap_item->oldchecksum = checksum; - return; - } - - if (!try_to_merge_with_zero_page(rmap_item, page)) - return; - tree_rmap_item = unstable_tree_search_insert(rmap_item, page, &tree_page); if (tree_rmap_item) {