From patchwork Wed Aug 3 10:05:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12935349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72AA2C19F28 for ; Wed, 3 Aug 2022 10:05:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1370A8E0002; Wed, 3 Aug 2022 06:05:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E6B46B0072; Wed, 3 Aug 2022 06:05:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF0C38E0002; Wed, 3 Aug 2022 06:05:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DEC3B6B0071 for ; Wed, 3 Aug 2022 06:05:46 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A7C251213FF for ; Wed, 3 Aug 2022 10:05:46 +0000 (UTC) X-FDA: 79757849892.14.1DF4364 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf14.hostedemail.com (Postfix) with ESMTP id 1F978100126 for ; Wed, 3 Aug 2022 10:05:23 +0000 (UTC) Received: by mail-pf1-f177.google.com with SMTP id q19so6667828pfg.8 for ; Wed, 03 Aug 2022 03:05:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=V7yMXZa3MywkOFBvlYMPqSqmCaZ5JNGIAqlkaSLZ/kM=; b=ipjbvHg48fVyW4jVX5gIjrnxkQ8/Uun3HS5plAciRpFp9N9V56WtAuDOfPwusWnA0M Ra3acdntu7vD9oDei3ZjYZSowPGpmY07igPy8HRpVLkeTkcf31OxvmQ+go0em31ADw2e xdPeXir+TMMAUQfNFeBjzEWoAsiIjC4BmO0xeqiO4pqM8SABLHZjevqyqoxRcIqtfqsO QpROXLlVD+2olAHOIGCTIU5YWCf1IkbU4LJf1AEX7ZQI0AHQPwQRKNc9+Tr6t91wITSV W9wIQoLg2vLppCaaIrBWQx3tzEu90iHp/9ztRKJxAbAa4Jj6VCf5dDziuLsHdXDuscqC Zwzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=V7yMXZa3MywkOFBvlYMPqSqmCaZ5JNGIAqlkaSLZ/kM=; b=8InVsFFSB/ACQR4ubaPzYPqTRjjf6dkaq01p2Cd8pduDtBHw93o/EO96QKiUuV5RJC Zl+Pp2iezAhsoHVsotJkAD4piLOqmKNLPYZ5hKmPX1e2SNNwf6ZAUWP4GlqGiiB3It7p XI61Bs4ZnX2Paz8q4uz9vsIoro61yqI4tMvKDOEZS3zS8D6z/wTnFR7ul/YwxLcEjtcm zjXA43EFgY2GaeewM9Z3UDSz4dMPVesPrlygXHIH9dMk9kAE/RPglZYm/Y1rYViFr8D0 dldRJL0CSpS8iRgHtuE2nnbD7+5He0TA646k3M978cUc9HCfQEvzGAUdo7QoT7WR1GNp JBCg== X-Gm-Message-State: AJIora+pUlkb91Vkp3dLxWGm34i6Xy5k6XWEq3OYbzMfRyjvLNXmaX0k rasv5OnfAu//gAmNBkCXbQs= X-Google-Smtp-Source: AGRyM1saowNGHUEc91GATroNAs1Ulpr9zGAqAodgHMmGAOrGXpFhOyI9NrSJEWMx+FVvZRavoC9blg== X-Received: by 2002:a63:c10a:0:b0:41b:f15:f9c4 with SMTP id w10-20020a63c10a000000b0041b0f15f9c4mr20763742pgf.72.1659521123115; Wed, 03 Aug 2022 03:05:23 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id k14-20020a170902d58e00b0016b90620910sm1455917plh.71.2022.08.03.03.05.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Aug 2022 03:05:22 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.con.cn To: akpm@linux-foundation.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, willy@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu xin , CGEL Subject: [RFC PATCH 1/4] ksm: propose a auto-run mode of ksm Date: Wed, 3 Aug 2022 10:05:14 +0000 Message-Id: <20220803100514.1653440-1-xu.xin16@zte.con.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803100306.1653382-1-xu.xin16@zte.con.cn> References: <20220803100306.1653382-1-xu.xin16@zte.con.cn> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659521142; a=rsa-sha256; cv=none; b=TsD0+H11IoWOotA3X5OyRmLfTipmshamdsSEGkjpitqUaIBaaTIOs9urqXnqNjJvEE6jC1 EufJO510JvzJM+6NMcYvafk/c9TPz7ygJZ2woBRUdMQ2AjFMq6Grxwx5u+LMjIqi/kmhmm g14+skHa3oyeZMEQP9Xd5xr2WmwYdyw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ipjbvHg4; spf=pass (imf14.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=temperror reason="query timed out" header.from=gmail.com (policy=temperror) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659521142; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V7yMXZa3MywkOFBvlYMPqSqmCaZ5JNGIAqlkaSLZ/kM=; b=KlutWqPvqWMmw+tZjHzMW0MdBXmzPNlxHA/V76ODP2tZi6gRDhzGEce0+JV9dEPu+t9lFN mSfkMJOga/PXOq5q5MwNMf1A0Rkdd9KTWrk4kNCdr1XRt3KLNTfCSv6HSL0bU6b75DLe3j YUwEvpdyxi/hE75B5lUd2pfwJ3945hw= X-Rspam-User: X-Stat-Signature: gug9q6bmrxhea78e196y1e5qy5zjgsrh X-Rspamd-Queue-Id: 1F978100126 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ipjbvHg4; spf=pass (imf14.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=temperror reason="query timed out" header.from=gmail.com (policy=temperror) X-Rspamd-Server: rspam02 X-HE-Tag: 1659521123-155292 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin Add a new running state auto-mode to ksm. This is to pave the way for subsequent real optimization features. Use it by: echo 8 > /sys/kernel/mm/ksm/run Signed-off-by: xu xin Signed-off-by: CGEL --- mm/ksm.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 2f315c69fa2c..c80d908221a4 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -290,6 +290,7 @@ static int ksm_nr_node_ids = 1; #define KSM_RUN_MERGE 1 #define KSM_RUN_UNMERGE 2 #define KSM_RUN_OFFLINE 4 +#define KSM_RUN_AUTO 8 static unsigned long ksm_run = KSM_RUN_STOP; static void wait_while_offlining(void); @@ -2416,7 +2417,9 @@ static void ksm_do_scan(unsigned int scan_npages) static int ksmd_should_run(void) { - return (ksm_run & KSM_RUN_MERGE) && !list_empty(&ksm_mm_head.mm_list); + if (!list_empty(&ksm_mm_head.mm_list)) + return ksm_run & KSM_RUN_AUTO || ksm_run & KSM_RUN_MERGE; + return 0; } static int ksm_scan_thread(void *nothing) @@ -2916,7 +2919,7 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, err = kstrtouint(buf, 10, &flags); if (err) return -EINVAL; - if (flags > KSM_RUN_UNMERGE) + if (flags > KSM_RUN_UNMERGE && flags != KSM_RUN_AUTO) return -EINVAL; /* @@ -2942,7 +2945,7 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, } mutex_unlock(&ksm_thread_mutex); - if (flags & KSM_RUN_MERGE) + if (flags & KSM_RUN_MERGE || flags & KSM_RUN_AUTO) wake_up_interruptible(&ksm_thread_wait); return count; From patchwork Wed Aug 3 10:05:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12935348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3028FC19F28 for ; Wed, 3 Aug 2022 10:05:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A5E448E0001; Wed, 3 Aug 2022 06:05:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0E366B0072; Wed, 3 Aug 2022 06:05:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AE438E0001; Wed, 3 Aug 2022 06:05:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 7875D6B0071 for ; Wed, 3 Aug 2022 06:05:38 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4236212130A for ; Wed, 3 Aug 2022 10:05:38 +0000 (UTC) X-FDA: 79757849556.10.0E452FB Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by imf26.hostedemail.com (Postfix) with ESMTP id 436D7140128 for ; Wed, 3 Aug 2022 10:05:36 +0000 (UTC) Received: by mail-pg1-f175.google.com with SMTP id 12so14705889pga.1 for ; Wed, 03 Aug 2022 03:05:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=C9VGA9sPog0r28l3bCbEGjn4o1fpHQlIN2nhYVLWw/k=; b=F0m9Q1ha0av70aTNZazPL/+nYhOyWG5MFlSFketERp6xdP5CSW0snaik6ZkvSq5eG0 ASzhGNd9PIVGQkeOyDWp7E6trgcnW6YQnzCJsnm3/xp4/cnzmdVd84VJmGEcFRvQWDpz Ta3UeNHGrT/OOlpZzTMrvC3yq0QN4Gz6nYpd31UTk6ZF2hht6W0p+Nk2o1oezlGPAetK xyW1x0aBUGYc3eu0LRIoxwxtt1+ihbYAyyZA59IRajKm7mDGFOtGTtonWxbI6xA8i++7 4N+tUoCcG83KFeLFJXYlc6vTDa6GT8Qy8XRZbdxAABre6Q42qnRXf9JfN4pH7f02ebqe KlyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=C9VGA9sPog0r28l3bCbEGjn4o1fpHQlIN2nhYVLWw/k=; b=0ixcIXby3es/yh1SzEXyNXPrV+wGHaaHlwnJ6GSiwB9Im5JFlwOTZsgYQY//O2ppT9 KScRjTUzukLBUD8+7WLWdPru1dubO8dKF8Kn6tFA0c3BAhPTXaW/YPjV0ak2QVYCcP0D 7Sg62Fsy61KEEBsdea05C4ZaxutrRMFx9uMfDELtMtRqFZ601NNmyIdKvPXWQJ8PlZDr xOyxqPGjA3VYDEUyjF9e7Uo+gkLEmb+Ooo4OJUw+2ey+03uCi6hx+r/stydTqlr1D8zl u1qH0XNsgr/HMEbS0vNooUVnRt15k1lVyFvXvJMWg8Q4c8vcZ6NfRMbB1C984FXenZ4D XHMw== X-Gm-Message-State: ACgBeo3321oKNNy7OykvmI4xAk83LeURAyfPnqYdKiHqiDEqpgPV00KA F+N2w+fGtogMinq6wvBmpSA= X-Google-Smtp-Source: AA6agR6kGDhUSD4pprsK8/LMWUFqIDLfXb2kskwuuEFcf+pw4VJ649BtqcFgbC/w1Qh3CAiRFsQpqA== X-Received: by 2002:a05:6a00:1f8b:b0:52d:5b9e:3ecf with SMTP id bg11-20020a056a001f8b00b0052d5b9e3ecfmr14659525pfb.48.1659521135990; Wed, 03 Aug 2022 03:05:35 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id w8-20020a1709026f0800b0016dd667d511sm1452653plk.252.2022.08.03.03.05.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Aug 2022 03:05:35 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.con.cn To: akpm@linux-foundation.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, willy@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu xin , CGEL Subject: [RFC PATCH 2/4] ksm: implement scan-enhanced algorithm of auto mode Date: Wed, 3 Aug 2022 10:05:30 +0000 Message-Id: <20220803100530.1653496-1-xu.xin16@zte.con.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803100306.1653382-1-xu.xin16@zte.con.cn> References: <20220803100306.1653382-1-xu.xin16@zte.con.cn> MIME-Version: 1.0 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659521137; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C9VGA9sPog0r28l3bCbEGjn4o1fpHQlIN2nhYVLWw/k=; b=vE/nqPWfpgwpvnwj2qhHxubI7vN1Ywval64+rWBpCsSFy5uY67yqkDJ9DOjs1awovnvAMM bQApaMD1PPSGKVT6RbOMSuKYGgIvSh0FoakM79mAdvqx+30iEElsMcYYExtkLHmqULvfru OgPJVHdCznvpZ+/fE0LSuB8KNssb49s= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=F0m9Q1ha; spf=pass (imf26.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659521137; a=rsa-sha256; cv=none; b=VAReRLGMtTGaqBKiWTZCVcQ2kCVHlWi9y7PmBu6HwpPHvIfhzWWeNNIkX8Ndub9QSk2Q54 b2uPdfFiNxRVvOpauOjiJgrMfzOg/wb31amujcPDERskViHrAp30hz5VTdnjOZhKe4kz0W US0aO3Xzit9oCpgw5ctfI254i8erPps= X-Rspam-User: Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=F0m9Q1ha; spf=pass (imf26.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.215.175 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: qow4upure796wwob5q9ftgba4tzrp7ek X-Rspamd-Queue-Id: 436D7140128 X-Rspamd-Server: rspam10 X-HE-Tag: 1659521136-536576 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin Implement the scan-enhanced algorithm of auto mode. In this algorithm, after every time of scanning, if new ksm pages are obtained, it will double pages_to_scan for the next scanning until the general multiplying factor is not less than max_scanning_factor. If no new ksm pages are obtained, then reset pages_to_scan to the default value. We add the sysfs klob of max_scanning_factor to limit scanning factor's excessive growth. Signed-off-by: CGEL Signed-off-by: xu xin --- mm/ksm.c | 88 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 86 insertions(+), 2 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index c80d908221a4..8acc893e4d61 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -131,6 +131,8 @@ struct mm_slot { * @address: the next address inside that to be scanned * @rmap_list: link to the next rmap to be scanned in the rmap_list * @seqnr: count of completed full scans (needed when removing unstable node) + * @new_ksmpages_of_this_scanning: count of the new merged KSM pages in the + * current scanning of mm_lists (cleared after ksm_do_scan() ends) * * There is only the one ksm_scan instance of this cursor structure. */ @@ -139,6 +141,7 @@ struct ksm_scan { unsigned long address; struct rmap_item **rmap_list; unsigned long seqnr; + unsigned long new_ksmpages_of_this_scanning; }; /** @@ -277,6 +280,20 @@ static unsigned int zero_checksum __read_mostly; /* Whether to merge empty (zeroed) pages with actual zero pages */ static bool ksm_use_zero_pages __read_mostly; +/* + * Work in auto-mode. + * Maximum number of multiplicative factor of pages_to_scan. + */ +static unsigned int ksm_max_scanning_factor = 32; + +/* + * Work in auto-mode. + * The multiplicative factor of pages_to_scan. + * Real pages to scan equals to the product of scanning_factor + * and pages_to_scan + */ +static unsigned int scanning_factor = 1; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -2031,6 +2048,8 @@ static void stable_tree_append(struct rmap_item *rmap_item, rmap_item->address |= STABLE_FLAG; hlist_add_head(&rmap_item->hlist, &stable_node->hlist); + ksm_scan.new_ksmpages_of_this_scanning++; + if (rmap_item->hlist.next) ksm_pages_sharing++; else @@ -2396,6 +2415,23 @@ static struct rmap_item *scan_get_next_rmap_item(struct page **page) return NULL; } +/* + * enhance_scanning_factor(): + * double the values of scanning_factor, but not more than + * ksm_max_scanning_factor. + */ +static inline void enhance_scanning_factor(void) +{ + scanning_factor = scanning_factor << 1; + if (scanning_factor > ksm_max_scanning_factor) + scanning_factor = ksm_max_scanning_factor; +} + +static inline void reset_scanning_factor(void) +{ + scanning_factor = 1; +} + /** * ksm_do_scan - the ksm scanner main worker function. * @scan_npages: number of pages we want to scan before we return. @@ -2432,10 +2468,31 @@ static int ksm_scan_thread(void *nothing) while (!kthread_should_stop()) { mutex_lock(&ksm_thread_mutex); wait_while_offlining(); - if (ksmd_should_run()) - ksm_do_scan(ksm_thread_pages_to_scan); + if (ksmd_should_run()) { + if (ksm_run & KSM_RUN_AUTO) + ksm_do_scan(ksm_thread_pages_to_scan * scanning_factor); + else + ksm_do_scan(ksm_thread_pages_to_scan); + } mutex_unlock(&ksm_thread_mutex); + /* + * If there are new ksm pages after scanning, then we + * can enhance scanning_factor to improve ksm_thread's + * pages_to_scan to speed up scanning. Otherwaise, we + * reset scanning_factor to be one, so that to recover + * the normal state because there is greater probability + * of getting no new KsmPages in the next scanning. + */ + if (ksm_run & KSM_RUN_AUTO) { + if (ksm_scan.new_ksmpages_of_this_scanning > 0) + enhance_scanning_factor(); + else + reset_scanning_factor(); + + ksm_scan.new_ksmpages_of_this_scanning = 0; + } + try_to_freeze(); if (ksmd_should_run()) { @@ -2952,6 +3009,32 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, } KSM_ATTR(run); +static ssize_t max_scanning_factor_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%u\n", ksm_max_scanning_factor); +} + +static ssize_t max_scanning_factor_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + unsigned int value; + int err; + + err = kstrtouint(buf, 10, &value); + if (err) + return -EINVAL; + + if (value < 1) + return -EINVAL; + + ksm_max_scanning_factor = value; + + return count; +} +KSM_ATTR(max_scanning_factor); + #ifdef CONFIG_NUMA static ssize_t merge_across_nodes_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) @@ -3162,6 +3245,7 @@ static struct attribute *ksm_attrs[] = { &sleep_millisecs_attr.attr, &pages_to_scan_attr.attr, &run_attr.attr, + &max_scanning_factor_attr.attr, &pages_shared_attr.attr, &pages_sharing_attr.attr, &pages_unshared_attr.attr, From patchwork Wed Aug 3 10:05:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12935350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE640C19F2A for ; Wed, 3 Aug 2022 10:06:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 752B28E0002; Wed, 3 Aug 2022 06:06:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 701136B0072; Wed, 3 Aug 2022 06:06:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F0E08E0002; Wed, 3 Aug 2022 06:06:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 4E5A36B0071 for ; Wed, 3 Aug 2022 06:06:13 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1738A1414AC for ; Wed, 3 Aug 2022 10:06:13 +0000 (UTC) X-FDA: 79757851026.25.86E2E29 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf06.hostedemail.com (Postfix) with ESMTP id D82AF18010F for ; Wed, 3 Aug 2022 10:05:51 +0000 (UTC) Received: by mail-pj1-f41.google.com with SMTP id t22so8218674pjy.1 for ; Wed, 03 Aug 2022 03:05:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=bANxufpLCUY+XWPiguyc6Xk+fdLhXPbcV4C2RPgI3FM=; b=CTgvu/TjNHKxvdz52lHLnsHWO3P5bEpln58ylmFson4kOhjVckhXarZUwPPy5FRDxZ Q0gr5DFAUCLSbgjz5aLllSNEp3WL7p5Of1F++OlrXMjVaPLhNqDfuOIez2sWrUilbs+s AymLnmzvsjgTu7EzeEts3SFZZ/VV71kIexvB+QdFaYuqj2BaK7Zk1SADeA2VMcfCQ+TI qN043yUVP5TVIwHSZVdvHmWMysAls8dn6s1GpYsMrX9xd7mbLRIJmD16zWyuoF6esrv1 9nbqR3ntLvsk0ZiT03CqdsDklDBBm7wggbihqhN7qgEmp/M3EXfwkuPRuZ8/3QFpi2hS +lQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=bANxufpLCUY+XWPiguyc6Xk+fdLhXPbcV4C2RPgI3FM=; b=QpBbBlWKwqSIZFWs+dzYPDw84ANDNw5X2+RzMeCyhN0qePIpIfeRgHuPpmiIREa/Dl S5VgyxCBFeFbYKTBnmJtEfZWDoL+PXihCPgzUN4FP1O/+AqDvTFgPbuC83Om1ZXBQ78F DZMPx7WfLMYqUsPkEO73EwHs65PhlJuLbKlEqPxQxFWJaQUejR+fdclipfPGpWQekuEX Mawv84qpiLplqU0qpC1V+x/WRz6HD0ZzOMZ58R3U3lk8M7LyUmBm+BJCbcnXj93rPY4X EIGzU31WASsXXnFOVUSneJENphfuJbiXCps48U7/fN8TcaM2MROxLlvbZEEPtqDYihTk oV3w== X-Gm-Message-State: ACgBeo1GPFNl2vh1IidTDuMCaYDleengG/c9+K/KYpA5ErrcxkgASUjZ k8VKkcQZI8TWTV0c2hs/qIc= X-Google-Smtp-Source: AA6agR6ej/DKKrC02wuJgBsr4b0OO0L2fAyintP2DPnrkyf8wCjR9D+JmfYcQs2tDO6/EciPZsS1Vg== X-Received: by 2002:a17:90b:2c13:b0:1f3:359a:c7d5 with SMTP id rv19-20020a17090b2c1300b001f3359ac7d5mr4162524pjb.72.1659521150810; Wed, 03 Aug 2022 03:05:50 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id t7-20020a170902e84700b0016e808dbe55sm1499003plg.96.2022.08.03.03.05.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Aug 2022 03:05:50 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.con.cn To: akpm@linux-foundation.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, willy@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu xin , CGEL Subject: [RFC PATCH 3/4] ksm: let ksmd work automatically with memory threshold Date: Wed, 3 Aug 2022 10:05:43 +0000 Message-Id: <20220803100543.1653550-1-xu.xin16@zte.con.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803100306.1653382-1-xu.xin16@zte.con.cn> References: <20220803100306.1653382-1-xu.xin16@zte.con.cn> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: m3r3gonqfggwngfw8bhooxiuasyy8zbs X-Rspamd-Queue-Id: D82AF18010F Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="CTgvu/Tj"; spf=temperror (imf06.hostedemail.com: error in processing during lookup of cgel.zte@gmail.com: DNS error) smtp.mailfrom=cgel.zte@gmail.com X-Rspamd-Server: rspam02 X-HE-Tag: 1659521151-478856 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin When memory is sufficient, merging pages to save memory is not very much needed, and it also inceases delays of COW for user application. So set a memory threshold, when free memory is lower than the threshold, ksmd will be triggered to compare and merge pages. And to avoid ping-pong effect due to the threshold, ksmd needs to try to merge pages until free memory is larger than (threshold + total_memory * 1/16). Before free memory is lower than the threshold, ksmd will still scan pages at a very low speed, to calculate their checksum but not to compare and merge pages. | | ----(Threshold + total_memory/16)-------- | | ------Threshold------ | | | |_____ksmd try to merge pages__| We also add a new sysfs klob auto_threshold_percent for user to be able to tune. Signed-off-by: xu xin Signed-off-by: CGEL --- mm/ksm.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 92 insertions(+), 3 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 8acc893e4d61..78819b56efec 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -294,6 +294,17 @@ static unsigned int ksm_max_scanning_factor = 32; */ static unsigned int scanning_factor = 1; +/* + * Work in auto-mode. + * Default ksm_auto_threshold_percent: 20, means 20%. When free memory + * is lower than total memory * ksm_auto_threshold_percent/100, ksmd will + * be triggered to compare and merge pages. + */ +unsigned int ksm_auto_threshold_percent = 20; + +/* Work in auto-mode. Whether trigger ksmd to compare and merge pages */ +static bool auto_triggered; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -2446,11 +2457,46 @@ static void ksm_do_scan(unsigned int scan_npages) rmap_item = scan_get_next_rmap_item(&page); if (!rmap_item) return; - cmp_and_merge_page(page, rmap_item); + if (ksm_run & KSM_RUN_AUTO && !auto_triggered) { + /* + * This should happens only when ksm_run is KSM_RUN_AUTO + * and free memory threshold still not reached. + * The reason to calculate it's checksum is to reduce the + * waiting time the rmap_item is added to unstable tree. + */ + rmap_item->oldchecksum = calc_checksum(page); + } else + cmp_and_merge_page(page, rmap_item); + put_page(page); } } +/* Work in auto mode, should reset auto_triggered ? */ +static bool should_untrigger_ksmd_to_merge(void) +{ + unsigned long total_ram_pages, free_pages; + + total_ram_pages = totalram_pages(); + free_pages = global_zone_page_state(NR_FREE_PAGES); + + return free_pages > (total_ram_pages * + ksm_auto_threshold_percent / 100) + + (total_ram_pages >> 4); +} + +/* Work in auto mode, should ksmd start to merge ? */ +static bool should_trigger_ksmd_to_merge(void) +{ + unsigned long total_ram_pages, free_pages; + + total_ram_pages = totalram_pages(); + free_pages = global_zone_page_state(NR_FREE_PAGES); + + return free_pages < (total_ram_pages * + ksm_auto_threshold_percent / 100); +} + static int ksmd_should_run(void) { if (!list_empty(&ksm_mm_head.mm_list)) @@ -2469,8 +2515,18 @@ static int ksm_scan_thread(void *nothing) mutex_lock(&ksm_thread_mutex); wait_while_offlining(); if (ksmd_should_run()) { - if (ksm_run & KSM_RUN_AUTO) - ksm_do_scan(ksm_thread_pages_to_scan * scanning_factor); + if (ksm_run & KSM_RUN_AUTO) { + /* + * If free memory is not lower than threshold, we only scan + * 5 pages and just calculate pages' checksum and not compare + * and merge them. Otherwise, do real scanning and merging as + * scanning-enhanced algorithm. + */ + if (!auto_triggered) + ksm_do_scan(5); + else + ksm_do_scan(ksm_thread_pages_to_scan * scanning_factor); + } else ksm_do_scan(ksm_thread_pages_to_scan); } @@ -2491,6 +2547,11 @@ static int ksm_scan_thread(void *nothing) reset_scanning_factor(); ksm_scan.new_ksmpages_of_this_scanning = 0; + + if (should_trigger_ksmd_to_merge()) + auto_triggered = true; + else if (should_untrigger_ksmd_to_merge()) + auto_triggered = false; } try_to_freeze(); @@ -3035,6 +3096,32 @@ static ssize_t max_scanning_factor_store(struct kobject *kobj, } KSM_ATTR(max_scanning_factor); +static ssize_t auto_threshold_percent_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%u\n", ksm_auto_threshold_percent); +} + +static ssize_t auto_threshold_percent_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + unsigned int value; + int err; + + err = kstrtouint(buf, 10, &value); + if (err) + return -EINVAL; + + if (value > 100) + return -EINVAL; + + ksm_auto_threshold_percent = value; + + return count; +} +KSM_ATTR(auto_threshold_percent); + #ifdef CONFIG_NUMA static ssize_t merge_across_nodes_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) @@ -3246,6 +3333,7 @@ static struct attribute *ksm_attrs[] = { &pages_to_scan_attr.attr, &run_attr.attr, &max_scanning_factor_attr.attr, + &auto_threshold_percent_attr.attr, &pages_shared_attr.attr, &pages_sharing_attr.attr, &pages_unshared_attr.attr, @@ -3277,6 +3365,7 @@ static int __init ksm_init(void) zero_checksum = calc_checksum(ZERO_PAGE(0)); /* Default to false for backwards compatibility */ ksm_use_zero_pages = false; + auto_triggered = false; err = ksm_slab_init(); if (err) From patchwork Wed Aug 3 10:05:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12935351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EC16C19F28 for ; Wed, 3 Aug 2022 10:06:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E51C8E0001; Wed, 3 Aug 2022 06:06:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 293FC6B0072; Wed, 3 Aug 2022 06:06:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 183208E0001; Wed, 3 Aug 2022 06:06:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 077DE6B0071 for ; Wed, 3 Aug 2022 06:06:25 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D6D011C6E7C for ; Wed, 3 Aug 2022 10:06:24 +0000 (UTC) X-FDA: 79757851488.24.00DB502 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf06.hostedemail.com (Postfix) with ESMTP id 8718118003E for ; Wed, 3 Aug 2022 10:06:05 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id gj1so1136148pjb.0 for ; Wed, 03 Aug 2022 03:06:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=N6470V84i8r/SrAbUOMPMzFLhAwImnUfo4s3+mmJyH4=; b=gIbeBO48HczBwJi8HhEhiZlbToMlHiOOoZJ/OiW7pomV3TCRTBIJFQek8ZQfpj8bdD 6Haf2rdTJfq8QgglHWwURPMsF80Fw/7k3DdMkbG8Ul8vDmYN68blcn53jsfywQg5AhEw rVZiDEMDNxTj5ZHYNli3kUiEUXEQzk6YQYyigG80fN7BU3SOgL8dwQxupJRkyohV7Hj3 /3ipKsYLv0Ccs0xzYfsu9YuYpJC/b2kwyR7oDx6/gXzPns/LzKz/64J0hR1ZEkf7ARAE IV/f9lRsZTgtCgxrOvsHw0fTO3SfZLEUokHTjt5RIypzKt6bGmoPYi6bGNIrgRZ6W7BH H2OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=N6470V84i8r/SrAbUOMPMzFLhAwImnUfo4s3+mmJyH4=; b=SseSl5r+lQI+cPoAfFoFhOVjXFjpFEaP1Ci4CXOqBcGCJDhEq3CUznS4+2oNQpjiRc 7W/kQ8qMzs/A0rqdvdmGffKgEiLd8bjbf7t599X6EdtjZb9/h718hSlKmpFr1trinABz eUQCsEA4hgpMhR/ZIS7FZO2UZAhTNOu9Lka2t5GcUJCmoWh4qw5ZSHvgtsqNWfXQOcyD 0ZqE44+D8tgyQ1iD+5THx5+bmGC6Oq2STRcP3UAcIMbWwoT7yfFqxJw88DJCKm47HLNH wMrQFLm64ykoDD7RCn0upBBOc+moXEavaigif9/tsjdqUgkX7Wv6VfqS2IhFpLBfic56 M+zg== X-Gm-Message-State: ACgBeo0+f9Ao3Qna5Cqhkfk6aUKUc4+QKVJ8J4EDdSnHnguvXdkSikK7 DcJklyOl/4kcAtraBwfD8Ag= X-Google-Smtp-Source: AA6agR6ATn+tw5xU/Xs1DoivKBqKZJJ2fvM7DM7gwnYkVkEDKR9V5dy4GqCzUk4ZFQmpDss/ft05zg== X-Received: by 2002:a17:90b:4d07:b0:1ef:521c:f051 with SMTP id mw7-20020a17090b4d0700b001ef521cf051mr4171718pjb.164.1659521164547; Wed, 03 Aug 2022 03:06:04 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id h3-20020a63df43000000b00416018b5bbbsm10809337pgj.76.2022.08.03.03.06.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 03 Aug 2022 03:06:04 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.con.cn To: akpm@linux-foundation.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, willy@infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu xin , CGEL Subject: [RFC PATCH 4/4] ksm: show ksmd status of auto mode Date: Wed, 3 Aug 2022 10:05:59 +0000 Message-Id: <20220803100559.1653604-1-xu.xin16@zte.con.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220803100306.1653382-1-xu.xin16@zte.con.cn> References: <20220803100306.1653382-1-xu.xin16@zte.con.cn> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: byisfftawrumq9ip5t9zpfhbkfjz1xop X-Rspamd-Queue-Id: 8718118003E Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=gIbeBO48; spf=temperror (imf06.hostedemail.com: error in processing during lookup of cgel.zte@gmail.com: DNS error) smtp.mailfrom=cgel.zte@gmail.com; dmarc=temperror reason="query timed out" header.from=gmail.com (policy=temperror) X-Rspamd-Server: rspam02 X-HE-Tag: 1659521165-63519 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin Add a sysfs interface of ksmd_status to show some details related with auto-mode. Signed-off-by: xu xin Signed-off-by: CGEL --- mm/ksm.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/mm/ksm.c b/mm/ksm.c index 78819b56efec..6417bc034a3d 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -3070,6 +3070,29 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, } KSM_ATTR(run); +static ssize_t ksmd_status_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + int len = 0; + + if (ksm_run & KSM_RUN_AUTO) { + len += sysfs_emit_at(buf, len, "mode: auto\n"); + len += sysfs_emit_at(buf, len, "auto_triggered %d\n", + auto_triggered); + len += sysfs_emit_at(buf, len, "scanning_factor: %u\n", + scanning_factor); + } else if (ksm_run & KSM_RUN_MERGE) + len += sysfs_emit_at(buf, len, "mode: on\n"); + else if (ksm_run & KSM_RUN_UNMERGE) + len += sysfs_emit_at(buf, len, "mode: unmerge\n"); + else + len += sysfs_emit_at(buf, len, "mode: off\n"); + + + return len; +} +KSM_ATTR_RO(ksmd_status); + static ssize_t max_scanning_factor_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -3332,6 +3355,7 @@ static struct attribute *ksm_attrs[] = { &sleep_millisecs_attr.attr, &pages_to_scan_attr.attr, &run_attr.attr, + &ksmd_status_attr.attr, &max_scanning_factor_attr.attr, &auto_threshold_percent_attr.attr, &pages_shared_attr.attr,