From patchwork Fri Aug 12 10:12:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12942103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A728FC25B0E for ; Fri, 12 Aug 2022 10:12:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0124F8E0005; Fri, 12 Aug 2022 06:12:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F03F98E0001; Fri, 12 Aug 2022 06:12:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCC1E8E0005; Fri, 12 Aug 2022 06:12:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D0C408E0001 for ; Fri, 12 Aug 2022 06:12:18 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9E937A1BD2 for ; Fri, 12 Aug 2022 10:12:18 +0000 (UTC) X-FDA: 79790525556.28.F26194C Received: from mail-pf1-f180.google.com (mail-pf1-f180.google.com [209.85.210.180]) by imf02.hostedemail.com (Postfix) with ESMTP id 3D93880192 for ; Fri, 12 Aug 2022 10:12:18 +0000 (UTC) Received: by mail-pf1-f180.google.com with SMTP id p125so564589pfp.2 for ; Fri, 12 Aug 2022 03:12:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=Zow/Qe/P2NEQDMvwyeUd/99vAJcANYj6amOMdI5Aoqg=; b=Mfg+p9sl9EKee4oeLXvUpcn98hONemFjE5MlOv9aqEVL98PWkiyFxu2JleR+qqciZf EAEc/jMEw1qYl9MZ4fuy81YcBMj39ao2+ky+HEGNAVDEmtevdQCgY9dUEi1473EOqVtO Ld0dhDUYIHPwcKxxqrND4fdBLqPHyTK0mXZd0EKWVmD7UxKYZULNdG13O+ZunaHZJZaQ +vbaIM/okQZMyB0YLly11rIXt+EAU675o9MvODyp94OA5XRbiYXvnj0fFhV5srQ9f/hk uEA1qpyRGt0etlJeYq/e0nz4yKpZnFsjUo3qQhmJguu7qoIAyGn2/eTZAAoBN2FWgCLO mRuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=Zow/Qe/P2NEQDMvwyeUd/99vAJcANYj6amOMdI5Aoqg=; b=tb6QUP7iSFz1+2HyW+2TqQEN492r+0rQc9HBAdhzpB3A51N+27B/ChV3dF2RjDyIcn XeV0joSG0qOm5XjiL2MhjfcFtFKE1hRduxAD5nMGZPIOICHw4wAXcc4U8me1vYyQOVln XauIqJ7LkXDHigazcnxIEYoDLMR8zr1Bb/SKkmXJggpm0au0BcoxT08AGJOYw9j3qYoz 9iY/LeoHj7WMj0keoncUIGe3fYyRvZJRIg6kA46IjnOv0Fg+q3L9Ax4x9DmqWveXrBah f2zX5ps9lu/HErPvmOtSNq9/wanTJvFzQ4mLT/RPXXi5BfIJSy4pd4g4gXGVHr64uRvr 9Vxw== X-Gm-Message-State: ACgBeo2eQST7xoitg/cHcfpK0ZcLlAsgLLobHgQqks5egg7i4sIhS2y/ EWa+Zs+5Kx/mgn+ZF2T8olI= X-Google-Smtp-Source: AA6agR5skhqPaePy5qtgLQ5EWyQuWhL4KUe6VyWBMJ1w9oMX6T2nxssa3zNoHs9qCSvhfdGWBnddZw== X-Received: by 2002:a65:4605:0:b0:41c:3d73:9385 with SMTP id v5-20020a654605000000b0041c3d739385mr2481969pgq.168.1660299137278; Fri, 12 Aug 2022 03:12:17 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id 196-20020a6218cd000000b0052b94e757ecsm1203269pfy.213.2022.08.12.03.12.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 03:12:17 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org, willy@infradead.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu.xin16@zte.com.cn, CGEL Subject: [PATCH v2 3/5] ksm: let ksmd auto-work with memory threshold Date: Fri, 12 Aug 2022 10:12:12 +0000 Message-Id: <20220812101212.41587-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220812101102.41422-1-xu.xin16@zte.com.cn> References: <20220812101102.41422-1-xu.xin16@zte.com.cn> MIME-Version: 1.0 ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Mfg+p9sl; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660299138; a=rsa-sha256; cv=none; b=Z1BRZGf1WA6v9ugcN/WhpFLFENTWQt5Ls3cIm229fZz5eMOJuWGHw/jvZeOWkywcazgVIQ MuBkk0UHqVqNNyH1z5SF0wKHazpnRwPHvHlcgezq0pe4CvMEpO1M7nSJXsYuxWoQuXyBMT vH05SDF3O16BA7fAAEuoDGw8lb+SvGk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660299138; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Zow/Qe/P2NEQDMvwyeUd/99vAJcANYj6amOMdI5Aoqg=; b=TUSybLKj8LHnGpHSfh/v3uO62+SpAw1pFSz9Lnh4i0gg4CwLo2wWwLVLbtZmA1xX2+2beN LMxfwu4uFJYz0WTQKFGjt3jCXqkmJBEbQWRCRohI2C3rNiMCGS9SLtmTENGZ3ucUM1g1eQ BeXab0U5Ig2KeuMQmBF8x4mvvHLv/eg= X-Rspam-User: Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Mfg+p9sl; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.210.180 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com X-Stat-Signature: 78ofcmrro7mhiu1k95updk4ixbinigf7 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 3D93880192 X-HE-Tag: 1660299138-907854 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin When memory is sufficient, merging pages to save memory is not very much needed, and it also inceases delays of COW for user application. So set a memory threshold, when free memory is lower than the threshold, ksmd will be triggered to compare and merge pages. And to avoid ping-pong effect due to the threshold, ksmd needs to try to merge pages until free memory is larger than (threshold + total_memory * 1/16). Before free memory is lower than the threshold, ksmd will still scan pages at a very low speed, to calculate their checksum but not to compare and merge pages. | | ----(Threshold + total_memory/16)-------- | | ------Threshold------ | | | |_____ksmd try to merge pages__| We also add a new sysfs klob auto_threshold_percent for user to be able to tune. Signed-off-by: xu xin Signed-off-by: CGEL --- mm/ksm.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 101 insertions(+), 2 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index f416f168a6da..c5fd4f520f4a 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -296,6 +296,17 @@ static unsigned int scanning_factor = INIT_SCANNING_FACTOR; #define DEFAULT_MAX_SCANNING_FACTOR 16 static unsigned int max_scanning_factor = DEFAULT_MAX_SCANNING_FACTOR; +/* + * Work in auto mode. + * Value: 0~100. Default 20 means "20%". When free memory is lower + * than this total memory * ksm_auto_threshold/100, auto_triggered + * will be set true. + */ +unsigned int ksm_auto_threshold = 20; + +/* Work in auto-mode. Whether trigger ksmd to compare and merge pages */ +static bool auto_triggered; + #ifdef CONFIG_NUMA /* Zeroed when merging across nodes is not allowed */ static unsigned int ksm_merge_across_nodes = 1; @@ -2431,11 +2442,61 @@ static void ksm_do_scan(unsigned int scan_npages) rmap_item = scan_get_next_rmap_item(&page); if (!rmap_item) return; - cmp_and_merge_page(page, rmap_item); + if (ksm_run & KSM_RUN_AUTO && !auto_triggered) { + /* + * This should happens only when ksm_run is KSM_RUN_AUTO + * and free memory threshold still not reached. + * The reason to calculate it's checksum is to reduce the + * waiting time the rmap_item is added to unstable tree. + */ + rmap_item->oldchecksum = calc_checksum(page); + } else + cmp_and_merge_page(page, rmap_item); + put_page(page); } } +#define RIGHT_SHIFT_FOUR_BIT 4 +/* Work in auto mode, should reset auto_triggered ? */ +static bool should_stop_ksmd_to_merge(void) +{ + unsigned long total_ram_pages, free_pages; + unsigned int threshold; + + total_ram_pages = totalram_pages(); + free_pages = global_zone_page_state(NR_FREE_PAGES); + threshold = READ_ONCE(ksm_auto_threshold); + + return free_pages > (total_ram_pages * threshold / 100) + + (total_ram_pages >> RIGHT_SHIFT_FOUR_BIT); +} + +/* Work in auto mode, should ksmd start to merge ? */ +static bool should_trigger_ksmd_to_merge(void) +{ + unsigned long total_ram_pages, free_pages; + unsigned int threshold; + + total_ram_pages = totalram_pages(); + free_pages = global_zone_page_state(NR_FREE_PAGES); + threshold = READ_ONCE(ksm_auto_threshold); + + return free_pages < (total_ram_pages * threshold / 100); +} + +static inline void trigger_ksmd_to_merge(void) +{ + if (!auto_triggered) + auto_triggered = true; +} + +static inline void stop_ksmd_to_merge(void) +{ + if (auto_triggered) + auto_triggered = false; +} + static int ksmd_should_run(void) { if (!list_empty(&ksm_mm_head.mm_list)) @@ -2478,6 +2539,8 @@ static unsigned int scan_enhanced_algorithm(unsigned int current_factor) return next_factor; } +#define SLOW_SCAN_PAGES 5 /* Used when ksmd is not triggered to merge*/ + static int ksm_scan_thread(void *nothing) { unsigned int sleep_ms; @@ -2490,7 +2553,10 @@ static int ksm_scan_thread(void *nothing) wait_while_offlining(); if (ksmd_should_run()) { if (ksm_run & KSM_RUN_AUTO) { - ksm_do_scan(ksm_thread_pages_to_scan * scanning_factor); + if (!auto_triggered) + ksm_do_scan(SLOW_SCAN_PAGES); + else + ksm_do_scan(ksm_thread_pages_to_scan * scanning_factor); scanning_factor = scan_enhanced_algorithm(scanning_factor); /* @@ -2498,6 +2564,11 @@ static int ksm_scan_thread(void *nothing) * updating scanning_factor by scan_enhanced_algorithm. */ ksm_scan.new_ksmpages = 0; + + if (should_trigger_ksmd_to_merge()) + trigger_ksmd_to_merge(); + else if (should_stop_ksmd_to_merge()) + stop_ksmd_to_merge(); } else ksm_do_scan(ksm_thread_pages_to_scan); } @@ -3047,6 +3118,32 @@ static ssize_t run_store(struct kobject *kobj, struct kobj_attribute *attr, } KSM_ATTR(run); +static ssize_t auto_threshold_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + return sysfs_emit(buf, "%u\n", ksm_auto_threshold); +} + +static ssize_t auto_threshold_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t count) +{ + unsigned int value; + int err; + + err = kstrtouint(buf, 10, &value); + if (err) + return -EINVAL; + + if (value > 100) + return -EINVAL; + + ksm_auto_threshold = value; + + return count; +} +KSM_ATTR(auto_threshold); + #ifdef CONFIG_NUMA static ssize_t merge_across_nodes_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) @@ -3258,6 +3355,7 @@ static struct attribute *ksm_attrs[] = { &pages_to_scan_attr.attr, &max_scanning_factor_attr.attr, &run_attr.attr, + &auto_threshold_attr.attr, &pages_shared_attr.attr, &pages_sharing_attr.attr, &pages_unshared_attr.attr, @@ -3289,6 +3387,7 @@ static int __init ksm_init(void) zero_checksum = calc_checksum(ZERO_PAGE(0)); /* Default to false for backwards compatibility */ ksm_use_zero_pages = false; + auto_triggered = false; err = ksm_slab_init(); if (err)