From patchwork Fri Aug 12 10:11:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: CGEL X-Patchwork-Id: 12942080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F6DAC25B0E for ; Fri, 12 Aug 2022 10:11:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F38C8E0002; Fri, 12 Aug 2022 06:11:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4A3228E0001; Fri, 12 Aug 2022 06:11:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36AF68E0002; Fri, 12 Aug 2022 06:11:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 271E38E0001 for ; Fri, 12 Aug 2022 06:11:10 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E52B91605A1 for ; Fri, 12 Aug 2022 10:11:09 +0000 (UTC) X-FDA: 79790522658.06.ADB418D Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf07.hostedemail.com (Postfix) with ESMTP id 966384018F for ; Fri, 12 Aug 2022 10:11:09 +0000 (UTC) Received: by mail-pj1-f53.google.com with SMTP id 15-20020a17090a098f00b001f305b453feso7929823pjo.1 for ; Fri, 12 Aug 2022 03:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc; bh=aYgtpe2qKchqYAVeaGiRPsk+O8CEAY4+LBGWObOMEO0=; b=Xuiun91BGib/zGfk8+xs/x42MxJnyZVMMGYUsQSaU70f3rzjFyRUvo3ep6P2yD3O8M erT+nwafT8sNELNBpih76eDx+wI5kRBqmDpGtu7p4yw5zUzEalosFQbHocoyTm3HELTc jP0gSfo+d/WOTxfoUZlDdft2/BkiJICBz0+0EemLlQg5Moh3nHW0nvUok6+CoOn0D1cV 6k+sUe5Usd91nnc2ogQKcE8JvTbu1w/W+KtPbEkB9bsQTPPnXS/foTOAn9Sbr5WXcmei P9ol9ggHPExmzkcJq8Hl6HjVK87NPEUYmZQPoxncdoXmEMKlNyMjVDt2QreEwsZzkUJY vsUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc; bh=aYgtpe2qKchqYAVeaGiRPsk+O8CEAY4+LBGWObOMEO0=; b=Sb6/zseZ4nWOI2h8MDTJtDZH6pEp98xax0onIpAzFYL+f7+SwySK+3SuQgpxrPJBJU ZRyFWopLPCAToUrs9hO7CoKt5PtFPYTPlV44spg2j/Wm/PWYwlx7qOnDvIxM33QBsB05 BIXeMOVICugzBTRQ4c752kbidJI8souf5kGhvRPxY9ZK96jNdc5cPXRtO7kRP6MNayI4 V1d2V7TgUTGdXoNXu0Wp5Ujc6Cwf+Ab7vGaXPISRGdMwCbMZQrHTESVhQFoaN5oWBx05 KK1VF0lF6RPhWScSnySLbzJPGzs4TUTXhIhhOxC6eF1j2S5luDS23Or19lELAP/sXvFv 4D+w== X-Gm-Message-State: ACgBeo3Xal5IdazTni4LWymfKfXLC8f3h3rb2uuTHcGWfejyibwnEQeh qnKmi53bEkLAE3o1cXFoKaQ= X-Google-Smtp-Source: AA6agR4Q7sWDYiFEdIaeHOgye9LDGjiiTXSD+0nlr16pHDJZFMeVUlzOS/Yui26vYMrKOvjTRkhPpg== X-Received: by 2002:a17:902:e54c:b0:171:4b29:d1e with SMTP id n12-20020a170902e54c00b001714b290d1emr3327280plf.39.1660299068440; Fri, 12 Aug 2022 03:11:08 -0700 (PDT) Received: from localhost.localdomain ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id 20-20020a621814000000b0052a297324cbsm1215561pfy.41.2022.08.12.03.11.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Aug 2022 03:11:07 -0700 (PDT) From: cgel.zte@gmail.com X-Google-Original-From: xu.xin16@zte.com.cn To: akpm@linux-foundation.org, willy@infradead.org Cc: hughd@google.com, izik.eidus@ravellosystems.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, xu.xin16@zte.com.cn Subject: [PATCH v2 0/5] propose auto-run mode of ksm and its tests Date: Fri, 12 Aug 2022 10:11:02 +0000 Message-Id: <20220812101102.41422-1-xu.xin16@zte.com.cn> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1660299069; a=rsa-sha256; cv=none; b=FnhC6LLEwheQltTjbWeKmfPlFlNcX3wys7WtmXhG6BEHDplX5aeDpu3uHdJFl/33TtwpQe LVYZ3DsL5BeTPeHyhyVVYySrckJm350M2jcwhGwjwIDb/RtO0b3E6kDuI+NgamvGAymQUx Xah+8tBPQhQZkSk/8iB8HmcBT+QU0fc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Xuiun91B; spf=pass (imf07.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1660299069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=aYgtpe2qKchqYAVeaGiRPsk+O8CEAY4+LBGWObOMEO0=; b=R6gIqM/KwKEjwW4n+YaYeOxNP9ELzLkDXDGUYYWgQaQ3mBEI0nmdPhbwf5MMOnZWFB75Re Pb4pym/tUcndtyLOwB0LeMQKMOpv0X64r6LNvAgH6UHXEiEnTOBvkpytXLeynJFF721va0 A7dxAJd/UkzM3G0YLKvkcsg4jhDvrko= X-Stat-Signature: 3m86o9i71e4sx1pfwamh4sti3mwb3aib X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 966384018F Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Xuiun91B; spf=pass (imf07.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-HE-Tag: 1660299069-146468 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: xu xin The following patch series bring a NEW running state "auto mode". In traditional KSM, whether ksmd works scanning and merging or not depends on the sysfs klob ksm_run. Most of time, letting ksmd run is not very much needed, for example, when memory is sufficient, because it increases the delays of COW for user applications and also consume extra cpu resource. Besides, the fixed pages_to_scan is not always good. When there are a lot of same pages, the default pages_to_scan makes ksmd so slow to merge them. The four patches try to optimize the above two points with a auto mode. It's a lightweight optimization to KSM, not like uksm or pksm and so on. The tests show: The convergence time of merging time of auto-mode reduces can low to HALF of normal-mode. The cpu consumption of auto-mode also get less than normal-mode. There are two type of TEST which have different shape of samepage areas. One is full-samepages vm areas, and the other is sparse-samepage areas. Both of the madvise area'size are 300MB. Here is test log: ****************** TEST 1 ****************** Now the shape of memory area is full-samepages: [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx] Start to test normal-run ksmd... ksm current cpu total slice: 15274 ksm normal-run's merging time: 45.596018 seconds ksm current cpu total slice: 16370 Start to test auto-run ksmd... ksm auto-run's merging time: 17.599391 seconds ksm current cpu total slice: 16488 ****************** TEST 2 ****************** Now the shape of memory area is sparse-samepages: [xx] [xx] [xx] Start to test normal-run ksmd... ksm current cpu total slice: 16532 ksm normal-run's merging time: 21.625451 seconds ksm current cpu total slice: 16833 Start to test auto-run ksmd... ksm auto-run's merging time: 17.684379 seconds ksm current cpu total slice: 16950 Change log ========== for v2: 1. We adjust the scan-enhanced algorithm, abstract its implementation into a single function, which is clearer and smoother. 2. Add tests of auto-mode of ksm to verify the effectiveness. xu xin (5): ksm: add a auto-run mode of ksm ksm: implement scan-enhanced algorithm ksm: let ksmd auto-work with memory threshold ksm: show ksmd status for auto mode ksm: add tests of ksm auto mode mm/ksm.c | 250 ++++++++++++++++++- tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile | 1 + tools/testing/selftests/vm/test-ksm-auto.c | 273 +++++++++++++++++++++ 4 files changed, 519 insertions(+), 6 deletions(-) create mode 100644 tools/testing/selftests/vm/test-ksm-auto.c