From patchwork Fri Jun 16 19:17:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13283224 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB5D0EB64D7 for ; Fri, 16 Jun 2023 19:18:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7939E8E0005; Fri, 16 Jun 2023 15:17:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6821C6B007B; Fri, 16 Jun 2023 15:17:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4D1A98E0005; Fri, 16 Jun 2023 15:17:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3B5756B0078 for ; Fri, 16 Jun 2023 15:17:55 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 1548A120DA4 for ; Fri, 16 Jun 2023 19:17:55 +0000 (UTC) X-FDA: 80909570910.08.00D1D86 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf11.hostedemail.com (Postfix) with ESMTP id 3D62340024 for ; Fri, 16 Jun 2023 19:17:52 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="paP/cMrr"; spf=pass (imf11.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686943073; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=95UoFLgZ6SxoqX8pXjdKD3hamOYJi8AF32Yw0fsYzB8=; b=Yl355iZNH5CC9h9NlO+wI2v/Dj9zfvxi0YMsrSA+kQZaTYePJU1A37YOy2Bt0d4wQX8sce byyKyszE13+PZJ5yymwVof8hPuKyNt5odMMMPMrhJ2adzNwJh19TaJ+7j0OkMxB8ygtjSQ sHfLUe6ewWZsXkxw5Mbm/eFxRcQMLO4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686943073; a=rsa-sha256; cv=none; b=5k7pbGI2RhK6S8wS6+AQHufjKTSxA/2aYjExleLyT5hfs2ER3UHnlXwAftEzy9izK8ofLw QWBIWqPGCl+DyYAtshAM1XJazC03QpiQU8/ji82ey1LVOC4OoZt/9vX745R5nkE9RN1IqB kIwg75WOJzzP1+1xpgHSMMBYZMn6gQQ= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="paP/cMrr"; spf=pass (imf11.hostedemail.com: domain of sj@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4551563CEC; Fri, 16 Jun 2023 19:17:52 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 27FFEC433CB; Fri, 16 Jun 2023 19:17:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686943071; bh=f3HOe9/kZR2u+JAC7ZFka5WaxvMhj+FaHkWQpsoLpnE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=paP/cMrrN2YC1qI+2IOL6lzfXW+Cgu9GTDAFTYdWea8Vi9SeJGajGut/HE8gpTI1G YPSdaHIqP/xEAjnvjXDzgDU7D82IMUJGXx8HhmVYuSXRfm7bP3Ec1KgfFqvXSgR1QJ NNMxIA79qpBl4DsDGX6d0dF5bWvxj+IjySwODmHLwaJTKDIOY4j/QV2/4SSNyUGrii 0FxdXjaDU/4yVD2D8dQ8cy8Lvw0xpXDuDSlxj4MUFEzG+fcnEI5J2GsaueAGO24eRA SlZ4VI+1u4Dfkw+hjNHxjGUfSwNalwcnN6/BsiETA6RsnAPLVcYAoGp0kKpCY69N7J c3HDybrTKMGmA== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , Jonathan Corbet , damon@lists.linux.dev, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/7] Docs/admin-guide/mm/damon/usage: link design document for DAMOS Date: Fri, 16 Jun 2023 19:17:40 +0000 Message-Id: <20230616191742.87531-6-sj@kernel.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230616191742.87531-1-sj@kernel.org> References: <20230616191742.87531-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3D62340024 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: mxm8qgqn6dyrm9fe9qps9jgdh6zhkyax X-HE-Tag: 1686943072-843963 X-HE-Meta: U2FsdGVkX19577MfO8kRu4vJhtnzA08Ero0JNDHOF/6zH1+aiLahhqPjns3Pek0TKbjcUnEQZj3+3zVpCGje9U9hiNlGG8Kpv8D8vDuZaJr8V/nRVeZ7dZiNvrE9z2mgiNCUG1r5e0618TKh4EtXKqP5PqZjIQTGCgGBSf5Q7UH5EaZgYcJdrh7jv/LuRrAwKMMNYkgRzlTBi1pL6/QlT++SrKtgpDqmhEpiMr/dLXjnGL2J/9Lc1zN0/AA8nAKPgSSggTGOl9wSHHmquIvqFd71TRh3CihAozSF7r2LWYZM6J6OKf/CrIrrEFZbZzm2iom/yatwKrhliYVTchasqL/2LnvEuHOkdmzeK3APFNy7wdAC4J8czLxN4rDRBO9owxxcRQ71WRFhRjDNen0G0rFRfb12K6DecTTtlt5HyHFrqgTElt1gYLQ3TVsDYrHV5Stxp8FYTtD2eiRXjfxoHZzjvJvx3H6N0sgnhgO34I4DrTYHYH3W8++uMLiEUcAThAn63sbl5Z+W0TS0HWNhZKhkmIEsbxcnstTr4eE4h1KS6kIRFXiB8ntw+3irXM/ExKB4WmBoqjkPRsYpTHK/cUubyYi8LYfnjx3wMo+DShmqyG0tR54eH4oA6qPB5KsYgt/f0sJnJB5ii2PBFFwa/ZT1LjX13IpD81EBF0HqHKCrdPEcUXsb0XuKxabmD/F/RwQqsxCcQiQeiAU6JNjTTr3ZVEjKkFpg6+Ks0wuzh0ustdNIJQClsMrvVIvy+beTV0GGAyexxuflbZo+uH+gPbhYVaXslOuwhXnXMj+5sGAQA7XbBAXMHD7O+cJpbmGd/Yrv9VtuMqykjXeYkx0J1Jwp03c9oPc9+V223IinVwjHA3/q2JN0LDLXLlAOMM7340/0efe0d3SEYTD8XDx6Ci/spC2fiGYSjg9tY0q3p5RhQNdDHajKS9ggTJIQwBkPdjUKWiV7DkzDR4fj+vf Qz47x1Gv nIds6bdxZg38RheuoXij0BYnmZM9xtAUpnmb9CER9j4WPdug4h2YH0sUeh5KCgQMqqLGlvN+Mkz52lPMjHG2p2iDo2cqbd2ckaMb+UXc38u1p2wguehsodO4CtzCZMe8upoAshOefBhF6GT6WIsAmK+WLxtkZh//FmRtlCO17MmhUw7GbpUBYT0TfdJdmknUiGLECLt1p2fkCeMTwHAbmPgfQTTBoEyK9KaedqBioPrve3f0sUoO3iQZrer/lMN4f3xsyIG+qu+2Zr9sPYTRKW49NdBzFrKZUydQzQdQhBnXmB+AN1vTp7Ub3D4P2ywLL01+WiZVd7zMa53GfRcqZD9FLOA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000018, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The background and concept of DAMOS is redundantly documented, in the design document and the usage document. Replace the duplicated ones in usage document with links to the design document. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 104 +++++++------------ Documentation/mm/damon/design.rst | 14 +++ 2 files changed, 51 insertions(+), 67 deletions(-) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst index 82a79838a47d..ea6a5dc8930e 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -257,12 +257,9 @@ be equal or smaller than ``start`` of directory ``N+1``. contexts//schemes/ --------------------- -For usual DAMON-based data access aware memory management optimizations, users -would normally want the system to apply a memory management action to a memory -region of a specific access pattern. DAMON receives such formalized operation -schemes from the user and applies those to the target memory regions. Users -can get and set the schemes by reading from and writing to files under this -directory. +The directory for DAMON-based Operation Schemes (:ref:`DAMOS +`). Users can get and set the schemes by reading from and +writing to files under this directory. In the beginning, this directory has only one file, ``nr_schemes``. Writing a number (``N``) to the file creates the number of child directories named ``0`` @@ -275,9 +272,9 @@ In each scheme directory, five directories (``access_pattern``, ``quotas``, ``watermarks``, ``filters``, ``stats``, and ``tried_regions``) and one file (``action``) exist. -The ``action`` file is for setting and getting what action you want to apply to -memory regions having specific access pattern of the interest. The keywords -that can be written to and read from the file and their meaning are as below. +The ``action`` file is for setting and getting the scheme's :ref:`action +`. The keywords that can be written to and read +from the file and their meaning are as below. Note that support of each action depends on the running DAMON operations set :ref:`implementation `. @@ -302,10 +299,8 @@ Note that support of each action depends on the running DAMON operations set schemes//access_pattern/ --------------------------- -The target access pattern of each DAMON-based operation scheme is constructed -with three ranges including the size of the region in bytes, number of -monitored accesses per aggregate interval, and number of aggregated intervals -for the age of the region. +The directory for the target access :ref:`pattern +` of the given DAMON-based operation scheme. Under the ``access_pattern`` directory, three directories (``sz``, ``nr_accesses``, and ``age``) each having two files (``min`` and ``max``) @@ -316,18 +311,8 @@ to and reading from the ``min`` and ``max`` files under ``sz``, schemes//quotas/ ------------------- -Optimal ``target access pattern`` for each ``action`` is workload dependent, so -not easy to find. Worse yet, setting a scheme of some action too aggressive -can cause severe overhead. To avoid such overhead, users can limit time and -size quota for each scheme. In detail, users can ask DAMON to try to use only -up to specific time (``time quota``) for applying the action, and to apply the -action to only up to specific amount (``size quota``) of memory regions having -the target access pattern within a given time interval (``reset interval``). - -When the quota limit is expected to be exceeded, DAMON prioritizes found memory -regions of the ``target access pattern`` based on their size, access frequency, -and age. For personalized prioritization, users can set the weights for the -three properties. +The directory for the :ref:`quotas ` of the given +DAMON-based operation scheme. Under ``quotas`` directory, three files (``ms``, ``bytes``, ``reset_interval_ms``) and one directory (``weights``) having three files @@ -335,23 +320,20 @@ Under ``quotas`` directory, three files (``ms``, ``bytes``, You can set the ``time quota`` in milliseconds, ``size quota`` in bytes, and ``reset interval`` in milliseconds by writing the values to the three files, -respectively. You can also set the prioritization weights for size, access -frequency, and age in per-thousand unit by writing the values to the three -files under the ``weights`` directory. +respectively. You can also set the :ref:`prioritization weights +` for size, access frequency, and age +in per-thousand unit by writing the values to the three files under the +``weights`` directory. schemes//watermarks/ ----------------------- -To allow easy activation and deactivation of each scheme based on system -status, DAMON provides a feature called watermarks. The feature receives five -values called ``metric``, ``interval``, ``high``, ``mid``, and ``low``. The -``metric`` is the system metric such as free memory ratio that can be measured. -If the metric value of the system is higher than the value in ``high`` or lower -than ``low`` at the memoent, the scheme is deactivated. If the value is lower -than ``mid``, the scheme is activated. +The directory for the :ref:`watermarks ` of the +given DAMON-based operation scheme. Under the watermarks directory, five files (``metric``, ``interval_us``, -``high``, ``mid``, and ``low``) for setting each value exist. You can set and +``high``, ``mid``, and ``low``) for setting the metric, the time interval +between check of the metric, and the three watermarks exist. You can set and get the five values by writing to the files, respectively. Keywords and meanings of those that can be written to the ``metric`` file are @@ -365,12 +347,8 @@ The ``interval`` should written in microseconds unit. schemes//filters/ -------------------- -Users could know something more than the kernel for specific types of memory. -In the case, users could do their own management for the memory and hence -doesn't want DAMOS bothers that. Users could limit DAMOS by setting the access -pattern of the scheme and/or the monitoring regions for the purpose, but that -can be inefficient in some cases. In such cases, users could set non-access -pattern driven filters using files in this directory. +The directory for the :ref:`filters ` of the given +DAMON-based operation scheme. In the beginning, this directory has only one file, ``nr_filters``. Writing a number (``N``) to the file creates the number of child directories named ``0`` @@ -597,15 +575,10 @@ update. Schemes ------- -For usual DAMON-based data access aware memory management optimizations, users -would simply want the system to apply a memory management action to a memory -region of a specific access pattern. DAMON receives such formalized operation -schemes from the user and applies those to the target processes. - -Users can get and set the schemes by reading from and writing to ``schemes`` -debugfs file. Reading the file also shows the statistics of each scheme. To -the file, each of the schemes should be represented in each line in below -form:: +Users can get and set the DAMON-based operation :ref:`schemes +` by reading from and writing to ``schemes`` debugfs file. +Reading the file also shows the statistics of each scheme. To the file, each +of the schemes should be represented in each line in below form:: @@ -614,8 +587,9 @@ You can disable schemes by simply writing an empty string to the file. Target Access Pattern ~~~~~~~~~~~~~~~~~~~~~ -The ```` is constructed with three ranges in below -form:: +The target access :ref:`pattern ` of the +scheme. The ```` is constructed with three ranges in +below form:: min-size max-size min-acc max-acc min-age max-age @@ -628,9 +602,9 @@ closed interval. Action ~~~~~~ -The ```` is a predefined integer for memory management actions, which -DAMON will apply to the regions having the target access pattern. The -supported numbers and their meanings are as below. +The ```` is a predefined integer for memory management :ref:`actions +`. The supported numbers and their meanings are as +below. - 0: Call ``madvise()`` for the region with ``MADV_WILLNEED``. Ignored if ``target`` is ``paddr``. @@ -646,10 +620,8 @@ supported numbers and their meanings are as below. Quota ~~~~~ -Optimal ``target access pattern`` for each ``action`` is workload dependent, so -not easy to find. Worse yet, setting a scheme of some action too aggressive -can cause severe overhead. To avoid such overhead, users can limit time and -size quota for the scheme via the ```` in below form:: +Users can set the :ref:`quotas ` of the given scheme +via the ```` in below form:: @@ -659,19 +631,17 @@ the action to memory regions of the ``target access pattern`` within the ```` bytes of memory regions within the ````. Setting both ```` and ```` zero disables the quota limits. -When the quota limit is expected to be exceeded, DAMON prioritizes found memory -regions of the ``target access pattern`` based on their size, access frequency, -and age. For personalized prioritization, users can set the weights for the -three properties in ```` in below form:: +For the :ref:`prioritization `, users +can set the weights for the three properties in ```` in below +form:: Watermarks ~~~~~~~~~~ -Some schemes would need to run based on current value of the system's specific -metrics like free memory ratio. For such cases, users can specify watermarks -for the condition.:: +Users can specify :ref:`watermarks ` of the +given scheme via ```` in below form:: diff --git a/Documentation/mm/damon/design.rst b/Documentation/mm/damon/design.rst index a98af99bb705..4bfdf1d30c4a 100644 --- a/Documentation/mm/damon/design.rst +++ b/Documentation/mm/damon/design.rst @@ -218,6 +218,8 @@ abstracted monitoring target memory area only for each of a user-specified time interval (``update interval``). +.. _damon_design_damos: + Operation Schemes ----------------- @@ -255,6 +257,8 @@ the access pattern of interest, and applies the user-desired operation actions to the regions as soon as found. +.. _damon_design_damos_action: + Operation Action ~~~~~~~~~~~~~~~~ @@ -277,6 +281,8 @@ characteristics. Hence, DAMOS resets the age of regions when an action is applied to those. +.. _damon_design_damos_access_pattern: + Target Access Pattern ~~~~~~~~~~~~~~~~~~~~~ @@ -288,6 +294,8 @@ region's three properties are in the ranges, DAMOS classifies it as one of the regions that the scheme is having an interest in. +.. _damon_design_damos_quotas: + Quotas ~~~~~~ @@ -305,6 +313,8 @@ can use for applying the action, and/or a maximum bytes of memory regions that the action can be applied within a user-specified time duration. +.. _damon_design_damos_quotas_prioritization: + Prioritization ^^^^^^^^^^^^^^ @@ -330,6 +340,8 @@ the weight will be respected are up to the underlying prioritization mechanism implementation. +.. _damon_design_damos_watermarks: + Watermarks ~~~~~~~~~~ @@ -350,6 +362,8 @@ is also deactivated. In this case, the DAMON worker thread only periodically checks the watermarks and therefore incurs nearly zero overhead. +.. _damon_design_damos_filters: + Filters ~~~~~~~