From patchwork Tue Aug 18 07:25:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11720171 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 588C6138C for ; Tue, 18 Aug 2020 07:29:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 18B7D2063A for ; Tue, 18 Aug 2020 07:29:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="bwJIpqHJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 18B7D2063A Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 517738D000A; Tue, 18 Aug 2020 03:29:02 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4EE418D0005; Tue, 18 Aug 2020 03:29:02 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3DBB98D000A; Tue, 18 Aug 2020 03:29:02 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id 27FE38D0005 for ; Tue, 18 Aug 2020 03:29:02 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DA0211DE8 for ; Tue, 18 Aug 2020 07:29:01 +0000 (UTC) X-FDA: 77162862882.30.fold34_12126d42701d Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id A1408180B3C8B for ; Tue, 18 Aug 2020 07:29:01 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,prvs=492513fb0=sjpark@amazon.com,,RULES_HIT:30003:30025:30051:30054:30064:30070:30071:30090,0,RBL:207.171.190.10:@amazon.com:.lbl8.mailshell.net-62.18.0.100 66.10.201.10;04ygujadhiq8sqnubcsaqdaicfc8uyc586wgmzejucc4mjxbbmrzmf9n5g7o8ak.ixhrec4mq6zzjmftdju8msng6ht5hodk9e9dsxbetmgwqkgr4g9qh9bpnsz99rr.h-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: fold34_12126d42701d X-Filterd-Recvd-Size: 12397 Received: from smtp-fw-33001.amazon.com (smtp-fw-33001.amazon.com [207.171.190.10]) by imf15.hostedemail.com (Postfix) with ESMTP for ; Tue, 18 Aug 2020 07:29:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1597735742; x=1629271742; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qGKT1F1qVkIMFajZCvJm8BTClvxWeUPf3j13t8snc08=; b=bwJIpqHJcHcZbfWUnNeEqZAnF8uxWIKZusrItqpv+LDodF1W3uEvb6Hd KYbbRe7s554DyWbWupRQM3U7AnOygQBuGxjAjCYH20F8bb8dRZvoKGJUJ YZGvRi/KHth7ZBG6XIYEd0LUlEK0iOnLO1eh+lpVdIT0cDxZO0gSq1EX1 0=; X-IronPort-AV: E=Sophos;i="5.76,326,1592870400"; d="scan'208";a="67576751" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2a-53356bf6.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 18 Aug 2020 07:28:56 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2a-53356bf6.us-west-2.amazon.com (Postfix) with ESMTPS id 35C96A1D50; Tue, 18 Aug 2020 07:28:53 +0000 (UTC) Received: from EX13D31EUA001.ant.amazon.com (10.43.165.15) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 18 Aug 2020 07:28:52 +0000 Received: from u3f2cd687b01c55.ant.amazon.com (10.43.162.73) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 18 Aug 2020 07:28:35 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v7 10/10] Docs/DAMON: Document physical memory monitoring support Date: Tue, 18 Aug 2020 09:25:01 +0200 Message-ID: <20200818072501.30396-11-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200818072501.30396-1-sjpark@amazon.com> References: <20200818072501.30396-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.162.73] X-ClientProxiedBy: EX13D38UWB002.ant.amazon.com (10.43.161.171) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Rspamd-Queue-Id: A1408180B3C8B X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates the DAMON documents for the physical memory monitoring support. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/usage.rst | 42 ++++++++++++++++---- Documentation/vm/damon/design.rst | 29 +++++++++----- Documentation/vm/damon/faq.rst | 5 +-- 3 files changed, 54 insertions(+), 22 deletions(-) diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst index cf0d44ce0ac9..3e2f1519c96a 100644 --- a/Documentation/admin-guide/mm/damon/usage.rst +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -10,15 +10,16 @@ DAMON provides below three interfaces for different users. This is for privileged people such as system administrators who want a just-working human-friendly interface. Using this, users can use the DAMON’s major features in a human-friendly way. It may not be highly tuned for - special cases, though. It supports only virtual address spaces monitoring. + special cases, though. It supports both virtual and physical address spaces + monitoring. - *debugfs interface.* This is for privileged user space programmers who want more optimized use of DAMON. Using this, users can use DAMON’s major features by reading from and writing to special debugfs files. Therefore, you can write and use your personalized DAMON debugfs wrapper programs that reads/writes the debugfs files instead of you. The DAMON user space tool is also a reference - implementation of such programs. It supports only virtual address spaces - monitoring. + implementation of such programs. It supports both virtual and physical + address spaces monitoring. - *Kernel Space Programming Interface.* This is for kernel space programmers. Using this, users can utilize every feature of DAMON most flexibly and efficiently by writing kernel space @@ -49,8 +50,10 @@ Recording Data Access Pattern The ``record`` subcommand records the data access pattern of target workloads in a file (``./damon.data`` by default). You can specify the target with 1) -the command for execution of the monitoring target process, or 2) pid of -running target process. Below example shows a command target usage:: +the command for execution of the monitoring target process, 2) pid of running +target process, or 3) the special keyword, 'paddr', if you want to monitor the +system's physical memory address space. Below example shows a command target +usage:: # cd /tools/damon/ # damo record "sleep 5" @@ -61,6 +64,15 @@ of the process. Below example shows a pid target usage:: # sleep 5 & # damo record `pidof sleep` +Finally, below example shows the use of the special keyword, 'paddr':: + + # damo record paddr + +In this case, the monitoring target regions defaults to the largetst 'System +RAM' region specified in '/proc/iomem' file. Note that the initial monitoring +target region is maintained rather than dynamically updated like the virtual +memory address spaces monitoring case. + The location of the recorded file can be explicitly set using ``-o`` option. You can further tune this by setting the monitoring attributes. To know about the monitoring attributes in detail, please refer to the @@ -319,20 +331,34 @@ check it again:: # cat target_ids 42 4242 +Users can also monitor the physical memory address space of the system by +writing a special keyword, "``paddr\n``" to the file. Because physical address +space monitoring doesn't support multiple targets, reading the file will show a +fake value, ``42``, as below:: + + # cd /damon + # echo paddr > target_ids + # cat target_ids + 42 + Note that setting the target ids doesn't start the monitoring. Initial Monitoring Target Regions --------------------------------- -In case of the debugfs based monitoring, DAMON automatically sets and updates -the monitoring target regions so that entire memory mappings of target -processes can be covered. However, users might want to limit the monitoring +In case of the virtual address space monitoring, DAMON automatically sets and +updates the monitoring target regions so that entire memory mappings of target +processes can be covered. However, users might want to limit the monitoring region to specific address ranges, such as the heap, the stack, or specific file-mapped area. Or, some users might know the initial access pattern of their workloads and therefore want to set optimal initial regions for the 'adaptive regions adjustment'. +In contrast, DAMON do not automatically sets and updates the monitoring target +regions in case of physical memory monitoring. Therefore, users should set the +monitoring target regions by themselves. + In such cases, users can explicitly set the initial monitoring target regions as they want, by writing proper values to the ``init_regions`` file. Each line of the input should represent one region in below form.:: diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/design.rst index 727d72093f8f..0666e19018fd 100644 --- a/Documentation/vm/damon/design.rst +++ b/Documentation/vm/damon/design.rst @@ -35,27 +35,34 @@ two parts: 1. Identification of the monitoring target address range for the address space. 2. Access check of specific address range in the target space. -DAMON currently provides the implementation of the primitives for only the -virtual address spaces. Below two subsections describe how it works. +DAMON currently provides the implementations of the primitives for the physical +and virtual address spaces. Below two subsections describe how those work. PTE Accessed-bit Based Access Check ----------------------------------- -The implementation for the virtual address space uses PTE Accessed-bit for -basic access checks. It finds the relevant PTE Accessed bit from the address -by walking the page table for the target task of the address. In this way, the -implementation finds and clears the bit for next sampling target address and -checks whether the bit set again after one sampling period. This could disturb -other kernel subsystems using the Accessed bits, namely Idle page tracking and -the reclaim logic. To avoid such disturbances, DAMON makes it mutually -exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` page -flags to solve the conflict with the reclaim logic, as Idle page tracking does. +Both of the implementations for physical and virtual address spaces use PTE +Accessed-bit for basic access checks. Only one difference is the way of +finding the relevant PTE Accessed bit(s) from the address. While the +implementation for the virtual address walks the page table for the target task +of the address, the implementation for the physical address walks every page +table having a mapping to the address. In this way, the implementations find +and clear the bit(s) for next sampling target address and checks whether the +bit(s) set again after one sampling period. This could disturb other kernel +subsystems using the Accessed bits, namely Idle page tracking and the reclaim +logic. To avoid such disturbances, DAMON makes it mutually exclusive with Idle +page tracking and uses ``PG_idle`` and ``PG_young`` page flags to solve the +conflict with the reclaim logic, as Idle page tracking does. VMA-based Target Address Range Construction ------------------------------------------- +This is only for the virtual address space primitives implementation. That for +the physical address space simply asks users to manually set the monitoring +target address ranges. + Only small parts in the super-huge virtual address space of the processes are mapped to the physical memory and accessed. Thus, tracking the unmapped address regions is just wasteful. However, because DAMON can deal with some diff --git a/Documentation/vm/damon/faq.rst b/Documentation/vm/damon/faq.rst index 088128bbf22b..6469d54c480f 100644 --- a/Documentation/vm/damon/faq.rst +++ b/Documentation/vm/damon/faq.rst @@ -43,10 +43,9 @@ constructions and actual access checks can be implemented and configured on the DAMON core by the users. In this way, DAMON users can monitor any address space with any access check technique. -Nonetheless, DAMON provides vma tracking and PTE Accessed bit check based +Nonetheless, DAMON provides vma/rmap tracking and PTE Accessed bit check based implementations of the address space dependent functions for the virtual memory -by default, for a reference and convenient use. In near future, we will -provide those for physical memory address space. +and the physical memory by default, for a reference and convenient use. Can I simply monitor page granularity?