From patchwork Tue Jun 8 11:52:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07919C47082 for ; Tue, 8 Jun 2021 11:53:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ADB9461351 for ; Tue, 8 Jun 2021 11:53:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADB9461351 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4E17A6B006E; Tue, 8 Jun 2021 07:53:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B8656B0070; Tue, 8 Jun 2021 07:53:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 331E36B0071; Tue, 8 Jun 2021 07:53:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0231.hostedemail.com [216.40.44.231]) by kanga.kvack.org (Postfix) with ESMTP id 0416D6B006E for ; Tue, 8 Jun 2021 07:53:14 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9E0078249980 for ; Tue, 8 Jun 2021 11:53:14 +0000 (UTC) X-FDA: 78230395908.09.095D70D Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf21.hostedemail.com (Postfix) with ESMTP id 22237E000259 for ; Tue, 8 Jun 2021 11:53:10 +0000 (UTC) Received: by mail-qt1-f182.google.com with SMTP id t17so14977544qta.11 for ; Tue, 08 Jun 2021 04:53:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vycya5wChR8O11SPvwfrbgcahHWj2DvmeexI6w+aeRE=; b=VMaxIBn2qqbfUX6zKsHmZqwhLISgllu71tVzGJYE73tltZNe5U2IHXkvAALM8X9Yj1 QM3OQAud62cywUIC6AA2ohHBOniYB7AMSqJnrYzHHVIDgDushX7SBkzGId48qa3AjzRL osZi/YBaZITPanGvi2HJfoTrXjd3jNWeGbktte7mqMWCHLIPFHH/l5wSv4LmrXs0PhBy UocXgZQFJuwOYCZ5NcVz90UnQE9nXlAJp3Q67VE0VDwxaxCOp8gwkG4G2nmGQvukBE6M 80bTzbpafanF3H7Mt6P/Xq1Tj2EKncenNryrmYDhwlXFe9EwBlIc+fgQPJ1LNxvKr93a AO1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vycya5wChR8O11SPvwfrbgcahHWj2DvmeexI6w+aeRE=; b=e8O9bW9WpNBxVxTbnZpyACJ5+wM8VnHMEqMQBWjMAaUj2KlHpnvgWMkE0WRLD1+TeA /xogfcRQnXHMszy6tqESoXLyvPzXoVyZQ2mlfMrdr5HeVePDtLO6Twv1Rf6vzPeFWEKG WBqHMUmgwd30BqrRk3jHbfQ6bPz6mmtkKCUbm+XuoYxWRro+tGt4lzjHRXFC7NWkKhr0 W4HjHQKhFJ+XRUbEUECDYkUb8rgV8nChrNp5q5aqHigaaOVJZZwl5Xb3TOW886UpzrfA yOwEKu/5nHGbfoAPDprd8rgq/FWsDrys1hzYikxrVNJM8ruYrYMhrnTxWrGK4uv2P8AX AfKw== X-Gm-Message-State: AOAM530PUaIynMyE+pQqdXFuJgksIbAOy8z2xj35wUsb45QwZD+502E7 ohf+mMlqpbUOJeS8AmYDm6Y= X-Google-Smtp-Source: ABdhPJx845C6iRrgasVW+zarBTjWb+SmovQC2wL/Jo6i9FC3w095hG/aJB033l3Kh9Gyf0GzT+yN5Q== X-Received: by 2002:ac8:604:: with SMTP id d4mr20607540qth.304.1623153193659; Tue, 08 Jun 2021 04:53:13 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:13 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 01/14] mm/damon/paddr: Support the pageout scheme Date: Tue, 8 Jun 2021 11:52:41 +0000 Message-Id: <20210608115254.11930-2-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 22237E000259 Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=VMaxIBn2; spf=pass (imf21.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.182 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: f4ik7rft9jrpoqui6ukk8g86hi9zuu46 X-HE-Tag: 1623153190-293263 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes the DAMON primitives for physical address space to support the pageout action for DAMON-based Operation Schemes. IOW, now the users can implement their own data access-aware reclamations for whole system using DAMOS. Signed-off-by: SeongJae Park --- mm/damon/paddr.c | 38 +++++++++++++++++++++++++++++++++++++- mm/damon/prmtv-common.c | 2 +- mm/damon/prmtv-common.h | 2 ++ 3 files changed, 40 insertions(+), 2 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index b92b07a3ce53..303db372e53b 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -7,6 +7,9 @@ #define pr_fmt(fmt) "damon-pa: " fmt +#include + +#include "../internal.h" #include "prmtv-common.h" /* @@ -85,6 +88,39 @@ bool damon_pa_target_valid(void *t) return true; } +int damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + unsigned long addr; + LIST_HEAD(page_list); + + if (scheme->action != DAMOS_PAGEOUT) + return -EINVAL; + + for (addr = r->ar.start; addr < r->ar.end; addr += PAGE_SIZE) { + struct page *page = damon_get_page(PHYS_PFN(addr)); + + if (!page) + continue; + + ClearPageReferenced(page); + test_and_clear_page_young(page); + if (isolate_lru_page(page)) { + put_page(page); + continue; + } + if (PageUnevictable(page)) { + putback_lru_page(page); + } else { + list_add(&page->lru, &page_list); + put_page(page); + } + } + reclaim_pages(&page_list); + cond_resched(); + return 0; +} + void damon_pa_set_primitives(struct damon_ctx *ctx) { ctx->primitive.init = NULL; @@ -94,5 +130,5 @@ void damon_pa_set_primitives(struct damon_ctx *ctx) ctx->primitive.reset_aggregated = NULL; ctx->primitive.target_valid = damon_pa_target_valid; ctx->primitive.cleanup = NULL; - ctx->primitive.apply_scheme = NULL; + ctx->primitive.apply_scheme = damon_pa_apply_scheme; } diff --git a/mm/damon/prmtv-common.c b/mm/damon/prmtv-common.c index 08e9318d67ed..01c1c1b37859 100644 --- a/mm/damon/prmtv-common.c +++ b/mm/damon/prmtv-common.c @@ -14,7 +14,7 @@ * The body of this function is stolen from the 'page_idle_get_page()'. We * steal rather than reuse it because the code is quite simple. */ -static struct page *damon_get_page(unsigned long pfn) +struct page *damon_get_page(unsigned long pfn) { struct page *page = pfn_to_online_page(pfn); diff --git a/mm/damon/prmtv-common.h b/mm/damon/prmtv-common.h index 939c41af6b59..ba0c4eecbb79 100644 --- a/mm/damon/prmtv-common.h +++ b/mm/damon/prmtv-common.h @@ -18,6 +18,8 @@ /* Get a random number in [l, r) */ #define damon_rand(l, r) (l + prandom_u32_max(r - l)) +struct page *damon_get_page(unsigned long pfn); + void damon_va_mkold(struct mm_struct *mm, unsigned long addr); bool damon_va_young(struct mm_struct *mm, unsigned long addr, unsigned long *page_sz); From patchwork Tue Jun 8 11:52:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65579C47082 for ; Tue, 8 Jun 2021 11:53:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F25606124B for ; Tue, 8 Jun 2021 11:53:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F25606124B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99CAF6B0070; Tue, 8 Jun 2021 07:53:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 973DE6B0071; Tue, 8 Jun 2021 07:53:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7047D6B0072; Tue, 8 Jun 2021 07:53:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0230.hostedemail.com [216.40.44.230]) by kanga.kvack.org (Postfix) with ESMTP id 3E5B36B0070 for ; Tue, 8 Jun 2021 07:53:17 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id DBE945923C00 for ; Tue, 8 Jun 2021 11:53:16 +0000 (UTC) X-FDA: 78230395992.14.4039B45 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf06.hostedemail.com (Postfix) with ESMTP id 8F565C00CBE4 for ; Tue, 8 Jun 2021 11:53:13 +0000 (UTC) Received: by mail-qt1-f170.google.com with SMTP id v6so6081265qta.9 for ; Tue, 08 Jun 2021 04:53:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bUDgBkSSnq/g6VE30aIh8yXuIULfQIu9mvYqZ40mOuc=; b=RnsxAEkoi0xWhUXm7BakyWuf1EJC9o3vDmabAovsZaoCo18rRYqjjVlCnPr0wXyJxY 3DysJ8D78S5Tzu+kbMC8nIByI5kfc5M7boHQ0OQkICEqWEyG81/KRtTPOkhgTW5/hH19 VALdLPwPcVWAp+5O5E2XcWjZ4a3jDRFgQH8T18HZTLWl0pOlxrra5k38xaDF8lj7TLPc ntKdg9Y9Hle8MbJAXSiqo+GfY5+APWrBmXTei3cGpfneO8OpuWaRDktVeTqqOFPWZ0N4 asPomQ6MfHxEHd2CK6bQWkxL1czaFw9wkOu574C3Gg/CtxBITQndvQ4bjozGKWadsFd/ M6GQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bUDgBkSSnq/g6VE30aIh8yXuIULfQIu9mvYqZ40mOuc=; b=B/hN25/zTcma6QNYE3qLmVeA+46PqUmxHudAXIMQvoHxQbuQYlsSbHM3sqBSRbeFiR 8kVAsr7OVATblQVbbd+DuZtVzriQr+a7OvL53Fkg5zlYR82+K6ZqlF8FWKVxWcPI1IS+ MwXEbQ2eYe1wVwXBmoG/UWGNJp+zazUNWiXHvu6f6pzpTjCH/X/2mYJ2PtL2Tx6TJVBk ztjts6zYc12O/MBri9ZBQLotve3hE7zUsqpvjdmJmhX7ee5pBOUMcyYUdA49mGffJ/ni jZLx+cTRKlxqlicW5VBTHBiL8Zvy6Rxj9B8iw/Sv6qUD3IO0hm1/INsOBEBUdmgauosA t5FA== X-Gm-Message-State: AOAM533z1tTzlHbCbqmlDDp1ZaFLFi+PxGNYyaetJya0ogw1xkqmIvWP xbm8wEckPEyVB/4I8tSP7uA= X-Google-Smtp-Source: ABdhPJx6uK0S6MA3y0u++av6kzxLhNBdZY26VvP4efiMSIscGrrM9G1vjWEgolfKHPhcrFguTfVplw== X-Received: by 2002:ac8:58ce:: with SMTP id u14mr21433954qta.83.1623153195874; Tue, 08 Jun 2021 04:53:15 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:15 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 02/14] mm/damon/damos: Make schemes aggressiveness controllable Date: Tue, 8 Jun 2021 11:52:42 +0000 Message-Id: <20210608115254.11930-3-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8F565C00CBE4 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=RnsxAEko; spf=pass (imf06.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: 5pbqsedt7wru67qthuupdts3x3qxpfe8 X-HE-Tag: 1623153193-310031 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park If there are too large memory regions fulfilling the target data access pattern of a DAMON-based operation scheme, applying the action of the scheme could consume too much CPU. To avoid that, this commit implements a limit for the action application speed. Using the feature, the client can set up to how much amount of memory regions the action could applied within specific time duration. Signed-off-by: SeongJae Park --- include/linux/damon.h | 34 ++++++++++++++++++++++++---- mm/damon/core.c | 52 ++++++++++++++++++++++++++++++++++++++----- mm/damon/dbgfs.c | 4 +++- 3 files changed, 79 insertions(+), 11 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 684a3603ddac..35068b0ece6f 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -87,6 +87,25 @@ enum damos_action { DAMOS_STAT, /* Do nothing but only record the stat */ }; +/** + * struct damos_speed_limit - Controls the aggressiveness of the given scheme. + * @sz: Scheme action amount limit in bytes. + * @ms: Scheme action amount charge duration. + * + * To avoid consuming too much CPU time for applying the &struct damos->action + * to large memory, DAMON applies it to only up to &sz bytes within &ms. + * + * If &sz is 0, the limit is disabled. + */ +struct damos_speed_limit { + unsigned long sz; + unsigned long ms; + +/* private: for limit accounting */ + unsigned long charged_sz; + unsigned long charged_from; +}; + /** * struct damos - Represents a Data Access Monitoring-based Operation Scheme. * @min_sz_region: Minimum size of target regions. @@ -96,13 +115,19 @@ enum damos_action { * @min_age_region: Minimum age of target regions. * @max_age_region: Maximum age of target regions. * @action: &damo_action to be applied to the target regions. + * @limit: Control the aggressiveness of this scheme. * @stat_count: Total number of regions that this scheme is applied. * @stat_sz: Total size of regions that this scheme is applied. * @list: List head for siblings. * - * For each aggregation interval, DAMON applies @action to monitoring target - * regions fit in the condition and updates the statistics. Note that both - * the minimums and the maximums are inclusive. + * For each aggregation interval, DAMON finds regions which fit in the + * condition (&min_sz_region, &max_sz_region, &min_nr_accesses, + * &max_nr_accesses, &min_age_region, &max_age_region) and applies &action to + * those. To avoid consuming too much CPU for the &action, &limit is used. + * + * After applying the &action to each region, &stat_count and &stat_sz is + * updated to reflect the number of regions and total size of regions that the + * &action is applied. */ struct damos { unsigned long min_sz_region; @@ -112,6 +137,7 @@ struct damos { unsigned int min_age_region; unsigned int max_age_region; enum damos_action action; + struct damos_speed_limit limit; unsigned long stat_count; unsigned long stat_sz; struct list_head list; @@ -335,7 +361,7 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action); + enum damos_action action, struct damos_speed_limit *limit); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index a33b3a3b9e57..4ac4a9aa2514 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -85,7 +85,7 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action) + enum damos_action action, struct damos_speed_limit *limit) { struct damos *scheme; @@ -103,6 +103,11 @@ struct damos *damon_new_scheme( scheme->stat_sz = 0; INIT_LIST_HEAD(&scheme->list); + scheme->limit.sz = limit->sz; + scheme->limit.ms = limit->ms; + scheme->limit.charged_sz = 0; + scheme->limit.charged_from = 0; + return scheme; } @@ -536,6 +541,9 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) } } +static void damon_split_region_at(struct damon_ctx *ctx, + struct damon_region *r, unsigned long sz_r); + static void damon_do_apply_schemes(struct damon_ctx *c, struct damon_target *t, struct damon_region *r) @@ -544,7 +552,14 @@ static void damon_do_apply_schemes(struct damon_ctx *c, unsigned long sz; damon_for_each_scheme(s, c) { + struct damos_speed_limit *limit = &s->limit; + + /* Check the limit */ + if (limit->sz && limit->charged_sz >= limit->sz) + continue; + sz = r->ar.end - r->ar.start; + /* Check the target regions condition */ if (sz < s->min_sz_region || s->max_sz_region < sz) continue; if (r->nr_accesses < s->min_nr_accesses || @@ -552,22 +567,47 @@ static void damon_do_apply_schemes(struct damon_ctx *c, continue; if (r->age < s->min_age_region || s->max_age_region < r->age) continue; - s->stat_count++; - s->stat_sz += sz; - if (c->primitive.apply_scheme) + + /* Apply the scheme */ + if (c->primitive.apply_scheme) { + if (limit->sz && limit->charged_sz + sz > limit->sz) { + sz = limit->sz - limit->charged_sz; + damon_split_region_at(c, r, sz); + } c->primitive.apply_scheme(c, t, r, s); + limit->charged_sz += sz; + } if (s->action != DAMOS_STAT) r->age = 0; + + /* Update stat */ + s->stat_count++; + s->stat_sz += sz; } } static void kdamond_apply_schemes(struct damon_ctx *c) { struct damon_target *t; - struct damon_region *r; + struct damon_region *r, *next_r; + struct damos *s; + + damon_for_each_scheme(s, c) { + struct damos_speed_limit *limit = &s->limit; + + if (!limit->sz) + continue; + + /* Reset charge window if the duration passed */ + if (time_after_eq(jiffies, limit->charged_from + + msecs_to_jiffies(limit->ms))) { + limit->charged_from = jiffies; + limit->charged_sz = 0; + } + } damon_for_each_target(t, c) { - damon_for_each_region(r, t) + damon_for_each_region_safe(r, next_r, t) damon_do_apply_schemes(c, t, r); } } diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 5b254eccdb43..4b45b69db697 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -310,6 +310,8 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, *nr_schemes = 0; while (pos < len && *nr_schemes < max_nr_schemes) { + struct damos_speed_limit limit = {}; + ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, &min_age, &max_age, &action, &parsed); @@ -322,7 +324,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(min_sz, max_sz, min_nr_a, max_nr_a, - min_age, max_age, action); + min_age, max_age, action, &limit); if (!scheme) goto fail; From patchwork Tue Jun 8 11:52:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C671FC4743F for ; Tue, 8 Jun 2021 11:53:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6457F61351 for ; Tue, 8 Jun 2021 11:53:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6457F61351 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 09B576B0071; Tue, 8 Jun 2021 07:53:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 074646B0072; Tue, 8 Jun 2021 07:53:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E2E646B0073; Tue, 8 Jun 2021 07:53:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0101.hostedemail.com [216.40.44.101]) by kanga.kvack.org (Postfix) with ESMTP id B3B596B0071 for ; Tue, 8 Jun 2021 07:53:19 -0400 (EDT) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 43AD498B2 for ; Tue, 8 Jun 2021 11:53:19 +0000 (UTC) X-FDA: 78230396118.16.FC986EA Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by imf16.hostedemail.com (Postfix) with ESMTP id 33C05801936A for ; Tue, 8 Jun 2021 11:53:16 +0000 (UTC) Received: by mail-qk1-f176.google.com with SMTP id c138so6939408qkg.5 for ; Tue, 08 Jun 2021 04:53:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1v3jF6xE2cadC0A5okidQotQQ/sADUe8F+zXLAj+QLM=; b=mDHQ2YMh1wx+gC9L9QO0GSL2MxjH0qEDBREBkT9xZfRPHTih2jw4S6xa1bbGfaOZrs CwcL3xX1gTSfuPCDIYBAJ66P1bZam83EPUrmm6/K0avsxZZU+kHgpf/6wf0J9FLkqWEP htvwgvuyJrf4saioMmd1RvXq0nVd1ZLxa/yZ+FKnH4GYUj7jJxm7KOZnW9MM+rZziU/r tzzFszZw0W1TZaqN8Pnlv7Ouo4k/mFznd8bXE86+0y3+kO3eGD3P4467F8iR97Md891y UYYGgQdVJbo0MKn67nGnjtbW0Kf7dwPRIkrbRqtZWF7TkacXxoZGHMED7KZLxPZ5v8B2 XZpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1v3jF6xE2cadC0A5okidQotQQ/sADUe8F+zXLAj+QLM=; b=AbRPIn//fKgvMZ1GFY2IKEckYDWf0thJhPrU52f4C/FLWqEC07K1sM/JbFDhliav9r /KShfywYl93ZmUigXz3xMwESVdBKhxPd7619ezHgEQd+z+3yoGH8CxBypT/AA34cV042 bMeQvL5qPrPKAqWadqdWUuFbUAx6tvlUcNgq7hNFQkWHJ3gfpnrjFLCUaOGpl3obko1u VlMchLWOuK5guTop6f3q60Q145h98W40si47Ou5pIE09MghLlisQ1/9Obl0O22pMxB3V izTMDg2qkoW4cfbwcZqfzgfuB6/y7Ag3M1BzOsc+PUA0QgkYcOrfbzCzj4TCWiKTBvZu dv0g== X-Gm-Message-State: AOAM530XcadmEOTjFolXJ4rvrC6qiztWaxWDNE2+LeJsxb793HJlc1+7 m5eVgv+xEO41jZZ75gmfm/o= X-Google-Smtp-Source: ABdhPJwUvb+KltsTXZs159GKaS7Z/4amSHZTojI8glTyC1QaSMs/x/Ym94/UGCYQevE7izxWu9Lo+g== X-Received: by 2002:a37:73c7:: with SMTP id o190mr6613475qkc.314.1623153198298; Tue, 08 Jun 2021 04:53:18 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:17 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 03/14] damon/core/schemes: Skip already charged targets and regions Date: Tue, 8 Jun 2021 11:52:43 +0000 Message-Id: <20210608115254.11930-4-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=mDHQ2YMh; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.176 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Stat-Signature: kxmc8mc4j4mank34kyu3zxd6eod8y1sr X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 33C05801936A X-HE-Tag: 1623153196-215070 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park If DAMOS stopped applying action to memory regions due to the speed limit, it does nothing until next charge window starts. Then, it starts the work from the beginning of the address space. If there is a huge memory region at the beginning of the address space and it fulfills the scheme target data access pattern, the action will applied to only the region. This commit mitigates the case by skipping memory regions that charged in previous charge window at the beginning of current charge window. Signed-off-by: SeongJae Park --- include/linux/damon.h | 5 +++++ mm/damon/core.c | 21 +++++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 35068b0ece6f..0df81dd2d560 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -104,6 +104,8 @@ struct damos_speed_limit { /* private: for limit accounting */ unsigned long charged_sz; unsigned long charged_from; + struct damon_target *charge_target_from; + unsigned long charge_addr_from; }; /** @@ -331,6 +333,9 @@ struct damon_ctx { #define damon_prev_region(r) \ (container_of(r->list.prev, struct damon_region, list)) +#define damon_last_region(t) \ + (list_last_entry(&t->regions_list, struct damon_region, list)) + #define damon_for_each_region(r, t) \ list_for_each_entry(r, &t->regions_list, list) diff --git a/mm/damon/core.c b/mm/damon/core.c index 4ac4a9aa2514..5864b63c18f9 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -107,6 +107,8 @@ struct damos *damon_new_scheme( scheme->limit.ms = limit->ms; scheme->limit.charged_sz = 0; scheme->limit.charged_from = 0; + scheme->limit.charge_target_from = NULL; + scheme->limit.charge_addr_from = 0; return scheme; } @@ -558,6 +560,21 @@ static void damon_do_apply_schemes(struct damon_ctx *c, if (limit->sz && limit->charged_sz >= limit->sz) continue; + if (limit->charge_target_from) { + if (t != limit->charge_target_from) + continue; + if (r == damon_last_region(t)) { + limit->charge_target_from = NULL; + limit->charge_addr_from = 0; + continue; + } + if (limit->charge_addr_from && + r->ar.start < limit->charge_addr_from) + continue; + limit->charge_target_from = NULL; + limit->charge_addr_from = 0; + } + sz = r->ar.end - r->ar.start; /* Check the target regions condition */ if (sz < s->min_sz_region || s->max_sz_region < sz) @@ -576,6 +593,10 @@ static void damon_do_apply_schemes(struct damon_ctx *c, } c->primitive.apply_scheme(c, t, r, s); limit->charged_sz += sz; + if (limit->sz && limit->charged_sz >= limit->sz) { + limit->charge_target_from = t; + limit->charge_addr_from = r->ar.end + 1; + } } if (s->action != DAMOS_STAT) r->age = 0; From patchwork Tue Jun 8 11:52:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AC33C4743F for ; Tue, 8 Jun 2021 11:53:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AFF576135A for ; Tue, 8 Jun 2021 11:53:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFF576135A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 52F2A6B0072; Tue, 8 Jun 2021 07:53:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 508766B0074; Tue, 8 Jun 2021 07:53:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 340956B0072; Tue, 8 Jun 2021 07:53:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id F35206B0072 for ; Tue, 8 Jun 2021 07:53:21 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8BD6A181AEF15 for ; Tue, 8 Jun 2021 11:53:21 +0000 (UTC) X-FDA: 78230396202.22.BD3590B Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) by imf04.hostedemail.com (Postfix) with ESMTP id CAD4B42D for ; Tue, 8 Jun 2021 11:53:18 +0000 (UTC) Received: by mail-qk1-f174.google.com with SMTP id j62so6042245qke.10 for ; Tue, 08 Jun 2021 04:53:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nFxPSjswgRnIxs2vNxK3L4JE5I+QJrKj8CTw993w1EM=; b=GVV3JJMQ0XCOg8jHkapL/ow7iREJv5SsbPj9zFzb0rwiLDvmbHCdQCuyTKqSKLk9KC 4L5th8pMvcUFnVHUf3ChYt4sxsIFIVvxy6UwidbuXx0b+pstL8nK8emHY/hYZLs8JGOd fT/Fiy2x8bf341SreDqjTIAVCbPiDhMDrZ2gQuo2B0fchSg+NGcH7dfDl+gSUeD0eBkS 2YRRfc/m81doMDzhpHJeBzJaLvlTjunzNXqJwiPc8sNE8nWw5T3gflkGPgduNsMPqXSN B5GG1BUAFSuGYVSgUB4cfmRosglzn++/cxc79q7VmvR33u4xfLLj2Ni2KrUJfw3bXNwH /XLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nFxPSjswgRnIxs2vNxK3L4JE5I+QJrKj8CTw993w1EM=; b=ZfqLpl7LYW0c0QmUAmRWa2YmWraz8Bzo2xTsPX+5AKV0oJ7XjJsJXLLlHygJdjbsKF FMK72NAikK5gbDcPFzdgnKEpEWDZauasyypnpGtjs1cn33seEggvEBvYVa7Qtax0eYGJ mynPP2mf/SVrWuFiKc2QkG1x9z7HfkEz2+kGrAVbGj5hMC9sGGQtdlUmv1N7Ws23x42M CajVRbmMv0FZ4ySN50WnJxAOkoTL8KQRVqIkzOyilRydVUQr129Ykoy7+KYRUmlfi8WU rrVCm6CHT251vVc6AeHCCB/5njIBPPQ5dXxoYVTYeME3M41iiqerRZCck+Yur9zEs3Q7 3LKw== X-Gm-Message-State: AOAM533bgpurqnd/w7eRtpxclwbFnyBlqZtDobpTLZ8AWGm2L0QcILVX ncv1bR9WKoMhy5Y99CRJh9w= X-Google-Smtp-Source: ABdhPJz0W4Tm3yxT5k/BicvuMfYTgFKUVIqEsXsSXgO0izcZdX2uEWYTaLsVGNAADQ7IvbEL4S0CMQ== X-Received: by 2002:a37:7287:: with SMTP id n129mr20201503qkc.457.1623153200545; Tue, 08 Jun 2021 04:53:20 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:20 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 04/14] mm/damon/dbgfs: Support schemes speed limit Date: Tue, 8 Jun 2021 11:52:44 +0000 Message-Id: <20210608115254.11930-5-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: CAD4B42D Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=GVV3JJMQ; spf=pass (imf04.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.174 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: z3yat3anbbiphqeu5edrgqxjfzciexyf X-HE-Tag: 1623153198-749224 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes the debugfs interface of DAMON to support the schemes speed limit by chaning the format of the input for the schemes file. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 4b45b69db697..ea6d4fdb57fa 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -227,11 +227,12 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) damon_for_each_scheme(s, c) { rc = scnprintf(&buf[written], len - written, - "%lu %lu %u %u %u %u %d %lu %lu\n", + "%lu %lu %u %u %u %u %d %lu %lu %lu %lu\n", s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, - s->action, s->stat_count, s->stat_sz); + s->action, s->limit.sz, s->limit.ms, + s->stat_count, s->stat_sz); if (!rc) return -ENOMEM; @@ -312,10 +313,11 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_speed_limit limit = {}; - ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u%n", + ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, - &min_age, &max_age, &action, &parsed); - if (ret != 7) + &min_age, &max_age, &action, &limit.sz, + &limit.ms, &parsed); + if (ret != 9) break; if (!damos_action_valid(action)) { pr_err("wrong action %d\n", action); @@ -1133,6 +1135,15 @@ static ssize_t dbgfs_monitor_on_write(struct file *file, return ret; } +/* + * v1: Add the scheme speed limit + */ +static ssize_t dbgfs_version_read(struct file *file, + char __user *buf, size_t count, loff_t *ppos) +{ + return simple_read_from_buffer(buf, count, ppos, "1\n", 2); +} + static const struct file_operations mk_contexts_fops = { .owner = THIS_MODULE, .write = dbgfs_mk_context_write, @@ -1149,13 +1160,18 @@ static const struct file_operations monitor_on_fops = { .write = dbgfs_monitor_on_write, }; +static const struct file_operations version_fops = { + .owner = THIS_MODULE, + .read = dbgfs_version_read, +}; + static int __init __damon_dbgfs_init(void) { struct dentry *dbgfs_root; const char * const file_names[] = {"mk_contexts", "rm_contexts", - "monitor_on"}; + "monitor_on", "version"}; const struct file_operations *fops[] = {&mk_contexts_fops, - &rm_contexts_fops, &monitor_on_fops}; + &rm_contexts_fops, &monitor_on_fops, &version_fops}; int i; dbgfs_root = debugfs_create_dir("damon", NULL); From patchwork Tue Jun 8 11:52:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23A11C4743E for ; Tue, 8 Jun 2021 11:53:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CB3DD61351 for ; Tue, 8 Jun 2021 11:53:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB3DD61351 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6DD756B0073; Tue, 8 Jun 2021 07:53:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6448C6B0074; Tue, 8 Jun 2021 07:53:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FB2A6B0075; Tue, 8 Jun 2021 07:53:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0252.hostedemail.com [216.40.44.252]) by kanga.kvack.org (Postfix) with ESMTP id EE7406B0073 for ; Tue, 8 Jun 2021 07:53:23 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 92AD0180AD80F for ; Tue, 8 Jun 2021 11:53:23 +0000 (UTC) X-FDA: 78230396286.01.82BC966 Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by imf29.hostedemail.com (Postfix) with ESMTP id 625B0544 for ; Tue, 8 Jun 2021 11:53:18 +0000 (UTC) Received: by mail-qv1-f50.google.com with SMTP id w4so3137348qvr.11 for ; Tue, 08 Jun 2021 04:53:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4ZGpG/OoDyFj8zWzsVNZahhh/PU2Xw0WQnYylGT37eU=; b=Yq42KPRH05BimDLNJKHdfHTmPCD/zIWdkIN6o+96suXi52Za+1HZdprMUeI5P+427S AEqy4LePzL2ryEKdqpcE4J98KUIVG4QY5h83SDWdBQfV5BOP2gulJgm4yAWQApUiumXo qzUEKKLq/VNGYJB1Y4FBTxNXM2oZMlHeFDNuwsyqs7kUJcmgzIdLwvZcLLwcHcPxmxSq VxLhAu9eLYgyfDhcBnCJv7V9o9gaKsU+rE7KMlucNQ939EfHDtb5RlpAg4mYxB/JGTiW lUjyT3mD02/BMiv49C2TrYj04olrTuDDS807u03bHDn6/ePDwndCtnrLg1OAW/7QpziZ DKyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4ZGpG/OoDyFj8zWzsVNZahhh/PU2Xw0WQnYylGT37eU=; b=JCWO1QGBTUfk7E8HFKlEcVzZEs5hByz6KEqGRWgZ1JbPLe7JszWDhkcUebVMCMTiiI AQMQ6gqqfL5Is6h19yJZyf7WqlGG1Wdv53GHhtIP+JFNqDtz33aE5BaIiQmh13hFzDkB g/BNAJdn2rRfM5CIjSLM79+r32uywFnXYMY8Kx46D3R0CECj3sGVuehEmP4PjHLW5CTn lRjMbKTcjCPRziR9L08PASeUQmi9POjMQMBPBsEYsA17HputIxcmUvRA6e1IOGdgdRjl 26dJqmAOfqFJ2FwugG0EP7vVSqt7+C55jrRkZcEKz9yW7Q4IFcJUiixilWbeSzy7u1zX SzeQ== X-Gm-Message-State: AOAM533Q5ibgpeblzCBqO26aKcJ8hQEqo/JX6UTAUMDaMtF6g03niCy+ d8mOkfFIR2gzKGw6tCi0YWQ= X-Google-Smtp-Source: ABdhPJzgDOgfXw9JK8azbDW7zgdkh/TmBzWQYC3OVdiC73Izm7VnyBqSB9ks1x4aSR7N2SPdb6QUKw== X-Received: by 2002:ad4:57c4:: with SMTP id y4mr23083873qvx.12.1623153202673; Tue, 08 Jun 2021 04:53:22 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:22 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 05/14] mm/damon/selftests: Support schemes speed limit Date: Tue, 8 Jun 2021 11:52:45 +0000 Message-Id: <20210608115254.11930-6-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 625B0544 X-Stat-Signature: jc9z4c8efy9ayn1zqzxu38f5jwixkog8 Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Yq42KPRH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.50 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-HE-Tag: 1623153198-335803 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates DAMON selftests to support updated schemes debugfs file format. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/debugfs_attrs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh index 61fd3e5598e9..012b0c1fdbd3 100755 --- a/tools/testing/selftests/damon/debugfs_attrs.sh +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -101,7 +101,7 @@ echo $ORIG_CONTENT > $file file="$DBGFS/schemes" ORIG_CONTENT=$(cat $file) -echo "1 2 3 4 5 6 3" > $file +echo "1 2 3 4 5 6 3 0 0" > $file if [ $? -ne 0 ] then echo "$file write fail" @@ -110,7 +110,7 @@ then fi echo "1 2 -3 4 5 6 3" > $file +3 4 5 6 3 0 0" > $file if [ $? -eq 0 ] then echo "$file multi line write success (expected fail)" From patchwork Tue Jun 8 11:52:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C7A8C47082 for ; Tue, 8 Jun 2021 11:53:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0F50E6128D for ; Tue, 8 Jun 2021 11:53:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F50E6128D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A86A36B0074; Tue, 8 Jun 2021 07:53:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A60E16B0075; Tue, 8 Jun 2021 07:53:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8134B6B0078; Tue, 8 Jun 2021 07:53:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0226.hostedemail.com [216.40.44.226]) by kanga.kvack.org (Postfix) with ESMTP id 4F2A76B0074 for ; Tue, 8 Jun 2021 07:53:26 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id F12739898 for ; Tue, 8 Jun 2021 11:53:25 +0000 (UTC) X-FDA: 78230396370.34.61EEC52 Received: from mail-qv1-f42.google.com (mail-qv1-f42.google.com [209.85.219.42]) by imf11.hostedemail.com (Postfix) with ESMTP id 3A33020010AA for ; Tue, 8 Jun 2021 11:53:22 +0000 (UTC) Received: by mail-qv1-f42.google.com with SMTP id u14so7675635qvq.6 for ; Tue, 08 Jun 2021 04:53:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=USupWkjstKLzS9GL1eZ+AKXClsG/TjR98Rj/j8aN4z0=; b=Z/Knr8Fgp9bWjg1dd0BIpTKDEBXVpsMpZ3+elPatEnn5m801o33WnyZYHWQifpPT7H qc3aLzdcU3Ixq28wk30Reufga9vdRaLhMmwJr9S/5hNxWiW7iGy1KAjSA79sIjLqrtAX B1tkfsqUNz51ADga1t7fk9SvFgXMEcX8Wkr5LTDu/FO1xhh49lt3/OG/SyARRO+M2Csx QnkbGAPXhi/8QlDRjM0YjD161hlsCuIGipgB9jLv3mUnG+pd8rMjwWm2B/Fq7bf2EQFT kjkNLRD1IcJNIdWbJ8IO5SCc73zy0UWM9g8tx5d8wVDJuW3YboYrWHjGyd+PgybgvzE4 yf4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=USupWkjstKLzS9GL1eZ+AKXClsG/TjR98Rj/j8aN4z0=; b=KySqRDX28pWRTI9+flC4tKW/d3vjwnw00tsi5FQiC2SOsCkA7HjpJ+3IIDcCcQ57wx CJ0JuiS4X65Uy9xdrqB4L5eBNP9MrgdYbW0BzuRvSYw/wQIU6U8XfY28GcPEyk1xEDbw 5mVzVMgMf0MuOK06YVI6RcOqiLZ6bMV8bWCpfe4RvellF+kIPPolq55XRrLRrDYJfu66 MrGgczPKO1x+RHgUExWMFGc0Q4AQR218Gi2+mKNLegDu0UggjrDFI4cFKBufivpWys+5 hHVMQYanSwePmcsugdiVqFOfP5g923vFRdnWZy3Aj0rVFGjxFzkn5sc3UHaU6GAwCrGq w06g== X-Gm-Message-State: AOAM530GVbPjf09hAfalpKbbN4GvsE0y9C5SMluUjEL1163AKL4fGe21 LQl9VoyLdS0fnhkAmNL4Fbk= X-Google-Smtp-Source: ABdhPJziaxjKD2gnNPTzevzkb3cN3YS/ADUNou87xRED4m+fbEP2mh4YVLkGZXxtdWHNNE0qodFlTQ== X-Received: by 2002:a0c:e185:: with SMTP id p5mr22740331qvl.17.1623153205024; Tue, 08 Jun 2021 04:53:25 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:24 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 06/14] mm/damon/schemes: Prioritize regions within speed limit Date: Tue, 8 Jun 2021 11:52:46 +0000 Message-Id: <20210608115254.11930-7-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3A33020010AA Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="Z/Knr8Fg"; spf=pass (imf11.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.42 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: 1jeiaggyqy3i9gjsmp3azuun8yyzfzjn X-HE-Tag: 1623153202-604081 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes DAMON to apply schemes to regions having higher priority first, if it cannot apply schemes to all regions due to the speed limit. The prioritization function should be implemented in each monitoring primitive. Those would commonly calculate the priority of the region using attributes of regions, namely 'size', 'nr_accesses', and 'age'. For example, some primitive would calculate the priority of each region using a weighted sum of 'nr_accesses' and 'age' of the region. The optimal weights would depend on give environments, so this commit allows it to be customizable. Nevertheless, the score calculation functions are only encouraged to respect the weights, not mandated. So, the customization might not work for some primitives. Signed-off-by: SeongJae Park --- include/linux/damon.h | 27 ++++++++++++++++- mm/damon/core.c | 69 ++++++++++++++++++++++++++++++++++++------- 2 files changed, 84 insertions(+), 12 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 0df81dd2d560..8f35bd94fc2b 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -14,6 +14,8 @@ /* Minimal region size. Every damon_region is aligned by this. */ #define DAMON_MIN_REGION PAGE_SIZE +/* Max priority score for DAMON-based operation schemes */ +#define DAMOS_MAX_SCORE (99) /** * struct damon_addr_range - Represents an address region of [@start, @end). @@ -92,8 +94,18 @@ enum damos_action { * @sz: Scheme action amount limit in bytes. * @ms: Scheme action amount charge duration. * + * @weight_sz: Weight of the region's size for prioritization. + * @weight_nr_accesses: Weight of the region's nr_accesses for prioritization. + * @weight_age: Weight of the region's age for prioritization. + * * To avoid consuming too much CPU time for applying the &struct damos->action - * to large memory, DAMON applies it to only up to &sz bytes within &ms. + * to large memory, DAMON applies it to only up to &sz bytes within &ms. For + * selecting regions within the limit, DAMON prioritizes current scheme's + * target memory regions using the given &struct + * damon_primitive->get_scheme_score. You could customize the prioritization + * logic for your environment by setting &weight_sz, &weight_nr_accesses, and + * &weight_age, because primitives are encouraged to respect those, though it's + * not mandatory. * * If &sz is 0, the limit is disabled. */ @@ -101,11 +113,18 @@ struct damos_speed_limit { unsigned long sz; unsigned long ms; + unsigned int weight_sz; + unsigned int weight_nr_accesses; + unsigned int weight_age; + /* private: for limit accounting */ unsigned long charged_sz; unsigned long charged_from; struct damon_target *charge_target_from; unsigned long charge_addr_from; + + unsigned long histogram[DAMOS_MAX_SCORE + 1]; + unsigned int min_score; }; /** @@ -155,6 +174,7 @@ struct damon_ctx; * @prepare_access_checks: Prepare next access check of target regions. * @check_accesses: Check the accesses to target regions. * @reset_aggregated: Reset aggregated accesses monitoring results. + * @get_scheme_score: Get the score of a region for a scheme. * @apply_scheme: Apply a DAMON-based operation scheme. * @target_valid: Determine if the target is valid. * @cleanup: Clean up the context. @@ -182,6 +202,8 @@ struct damon_ctx; * of its update. The value will be used for regions adjustment threshold. * @reset_aggregated should reset the access monitoring results that aggregated * by @check_accesses. + * @get_scheme_score should return the priority score of a region for a scheme + * as an integer in [0, &DAMOS_MAX_SCORE]. * @apply_scheme is called from @kdamond when a region for user provided * DAMON-based operation scheme is found. It should apply the scheme's action * to the region. This is not used for &DAMON_ARBITRARY_TARGET case. @@ -196,6 +218,9 @@ struct damon_primitive { void (*prepare_access_checks)(struct damon_ctx *context); unsigned int (*check_accesses)(struct damon_ctx *context); void (*reset_aggregated)(struct damon_ctx *context); + int (*get_scheme_score)(struct damon_ctx *context, + struct damon_target *t, struct damon_region *r, + struct damos *scheme); int (*apply_scheme)(struct damon_ctx *context, struct damon_target *t, struct damon_region *r, struct damos *scheme); bool (*target_valid)(void *target); diff --git a/mm/damon/core.c b/mm/damon/core.c index 5864b63c18f9..15bcd05670d1 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -12,6 +12,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -105,11 +106,13 @@ struct damos *damon_new_scheme( scheme->limit.sz = limit->sz; scheme->limit.ms = limit->ms; + scheme->limit.weight_sz = limit->weight_sz; + scheme->limit.weight_nr_accesses = limit->weight_nr_accesses; + scheme->limit.weight_age = limit->weight_age; scheme->limit.charged_sz = 0; scheme->limit.charged_from = 0; scheme->limit.charge_target_from = NULL; scheme->limit.charge_addr_from = 0; - return scheme; } @@ -546,6 +549,28 @@ static void kdamond_reset_aggregated(struct damon_ctx *c) static void damon_split_region_at(struct damon_ctx *ctx, struct damon_region *r, unsigned long sz_r); +static bool __damos_valid_target(struct damon_region *r, struct damos *s) +{ + unsigned long sz; + + sz = r->ar.end - r->ar.start; + return s->min_sz_region <= sz && sz <= s->max_sz_region && + s->min_nr_accesses <= r->nr_accesses && + r->nr_accesses <= s->max_nr_accesses && + s->min_age_region <= r->age && r->age <= s->max_age_region; +} + +static bool damos_valid_target(struct damon_ctx *c, struct damon_target *t, + struct damon_region *r, struct damos *s) +{ + bool ret = __damos_valid_target(r, s); + + if (!ret || !s->limit.sz || !c->primitive.get_scheme_score) + return ret; + + return c->primitive.get_scheme_score(c, t, r, s) >= s->limit.min_score; +} + static void damon_do_apply_schemes(struct damon_ctx *c, struct damon_target *t, struct damon_region *r) @@ -575,17 +600,11 @@ static void damon_do_apply_schemes(struct damon_ctx *c, limit->charge_addr_from = 0; } - sz = r->ar.end - r->ar.start; - /* Check the target regions condition */ - if (sz < s->min_sz_region || s->max_sz_region < sz) - continue; - if (r->nr_accesses < s->min_nr_accesses || - s->max_nr_accesses < r->nr_accesses) - continue; - if (r->age < s->min_age_region || s->max_age_region < r->age) + if (!damos_valid_target(c, t, r, s)) continue; /* Apply the scheme */ + sz = r->ar.end - r->ar.start; if (c->primitive.apply_scheme) { if (limit->sz && limit->charged_sz + sz > limit->sz) { sz = limit->sz - limit->charged_sz; @@ -615,16 +634,44 @@ static void kdamond_apply_schemes(struct damon_ctx *c) damon_for_each_scheme(s, c) { struct damos_speed_limit *limit = &s->limit; + unsigned long cumulated_sz; + unsigned int score, max_score = 0; if (!limit->sz) continue; /* Reset charge window if the duration passed */ - if (time_after_eq(jiffies, limit->charged_from + - msecs_to_jiffies(limit->ms))) { + if (time_after_eq(jiffies, s->limit.charged_from + + msecs_to_jiffies(s->limit.ms))) { limit->charged_from = jiffies; limit->charged_sz = 0; } + + if (!c->primitive.get_scheme_score) + continue; + + /* Fill up the score histogram */ + memset(limit->histogram, 0, sizeof(limit->histogram)); + damon_for_each_target(t, c) { + damon_for_each_region(r, t) { + if (!__damos_valid_target(r, s)) + continue; + score = c->primitive.get_scheme_score( + c, t, r, s); + limit->histogram[score] += + r->ar.end - r->ar.start; + if (score > max_score) + max_score = score; + } + } + + /* Set the min score limit */ + for (cumulated_sz = 0, score = max_score; ; score--) { + cumulated_sz += limit->histogram[score]; + if (cumulated_sz >= limit->sz || !score) + break; + } + limit->min_score = score; } damon_for_each_target(t, c) { From patchwork Tue Jun 8 11:52:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8800EC48BCD for ; Tue, 8 Jun 2021 11:53:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 38B5C6128D for ; Tue, 8 Jun 2021 11:53:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 38B5C6128D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CF7216B0075; Tue, 8 Jun 2021 07:53:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5AD86B0078; Tue, 8 Jun 2021 07:53:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD3046B007B; Tue, 8 Jun 2021 07:53:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id 7B6C16B0075 for ; Tue, 8 Jun 2021 07:53:28 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 18EE0181AEF15 for ; Tue, 8 Jun 2021 11:53:28 +0000 (UTC) X-FDA: 78230396496.01.9824967 Received: from mail-qt1-f169.google.com (mail-qt1-f169.google.com [209.85.160.169]) by imf23.hostedemail.com (Postfix) with ESMTP id 10574A0001CF for ; Tue, 8 Jun 2021 11:53:24 +0000 (UTC) Received: by mail-qt1-f169.google.com with SMTP id a15so758333qtx.13 for ; Tue, 08 Jun 2021 04:53:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+6bCyXjso+WpVBQaujE86WjgkNPOriN2PNEeNMasASY=; b=Z5y/g8TDzpDeUlu2xE8q86uL+v7AvXqMVqOPhy85ZrOfsposhhFmoDYqcr1j8ec3Eb QIdIWbxTEnqqLfMRrEfLHJvP+gcagLwfcrJLau9EBjixbCHcmTOTKAi1vNHwd3/YiwF4 8xYlxLG1oyvZaoa1K7Clp3os9xxLmW3HlurRe730YQdmuW9Mkg6v7CE3IHjgx9CNqg+V XB41V6n4ts3VkO7dL7btbje0KmdVWsMWjvQzapMTEFYBnvc1loE5qC4VS/eXMCKC4p/7 vZf9momFfZ8ZfGhEG1SPyYwB9GpdFCyCykX2JqNm0HtPEAGHe9OBijNCQcVVbnrSHuJ6 lNIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+6bCyXjso+WpVBQaujE86WjgkNPOriN2PNEeNMasASY=; b=s6/og6hmLNrKELwsnfLZJ6irHZDd6okkWdHvOqGFZ9XdthaqY0GNZhTu4+IHQq/8Lh ptOMJucvg8BPXxdEL0hGdUs4Q8K9Goun3RN2+Xlu8bbpFODH//qwlkmNJpiD+gJypBRv YnOJqFroim2eXPgiJJ/iJOzpl9SNDzGAjV6Lujp7J+G9Sl/gaJAllQJcgu8KC1bejbLX pvOP79K52Acv4uph0TRW9gYGWazibE51ZH28c2R+jJKJ2X+O260PNKri3EaUV7CnDAda 68KhNKONNZH/o5fxgl2tFWvpKGV1okO85xZ4rHRtKgj4BTgBLFiX0d55PNQ4+o6Eikwv 7U4A== X-Gm-Message-State: AOAM5302Vaft71IM/N251BD3UKHzGjPWFDROIl75p2Z8mZBaRWwmc7PT RmI02lKo61J/YQvoWq6LwJ0= X-Google-Smtp-Source: ABdhPJzXaV675n2DfDHG1S32dUgL3bTc/YEy+K40RwUoByCs65CoLTJ/Cd4bBQsty9S7vmVcDOiOfA== X-Received: by 2002:ac8:670f:: with SMTP id e15mr20699929qtp.291.1623153207187; Tue, 08 Jun 2021 04:53:27 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:26 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 07/14] mm/damon/vaddr,paddr: Support pageout prioritization Date: Tue, 8 Jun 2021 11:52:47 +0000 Message-Id: <20210608115254.11930-8-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="Z5y/g8TD"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.169 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: qtgaeyghkkek6abn18uixzgeiomu97hw X-Rspamd-Queue-Id: 10574A0001CF X-HE-Tag: 1623153204-645316 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit makes the default monitoring primitives for virtual address spaces and the physical address sapce to support memory regions prioritization for 'PAGEOUT' DAMOS action. It calculates hotness of each region as weighted sum of 'nr_accesses' and 'age' of the region and get the priority score as reverse of the hotness, so that cold regions can be paged out first. Signed-off-by: SeongJae Park --- include/linux/damon.h | 4 ++++ mm/damon/paddr.c | 14 +++++++++++++ mm/damon/prmtv-common.c | 46 +++++++++++++++++++++++++++++++++++++++++ mm/damon/prmtv-common.h | 3 +++ mm/damon/vaddr.c | 15 ++++++++++++++ 5 files changed, 82 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index 8f35bd94fc2b..565f49d8ba44 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -428,6 +428,8 @@ bool damon_va_target_valid(void *t); void damon_va_cleanup(struct damon_ctx *ctx); int damon_va_apply_scheme(struct damon_ctx *context, struct damon_target *t, struct damon_region *r, struct damos *scheme); +int damon_va_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme); void damon_va_set_primitives(struct damon_ctx *ctx); #endif /* CONFIG_DAMON_VADDR */ @@ -438,6 +440,8 @@ void damon_va_set_primitives(struct damon_ctx *ctx); void damon_pa_prepare_access_checks(struct damon_ctx *ctx); unsigned int damon_pa_check_accesses(struct damon_ctx *ctx); bool damon_pa_target_valid(void *t); +int damon_pa_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme); void damon_pa_set_primitives(struct damon_ctx *ctx); #endif /* CONFIG_DAMON_PADDR */ diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 303db372e53b..99a579e8d046 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -121,6 +121,19 @@ int damon_pa_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, return 0; } +int damon_pa_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + switch (scheme->action) { + case DAMOS_PAGEOUT: + return damon_pageout_score(context, r, scheme); + default: + break; + } + + return DAMOS_MAX_SCORE; +} + void damon_pa_set_primitives(struct damon_ctx *ctx) { ctx->primitive.init = NULL; @@ -131,4 +144,5 @@ void damon_pa_set_primitives(struct damon_ctx *ctx) ctx->primitive.target_valid = damon_pa_target_valid; ctx->primitive.cleanup = NULL; ctx->primitive.apply_scheme = damon_pa_apply_scheme; + ctx->primitive.get_scheme_score = damon_pa_scheme_score; } diff --git a/mm/damon/prmtv-common.c b/mm/damon/prmtv-common.c index 01c1c1b37859..ca637a2bf7d8 100644 --- a/mm/damon/prmtv-common.c +++ b/mm/damon/prmtv-common.c @@ -236,3 +236,49 @@ bool damon_pa_young(unsigned long paddr, unsigned long *page_sz) *page_sz = result.page_sz; return result.accessed; } + +#define DAMON_MAX_SUBSCORE (100) +#define DAMON_MAX_AGE_IN_LOG (32) + +int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s) +{ + unsigned int max_nr_accesses; + int freq_subscore; + unsigned int age_in_sec; + int age_in_log, age_subscore; + unsigned int freq_weight = s->limit.weight_nr_accesses; + unsigned int age_weight = s->limit.weight_age; + int hotness; + + max_nr_accesses = c->aggr_interval / c->sample_interval; + freq_subscore = r->nr_accesses * DAMON_MAX_SUBSCORE / max_nr_accesses; + + age_in_sec = (unsigned long)r->age * c->aggr_interval / 1000000; + for (age_in_log = 0; age_in_log < DAMON_MAX_AGE_IN_LOG && age_in_sec; + age_in_log++, age_in_sec >>= 1) + ; + + /* If frequency is 0, higher age means it's colder */ + if (freq_subscore == 0) + age_in_log *= -1; + + /* + * Now age_in_log is in [-DAMON_MAX_AGE_IN_LOG, DAMON_MAX_AGE_IN_LOG]. + * Scale it to be in [0, 100] and set it as age subscore. + */ + age_in_log += DAMON_MAX_AGE_IN_LOG; + age_subscore = age_in_log * DAMON_MAX_SUBSCORE / + DAMON_MAX_AGE_IN_LOG / 2; + + hotness = (freq_weight * freq_subscore + age_weight * age_subscore); + if (freq_weight + age_weight) + hotness /= freq_weight + age_weight; + /* + * Transform it to fit in [0, DAMOS_MAX_SCORE] + */ + hotness = hotness * DAMOS_MAX_SCORE / DAMON_MAX_SUBSCORE; + + /* Return coldness of the region */ + return DAMOS_MAX_SCORE - hotness; +} diff --git a/mm/damon/prmtv-common.h b/mm/damon/prmtv-common.h index ba0c4eecbb79..b27c4e94917e 100644 --- a/mm/damon/prmtv-common.h +++ b/mm/damon/prmtv-common.h @@ -26,3 +26,6 @@ bool damon_va_young(struct mm_struct *mm, unsigned long addr, void damon_pa_mkold(unsigned long paddr); bool damon_pa_young(unsigned long paddr, unsigned long *page_sz); + +int damon_pageout_score(struct damon_ctx *c, struct damon_region *r, + struct damos *s); diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index de54ca70955d..cc70991076be 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -525,6 +525,20 @@ int damon_va_apply_scheme(struct damon_ctx *ctx, struct damon_target *t, return damos_madvise(t, r, madv_action); } +int damon_va_scheme_score(struct damon_ctx *context, struct damon_target *t, + struct damon_region *r, struct damos *scheme) +{ + + switch (scheme->action) { + case DAMOS_PAGEOUT: + return damon_pageout_score(context, r, scheme); + default: + break; + } + + return DAMOS_MAX_SCORE; +} + void damon_va_set_primitives(struct damon_ctx *ctx) { ctx->primitive.init = damon_va_init; @@ -535,6 +549,7 @@ void damon_va_set_primitives(struct damon_ctx *ctx) ctx->primitive.target_valid = damon_va_target_valid; ctx->primitive.cleanup = damon_va_cleanup; ctx->primitive.apply_scheme = damon_va_apply_scheme; + ctx->primitive.get_scheme_score = damon_va_scheme_score; } #include "vaddr-test.h" From patchwork Tue Jun 8 11:52:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96AA8C47082 for ; Tue, 8 Jun 2021 11:53:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4BE126128D for ; Tue, 8 Jun 2021 11:53:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BE126128D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E31076B0078; Tue, 8 Jun 2021 07:53:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E06556B007B; Tue, 8 Jun 2021 07:53:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0D396B007D; Tue, 8 Jun 2021 07:53:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 864C36B0078 for ; Tue, 8 Jun 2021 07:53:30 -0400 (EDT) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 2D10E180AD807 for ; Tue, 8 Jun 2021 11:53:30 +0000 (UTC) X-FDA: 78230396580.25.C7FE990 Received: from mail-qv1-f47.google.com (mail-qv1-f47.google.com [209.85.219.47]) by imf30.hostedemail.com (Postfix) with ESMTP id 359BAE0004C0 for ; Tue, 8 Jun 2021 11:53:28 +0000 (UTC) Received: by mail-qv1-f47.google.com with SMTP id w4so3137506qvr.11 for ; Tue, 08 Jun 2021 04:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=exn5/9MVsYyepOBbFagm+4+x6JfQKNCd23hgVVQiw28=; b=rW6VO0cYPPeohtegDgnMo56MynBg4r79O6s7fPi4Hdgb4YViiGit5ckAJxLvBxTHKG I8FeR8VtXvr2EsTbwvkB1bkBqhxzyiVvac7rSpDQhZj/P7OOAUfHdMCQkqlAkdbbLgr3 kQFCw79tAR2RAt/tCkDe6g4r0ptta+kQEJ6fYily/2qY5EUGH27vOKm7tarHYyj7spii Yeoje3bC0c7/17CK7Hy2F44baCEKYEdfvrHfIh248bmIgT/c6nc9e4OBf60PVslSTSj2 x08zgiiqh6DG0CuZvXn+j0UXu6G+Q3T8n4pat88EsZSJey77bIC0MZiVQDfFutphW8K2 ySaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=exn5/9MVsYyepOBbFagm+4+x6JfQKNCd23hgVVQiw28=; b=T0A2z6Q8z2xrrZyH4d1QJX9pmux/VQhJS5xvKVCET8FHdDefCHtcJc+lY6KjfRHmLV ZF6ccgZ2Fgy+676dEo/Z15AjLZOeJWZL253TpkKk9QN5+O6Nm4Kuut3nkB1I56r5/4Yc MbzsOudsRKTKopTcwZv5HAYJDMvu+gcbcPAHTTpVOigmiQXNJbWa2GCjTn0ddbaHbg27 ZDLF7qo7m1fkVfALNGGeZR89CC5My2CdE/RlCOjZeJroYXbP9kkNQlqQG2gXtiAbnoOL n9aF2gNKlwJEarpm72Dcq7pkt3iziOAyfKmO3R1zkJF5W4IdqWWBNY06RQLxRNDTrpin 8oZg== X-Gm-Message-State: AOAM532wFVNUcrwodu1XbUBCTMbzQe/ccytEmzm3uYhHrPPkC+8GVV9p HKDwpoXyqSntgYBxKHjBqjA= X-Google-Smtp-Source: ABdhPJywm2xA13cQrhGtHNB/ZXuL7WSc2fz2N2COdktA1EihuHv9PfAyl9AWb4EmC9kEbP0RMKU+Zg== X-Received: by 2002:a0c:a99c:: with SMTP id a28mr22979057qvb.33.1623153209315; Tue, 08 Jun 2021 04:53:29 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:28 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 08/14] mm/damon/dbgfs: Support prioritization weights Date: Tue, 8 Jun 2021 11:52:48 +0000 Message-Id: <20210608115254.11930-9-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=rW6VO0cY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf30.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.47 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: g7tfkwta7fb18acbfrx4sjgf5pt4r47o X-Rspamd-Queue-Id: 359BAE0004C0 X-HE-Tag: 1623153208-594301 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit allows DAMON debugfs interface users set the prioritization weights by putting three more numbers to the 'schemes' file. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index ea6d4fdb57fa..b90287b1e576 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -227,11 +227,14 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) damon_for_each_scheme(s, c) { rc = scnprintf(&buf[written], len - written, - "%lu %lu %u %u %u %u %d %lu %lu %lu %lu\n", + "%lu %lu %u %u %u %u %d %lu %lu %u %u %u %lu %lu\n", s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, s->action, s->limit.sz, s->limit.ms, + s->limit.weight_sz, + s->limit.weight_nr_accesses, + s->limit.weight_age, s->stat_count, s->stat_sz); if (!rc) return -ENOMEM; @@ -313,11 +316,14 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_speed_limit limit = {}; - ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu%n", + ret = sscanf(&str[pos], + "%lu %lu %u %u %u %u %u %lu %lu %u %u %u%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, &min_age, &max_age, &action, &limit.sz, - &limit.ms, &parsed); - if (ret != 9) + &limit.ms, &limit.weight_sz, + &limit.weight_nr_accesses, &limit.weight_age, + &parsed); + if (ret != 12) break; if (!damos_action_valid(action)) { pr_err("wrong action %d\n", action); @@ -1141,7 +1147,7 @@ static ssize_t dbgfs_monitor_on_write(struct file *file, static ssize_t dbgfs_version_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { - return simple_read_from_buffer(buf, count, ppos, "1\n", 2); + return simple_read_from_buffer(buf, count, ppos, "2\n", 2); } static const struct file_operations mk_contexts_fops = { From patchwork Tue Jun 8 11:52:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E17B9C47082 for ; Tue, 8 Jun 2021 11:53:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9546C6124B for ; Tue, 8 Jun 2021 11:53:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9546C6124B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 35AC66B007B; Tue, 8 Jun 2021 07:53:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2BC9A6B007D; Tue, 8 Jun 2021 07:53:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09AD56B007E; Tue, 8 Jun 2021 07:53:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0194.hostedemail.com [216.40.44.194]) by kanga.kvack.org (Postfix) with ESMTP id CCF936B007B for ; Tue, 8 Jun 2021 07:53:32 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 65CA298AB for ; Tue, 8 Jun 2021 11:53:32 +0000 (UTC) X-FDA: 78230396664.34.F008F81 Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf26.hostedemail.com (Postfix) with ESMTP id 51B5E40002E4 for ; Tue, 8 Jun 2021 11:53:30 +0000 (UTC) Received: by mail-qk1-f169.google.com with SMTP id f70so4608222qke.13 for ; Tue, 08 Jun 2021 04:53:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=D58QvjfAEUuFa2sQo4m4/NULizMsxON2k/Lqjog3pKs=; b=ShuEUzDpgEGR2MxV12KhZZRzCcu1EdFYf3mzV5BCeU1K6sktfwuBpq3jXPok+eJYfP Wag9ue5iMXqj96twwC5Pp1lfV6Wo7cFpfxfknl83s4ZsoRm4+00TbMdo6eXh73DZY4xv RyBF/QOH79FLuYYiwrROMm0FbnB47S69UIkNKePmXmjc86jwCM5Aj+BIcZAWYYkOHwt/ oAbrRlBqpNsnWodJXJMVE4Nkgin/tg49WmLvzSVkhkourZ8PqVS+A6O44PtJE04+P/Si W81wvhAAJp2es8cdX5Q+MS7G91Tm26cX+N3TXWt/aKWggARqaMQ4XIfz0vZrBpdExjwt 165Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=D58QvjfAEUuFa2sQo4m4/NULizMsxON2k/Lqjog3pKs=; b=dwyd9j+1OSCNhCpxq5p4y04d7k7/XgwdISp/TAj9BBpVI4tAKDd8D9R3VdimSRCUh+ 05Q3RhLiLSLAF7iBlohpp820AhZXzWiha3xrvrbF2rkpOiwSde82fZ3cZAspFS5/QbLm 6JDxTOSLtMvnh9YrcjQ2HuzmiYOvwlOAOuIVNv62oM5s3LYhsGABUMFjj0cZMihA7mot 0vZ9/8JzpmpPYhJyWDxVkQAR1PeDz4w3VAvBVSskbs3U6JmRj9N3VFbC0bheeEI7hGwp frojd3uyLYfXACbZ4OHeq0lFsQ9Cg0xp2BqEasLcdZKKE9mXNZd5EOW0GKxB9rms8siJ bxvw== X-Gm-Message-State: AOAM531Sh/iRafF+/b735crgmzMI/8ChfGmc2i0dT+4yigMN1LJsLjmI KF2yOOw2ArcXiFuuQTEpfwE= X-Google-Smtp-Source: ABdhPJySXLhnY1lAPd5sDJ+FjzYt3zkT5ln+T3NwujiPhhhQ8CUb3OuZ6Hz97KH4ZYFpjvEbU5PK9w== X-Received: by 2002:a37:43cd:: with SMTP id q196mr426583qka.486.1623153211595; Tue, 08 Jun 2021 04:53:31 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:31 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 09/14] tools/selftests/damon: Update for regions prioritization of schemes Date: Tue, 8 Jun 2021 11:52:49 +0000 Message-Id: <20210608115254.11930-10-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=ShuEUzDp; spf=pass (imf26.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.169 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: jbi69tgtfyursscuforeibosttxmadn6 X-Rspamd-Queue-Id: 51B5E40002E4 X-Rspamd-Server: rspam06 X-HE-Tag: 1623153210-609586 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates the DAMON selftests for 'schemes' debugfs file, as the file format is updated. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/debugfs_attrs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh index 012b0c1fdbd3..262034d8efa5 100755 --- a/tools/testing/selftests/damon/debugfs_attrs.sh +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -101,7 +101,7 @@ echo $ORIG_CONTENT > $file file="$DBGFS/schemes" ORIG_CONTENT=$(cat $file) -echo "1 2 3 4 5 6 3 0 0" > $file +echo "1 2 3 4 5 6 3 0 0 1 2 3" > $file if [ $? -ne 0 ] then echo "$file write fail" @@ -110,7 +110,7 @@ then fi echo "1 2 -3 4 5 6 3 0 0" > $file +3 4 5 6 3 0 0 1 2 3" > $file if [ $? -eq 0 ] then echo "$file multi line write success (expected fail)" From patchwork Tue Jun 8 11:52:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37ECEC47082 for ; Tue, 8 Jun 2021 11:53:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DEF326128D for ; Tue, 8 Jun 2021 11:53:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DEF326128D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7EF666B007D; Tue, 8 Jun 2021 07:53:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 750106B007E; Tue, 8 Jun 2021 07:53:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57DD26B0080; Tue, 8 Jun 2021 07:53:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id 25A376B007D for ; Tue, 8 Jun 2021 07:53:35 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B8873181AEF1F for ; Tue, 8 Jun 2021 11:53:34 +0000 (UTC) X-FDA: 78230396748.17.559B978 Received: from mail-qv1-f53.google.com (mail-qv1-f53.google.com [209.85.219.53]) by imf07.hostedemail.com (Postfix) with ESMTP id F3CF1A000151 for ; Tue, 8 Jun 2021 11:53:30 +0000 (UTC) Received: by mail-qv1-f53.google.com with SMTP id u14so7675825qvq.6 for ; Tue, 08 Jun 2021 04:53:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=WVonKJ4i5Mlrmmds3qLXuhbIENuPVaS0OBd/5Gs8AtA=; b=EtGtB2ASZxNFwo6uJCfR+SVHMMDFBUSjSDeogPB22aA4Hn1nTbYYcpyFg/2p1w6ERB itebYa0De9syGtSRM39U6UwNhoDo3arFpRqKYM5Z+360VypubR7yRR+DFBJgLdJT7BDW zpFpjPGclqC+p2OhduAYWQUepFjscecAXBkci76RR0XbRQRZ0d+FBNGaEd2tU7xciDCn qDBjHg+n1ksx2pKk85yTqcaACDgkwqO0wc/zyEjEYwgC1eQDu/k4cB5DUZ9KtdHpR2bt b0Z4DjXWopNfYuIK+/FDX13bhk3uuLj5USuUKzsFEPNgonXQvRndrGbiPG2a1PLgva8v i9GA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=WVonKJ4i5Mlrmmds3qLXuhbIENuPVaS0OBd/5Gs8AtA=; b=lqvZu7WASvxMlzQ4KSSp2dTHP9OY+KtZDDvMMF8l7hgu0RCUC9aVO9oEdTXwrNBBp/ 1I86mhYH1un91mNSqCnLLRbT5Etx/qWZ0XMgiO1FEpSBhn13gAHPGCcMdlxH5BhVEr68 QZV+bLSTvu3PCYWNZxvljAqOaZInD42O44zaDZ73KRAaD6pdKn4cvKPIdsla6d4j7zdR 8A5cFbVxOjYgY5B7aTkDGtbwzGqLX1ESr+ycYL27bKbAj75sjnHkmUcNZRovwaEjvAam SUe7f8GxYb4V6fwisXG0hHwxoWhwropA48BrS3hiiBgrt5Q5E/Ne4Imucv9pE4MTIFRh Tojg== X-Gm-Message-State: AOAM5318praSxEMwBsSBt/UqE+tK9WWwqieWzkWoqO/5+YD4uBsiXIn6 y++JVSimq337eBB6PvZr5Ms= X-Google-Smtp-Source: ABdhPJxVwq/GfwJZCOSdbHXBj13HB+PhiYpZM43LqrxUExQowmLT9j5y33BSt64hZjEkLnDZHo/+PQ== X-Received: by 2002:ad4:4084:: with SMTP id l4mr18117248qvp.37.1623153213794; Tue, 08 Jun 2021 04:53:33 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:33 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 10/14] mm/damon/schemes: Activate schemes based on a watermarks mechanism Date: Tue, 8 Jun 2021 11:52:50 +0000 Message-Id: <20210608115254.11930-11-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=EtGtB2AS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf07.hostedemail.com: domain of sj38park@gmail.com designates 209.85.219.53 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Stat-Signature: uhrkdqtctyq5kx594tusgiyx19qm54og X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F3CF1A000151 X-HE-Tag: 1623153210-905765 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park DAMON-based operation schemes need to be manually turned on and off. In some use cases, however, the condition for turning a scheme on and off would depend on the system's situation. For example, schemes for proactive pages reclamation would need to be turned on when some memory pressure is detected, and turned off when the system has enough free memory. For easier control of schemes activation based on the system situation, this commit introduces a watermarks-based mechanism. The client can describe the watermark metric (e.g., amount of free memory in the system), watermark check interval, and three watermarks, namely high, mid, and low. If the scheme is deactivated, it only gets the metric and compare that to the three watermarks for every check interval. If the metric is higher than the high watermark, the scheme is deactivated. If the metric is between the mid watermark and the low watermark, the scheme is activated. If the metric is lower than the low watermark, the scheme is deactivated again. This is to allow users fall back to traditional page-granularity mechanisms. Signed-off-by: SeongJae Park --- include/linux/damon.h | 52 ++++++++++++++++++++++- mm/damon/core.c | 98 ++++++++++++++++++++++++++++++++++++++++++- mm/damon/dbgfs.c | 5 ++- 3 files changed, 152 insertions(+), 3 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 565f49d8ba44..2edd84e98056 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -127,6 +127,45 @@ struct damos_speed_limit { unsigned int min_score; }; +/** + * enum damos_wmark_metric - Represents the watermark metric. + * + * @DAMOS_WMARK_NONE: Ignore the watermarks of the given scheme. + * @DAMOS_WMARK_FREE_MEM_RATE: Free memory rate of the system in [0,1000]. + */ +enum damos_wmark_metric { + DAMOS_WMARK_NONE, + DAMOS_WMARK_FREE_MEM_RATE, +}; + +/** + * struct damos_watermarks - Controls when a given scheme should be activated. + * @metric: Metric for the watermarks. + * @interval: Watermarks check time interval in microseconds. + * @high: High watermark. + * @mid: Middle watermark. + * @low: Low watermark. + * + * If &metric is &DAMOS_WMARK_NONE, the scheme is always active. Being active + * means DAMON does monitoring and applying the action of the scheme to + * appropriate memory regions. Else, DAMON checks &metric of the system for at + * least every &interval microseconds and works as below. + * + * If &metric is higher than &high, the scheme is inactivated. If &metric is + * between &mid and &low, the scheme is activated. If &metric is lower than + * &low, the scheme is inactivated. + */ +struct damos_watermarks { + enum damos_wmark_metric metric; + unsigned long interval; + unsigned long high; + unsigned long mid; + unsigned long low; + +/* private: */ + bool activated; +}; + /** * struct damos - Represents a Data Access Monitoring-based Operation Scheme. * @min_sz_region: Minimum size of target regions. @@ -137,6 +176,7 @@ struct damos_speed_limit { * @max_age_region: Maximum age of target regions. * @action: &damo_action to be applied to the target regions. * @limit: Control the aggressiveness of this scheme. + * @wmarks: Watermarks for automated (in)activation of this scheme. * @stat_count: Total number of regions that this scheme is applied. * @stat_sz: Total size of regions that this scheme is applied. * @list: List head for siblings. @@ -146,6 +186,14 @@ struct damos_speed_limit { * &max_nr_accesses, &min_age_region, &max_age_region) and applies &action to * those. To avoid consuming too much CPU for the &action, &limit is used. * + * To do the work only when needed, schemes can be activated for specific + * system situations using &wmarks. If all schemes that registered to the + * monitoring context are inactive, DAMON stops monitoring either, and just + * repeatedly checks the watermarks. + * + * If all schemes that registered to a &struct damon_ctx are inactive, DAMON + * stops monitoring and just repeatedly checks the watermarks. + * * After applying the &action to each region, &stat_count and &stat_sz is * updated to reflect the number of regions and total size of regions that the * &action is applied. @@ -159,6 +207,7 @@ struct damos { unsigned int max_age_region; enum damos_action action; struct damos_speed_limit limit; + struct damos_watermarks wmarks; unsigned long stat_count; unsigned long stat_sz; struct list_head list; @@ -391,7 +440,8 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action, struct damos_speed_limit *limit); + enum damos_action action, struct damos_speed_limit *limit, + struct damos_watermarks *wmarks); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index 15bcd05670d1..a754dd1f539d 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -86,7 +87,8 @@ struct damos *damon_new_scheme( unsigned long min_sz_region, unsigned long max_sz_region, unsigned int min_nr_accesses, unsigned int max_nr_accesses, unsigned int min_age_region, unsigned int max_age_region, - enum damos_action action, struct damos_speed_limit *limit) + enum damos_action action, struct damos_speed_limit *limit, + struct damos_watermarks *wmarks) { struct damos *scheme; @@ -113,6 +115,14 @@ struct damos *damon_new_scheme( scheme->limit.charged_from = 0; scheme->limit.charge_target_from = NULL; scheme->limit.charge_addr_from = 0; + + scheme->wmarks.metric = wmarks->metric; + scheme->wmarks.interval = wmarks->interval; + scheme->wmarks.high = wmarks->high; + scheme->wmarks.mid = wmarks->mid; + scheme->wmarks.low = wmarks->low; + scheme->wmarks.activated = true; + return scheme; } @@ -581,6 +591,9 @@ static void damon_do_apply_schemes(struct damon_ctx *c, damon_for_each_scheme(s, c) { struct damos_speed_limit *limit = &s->limit; + if (!s->wmarks.activated) + continue; + /* Check the limit */ if (limit->sz && limit->charged_sz >= limit->sz) continue; @@ -637,6 +650,9 @@ static void kdamond_apply_schemes(struct damon_ctx *c) unsigned long cumulated_sz; unsigned int score, max_score = 0; + if (!s->wmarks.activated) + continue; + if (!limit->sz) continue; @@ -876,6 +892,83 @@ static bool kdamond_need_stop(struct damon_ctx *ctx) return true; } +static unsigned long damos_wmark_metric_value(enum damos_wmark_metric metric) +{ + struct sysinfo i; + + switch (metric) { + case DAMOS_WMARK_FREE_MEM_RATE: + si_meminfo(&i); + return i.freeram * 1000 / i.totalram; + default: + break; + } + return -EINVAL; +} + +/* + * Returns zero if the scheme is active. Else, returns time to wait for next + * watermark check in micro-seconds. + */ +static unsigned long damos_wmark_wait_us(struct damos *scheme) +{ + unsigned long metric; + + if (scheme->wmarks.metric == DAMOS_WMARK_NONE) + return 0; + + metric = damos_wmark_metric_value(scheme->wmarks.metric); + /* higher than high watermark or lower than low watermark */ + if (metric > scheme->wmarks.high || scheme->wmarks.low > metric) { + if (scheme->wmarks.activated) + pr_info("inactivate a scheme (%d) for %s wmark\n", + scheme->action, + metric > scheme->wmarks.high ? + "high" : "low"); + scheme->wmarks.activated = false; + return scheme->wmarks.interval; + } + + /* inactive and higher than middle watermark */ + if ((scheme->wmarks.high >= metric && metric >= scheme->wmarks.mid) && + !scheme->wmarks.activated) + return scheme->wmarks.interval; + + if (!scheme->wmarks.activated) + pr_info("activate a scheme (%d)\n", scheme->action); + scheme->wmarks.activated = true; + return 0; +} + +static void kdamond_usleep(unsigned long usecs) +{ + if (usecs > 100 * 1000) + schedule_timeout_interruptible(usecs_to_jiffies(usecs)); + else + usleep_range(usecs, usecs + 1); +} + +/* Returns negative error code if it's not activated but should return */ +static int kdamond_wait_activation(struct damon_ctx *ctx) +{ + struct damos *s; + unsigned long wait_time; + unsigned long min_wait_time = 0; + + while (!kdamond_need_stop(ctx)) { + damon_for_each_scheme(s, ctx) { + wait_time = damos_wmark_wait_us(s); + if (!min_wait_time || wait_time < min_wait_time) + min_wait_time = wait_time; + } + if (!min_wait_time) + return 0; + + kdamond_usleep(min_wait_time); + } + return -EBUSY; +} + static void set_kdamond_stop(struct damon_ctx *ctx) { mutex_lock(&ctx->kdamond_lock); @@ -904,6 +997,9 @@ static int kdamond_fn(void *data) sz_limit = damon_region_sz_limit(ctx); while (!kdamond_need_stop(ctx)) { + if (kdamond_wait_activation(ctx)) + continue; + if (ctx->primitive.prepare_access_checks) ctx->primitive.prepare_access_checks(ctx); if (ctx->callback.after_sampling && diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index b90287b1e576..1680fb1be8e1 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -315,6 +315,9 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, *nr_schemes = 0; while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_speed_limit limit = {}; + struct damos_watermarks wmarks = { + .metric = DAMOS_WMARK_NONE, + }; ret = sscanf(&str[pos], "%lu %lu %u %u %u %u %u %lu %lu %u %u %u%n", @@ -332,7 +335,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(min_sz, max_sz, min_nr_a, max_nr_a, - min_age, max_age, action, &limit); + min_age, max_age, action, &limit, &wmarks); if (!scheme) goto fail; From patchwork Tue Jun 8 11:52:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81DE6C4743D for ; Tue, 8 Jun 2021 11:53:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 306CF6128D for ; Tue, 8 Jun 2021 11:53:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 306CF6128D Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CAD2C6B007E; Tue, 8 Jun 2021 07:53:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C5B9E6B0080; Tue, 8 Jun 2021 07:53:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD3306B0081; Tue, 8 Jun 2021 07:53:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id 77C436B007E for ; Tue, 8 Jun 2021 07:53:37 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 89FC798AB for ; Tue, 8 Jun 2021 11:53:36 +0000 (UTC) X-FDA: 78230396832.30.5B4BC10 Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf21.hostedemail.com (Postfix) with ESMTP id EAC57E000254 for ; Tue, 8 Jun 2021 11:53:31 +0000 (UTC) Received: by mail-qk1-f169.google.com with SMTP id k11so18189586qkk.1 for ; Tue, 08 Jun 2021 04:53:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nfDK/Bf/e4nvbxsoi7WegXjXDWkjyIFNxz/IfnwgRq4=; b=S44HjXvS2x2lWj9hdsU9WLr8ZmLz0+lYh2oBquC7n8yJ/GzJEXet10ZLvg+3L6YiBl yDtpC7bRtYeB0FsFOUs1+a5WfGCsnsmvazHjYqTlXKNXT+8a6Dmn4kyrKa9zkCeWbtqq krjoTyaA60Nxiru4lrNr/WljhplRXYVo7LyTI0/d2kquUgBxHy4C+Bsa/LoJOBfDZykZ hkwdmS/KPpYq6IXAyrcPajFw69paO8z0SYNwX9YBKA2vSgcsGpgZAcXj0Brlph5NjLzy D4z4aZRSPVR2EDXnpfNggQUhnt7jROenE8OUOJeq4a2KfZtEZQLxi73rSgDwqRN5VRKI uWXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nfDK/Bf/e4nvbxsoi7WegXjXDWkjyIFNxz/IfnwgRq4=; b=BiLcmSEHRjlxd7TuFHQnQ45pA+2MbvcHs7iUmbkViEOaBEBCxuyRDvnQWyPX5cgD6V MZHRysfR7p6/wYGVdxS10WCSSZfqiAEaOO7Semedke9NIdHdC+r6ZfU2MnlZ0lk/KpOB XBAbnvDLrFmmNsYKsNaQaKTYbDHxtwGVzDG+qR9FUQfUBaYO8O+JVyst7o2phzuqqIz7 Z4V02F9iWS77lpVWboX0DRIetiqXQO0I4j5Q7NlUaq/0PEmxkd04Uyojd4pA4KHMkTwd sgyuKNls2zp+sTG8U5FNwz0FAT5RHwWmMuiuSLbwy8stUJ4/u9vmC9sxvOHfRDEr6ttT UDZQ== X-Gm-Message-State: AOAM532/UNXr0o+ush9XCk6pDeQSz617oRP28Q6hXuDBn+9fVFjypWsy bxo6Yej/K1hDPxFNeHHevdY= X-Google-Smtp-Source: ABdhPJzOyu/UnvRdGVglHVJsejFgTwP8+46Uca/ivoGXirXLN/SSCSYd/i426/NRJMQcb02HTw+ddw== X-Received: by 2002:a37:7f82:: with SMTP id a124mr20833558qkd.419.1623153215611; Tue, 08 Jun 2021 04:53:35 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:35 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 11/14] mm/damon/dbgfs: Support watermarks Date: Tue, 8 Jun 2021 11:52:51 +0000 Message-Id: <20210608115254.11930-12-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=S44HjXvS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf21.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.169 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: onbf3ej6dhg99twoqy7i7bsem4q185rs X-Rspamd-Queue-Id: EAC57E000254 X-HE-Tag: 1623153211-326054 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates DAMON debugfs interface to support the watermarks based schemes activation. For this, now 'schemes' file receives five more values. Signed-off-by: SeongJae Park --- mm/damon/dbgfs.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 1680fb1be8e1..768ef3eb9550 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -227,7 +227,7 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) damon_for_each_scheme(s, c) { rc = scnprintf(&buf[written], len - written, - "%lu %lu %u %u %u %u %d %lu %lu %u %u %u %lu %lu\n", + "%lu %lu %u %u %u %u %d %lu %lu %u %u %u %d %lu %lu %lu %lu %lu %lu\n", s->min_sz_region, s->max_sz_region, s->min_nr_accesses, s->max_nr_accesses, s->min_age_region, s->max_age_region, @@ -235,6 +235,8 @@ static ssize_t sprint_schemes(struct damon_ctx *c, char *buf, ssize_t len) s->limit.weight_sz, s->limit.weight_nr_accesses, s->limit.weight_age, + s->wmarks.metric, s->wmarks.interval, + s->wmarks.high, s->wmarks.mid, s->wmarks.low, s->stat_count, s->stat_sz); if (!rc) return -ENOMEM; @@ -315,18 +317,18 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, *nr_schemes = 0; while (pos < len && *nr_schemes < max_nr_schemes) { struct damos_speed_limit limit = {}; - struct damos_watermarks wmarks = { - .metric = DAMOS_WMARK_NONE, - }; + struct damos_watermarks wmarks; ret = sscanf(&str[pos], - "%lu %lu %u %u %u %u %u %lu %lu %u %u %u%n", + "%lu %lu %u %u %u %u %u %lu %lu %u %u %u %u %lu %lu %lu %lu%n", &min_sz, &max_sz, &min_nr_a, &max_nr_a, &min_age, &max_age, &action, &limit.sz, &limit.ms, &limit.weight_sz, &limit.weight_nr_accesses, &limit.weight_age, + &wmarks.metric, &wmarks.interval, + &wmarks.high, &wmarks.mid, &wmarks.low, &parsed); - if (ret != 12) + if (ret != 17) break; if (!damos_action_valid(action)) { pr_err("wrong action %d\n", action); From patchwork Tue Jun 8 11:52:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C540C48BCF for ; Tue, 8 Jun 2021 11:53:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D28626124B for ; Tue, 8 Jun 2021 11:53:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D28626124B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 57F0A6B0080; Tue, 8 Jun 2021 07:53:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 556616B0081; Tue, 8 Jun 2021 07:53:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3A7DD6B0082; Tue, 8 Jun 2021 07:53:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0220.hostedemail.com [216.40.44.220]) by kanga.kvack.org (Postfix) with ESMTP id 0B9B96B0080 for ; Tue, 8 Jun 2021 07:53:38 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 99F2112E8 for ; Tue, 8 Jun 2021 11:53:38 +0000 (UTC) X-FDA: 78230396916.26.84E057E Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) by imf25.hostedemail.com (Postfix) with ESMTP id 985386000143 for ; Tue, 8 Jun 2021 11:53:35 +0000 (UTC) Received: by mail-qk1-f172.google.com with SMTP id i68so16231554qke.3 for ; Tue, 08 Jun 2021 04:53:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=AiQc0DD6N2aSgRO8sD4OJ1ER+4IqRjuDkiQMrkgaEeg=; b=k5//IFbGlIuVgYSa6FzehkG51crKgMNwddzs48OKNhdq41KBFjeAc+d7IqqQanR+0a DKyHTYSqwqOVCjiW+0JrTEJe4UI1VVQxU3hEaZJaD6Y2QVZexa/U0m14kDJ0B2M/Q5E/ PSFiG/H3D3E8jZX+0LrBj147VcDHTkjLwXBv6e18EhssO1JyUiaGZddqu/ucyXDZE/Sc Dt0fSbVBy3JQp3tHob9U22nwtWx7uAtEtHu6eUudqNySnSwYq0jgAWMn48oz/b6r0EnT Upae5mE3BZS4haahHto74ofLD2O3ePKTvj2OCusuWHSSp4R0rt8pCqCTSMtFtlH0DVJw PG6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=AiQc0DD6N2aSgRO8sD4OJ1ER+4IqRjuDkiQMrkgaEeg=; b=mUfeuXllbG+GvjfdzDmCdRS4AUiQ2GcElHmxgJ60jnLZjP/Q6c73LC2PKJICUkjEIA zmd9mU3vp7xYZLJSwAeBwybhOr59c8QiGvdxc/1GDM864Tyx+zmBYn/rcpog41qn7BiY GOpPUG2/zRC5Kd0BJ31IJ26YouNRiscb2fDbqyPco0rCYn7Wz8Fa6nZj/uWASG17B1/T iEwHQpzC0Wt1EZNor2qbPNq4DudtUR6+r8+0QI/wWD2+XTEsl2MdBGprfnrTOSPoODn0 hjgp5sSO4nKeejQc2L6JNEQ/dIyp7tceq8/gcK2/b32PhyEDbgp9NGOWqBNXPwXnws1K p97g== X-Gm-Message-State: AOAM531s6+3VWVdCkW10PDohU3VCDAVrvt5YgsNMqUs7nQUC0gwwwozo GIbBS+hxUJRfMo/C9Hprud8= X-Google-Smtp-Source: ABdhPJxDkNGIe9IynLDSfOi1J0bg0wZyaqtIp1Vv08ZHqor0JrhzLD/Dfg735w4QdgnUypSVRdqjsQ== X-Received: by 2002:a37:a143:: with SMTP id k64mr19898791qke.356.1623153217698; Tue, 08 Jun 2021 04:53:37 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:37 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 12/14] selftests/damon: Support watermarks Date: Tue, 8 Jun 2021 11:52:52 +0000 Message-Id: <20210608115254.11930-13-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 985386000143 Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b="k5//IFbG"; spf=pass (imf25.hostedemail.com: domain of sj38park@gmail.com designates 209.85.222.172 as permitted sender) smtp.mailfrom=sj38park@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Stat-Signature: i14gg8yejeajpfzupaemyducth6wmnyp X-HE-Tag: 1623153215-893900 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit updates DAMON selftests for 'schemes' debugfs file to reflect the changes in the format. Signed-off-by: SeongJae Park --- tools/testing/selftests/damon/debugfs_attrs.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh index 262034d8efa5..90440cb3aee8 100755 --- a/tools/testing/selftests/damon/debugfs_attrs.sh +++ b/tools/testing/selftests/damon/debugfs_attrs.sh @@ -101,7 +101,7 @@ echo $ORIG_CONTENT > $file file="$DBGFS/schemes" ORIG_CONTENT=$(cat $file) -echo "1 2 3 4 5 6 3 0 0 1 2 3" > $file +echo "1 2 3 4 5 6 3 0 0 1 2 3 1 100 3 2 1" > $file if [ $? -ne 0 ] then echo "$file write fail" @@ -110,7 +110,7 @@ then fi echo "1 2 -3 4 5 6 3 0 0 1 2 3" > $file +3 4 5 6 3 0 0 1 2 3 1 100 3 2 1" > $file if [ $? -eq 0 ] then echo "$file multi line write success (expected fail)" From patchwork Tue Jun 8 11:52:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 999BBC47082 for ; Tue, 8 Jun 2021 11:53:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 38E106124B for ; Tue, 8 Jun 2021 11:53:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 38E106124B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D20FD6B0081; Tue, 8 Jun 2021 07:53:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA8416B0082; Tue, 8 Jun 2021 07:53:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A39C56B0083; Tue, 8 Jun 2021 07:53:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id 727E16B0081 for ; Tue, 8 Jun 2021 07:53:41 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 13713181AEF15 for ; Tue, 8 Jun 2021 11:53:41 +0000 (UTC) X-FDA: 78230397042.02.04703C5 Received: from mail-qt1-f170.google.com (mail-qt1-f170.google.com [209.85.160.170]) by imf02.hostedemail.com (Postfix) with ESMTP id 3070C4202A07 for ; Tue, 8 Jun 2021 11:53:39 +0000 (UTC) Received: by mail-qt1-f170.google.com with SMTP id l17so10853644qtq.12 for ; Tue, 08 Jun 2021 04:53:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=398gdr4VtCF+1sg053gsMKOByn0sHBy2oJE8x1WnkHQ=; b=sWzl+HSYm14Y2JCiHQTZIiBbRP/Go6POhwFNbxLUEizq6jwCNtb3N+7j74uqiMxyCR 9wBui64Z84R94tMoBjjqvV0rwB1pDZhj3A6HOFxWDMVhI5T+If0RBrnusBBACRzm524G acNK8saWVwcWDf4Iw6QN79fEP91pD3TGxje2LFBOPwgMJyemMXw/vICB+RjDfxPvAtAF LvUq9N18SLNc6Ud65gxoa72XWUDFI7rNeiaG1gtRPrUMB3x8kfqKh23bL88q9bWGc19I n96XgsBDlYi4mV7JmcXY69o3ikfsYermw+85KUvrN1ydOdsrNLcd9yjP7dkOkLh8fOGE 0e1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=398gdr4VtCF+1sg053gsMKOByn0sHBy2oJE8x1WnkHQ=; b=PuR1ts9c3clRnUIcRWcx8s+Iw18kBb2GKOfYa6sXc6u41l0j6vkoifIVZ9pKHNYTxm W+J8ab2N3+cKrl2/P88ZXA6wKVAjDgQygM4eqxvJduPNxY2W53Xr2P0yzoG6hNKhNIgh Ca8npIvG3RU2IHGY91smZpVzcTjYIbcMjE9sQH9QE3KjilWNPFTiB38PRjx1US2S+BZD a6biCdog46w0ZACKSiTWrqpNsUB8u4DYlKR1pooHviF4PyNRStCWSZzL/m1RqbnKR9iG qUDfARe5LnHaX/J+bSN4fLoHCYRr1a6Jdbg8HmbhLIFWVlNGcytBL73mlq65oagED1K7 l6lQ== X-Gm-Message-State: AOAM531ZMV85U0R6YIxtSR8DO6si+hpcH/+R2jKz9dIMRTwAhSVF0BCV aLH742qVcMcPE3QzZzZUMYM= X-Google-Smtp-Source: ABdhPJwsJCFqvJJM0lNHOCetRYsho5O5kU6oYNdvKczSP2b0mSKdI9FoPUyKUc0qEPMIrtJzgzcffg== X-Received: by 2002:ac8:59c9:: with SMTP id f9mr20842952qtf.228.1623153219936; Tue, 08 Jun 2021 04:53:39 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:39 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 13/14] mm/damon: Introduce DAMON-based reclamation Date: Tue, 8 Jun 2021 11:52:53 +0000 Message-Id: <20210608115254.11930-14-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=sWzl+HSY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.170 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: 8xbqfpdg4xionwz1phpia1br3knzkyd6 X-Rspamd-Queue-Id: 3070C4202A07 X-HE-Tag: 1623153218-78185 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit implements a new kernel subsystem that finds cold memory regions using DAMON and reclaims those immediately. It is intended to be used as proactive lightweigh reclamation logic for light memory pressure. For heavy memory pressure, it could be inactivated and fall back to the traditional page-scanning based reclamation. It's implemented on top of DAMON framework to use the DAMON-based Operation Schemes (DAMOS) feature. It utilizes all the DAMOS features including speed limit, prioritization, and watermarks. It could be enabled and tuned in build time via the kernel configuration, in boot time via the kernel boot parameter, and in run time via its module parameter ('/sys/module/damon_reclaim/parameters/') interface. Signed-off-by: SeongJae Park --- mm/damon/Kconfig | 12 ++ mm/damon/Makefile | 1 + mm/damon/reclaim.c | 330 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 343 insertions(+) create mode 100644 mm/damon/reclaim.c diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig index eeefb5b633b6..d9cd88810279 100644 --- a/mm/damon/Kconfig +++ b/mm/damon/Kconfig @@ -84,4 +84,16 @@ config DAMON_DBGFS_KUNIT_TEST If unsure, say N. +config DAMON_RECLAIM + bool "Build DAMON-based reclaim (DAMON_RECLAIM)" + depends on DAMON_PADDR + help + This builds the DAMON-based reclamation subsystem. It finds pages + that not accessed for a long time (cold) using DAMON and reclaim + those if enabled. + + This is suggested to be used as a proactive and lightweight + reclamation under light memory pressure, while the traditional page + scanning-based reclamation is used for heavy pressure. + endmenu diff --git a/mm/damon/Makefile b/mm/damon/Makefile index 017799e5670a..39433e7d570c 100644 --- a/mm/damon/Makefile +++ b/mm/damon/Makefile @@ -5,3 +5,4 @@ obj-$(CONFIG_DAMON_VADDR) += prmtv-common.o vaddr.o obj-$(CONFIG_DAMON_PADDR) += prmtv-common.o paddr.o obj-$(CONFIG_DAMON_PGIDLE) += prmtv-common.o pgidle.o obj-$(CONFIG_DAMON_DBGFS) += dbgfs.o +obj-$(CONFIG_DAMON_RECLAIM) += reclaim.o diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c new file mode 100644 index 000000000000..a43cb61f18ff --- /dev/null +++ b/mm/damon/reclaim.c @@ -0,0 +1,330 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DAMON-based page reclamation + * + * Author: SeongJae Park + */ + +#define pr_fmt(fmt) "damon-reclaim: " fmt + +#include +#include +#include +#include +#include + +#ifdef MODULE_PARAM_PREFIX +#undef MODULE_PARAM_PREFIX +#endif +#define MODULE_PARAM_PREFIX "damon_reclaim." + +/* + * Enable or disable DAMON_RECLAIM. + * + * You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. + * Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could + * do no real monitoring and reclamation due to the watermarks-based activation + * condition. Refer to below descriptions for the watermarks parameter for + * this. + */ +static bool enabled __read_mostly; +module_param(enabled, bool, 0600); + +/* + * Time threshold for cold memory regions identification in microseconds. + * + * If a memory region is not accessed for this or longer time, DAMON_RECLAIM + * identifies the region as cold, and reclaims. 5 seconds by default. + */ +static unsigned long min_age __read_mostly = 5000000; +module_param(min_age, ulong, 0600); + +/* + * Maximum bytes of memory that can be reclaimed in a charging window. + * + * DAMON_RECLAIM charges amount of memory which has reclaimed within each + * charging time window and makes no more than this limit is charged. This + * could be useful for limiting CPU consumption of DAMON_RECLAIM. 1 GiB by + * default. + */ +static unsigned long limit_sz __read_mostly = 1024 * 1024 * 1024; +module_param(limit_sz, ulong, 0600); + +/* + * The amount of reclaimed memory charging window in milliseconds. + * + * The charging time window for limit_sz. 1 second by default. + */ +static unsigned long limit_ms __read_mostly = 1000; +module_param(limit_ms, ulong, 0600); + +/* + * The watermarks check time interval in microseconds. + * + * Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is + * enabled but inactive due to its watermarks rule. 5 seconds by default. + */ +static unsigned long wmarks_interval __read_mostly = 5000000; +module_param(wmarks_interval, ulong, 0600); + +/* + * Free memory rate (per thousand) for the high watermark. + * + * If free memory of the system in bytes per thousand bytes is higher than + * this, DAMON_RECLAIM becomes inactive, so it does nothing but periodically + * checks the watermarks. 500 (50%) by default. + */ +static unsigned long wmarks_high __read_mostly = 500; +module_param(wmarks_high, ulong, 0600); + +/* + * Free memory rate (per thousand) for the middle watermark. + * + * If free memory of the system in bytes per thousand bytes is between this and + * the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring + * and the reclaiming. 400 (40%) by default. + */ +static unsigned long wmarks_mid __read_mostly = 400; +module_param(wmarks_mid, ulong, 0600); + +/* + * Free memory rate (per thousand) for the low watermark. + * + * If free memory of the system in bytes per thousand bytes is lower than this, + * DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks + * the watermarks. In the case, the system falls back to the LRU-based page + * granularity reclamation logic. 200 (20%) by default. + */ +static unsigned long wmarks_low __read_mostly = 200; +module_param(wmarks_low, ulong, 0600); + +/* + * Sampling interval for the monitoring in microseconds. + * + * The sampling interval of DAMON for the cold memory monitoring. Please refer + * to the DAMON documentation for more detail. 5 ms by default. + */ +static unsigned long sample_interval __read_mostly = 5000; +module_param(sample_interval, ulong, 0600); + +/* + * Aggregation interval for the monitoring in microseconds. + * + * The aggregation interval of DAMON for the cold memory monitoring. Please + * refer to the DAMON documentation for more detail. 100 ms by default. + */ +static unsigned long aggr_interval __read_mostly = 100000; +module_param(aggr_interval, ulong, 0600); + +/* + * Minimum number of monitoring regions. + * + * The minimal number of monitoring regions of DAMON for the cold memory + * monitoring. This can be used to set lower-bound of the monitoring quality. + * But, setting this too high could result in increased monitoring overhead. + * Please refer to the DAMON documentation for more detail. 10 by default. + */ +static unsigned long min_nr_regions __read_mostly = 10; +module_param(min_nr_regions, ulong, 0600); + +/* + * Maximum number of monitoring regions. + * + * The maximum number of monitoring regions of DAMON for the cold memory + * monitoring. This can be used to set upper-bound of the monitoring overhead. + * However, setting this too low could result in bad monitoring quality. + * Please refer to the DAMON documentation for more detail. 1000 by default. + */ +static unsigned long max_nr_regions __read_mostly = 1000; +module_param(max_nr_regions, ulong, 0600); + +/* + * Start of the target memory region in physical address. + * + * The start physical address of memory region that DAMON_RECLAIM will do work + * against. By default, biggest System RAM is used as the region. + */ +static unsigned long monitor_region_start __read_mostly; +module_param(monitor_region_start, ulong, 0600); + +/* + * End of the target memory region in physical address. + * + * The end physical address of memory region that DAMON_RECLAIM will do work + * against. By default, biggest System RAM is used as the region. + */ +static unsigned long monitor_region_end __read_mostly; +module_param(monitor_region_end, ulong, 0600); + +/* + * PID of the DAMON thread + * + * If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. + * Else, -1. + */ +static int kdamond_pid __read_mostly = -1; +module_param(kdamond_pid, int, 0400); + +static struct damon_ctx *ctx; +static struct damon_target *target; + +struct damon_reclaim_ram_walk_arg { + unsigned long start; + unsigned long end; +}; + +static int walk_system_ram(struct resource *res, void *arg) +{ + struct damon_reclaim_ram_walk_arg *a = arg; + + if (a->end - a->start < res->end - res->start) { + a->start = res->start; + a->end = res->end; + } + return 0; +} + +/* + * Find biggest 'System RAM' resource and store its start and end address in + * @start and @end, respectively. If no System RAM is found, returns false. + */ +static bool get_monitoring_region(unsigned long *start, unsigned long *end) +{ + struct damon_reclaim_ram_walk_arg arg = {}; + + walk_system_ram_res(0, ULONG_MAX, &arg, walk_system_ram); + if (arg.end <= arg.start) + return false; + + *start = arg.start; + *end = arg.end; + return true; +} + +static struct damos *damon_reclaim_new_scheme(void) +{ + struct damos_watermarks wmarks = { + .metric = DAMOS_WMARK_FREE_MEM_RATE, + .interval = wmarks_interval, + .high = wmarks_high, + .mid = wmarks_mid, + .low = wmarks_low, + }; + struct damos_speed_limit limit = { + /* Do not page out more than limit_sz bytes within limit_ms */ + .sz = limit_sz, + .ms = limit_ms, + /* Within the limit, page out older regions first. */ + .weight_sz = 0, + .weight_nr_accesses = 0, + .weight_age = 1 + }; + struct damos *scheme = damon_new_scheme( + /* Find regions having PAGE_SIZE or larger size */ + PAGE_SIZE, ULONG_MAX, + /* and not accessed at all */ + 0, 0, + /* for min_age or more micro-seconds, and */ + min_age / aggr_interval, UINT_MAX, + /* page out those, as soon as found */ + DAMOS_PAGEOUT, + /* under the speed limit. */ + &limit, + /* (De)activate this according to the watermarks. */ + &wmarks); + + return scheme; +} + +static int damon_reclaim_turn(bool on) +{ + struct damon_region *region; + struct damos *scheme; + int err; + + if (!on) { + err = damon_stop(&ctx, 1); + if (!err) + kdamond_pid = -1; + return err; + } + + err = damon_set_attrs(ctx, sample_interval, aggr_interval, 0, + min_nr_regions, max_nr_regions); + if (err) + return err; + + if (monitor_region_start > monitor_region_end) + return -EINVAL; + if (!monitor_region_start && !monitor_region_end && + !get_monitoring_region(&monitor_region_start, + &monitor_region_end)) + return -EINVAL; + /* DAMON will free this on its own when finish monitoring */ + region = damon_new_region(monitor_region_start, monitor_region_end); + if (!region) + return -ENOMEM; + damon_add_region(region, target); + + /* Will be freed by 'damon_set_schemes()' below */ + scheme = damon_reclaim_new_scheme(); + if (!scheme) + goto free_region_out; + err = damon_set_schemes(ctx, &scheme, 1); + if (err) + goto free_scheme_out; + + err = damon_start(&ctx, 1); + if (!err) { + kdamond_pid = ctx->kdamond->pid; + return 0; + } + +free_scheme_out: + damon_destroy_scheme(scheme); +free_region_out: + damon_destroy_region(region); + return err; +} + +#define ENABLE_CHECK_INTERVAL_MS 1000 +static struct delayed_work damon_reclaim_timer; +static void damon_reclaim_timer_fn(struct work_struct *work) +{ + static bool last_enabled; + bool now_enabled; + + now_enabled = enabled; + if (last_enabled != now_enabled) { + if (!damon_reclaim_turn(now_enabled)) + last_enabled = now_enabled; + else + enabled = last_enabled; + } + + schedule_delayed_work(&damon_reclaim_timer, + msecs_to_jiffies(ENABLE_CHECK_INTERVAL_MS)); +} +static DECLARE_DELAYED_WORK(damon_reclaim_timer, damon_reclaim_timer_fn); + +static int __init damon_reclaim_init(void) +{ + ctx = damon_new_ctx(DAMON_ADAPTIVE_TARGET); + if (!ctx) + return -ENOMEM; + + damon_pa_set_primitives(ctx); + + /* 4242 means nothing but fun */ + target = damon_new_target(4242); + if (!target) { + damon_destroy_ctx(ctx); + return -ENOMEM; + } + damon_add_target(ctx, target); + + schedule_delayed_work(&damon_reclaim_timer, 0); + return 0; +} + +module_init(damon_reclaim_init); From patchwork Tue Jun 8 11:52:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 12306603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76DDAC4743F for ; Tue, 8 Jun 2021 11:53:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 210686124B for ; Tue, 8 Jun 2021 11:53:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 210686124B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B9A846B0082; Tue, 8 Jun 2021 07:53:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B72806B0083; Tue, 8 Jun 2021 07:53:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 975846B0085; Tue, 8 Jun 2021 07:53:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id 6750E6B0082 for ; Tue, 8 Jun 2021 07:53:43 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0F76298A9 for ; Tue, 8 Jun 2021 11:53:43 +0000 (UTC) X-FDA: 78230397126.35.81D82E6 Received: from mail-qt1-f173.google.com (mail-qt1-f173.google.com [209.85.160.173]) by imf01.hostedemail.com (Postfix) with ESMTP id 6ACDC5001533 for ; Tue, 8 Jun 2021 11:53:39 +0000 (UTC) Received: by mail-qt1-f173.google.com with SMTP id t17so14978519qta.11 for ; Tue, 08 Jun 2021 04:53:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pJTwc2+8OmVO0WYs0LS9qDVADcgdZ32QIgWE+KhO6qY=; b=Ed5PIja2L89hTEZqu5p9+ymg2BMEhGDZZ1DfUAX1zG4emZcFOuPPjaUF7Rg6Oq4yZ1 Bspb2FZhAzvQMP1K6Ecmsv+Yba8KIxnjz/eW73Jgf0JL81xq5iztd86lXEBm21A+8bYf E3IGDYaBcXVJKr5jEc9fPBh9+Pr8VUkWsRDwdSeVul6n3PVOP88PX/sDRCijZ/ok6J6O /zZwsDZIBDqVwA7+pncCkr/fPE0DbEgBGcqC+tJPjhZReHvs/+QoFFeybt/8F0LNnHj/ mvjGiGd5H047N4ckB/IqIyM630FO2IFGOq+o/kU2IfpJd7lvfWNLwpdIW/jqONIEMOyN MYow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pJTwc2+8OmVO0WYs0LS9qDVADcgdZ32QIgWE+KhO6qY=; b=kz50m+mWhFR3L5aXjNguXR1xYaKRjX9A6Fy/S3sDIFxMWXuGIbE+VW2/C98xECa6P2 SpBpQnym3Mcmnb/ZLjo+1pGMEYzxvhgnvTQ8xWtr31d2GCv3JT0y0H5bewvlEaXEJwR0 ivq2twwZTp/LaW2TehFeso2GBfRFoJgzfcHsWhi7t4OjQYAe51Oy86SwiOXWj1opCJbI rWjoN4Xalz+m6eoO+C9R1tvu2v09hpqRPNzKtzjX01VCJyGusSlUPVLSVvb+dhHp6tQk VEfJ/6jACCB8beE/WMYmh+HspeaPrDA5C4n+DShSLRyk9BiiRmcK0Uq9YkCMbBWD6ypz S1PQ== X-Gm-Message-State: AOAM530V2iLskHH5xyPiiSQvU0zkmI4Cxur+5HcUeEe55bADUjwrs+0Y /gAe3GyoAoUxdnhBqXHxfxQ= X-Google-Smtp-Source: ABdhPJyqXfjoMNVb/P2WbZokEeqnYl1dmsZF5orqEbrbRbGD7efJeeKKimDTO5AEr2MfND2mDEzgTw== X-Received: by 2002:ac8:67c2:: with SMTP id r2mr6361570qtp.131.1623153222062; Tue, 08 Jun 2021 04:53:42 -0700 (PDT) Received: from localhost.localdomain (ec2-35-169-212-159.compute-1.amazonaws.com. [35.169.212.159]) by smtp.gmail.com with ESMTPSA id h4sm2133927qth.66.2021.06.08.04.53.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Jun 2021 04:53:41 -0700 (PDT) From: SeongJae Park To: akpm@linux-foundation.org Cc: SeongJae Park , Jonathan.Cameron@Huawei.com, acme@kernel.org, alexander.shishkin@linux.intel.com, amit@kernel.org, benh@kernel.crashing.org, brendanhiggins@google.com, corbet@lwn.net, david@redhat.com, dwmw@amazon.com, elver@google.com, fan.du@intel.com, foersleo@amazon.de, greg@kroah.com, gthelen@google.com, guoju.fgj@alibaba-inc.com, jgowans@amazon.com, mgorman@suse.de, minchan@kernel.org, mingo@redhat.com, namhyung@kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, rostedt@goodmis.org, rppt@kernel.org, shakeelb@google.com, shuah@kernel.org, sj38.park@gmail.com, snu@zelle79.org, vbabka@suse.cz, vdavydov.dev@gmail.com, zgf574564920@gmail.com, linux-damon@amazon.com, linux-mm@kvack.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [RFC v2 14/14] Documentation/admin-guide/mm/damon: Add a document for DAMON_RECLAIM Date: Tue, 8 Jun 2021 11:52:54 +0000 Message-Id: <20210608115254.11930-15-sj38.park@gmail.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210608115254.11930-1-sj38.park@gmail.com> References: <20210608115254.11930-1-sj38.park@gmail.com> Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Ed5PIja2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of sj38park@gmail.com designates 209.85.160.173 as permitted sender) smtp.mailfrom=sj38park@gmail.com X-Rspamd-Server: rspam02 X-Stat-Signature: uxa9r8dzmweemzfh8cdtcpm4cerkeogi X-Rspamd-Queue-Id: 6ACDC5001533 X-HE-Tag: 1623153219-722495 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit adds an admin-guide document for DAMON-based Reclamation. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/index.rst | 1 + .../admin-guide/mm/damon/reclaim.rst | 211 ++++++++++++++++++ 2 files changed, 212 insertions(+) create mode 100644 Documentation/admin-guide/mm/damon/reclaim.rst diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/admin-guide/mm/damon/index.rst index 0baae7a5402b..e5b4da13f60b 100644 --- a/Documentation/admin-guide/mm/damon/index.rst +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -13,3 +13,4 @@ Using this, users can analyze and optimize their systems. start guide usage + reclaim diff --git a/Documentation/admin-guide/mm/damon/reclaim.rst b/Documentation/admin-guide/mm/damon/reclaim.rst new file mode 100644 index 000000000000..b4fe0c16f72e --- /dev/null +++ b/Documentation/admin-guide/mm/damon/reclaim.rst @@ -0,0 +1,211 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +DAMON-based Reclamation +======================= + +DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to +be used for proactive and lightweight reclamation under light memory pressure. +It doesn't aim to replace the LRU-list based page_granularity reclamation, but +to be selectively used for different level of memory pressure and requirements. + +Where Proactive Reclamation is Required? +======================================== + +On general memory over-committed systems, proactively reclaiming cold pages +helps saving memory and reducing latency spikes that incurred by the direct +reclaim of the process or CPU consumption of kswapd, while incurring only +minimal performance degradation [1]_ [2]_ . + +Free Pages Reporting [3]_ based memory over-commit virtualization systems are +good example of the cases. In such systems, the guest VMs reports their free +memory to host, and the host reallocates the reported memory to other guests. +As a result, the memory of the systems are fully utilized. However, the +guests could be not so memory-frugal, mainly because some kernel subsystems and +user-space applications are designed to use as much memory as available. Then, +guests could report only small amount of memory as free to host, results in +memory utilization drop of the systems. Running the proactive reclamation in +guests could mitigate this problem. + +How It Works? +============= + +DAMON_RECLAIM finds memory regions that didn't accessed for specific time +duration and page out. To avoid it consuming too much CPU for the paging out +operation, a speed limit can be configured. Under the speed limit, it pages +out memory regions that didn't accessed longer time first. System +administrators can also configure under what situation this scheme should +automatically activated and deactivated with three memory pressure watermarks. + +Interface: Module Parameters +============================ + +To use this feature, you should first ensure your system is running on a kernel +that is built with ``CONFIG_DAMON_RECLAIM=y``. + +To let sysadmins enable or disable it and tune for the given system, +DAMON_RECLAIM utilizes module parameters. That is, you can put +``damon_reclaim.=`` on the kernel boot command line or write +proper values to ``/sys/modules/damon_reclaim/parameters/`` files. + +Note that the parameter values except ``enabled`` are applied only when +DAMON_RECLAIM starts. Therefore, if you want to apply new parameter values in +runtime and DAMON_RECLAIM is already enabled, you should disable and re-enable +it via ``enabled`` parameter file. Writing of the new values to proper +parameter values should be done before the re-enablement. + +Below are the description of each parameter. + +enabled +------- + +Enable or disable DAMON_RECLAIM. + +You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``. +Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do +no real monitoring and reclamation due to the watermarks-based activation +condition. Refer to below descriptions for the watermarks parameter for this. + +min_age +------- + +Time threshold for cold memory regions identification in microseconds. + +If a memory region is not accessed for this or longer time, DAMON_RECLAIM +identifies the region as cold, and reclaims it. + +limit_sz +-------- + +Maximum bytes of memory that can be reclaimed in a charging window. + +DAMON_RECLAIM counts amount of memory which has reclaimed within current +charging time window and avoids reclaiming more than this limit in current time +window. This could be useful for limiting CPU consumption of DAMON_RECLAIM. + +limit_ms +-------- + +The reclaimed memory charging window in milliseconds. + +wmarks_interval +--------------- + +Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is +enabled but inactive due to its watermarks rule. + +wmarks_high +----------- + +Free memory rate (per thousand) for the high watermark. + +If free memory of the system in bytes per thousand bytes is higher than this, +DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks +the watermarks. + +wmarks_mid +---------- + +Free memory rate (per thousand) for the middle watermark. + +If free memory of the system in bytes per thousand bytes is between this and +the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and +the reclaiming. + +wmarks_low +---------- + +Free memory rate (per thousand) for the low watermark. + +If free memory of the system in bytes per thousand bytes is lower than this, +DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the +watermarks. In the case, the system falls back to the LRU-list based page +granularity reclamation logic. + +sample_interval +--------------- + +Sampling interval for the monitoring in microseconds. + +The sampling interval of DAMON for the cold memory monitoring. Please refer to +the DAMON documentation (:doc:`usage`) for more detail. + +aggr_interval +------------- + +Aggregation interval for the monitoring in microseconds. + +The aggregation interval of DAMON for the cold memory monitoring. Please +refer to the DAMON documentation (:doc:`usage`) for more detail. + +min_nr_regions +-------------- + +Minimum number of monitoring regions. + +The minimal number of monitoring regions of DAMON for the cold memory +monitoring. This can be used to set lower-bound of the monitoring quality. +But, setting this too high could result in increased monitoring overhead. +Please refer to the DAMON documentation (:doc:`usage`) for more detail. + +max_nr_regions +-------------- + +Maximum number of monitoring regions. + +The maximum number of monitoring regions of DAMON for the cold memory +monitoring. This can be used to set upper-bound of the monitoring overhead. +However, setting this too low could result in bad monitoring quality. Please +refer to the DAMON documentation (:doc:`usage`) for more detail. + +monitor_region_start +-------------------- + +Start of target memory region in physical address. + +The start physical address of memory region that DAMON_RECLAIM will do work +against. That is, DAMON_RECLAIM will find cold memory regions in this region +and reclaims. By default, biggest System RAM is used as the region. + +monitor_region_end +------------------ + +End of target memory region in physical address. + +The end physical address of memory region that DAMON_RECLAIM will do work +against. That is, DAMON_RECLAIM will find cold memory regions in this region +and reclaims. By default, biggest System RAM is used as the region. + +kdamond_pid +----------- + +PID of the DAMON thread. + +If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else, +-1. + +Example +======= + +Below runtime example commands make DAMON_RECLAIM to find memory regions that +not accessed for 30 seconds or more and pages out. The reclamation is limited +to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too +much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do +nothing if the system's free memory rate is more than 50%, but start the real +works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and +therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to +do nothing again, so that we can fall back to the LRU-list based page +granularity reclamation. :: + + # cd /sys/modules/damon_reclaim/parameters + # echo 30000000 > min_age + # echo $((1 * 1024 * 1024 * 1024)) > limit_sz + # echo 1000 > limit_ms + # echo 500 > wmarks_high + # echo 400 > wmarks_mid + # echo 200 > wmarks_low + # echo Y > enabled + +.. [1] https://research.google/pubs/pub48551/ +.. [2] https://lwn.net/Articles/787611/ +.. [3] https://www.kernel.org/doc/html/latest/vm/free_page_reporting.html