From patchwork Sun Nov 12 19:46:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13453384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4168BC4167B for ; Sun, 12 Nov 2023 19:46:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B16A58D001F; Sun, 12 Nov 2023 14:46:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AC6368D001D; Sun, 12 Nov 2023 14:46:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 918328D0020; Sun, 12 Nov 2023 14:46:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5ABC28D001D for ; Sun, 12 Nov 2023 14:46:21 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2EB33120445 for ; Sun, 12 Nov 2023 19:46:21 +0000 (UTC) X-FDA: 81450333762.27.07D3DBC Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf17.hostedemail.com (Postfix) with ESMTP id CD73840007 for ; Sun, 12 Nov 2023 19:46:18 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XkjUxCxf; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699818379; a=rsa-sha256; cv=none; b=zHRV3jGZreU/Jo3vYc/sUMbi8arxcuGXMPvqWReSZkYbYWnptLqsnwooAhq1545hBS9gYL SUb9qdLObw7L8ORCgtanNujS9hFfLb2UAeQILiWRsNNRtstS4R9HRoCgnWBJJHABNST3m+ zAvivXnuFiSYl1LVT2KG9Jv34rcH1hA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=XkjUxCxf; spf=pass (imf17.hostedemail.com: domain of sj@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699818379; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gxGV2K3I6gjEYhXMANRPr9ZZAD0VMOdDCMI2p6R70Mg=; b=OJDxCiL2S+XPAuiU+9Tx1OjZYktrWHl3VfJ1E6KQAQK5yML0qA01ezDFVoN6IoZ7zXs2z/ RlEMUJRSZEohJMseNsw1wd7l/iuItJyxNXiAFkRYWAdcteQuvrj1j4Yl3rUhohCkCLn9Y+ ONaP95HHlBHpGNgkV/ar6Teg8doGCIA= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 9D624CE0F1C; Sun, 12 Nov 2023 19:46:14 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51465C433CB; Sun, 12 Nov 2023 19:46:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1699818373; bh=KH5zwAcadi379iK70CZqM3VL8+sPUwrVnpfkvLQxuio=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XkjUxCxfX1eDI7+ZUfgU0fc8n+Jbbw4SnAXuDwRc0gK2X9MJSQ/Im4hUdsXf8ulo1 kWBRMqtgQ7mu/hCku3ihWgdbs9bRZ30guaTnrITIWYBeVXdhn3cXqcBZSX15bmxyoH kZx70B5f6Wihw8EQUrGMeKL+8/OZLa/P2mdO7LyxR+hxkpW7dAqzS2x0wMNQHUsD+V vkYfSwqBIzDI90NSXv8G57GkDDvA/kKI18Wh2GuRYSi6dSIl7rS/X1aIZzaRu61F4o 3NpMhRv3qxln2doao3lETfko0/RzXJVEi2a08qylRuH1pAmVvXB8CVyZXk20qiarmm 8SIzWD3FOpPCw== From: SeongJae Park To: Cc: SeongJae Park , Andrew Morton , damon@lists.linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [RFC PATCH 1/8] mm/damon/core: implement goal-oriented feedback-driven quota auto-tuning Date: Sun, 12 Nov 2023 19:46:00 +0000 Message-Id: <20231112194607.61399-2-sj@kernel.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231112194607.61399-1-sj@kernel.org> References: <20231112194607.61399-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: CD73840007 X-Stat-Signature: ohqqrjde4cga87wdkpb9t8hng996fms7 X-Rspam-User: X-HE-Tag: 1699818378-907108 X-HE-Meta: U2FsdGVkX19oT8iBLBWyP9b2Q/uSe/7DUM/2SR7DEr9twCf8L5ZpPizUipT2pMpgK+1XZah7aRVjhWsyoepzUboHhEtrg5HJMQdwowqF+kepW2anaAST5rByDJPgnRGjWPuc6c0Sb28FUbzsMnjvwadhw/dxyOkiYFgb/1LLUsMqAK8HmPw8KoMKsL769cO8bzJ9B0QwnJ0yGYBcmR1Qp/ZyP8sQz51S5hTHuGV/3gP6onvrU93gu0JMHczoq6AZ/ORjjsyX/K5qsCY6imQbhcqIhbk4n1xFsHi0a8U+b+imdxtrb1W9GGSA4sGFK0fvew16frovynZVGR9lMtb8smcKYmTQrn6KJs4Z/nEhpwtQPROs3rMNLsaIAK4dWnGG2SXhLH43LPMxs7NFjvvCbDXXMPTSbajmHX96M5LaFWa3VNJvspqUHl0Nq2NGDwPuU/IuQj4GSAHNKFfiXjZ+y3+qAi4c/k1VFvmXFZuSImBwk4JSpqB7gqn+idCXEGj3nfNlNcnbSTZk3QEnBJUaMP53rl8nSS+v9zGJ1DugInw8eix+x/0RTcVaZW7uXwZ7O2bJVenKNilLmuBcBhojgRNHWyV5LrGpZNgypf3hMCaSQofLWUwZflZJKTM+hSolzYlNJZPeDQOE2bDEZvmtnb9iamodPHbgs3Uv+cF8msJ05K/ASvtrX0VDC3SNPi00UnRf8qeNAKV8e2r+utZsgC8OXl3zcnCTt5mH7A6+gXCdqQ0sfS2RtQrHXzEqSUGN0irDOFmrlO81dHYDAhNrKGAMK56XLn9woANXz0VE0xdjmVCXGKwaSpn+KX6iRFfpyhD9m67AK9A/KvkRxSIZHvrKHfvQYHC5BDA4rcYm7q3AaD70bbbNs4DXPH5Xy6Jcxeq/THjmqSOpQkquR8K/jCCJNQmu4cksEiZ8QUv7NEFtAUfKrSQmGECDoLU1oGZKg9vzs/ppoVRAn19+YZ0 jTRwqSwE /Aao+S6ZOSdrwIZYdBMT7pcOE931/sWA6JIA2Gnt7xRCf7S1lIxC/bo7l7CWBgWbWY4UB/PFHUI+dMFBb/ly8Maa/Cw2oF3XOOjwi8dpatzYNCXqHkRcbkrTMUYMIO58+J6Eqk+2Wi4dY2u2fUzgLc/n5/4t/jEB+ljfpN8vVyN7usroUWNCKw/OFCTt1Mc3w3w+as08SJT9XmZJyhia6oLH7vU8+ilwKJBh2Az86+SKSap4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Users can effectively control the upper-most aggressiveness of DAMOS schemes using the quota feature. The quota provides best result under the aggressiveness limit since it prioritizes regions based on the access pattern. Finding the best value, which could depend on dynamic characteristics of the system and the workloads, is still challenging, though. Implement a simple feedback-driven tuning mechanism and use it for automatic tuning of DAMOS quota. The implementation allows users to provide the feedback by setting a feedback score returning callback function. Then DAMOS periodically adjusts the quota based on the return value of the function calls. Note that the absolute-value based time/size quotas still work as the maximum hard limit of the scheme's aggressiveness. The feedback-driven auto-tuned quota is applied only if it is not exceeding the manually set maximum limit. Same for the scheme-target access pattern and filters like other features. Signed-off-by: SeongJae Park --- include/linux/damon.h | 19 +++++++++++++ mm/damon/core.c | 65 +++++++++++++++++++++++++++++++++++++------ 2 files changed, 75 insertions(+), 9 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 89f5ca041848..9d46e0b39456 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -136,6 +136,8 @@ enum damos_action { * @weight_nr_accesses: Weight of the region's nr_accesses for prioritization. * @weight_age: Weight of the region's age for prioritization. * + * @get_score: Feedback function for self-tuning quota. + * * To avoid consuming too much CPU time or IO resources for applying the * &struct damos->action to large memory, DAMON allows users to set time and/or * size quotas. The quotas can be set by writing non-zero values to &ms and @@ -153,6 +155,17 @@ enum damos_action { * You could customize the prioritization logic by setting &weight_sz, * &weight_nr_accesses, and &weight_age, because monitoring operations are * encouraged to respect those. + * + * If @get_score function pointer is set, DAMON calls it back and get the + * return value of it for every @reset_interval. Then, DAMON adjusts the + * effective quota using the return value as a feedback score to the current + * quota, using its internal feedback loop algorithm. + * + * The feedback loop algorithem assumes the quota input and the feedback score + * output are in a positive proportional relationship, and the goal of the + * tuning is getting the feedback screo value of 10,000. If @ms and/or @sz are + * set together, those work as a hard limit quota. If neither @ms nor @sz are + * set, the mechanism starts from the quota of one byte. */ struct damos_quota { unsigned long ms; @@ -163,6 +176,9 @@ struct damos_quota { unsigned int weight_nr_accesses; unsigned int weight_age; + unsigned long (*get_score)(void *arg); + void *get_score_arg; + /* private: */ /* For throughput estimation */ unsigned long total_charged_sz; @@ -179,6 +195,9 @@ struct damos_quota { /* For prioritization */ unsigned long histogram[DAMOS_MAX_SCORE + 1]; unsigned int min_score; + + /* For feedback loop */ + unsigned long esz_bp; }; /** diff --git a/mm/damon/core.c b/mm/damon/core.c index c451e7dfcd64..4d4e4ebbbb2a 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -1086,26 +1086,73 @@ static void damon_do_apply_schemes(struct damon_ctx *c, } } -/* Shouldn't be called if quota->ms and quota->sz are zero */ +/* + * damon_feed_loop_next_input() - get next input to achieve a target score. + * @last_input The last input. + * @score Current score that made with @last_input. + * + * Calculate next input to achieve the target score, based on the last input + * and current score. Assuming the input and the score are positively + * proportional, calculate how much compensation should be added to or + * subtracted from the last input as a proportion of the last input. Avoid + * next input always being zero by setting it non-zero always. In short form + * (assuming support of float and signed calculations), the algorithm is as + * below. + * + * next_input = max(last_input * ((goal - current) / goal + 1), 1) + * + * For simple implementation, we assume the target score is always 10,000. The + * caller should adjust @score for this. + * + * Returns next input that assumed to achieve the target score. + */ +static unsigned long damon_feed_loop_next_input(unsigned long last_input, + unsigned long score) +{ + const unsigned long goal = 10000; + unsigned long score_goal_diff = max(goal, score) - min(goal, score); + unsigned long score_goal_diff_bp = score_goal_diff * 10000 / goal; + unsigned long compensation = last_input * score_goal_diff_bp / 10000; + /* Set minimum input as 10000 to avoid compensation be zero */ + const unsigned long min_input = 10000; + + if (goal > score) + return last_input + compensation; + if (last_input > compensation + min_input) + return last_input - compensation; + return min_input; +} + +/* Shouldn't be called if quota->ms, quota->sz, and quota->get_score unset */ static void damos_set_effective_quota(struct damos_quota *quota) { unsigned long throughput; unsigned long esz; - if (!quota->ms) { + if (!quota->ms && !quota->get_score) { quota->esz = quota->sz; return; } - if (quota->total_charged_ns) - throughput = quota->total_charged_sz * 1000000 / - quota->total_charged_ns; - else - throughput = PAGE_SIZE * 1024; - esz = throughput * quota->ms; + if (quota->get_score) { + quota->esz_bp = damon_feed_loop_next_input( + max(quota->esz_bp, 10000UL), + quota->get_score(quota->get_score_arg)); + esz = quota->esz_bp / 10000; + } + + if (quota->ms) { + if (quota->total_charged_ns) + throughput = quota->total_charged_sz * 1000000 / + quota->total_charged_ns; + else + throughput = PAGE_SIZE * 1024; + esz = min(throughput * quota->ms, esz); + } if (quota->sz && quota->sz < esz) esz = quota->sz; + quota->esz = esz; } @@ -1117,7 +1164,7 @@ static void damos_adjust_quota(struct damon_ctx *c, struct damos *s) unsigned long cumulated_sz; unsigned int score, max_score = 0; - if (!quota->ms && !quota->sz) + if (!quota->ms && !quota->sz && !quota->get_score) return; /* New charge window starts */