From patchwork Fri Jan 3 17:43:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 13925761 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3266E7718F for ; Fri, 3 Jan 2025 17:44:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 568F96B0095; Fri, 3 Jan 2025 12:44:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F4096B0093; Fri, 3 Jan 2025 12:44:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 27FC06B0096; Fri, 3 Jan 2025 12:44:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DA7376B008A for ; Fri, 3 Jan 2025 12:44:16 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8D1051A098C for ; Fri, 3 Jan 2025 17:44:16 +0000 (UTC) X-FDA: 82966863924.02.2E7D2A5 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf09.hostedemail.com (Postfix) with ESMTP id 8BA5C14000E for ; Fri, 3 Jan 2025 17:43:39 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="qpu/k9+9"; spf=pass (imf09.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735926206; a=rsa-sha256; cv=none; b=aFFCfWc7n+PJiUgJqU4Fxz3HLXsfhkNWmQ540NPBs28XJFW9aHiRHv/35yb+THY/j41A5T wNn1msysxijL4f5K54VArhojvJ9Oc8XtSWzAu8axjkt5SUZ6Kc0o73VuJbKWRqdIB1ntOX g6BXEJHYVFwi7sjP31ewg+D49Md8plw= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="qpu/k9+9"; spf=pass (imf09.hostedemail.com: domain of sj@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=sj@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735926206; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GL3K3VeEw3sBPlc5hG0f1xiMIwuovutSxEQCsmYLfb4=; b=CUusfrNtTswyFGDvEkm4UW8zIN2rd9SQBCwAvsjAOeGwEgJG+47WH8XzGm9tWwNISy3lVe wT56XVa90NlKGj2knjnDLeJ/Tpw51Aq3cpsIwzXlFsF3a26eOnRWyP8ORQUEhpLH0a+Im5 rE7yLQkgFaiQf5gyWWCjetVrmDSIgOk= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 50B4DA41033; Fri, 3 Jan 2025 17:42:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1274DC4CECE; Fri, 3 Jan 2025 17:44:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1735926254; bh=JNkpoBSJiwe93XL962RE5R1ynJv2g3n6CCV3Kyol5/0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qpu/k9+9YSU7eTA3ClaCp7f1fEbo8Y70eSTguSBhya0DXCc0VjaqGUxE1sa356iNY 4btMs7/D2rf/yQsum03Nzf1vbAp7ZG2QdAApH6XDkNcK7gttcahF2zLBq+tRTfCxgi aLM8AWc/Rde2IKCQAkeCWL1KoedEMklGBEE4EZtzWh4lXHqPt94AtjoqsemaQXcriZ UgMfZgATz0uIPr+Knfg1C43PytEkgyYZkYdsf59EMqs+nN4MVFV5WK+v5h/c2GybEN 8EgB9v4qbBlrVEKKPhBIVJhZAW0XHaQ1uVAJzFs9btRwRl1SU/MeMm4dkUbTTlOVZ2 5eogb+SVumsOw== From: SeongJae Park To: Andrew Morton Cc: SeongJae Park , damon@lists.linux.dev, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 03/10] mm/damon/core: introduce damon_call() Date: Fri, 3 Jan 2025 09:43:53 -0800 Message-Id: <20250103174400.54890-4-sj@kernel.org> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250103174400.54890-1-sj@kernel.org> References: <20250103174400.54890-1-sj@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 8BA5C14000E X-Stat-Signature: n1tmzxqio1siejsj58hmng1kgd7pk1qc X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1735926219-757945 X-HE-Meta: U2FsdGVkX18Djx8I3XA7HhETKuDiItKPVb50iLPL3inmvgk7xLm4TzbSY8v9+QT8Q3S0h5dmwxDqY5T/se2JPKfSsR/+sGsgLt5rLVaFzMUG4y132dZRpxfd2CUkgbKX2PFhpaG4mtcu6cN5gmUOTZK9Q9rcHJFeYlui1XqKjAFIW7D4f7nL5hbbc/5MsSXjrdIHvT1zJYmTgQjEsrxJuzbznlWkOeG7piHdXN2hmvMEDeUZiRgtqt/I06pQWZ1+fc39aB5jjqMmGapGlGxax5kFyUlrpBElpSsv/yS5LBnDgxjLs5SOsNQUg+8xS6sFlYcOnqiJTi2Ib69xGMTz4pg08XnFVpNcxpw7bsl6jrC0DevuGAR/ygLJv9bCvfIgrEeEArY15fIspEw6q7U6qEAXwSK9DZecwDo48bQZTd9Tr1xHOYHoDoG7FyKrRSxMATF15daUZnCK/NCSw0ueMWSp3XTc1g8DiWqbrHxGZXN74qta6uRQfttu33fSGoL4rgIE7Nmy2J3EXEWVWOsnCi5Z+W716Ukhx991xYfSylrC4ElU76JGJC+ZSh+hlhu2rmJijiSSfKyG068DS1eMEyRP7cLCJkMsJlEeB8JS/6ZEK7qmwTblY/6pN4hmO0Xe0I6IVi/ixPa1Xcl+ekA3LwSjz+Yveexj6BxpYORJG47SL+aiL45O+RLMaUqfIhky9WonDSt4jhU/y2AtXwQprIkQ2TtT80/+FGzXrDapBYam+bym46jD9PC68uj9ycVERSSrg0q5nawkDjHWPuyzROr/E/EjZ04MbBfmBbv5D1/2OJQo7RdUCNrFDxwXpP+Kn975Lbn/nUt7tYR9JZE5UP4pfFNeUXlB1ww1GpW5BCXthhxt2igre5IjH0M3/+Fmh9ZYMPNYZU8/4GHVyaO1EOs4xze1jBJpn3Tmv2Zlp1/i9MlQOUTWzbjx7TAwC+7qVmjf+eMS0DEh7U4m/KU DVzShcMI LhxCHOTx8/DCuYfn7t/jmq6150yHYo00qTRD05bIpGXBmWz6aEv4t8bsTQbYEjFD1bJ8gXdeLjD8uIJrsInXNGCgPoUeb2Ozek5BddV7YrgBZRqdiBUqTUke8IFSe7Y6og2zA7FzxvayagfFHFW4TwShgU+TwFtqIIqUvNNWGiPhCa4rQkWbObTapBVdKxh/NkD9JDUVlsoWFR7t6NWhBSPdwEFmBCrIVqnz7Xzr6uz48E2D+Oi00lqn0T7yhu9l3rNxlP5MPseZzvqAKBgjWXM325g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce a new DAMON core API function, damon_call(). It aims to replace some damon_callback usages that access damon_ctx of ongoing kdamond with additional synchronizations. It receives a function pointer, let the parallel kdamond invokes the function, and returns after the invocation is finished, or canceled due to some races. kdamond invokes the function inside the main loop after sampling is done. If it is deactivated by DAMOS watermarks or already out of the main loop, mark the request as canceled so that damon_call() can wakeup and return. Signed-off-by: SeongJae Park --- include/linux/damon.h | 26 +++++++++++++ mm/damon/core.c | 86 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) diff --git a/include/linux/damon.h b/include/linux/damon.h index a67f2c4940e9..ac2d42a50751 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -552,6 +552,27 @@ struct damon_callback { void (*before_terminate)(struct damon_ctx *context); }; +/* + * struct damon_call_control - Control damon_call(). + * + * @fn: Function to be called back. + * @data: Data that will be passed to @fn. + * @return_code: Return code from @fn invocation. + * + * Control damon_call(), which requests specific kdamond to invoke a given + * function. Refer to damon_call() for more details. + */ +struct damon_call_control { + int (*fn)(void *data); + void *data; + int return_code; +/* private: internal use only */ + /* informs if the kdamond finished handling of the request */ + struct completion completion; + /* informs if the kdamond canceled @fn infocation */ + bool canceled; +}; + /** * struct damon_attrs - Monitoring attributes for accuracy/overhead control. * @@ -632,6 +653,9 @@ struct damon_ctx { /* for scheme quotas prioritization */ unsigned long *regions_score_histogram; + struct damon_call_control *call_control; + struct mutex call_control_lock; + /* public: */ struct task_struct *kdamond; struct mutex kdamond_lock; @@ -779,6 +803,8 @@ static inline unsigned int damon_max_nr_accesses(const struct damon_attrs *attrs int damon_start(struct damon_ctx **ctxs, int nr_ctxs, bool exclusive); int damon_stop(struct damon_ctx **ctxs, int nr_ctxs); +int damon_call(struct damon_ctx *ctx, struct damon_call_control *control); + int damon_set_region_biggest_system_ram_default(struct damon_target *t, unsigned long *start, unsigned long *end); diff --git a/mm/damon/core.c b/mm/damon/core.c index 5192ee29f6cf..97f19ec4179c 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -504,6 +504,7 @@ struct damon_ctx *damon_new_ctx(void) ctx->next_ops_update_sis = 0; mutex_init(&ctx->kdamond_lock); + mutex_init(&ctx->call_control_lock); ctx->attrs.min_nr_regions = 10; ctx->attrs.max_nr_regions = 1000; @@ -1162,6 +1163,54 @@ int damon_stop(struct damon_ctx **ctxs, int nr_ctxs) return err; } +static bool damon_is_running(struct damon_ctx *ctx) +{ + bool running; + + mutex_lock(&ctx->kdamond_lock); + running = ctx->kdamond != NULL; + mutex_unlock(&ctx->kdamond_lock); + return running; +} + +/** + * damon_call() - Invoke a given function on DAMON worker thread (kdamond). + * @ctx: DAMON context to call the function for. + * @control: Control variable of the call request. + * + * Ask DAMON worker thread (kdamond) of @ctx to call a function with an + * argument data that respectively passed via &damon_call_control->fn and + * &damon_call_control->data of @control, and wait until the kdamond finishes + * handling of the request. + * + * The kdamond executes the function with the argument in the main loop, just + * after a sampling of the iteration is finished. The function can hence + * safely access the internal data of the &struct damon_ctx without additional + * synchronization. The return value of the function will be saved in + * &damon_call_control->return_code. + * + * Return: 0 on success, negative error code otherwise. + */ +int damon_call(struct damon_ctx *ctx, struct damon_call_control *control) +{ + init_completion(&control->completion); + control->canceled = false; + + mutex_lock(&ctx->call_control_lock); + if (ctx->call_control) { + mutex_unlock(&ctx->call_control_lock); + return -EBUSY; + } + ctx->call_control = control; + mutex_unlock(&ctx->call_control_lock); + if (!damon_is_running(ctx)) + return -EINVAL; + wait_for_completion(&control->completion); + if (control->canceled) + return -ECANCELED; + return 0; +} + /* * Reset the aggregated monitoring results ('nr_accesses' of each region). */ @@ -1917,6 +1966,39 @@ static void kdamond_usleep(unsigned long usecs) usleep_range_idle(usecs, usecs + 1); } +/* + * kdamond_call() - handle damon_call_control. + * @ctx: The &struct damon_ctx of the kdamond. + * @cancel: Whether to cancel the invocation of the function. + * + * If there is a &struct damon_call_control request that registered via + * &damon_call() on @ctx, do or cancel the invocation of the function depending + * on @cancel. @cancel is set when the kdamond is deactivated by DAMOS + * watermarks, or the kdamond is already out of the main loop and therefore + * will be terminated. + */ +static void kdamond_call(struct damon_ctx *ctx, bool cancel) +{ + struct damon_call_control *control; + int ret = 0; + + mutex_lock(&ctx->call_control_lock); + control = ctx->call_control; + mutex_unlock(&ctx->call_control_lock); + if (!control) + return; + if (cancel) { + control->canceled = true; + } else { + ret = control->fn(control->data); + control->return_code = ret; + } + complete(&control->completion); + mutex_lock(&ctx->call_control_lock); + ctx->call_control = NULL; + mutex_unlock(&ctx->call_control_lock); +} + /* Returns negative error code if it's not activated but should return */ static int kdamond_wait_activation(struct damon_ctx *ctx) { @@ -1941,6 +2023,7 @@ static int kdamond_wait_activation(struct damon_ctx *ctx) if (ctx->callback.after_wmarks_check && ctx->callback.after_wmarks_check(ctx)) break; + kdamond_call(ctx, true); } return -EBUSY; } @@ -2011,6 +2094,7 @@ static int kdamond_fn(void *data) if (ctx->callback.after_sampling && ctx->callback.after_sampling(ctx)) break; + kdamond_call(ctx, false); kdamond_usleep(sample_interval); ctx->passed_sample_intervals++; @@ -2072,6 +2156,8 @@ static int kdamond_fn(void *data) ctx->kdamond = NULL; mutex_unlock(&ctx->kdamond_lock); + kdamond_call(ctx, true); + mutex_lock(&damon_lock); nr_running_ctxs--; if (!nr_running_ctxs && running_exclusive_ctxs)