From patchwork Fri Apr 5 06:08:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7239CCD11C2 for ; Fri, 5 Apr 2024 06:24:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B6886B00AA; Fri, 5 Apr 2024 02:24:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 03C6D6B00AB; Fri, 5 Apr 2024 02:24:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF9B96B00AE; Fri, 5 Apr 2024 02:24:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C525F6B00AA for ; Fri, 5 Apr 2024 02:24:22 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6433C160941 for ; Fri, 5 Apr 2024 06:24:22 +0000 (UTC) X-FDA: 81974488764.29.5B541EF Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf23.hostedemail.com (Postfix) with ESMTP id 58A66140011 for ; Fri, 5 Apr 2024 06:24:20 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712298260; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y9OC4YXYFjc+2FMc0Wyr+afdOtmeZZzGlGXJ/Mb/OPk=; b=fJTgaR1PxI14PUR574fJ8FHpCu8HA+9lxT9qaVOkB6Arlyz4HNwaLDzgigV6LUBH7DOMjs WYvUcbypgx5rsdBd/A5n/hVpDQ5Y9mNabUSk0qR2RCsKxKwiSReZunRjQtKrpMsAEwCLkW bvi7P1BlnHUweiAViqYG4UZfPkjH/kQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712298260; a=rsa-sha256; cv=none; b=lMayIrjyKU02CcBoNlyjvpLwSJleJDa2KscwauHPptWd2rQjjbNn4Gpn6PnNb9jcp/QMRS Il44t98oB2eKdmQP4eyvPEHczs7Jjp42c4RwLL7DEYpvPRsOIFPsBXGLr7mQeW2R/vTTxl eIyGZi0+N6rh77vp6XWckdulfWdfLOQ= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-86-660f957f95fe From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 1/7] mm/damon/paddr: refactor DAMOS_PAGEOUT with migration_mode Date: Fri, 5 Apr 2024 15:08:50 +0900 Message-ID: <20240405060858.2818-2-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRmVeSWpSXmKPExsXC9ZZnkW79VP40g0kXlS0m9hhYzFm/hs1i 140Qi/sPXrNb/N97jNHiyf/frBYnbjayWXR+X8picXnXHDaLe2v+s1ocWX+WxWLz2TPMFouX q1ns63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95jdxD2WHr6DZvHhiYgsXPWXXaPln23 2D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3yweOx9aerzYPJPRo7f5HZvH501yAXxR XDYpqTmZZalF+nYJXBlzGjYwF7TwVsz7sYatgfEKVxcjJ4eEgIlE94t9jF2MHGD27WdaIGE2 ATWJKy8nMYGERQQcJFZ9Vehi5OJgFvjPLHG59xcrSI2wQJhE0/JXYDaLgKrE2n13WUBsXgEz if1HXzJBjNeUeLz9JzuIzSlgLjHv8QYwWwio5tPBY+wQ9YISJ2c+AetlFpCXaN46mxlkmYTA KXaJzzua2SAGSUocXHGDZQIj/ywkPbOQ9CxgZFrFKJSZV5abmJljopdRmZdZoZecn7uJERh1 y2r/RO9g/HQh+BCjAAejEg+vx1y+NCHWxLLiytxDjBIczEoivN0OvGlCvCmJlVWpRfnxRaU5 qcWHGKU5WJTEeY2+lacICaQnlqRmp6YWpBbBZJk4OKUaGNme7eV0WT0z7nfAmw2pXAvTP8Vu CT4TmjfVRrQ3T+R5eFryrkTPFsOtAtE8t7+/Yj03c8H2Q6wzjf+pZ6o/VkgyuF7yM2rhmy1b nUv8Wh6n3f9UEZi6mF3/9bF/M1xeJAQynvCJ810UGajY/y5m/ecPG1MkW1YeSlYRtf3w4TaP +yKRSfa2k5VYijMSDbWYi4oTASXCDe+2AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT7d+Kn+awd2dohYTewws5qxfw2ax 60aIxf0Hr9kt/u89xmjx5P9vVosTNxvZLD4/e81s0fnkO6PF4bknWS06vy9lsbi8aw6bxb01 /1ktjqw/y2Kx+ewZZovFy9UsDl17zmqxr+MBk8Xhr2+YLCZfWsBm8WLKGUaLk7Mms1jMPnqP 3UHcY+npN2weG5qAxM5Zd9k9WvbdYvdYsKnUo+XIW1aPxXteMnlsWtXJ5rHp0yR2jxMzfrN4 7Hxo6fFi80xGj97md2we3257eCx+8YHJ4/MmuQCBKC6blNSczLLUIn27BK6MOQ0bmAtaeCvm /VjD1sB4hauLkYNDQsBE4vYzrS5GTg42ATWJKy8nMYGERQQcJFZ9Vehi5OJgFvjPLHG59xcr SI2wQJhE0/JXYDaLgKrE2n13WUBsXgEzif1HXzKB2BICmhKPt/9kB7E5Bcwl5j3eAGYLAdV8 OniMHaJeUOLkzCdgvcwC8hLNW2czT2DkmYUkNQtJagEj0ypGkcy8stzEzBxTveLsjMq8zAq9 5PzcTYzASFtW+2fiDsYvl90PMQpwMCrx8HrM5UsTYk0sK67MPcQowcGsJMLb7cCbJsSbklhZ lVqUH19UmpNafIhRmoNFSZzXKzw1QUggPbEkNTs1tSC1CCbLxMEp1cC4e7Y008H3Puvn802q KLBwYKqy+vuxWH3tMi73NL1wTbvn/h7l23ofn17MY7ciq/dfWHvN5Ztz96c+j6jVe141oa7V TanTctfMV50q3bOyr9jJ3JUvuflup6jOxUVSO+1jJnOFVwrx3fBly9J+/6hNOyshcXX1WYkp R14Y7LGYEbY0iJU/r1qJpTgj0VCLuag4EQAvLqFpsAIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 58A66140011 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: ecc8y3oseq1mbfcrrnqmrtizek8xzwrt X-HE-Tag: 1712298260-392410 X-HE-Meta: U2FsdGVkX1+c6XBwR/LVfE/DMW+Dg4XPPgPTJYLDZ3vmtOyHJibwzEesMPWPcDSKcDpthfWy0mHmMsFptDKnxsgE++be0s8DtpMW2mXWA3YLWo0nbCVQg7SYAF6DncZsrTxmweBGcy9Mj692ltvhJfGDKjQTRU1SSRcff/ZMhLRocEp6752prA34bG9NpP9pFXJS5f3YQrfz0aq8j28CtEsRuCoGNhngTyGt6Zghf4b4gJy1HGk5ExksgrC5W7Ewo2O74OadHrkcoXI0A60MLSD3MwPE+vfgpeBpwlMMS2gvjS8JJYCtiwdceIrEEMr1tFpRoXqh5C5M2uBu8xwH41Qqxu2MIyV+OLhbEPH4jakaWtZ1SkgCqEozGfp0JuO6Q9EKMSJvmwAX4D6q9lO2bYavgLgROHE/DZo9uPGSGahu35m+gBwZfsUyKP7rmZZXabnP6pASCE/haFUgkEHK3JSHImHQcBVQfno8mNOFlpbSgxnbrHweY7CE9c7DRzojzj2QL2h28rzyAgPmvKwLjXEtjDAP+RTvj+vs3Oyq3M4z7mdmZ9YZDbR4cz3mZZFdpjcr/e3KiBb3yJ5rNDSmHiVLFYVxP9A+aKSBHsRFRFz+UU/ufp5LFxXPRPGZZ5EsP1zt8fM5V2pc3tcfHW3XBRUM5h07jSP0Cw6vnoKMu41iqZkm+hM3XYLZJWOvYNdLLF8u1kdezDxGExwVkhwkcm08lFqhCiQvKa1f/KUfn61EZJOgDHOUM6OBqhn86Br7J2hJED6O02t0bWxhGME1iYDzCEn1ZkZ0v/MKTtNQoBF0kssLhihulVzKhY6mPUJCnCVPGOFfFbsshd/t34ilxpW1IKltuWl1tomZKr3GSxDcFuePUstEFih1wqiLCoMFeBj+EEvyBQMkLxxDPkZGiwVyZSLHnbF25QrTzwhCGxVZIcca01l1q0s6Rf81vl+y8vkFihGZrbkjFTXva/Q Vz/Rvcci LsVAwOgBfMWrDbbJ59gFeoKXkOVXeLLx/34Ya3pN3Ka+Cl9F/GELs/Ylvr+IWhs6VE0dAvkirPaxr+Wgdltd38FWR42NWCAi1R70dlLpMzRnABZnOIOJ+M01fIW6gSVoAwlISzdIZboMuAVSC7+Q8WRyqNA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation patch that introduces migration modes. The damon_pa_pageout is renamed to damon_pa_migrate and it receives an extra argument for migration_mode. No functional changes applied. Signed-off-by: Honggyu Kim --- mm/damon/paddr.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 081e2a325778..277a1c4d833c 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -224,7 +224,12 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) return false; } -static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) +enum migration_mode { + MIG_PAGEOUT, +}; + +static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, + enum migration_mode mm) { unsigned long addr, applied; LIST_HEAD(folio_list); @@ -249,7 +254,14 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) put_folio: folio_put(folio); } - applied = reclaim_pages(&folio_list); + switch (mm) { + case MIG_PAGEOUT: + applied = reclaim_pages(&folio_list); + break; + default: + /* Unexpected migration mode. */ + return 0; + } cond_resched(); return applied * PAGE_SIZE; } @@ -297,7 +309,7 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, { switch (scheme->action) { case DAMOS_PAGEOUT: - return damon_pa_pageout(r, scheme); + return damon_pa_migrate(r, scheme, MIG_PAGEOUT); case DAMOS_LRU_PRIO: return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: From patchwork Fri Apr 5 06:08:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618572 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1A0DCD11C2 for ; Fri, 5 Apr 2024 06:24:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 82E626B00A9; Fri, 5 Apr 2024 02:24:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DD496B00AA; Fri, 5 Apr 2024 02:24:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67DC06B00AB; Fri, 5 Apr 2024 02:24:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 498246B00A9 for ; Fri, 5 Apr 2024 02:24:20 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 046941210CA for ; Fri, 5 Apr 2024 06:24:19 +0000 (UTC) X-FDA: 81974488680.28.08FD031 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf23.hostedemail.com (Postfix) with ESMTP id DAB8614000C for ; Fri, 5 Apr 2024 06:24:15 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712298258; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+zGyhrR9T2NFQOurh94KcoqTuEhCHBx1Vh9SP6FeliA=; b=EfQbb8cx/xOCZAR+SghJZ1qpD7NrKycROJp84O94AqP5VFssxAotCybVniLWQ4I1ILgLDz hC5HDX9hID4bPSGKB087hjnPfd1DAPc7ljByUsv8f8QzMQO7idjroD1ZAv1/U/G7sMpk6e 5QeWYabamVLogJas3OERsy2fR6iXm6I= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712298258; a=rsa-sha256; cv=none; b=AGk6h1T9qtk4kt8FXapw8izc9Ik9qgQexwIMAAgidMAFkvVEcvc3EcMI0PpMUnSOrvPJrO r4A8BxJXnFiNgy9djdz+ZyWQEFJaMZ1qLwsJ0xpqXWi7fw9i1aCuVzTDjyzPd1gX1WFXdj onzzno116vNTOBpiqvjoCyirYJORuZo= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-8f-660f9580d017 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 2/7] mm: make alloc_demote_folio externally invokable for migration Date: Fri, 5 Apr 2024 15:08:51 +0900 Message-ID: <20240405060858.2818-3-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRmVeSWpSXmKPExsXC9ZZnkW7DVP40g7N/TC0m9hhYzFm/hs1i 140Qi/sPXrNb/N97jNHiyf/frBYnbjayWXR+X8picXnXHDaLe2v+s1ocWX+WxWLz2TPMFouX q1ns63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95jdxD2WHr6DZvHhiYgsXPWXXaPln23 2D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3yweOx9aerzYPJPRo7f5HZvH501yAXxR XDYpqTmZZalF+nYJXBmHjj9hKTgkWHHr/VzGBsbNfF2MHBwSAiYSE85KdjFygpl31zazgdhs AmoSV15OYgIpERFwkFj1VaGLkYuDWeA/s8Tl3l+sIDXCAlESS0+8BLNZBFQlFty7D9bLK2Am 8e70O3aImZoSj7f/BLM5Bcwl5j3eAGYLAdV8OniMHaJeUOLkzCcsIDazgLxE89bZzCDLJASO sUv03PrOCDFIUuLgihssExj5ZyHpmYWkZwEj0ypGocy8stzEzBwTvYzKvMwKveT83E2MwKhb VvsnegfjpwvBhxgFOBiVeHg95vKlCbEmlhVX5h5ilOBgVhLh7XbgTRPiTUmsrEotyo8vKs1J LT7EKM3BoiTOa/StPEVIID2xJDU7NbUgtQgmy8TBKdXAWOatH6b/L89C8d/CqDVemTeuZH8+ X/5rodYsN9EFLz8aP2P9dG2NeM3+Gw9UlRUvNYRLdK37oDmjOCw8Rj346zMztQ0NSTc2GFY3 rWqOqVx95YSpS9aMAzIz7JZYvn3SuOnK6xcNrfv3bfm08e2e757Pvkaw9V0tdLlQbf7ggO0N LdH+Jg6lGCWW4oxEQy3mouJEAHAPeW+2AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT7dhKn+awclHGhYTewws5qxfw2ax 60aIxf0Hr9kt/u89xmjx5P9vVosTNxvZLD4/e81s0fnkO6PF4bknWS06vy9lsbi8aw6bxb01 /1ktjqw/y2Kx+ewZZovFy9UsDl17zmqxr+MBk8Xhr2+YLCZfWsBm8WLKGUaLk7Mms1jMPnqP 3UHcY+npN2weG5qAxM5Zd9k9WvbdYvdYsKnUo+XIW1aPxXteMnlsWtXJ5rHp0yR2jxMzfrN4 7Hxo6fFi80xGj97md2we3257eCx+8YHJ4/MmuQCBKC6blNSczLLUIn27BK6MQ8efsBQcEqy4 9X4uYwPjZr4uRk4OCQETibtrm9lAbDYBNYkrLycxdTFycIgIOEis+qrQxcjFwSzwn1nicu8v VpAaYYEoiaUnXoLZLAKqEgvu3Qfr5RUwk3h3+h07xExNicfbf4LZnALmEvMebwCzhYBqPh08 xg5RLyhxcuYTFhCbWUBeonnrbOYJjDyzkKRmIUktYGRaxSiSmVeWm5iZY6pXnJ1RmZdZoZec n7uJERhry2r/TNzB+OWy+yFGAQ5GJR5ej7l8aUKsiWXFlbmHGCU4mJVEeLsdeNOEeFMSK6tS i/Lji0pzUosPMUpzsCiJ83qFpyYICaQnlqRmp6YWpBbBZJk4OKUaGLdVHdhTwN9+yD3Jw63+ JXvOuqMRXl+z9825+mHGKZ8WR529Mb90dG3cTi3O4ckKf3TTi/vew4yrTr3Vy024a479SPhf db/Q7PuhRReyTaTnxlZs65JO8bgv9NmodPkpmVpf37t926983XPrcXDdBIZD6Wn5jCF5ggfs znZ9+DS3LaY2M776jxJLcUaioRZzUXEiAFYH+5KxAgAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: DAB8614000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: bqdh73ax7qtksbocez4maxdopgu31oc6 X-HE-Tag: 1712298255-693941 X-HE-Meta: U2FsdGVkX1/xy6g/57fTVE+dhO0fAhi/qABu0sa6GVqISWY//VZPv+TxGsRnZWXO9GYGT4DuAeHQKCsyIIctKsIZxCmL+vWq2hFJa4nxNKrx6bcVLsAiMJKKbcMWExdTbPxPfywWgOoueDpZcR3JmaJS4YDOacq6WRc1gDwpZ8FH7VGgUdDRb2pH1p6MhZZ7wlDNkiBtcTsUl/S00jk5WDPiCl7gGMJjJpz6hipg5GanoXd88PrkvDwtCPGbCoLu4P+KZFi1l5VJ+F8lrMmJsUqrnN78eoFbLYSO7H1kGn6sRDTJC+Tr7UWi8+9jf3MXVmhL49kghiuKuVBJvothucWyEW+PR3Qe/Tin5o8StQotUsZaJ4iFDo69kdRtoLkZjMATmCaH1lk3ABtnZq0eKTRB/foMREyyaXQLik2mLMMvARKiYHgMBRD8z5dAKdLIaPzOO/hMf8Z/Yvi1uYWljIfXIwsSBUA3PnL9NAMANm8YphbmE7AWs8V88H+Sizs7cX3Nty7oF1LmC5rY+R0LMi/ojeGRL2tE9xpg+i2M0MRanDKMh54MsbIVNyE3U/bFt248vaFnaSM4Mu8/XqQlDBEjzt3mpEwea2ZgxP1ghQFwII3PtuyTnJqbU9wZMl/3siXfDCYWZqTfg4qTf31agQcVQANc/hvogqK9dyYmsIk0DmdeAsF0vJd0Z6sMtSOys+L0UTrP9L3ZesDsYn5K82cTvGi8qEjdgPyvDAmTZTggXeKBc1jfy+8MevZCqdj57QZgpM64s0uoOK1ToG0Hxw16yU8K42D9dL77QQOuykD/kadoyQzRoRNcxSt6BDaJc1oFgf8utaKGwL0pi3msYpqyvSedC9amTyAkoD63t99kHHv6gJ8XR35RnO1NKPf9MVmnaEzzDxARfqf8Top746YvELeCvikbBjsL50kqardoXHqERnyGy6ieZZlDQQe7tKOPdsoFQDcvOzkzhsa zFeBEt4Q RacttSj/tYEyCVrvTVrYXm+at0di83bFFTfexrGNfLzVbyaDUVX/mqqMJTC2Aqf6j2XXdF258bFwixqAjvM8ik0xg1d3m5WxuqgvHPdfT0DncP+qp1Ap8s5IOhGGomF6+OutQWk4v/7w1zTbszEfQeSEbmg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The alloc_demote_folio can be used out of vmscan.c so it'd be better to remove static keyword from it. This function can also be used for both demotion and promotion so it'd be better to rename it from alloc_demote_folio to alloc_migrate_folio. Signed-off-by: Honggyu Kim Reviewed-by: SeongJae Park --- mm/internal.h | 1 + mm/vmscan.c | 10 +++++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index f309a010d50f..c96ff9bc82d0 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -866,6 +866,7 @@ extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long); extern void set_pageblock_order(void); +struct folio *alloc_migrate_folio(struct folio *src, unsigned long private); unsigned long reclaim_pages(struct list_head *folio_list); unsigned int reclaim_clean_pages_from_list(struct zone *zone, struct list_head *folio_list); diff --git a/mm/vmscan.c b/mm/vmscan.c index 4255619a1a31..9e456cac03b4 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -910,8 +910,7 @@ static void folio_check_dirty_writeback(struct folio *folio, mapping->a_ops->is_dirty_writeback(folio, dirty, writeback); } -static struct folio *alloc_demote_folio(struct folio *src, - unsigned long private) +struct folio *alloc_migrate_folio(struct folio *src, unsigned long private) { struct folio *dst; nodemask_t *allowed_mask; @@ -935,6 +934,11 @@ static struct folio *alloc_demote_folio(struct folio *src, if (dst) return dst; + /* + * Allocation failed from the target node so try to allocate from + * fallback nodes based on allowed_mask. + * See fallback_alloc() at mm/slab.c. + */ mtc->gfp_mask &= ~__GFP_THISNODE; mtc->nmask = allowed_mask; @@ -973,7 +977,7 @@ static unsigned int demote_folio_list(struct list_head *demote_folios, node_get_allowed_targets(pgdat, &allowed_mask); /* Demotion ignores all cpuset and mempolicy settings */ - migrate_pages(demote_folios, alloc_demote_folio, NULL, + migrate_pages(demote_folios, alloc_migrate_folio, NULL, (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, &nr_succeeded); From patchwork Fri Apr 5 06:08:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6EC2DCD1296 for ; Fri, 5 Apr 2024 06:09:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB6A26B009D; Fri, 5 Apr 2024 02:09:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E40806B00F2; Fri, 5 Apr 2024 02:09:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1DED6B00F3; Fri, 5 Apr 2024 02:09:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A47306B009D for ; Fri, 5 Apr 2024 02:09:13 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5D19681091 for ; Fri, 5 Apr 2024 06:09:13 +0000 (UTC) X-FDA: 81974450586.23.B5BB6F3 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf07.hostedemail.com (Postfix) with ESMTP id 6365140016 for ; Fri, 5 Apr 2024 06:09:11 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf07.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712297351; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mNgAPJ+qIlwiGuqJ6g7Mm+OsST2uRHDmb//xZNucxeU=; b=SDs6YGYuVaLB+Tr5+ExceJkPY/kNdItTokQscdi32MvEAAOQDEBsjrdpGaFujAed4vnEsE cuOzoTXO8oq0Lk1/ykMnHu7yAj3Ls0KAVQSvdZoHLPJB8tD8A/WPcz6LMpKE2kcdycmZU3 Rpp09loVsKX0mbi4QcBmqsT55sspTCU= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf07.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712297351; a=rsa-sha256; cv=none; b=gbDLAVMz7Ig2jwo9SX/3/W21LCEsefMWitpPEBlpYbRRZiO1jwIgdeblJk2u9rt9y1flPr x6q+uhKyoxsyjlbWu1mAQ5jEuQnJ9OxcKTLSOJbRIFRYcDPhV400yblt6FqO0NvJIdKhAs 2zXItxJx/cwLqlHehNTKT8+rYdoyUKY= X-AuditID: a67dfc5b-d6dff70000001748-9f-660f958295ec From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 3/7] mm/damon/sysfs-schemes: add target_nid on sysfs-schemes Date: Fri, 5 Apr 2024 15:08:52 +0900 Message-ID: <20240405060858.2818-4-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrIIsWRmVeSWpSXmKPExsXC9ZZnkW7TVP40gx0nhCwm9hhYzFm/hs1i 140Qi/sPXrNb/N97jNHiyf/frBYnbjayWXR+X8picXnXHDaLe2v+s1ocWX+WxWLz2TPMFouX q1ns63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95jdxD2WHr6DZvHhiYgsXPWXXaPln23 2D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3yweOx9aerzYPJPRo7f5HZvH501yAXxR XDYpqTmZZalF+nYJXBmbHy5jLbhjVXFg5hLmBsZ9Bl2MHBwSAiYSsw+FdjFygpmb3y1lBbHZ BNQkrrycxARSIiLgILHqq0IXIxcHs8B/ZonLvb9YQeLCAsES8655g5SzCKhKNPRtBQvzCphJ bJoaBzFRU+Lx9p/sIDangLnEvMcbwGwhoJJPB4+B2bwCghInZz5hAbGZBeQlmrfOZgZZJSFw il1i2tNtrBCDJCUOrrjBMoGRfxaSnllIehYwMq1iFMrMK8tNzMwx0cuozMus0EvOz93ECIy4 ZbV/oncwfroQfIhRgINRiYfXYy5fmhBrYllxZe4hRgkOZiUR3m4H3jQh3pTEyqrUovz4otKc 1OJDjNIcLErivEbfylOEBNITS1KzU1MLUotgskwcnFINjIH+PVPYlZRNStfYajLPX5ez+Vl5 y04jjwWvZlpHie35adks+d2wcdm26atc887yvBISNp248c5zDfeYv63rnFV95Zm4tv29zcon G2WyuOJXw6LVLEXz54mGHfw8leXBtjC93bIyunOnyWqEzRCtucxbeqrMvP7ChFdzn8dqaemZ Bej2tDRdVmIpzkg01GIuKk4EAASFmLK0AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprDIsWRmVeSWpSXmKPExsXCNUNLT7dpKn+aQV8Ts8XEHgOLOevXsFns uhFicf/Ba3aL/3uPMVo8+f+b1eLEzUY2i8/PXjNbdD75zmhxeO5JVovO70tZLC7vmsNmcW/N f1aLI+vPslhsPnuG2WLxcjWLQ9ees1rs63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95j dxD3WHr6DZvHhiYgsXPWXXaPln232D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3ywe Ox9aerzYPJPRo7f5HZvHt9seHotffGDy+LxJLkAgissmJTUnsyy1SN8ugStj88NlrAV3rCoO zFzC3MC4z6CLkZNDQsBEYvO7pawgNpuAmsSVl5OYuhg5OEQEHCRWfVXoYuTiYBb4zyxxufcX K0hcWCBYYt41b5ByFgFViYa+rWBhXgEziU1T4yAmako83v6THcTmFDCXmPd4A5gtBFTy6eAx MJtXQFDi5MwnLCA2s4C8RPPW2cwTGHlmIUnNQpJawMi0ilEkM68sNzEzx1SvODujMi+zQi85 P3cTIzDOltX+mbiD8ctl90OMAhyMSjy8HnP50oRYE8uKK3MPMUpwMCuJ8HY78KYJ8aYkVlal FuXHF5XmpBYfYpTmYFES5/UKT00QEkhPLEnNTk0tSC2CyTJxcEo1MD6Uu/581YT/bsqSy4IC ReRdy5YveLJV58GjZdvu3P5vfeDwy4eNLy6sbd8bYsf4TIxv2dT+5Sbb7K8lmcuf/+8kmTuN c0ZwiBSrVO6JQw+373+8fbIDQ82jrRt2X24KatvRbii07+iqW4+/6yTmfX7Zu/HSsSTtq+yT W6yPR3eWHc78fm9bUu9nJZbijERDLeai4kQA5lJONa8CAAA= X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 6365140016 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 3ndynzirfkets14oru3686ugahk7mki4 X-HE-Tag: 1712297351-332614 X-HE-Meta: U2FsdGVkX18UTY41Iil693QB3Br9AZCTEDBtLKO3g8JgSkNdVi4FsH1F6/yxTH6Iz20XQ7IuEkvrx7l+NG8Gqhi++yCjEsFvbdafyhxZsBMR7TSRceupo8n1rZ5AMjRhUhZHEddIYFeUDMeZnPH1sNv/oUzMvEze0jDOGA6fiOK99dH26c4RsrVFYwoIb4caKVL1SEMee80Xls3N/iMCt58FbpLLgZPJY3hTp1SLNhfqE21R1DLbL+KtSny/U0i0wTPTV4Ru/nWJ4IGJo7QDRTyGvEoH/htS112o22KRkhzZlO8E+GqpnyTfWHt3PlqhYWmVZRNd+FL7W1XG188h6sIhnj8caXclYceoKs3juERc12AoOmzygmGRasFfhkoSny3AO0/eyKiJCqFBagn2WzMaTvzCW9r9/lEtjule+l7v3qUQFJGRvoGuzJczF3d3QzoURMisn2RQiz4PpWrO54UbAKeq+VY0NNrRIEsiJVsB5SlxaHeUh85nFKV7NvP2W0MFSnwXnN1EWM5Z2PPD5YEc46YHBZfFFhy8Cb2wGBwUzJDyfQ6G54CQGfrJJHztXDL5h0ieIepXynXXDS5Azy7CIktTdu86kh6NnUJhfqZKS6sekrOYBtyUPofO35yXKaixz/XV+kArKLWuQ7JsSgfdUc+ZCtEVcu/EhMjKNVO/yIZLe8Sb+f1P9JSbzBCQRtDO+Qvkdhyuao0gbvLhJYAZ95HQ7M9ovgV2lWm9jxie1seE0emBSWUfQZKj3J2ssSFGXdRvd8qbz8Qab8H4n8sZdYaLsjSKCQ84+Fyx/SB1mTQI9dZhrLqbeneKoqYHBZRkImcbiw/UQ5s8aLoJ2JzTGq4MnzprTL4XfxHflhvqc9+llLQCiOp1Y+xUnZFX5lD5waGUUkQSZ5TA2D8gEYKegUaFEEZZXUP+w5uIqKM25gvWHo69gDj46HDYplcjiCB0q+9+qNPSSc2Xa5d uoGqS6X6 f8BaAXG+2IS7RryBoxqwqA2zXEUAcrwn6kb3oLBxPnQJt/HSMCBmYPLUPyudnmnoaNPSmUlAKe8Q1dVYdf/GJva2GJXnV5shbzfpyCeGtMmDmwLwEjnf6gwEKUqx82fpXo4YYrCyWrZq6Oj+fL/OcmvbB/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeongtak Ji This patch adds target_nid under /sys/kernel/mm/damon/admin/kdamonds//contexts//schemes// The 'target_nid' can be used as the destination node for DAMOS actions such as DAMOS_MIGRATE_{HOT,COLD} in the follow up patches. Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- include/linux/damon.h | 11 ++++++++++- mm/damon/core.c | 5 ++++- mm/damon/dbgfs.c | 2 +- mm/damon/lru_sort.c | 3 ++- mm/damon/reclaim.c | 3 ++- mm/damon/sysfs-schemes.c | 33 ++++++++++++++++++++++++++++++++- 6 files changed, 51 insertions(+), 6 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 5881e4ac30be..24ea33a03d5d 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -337,6 +337,7 @@ struct damos_access_pattern { * @apply_interval_us: The time between applying the @action. * @quota: Control the aggressiveness of this scheme. * @wmarks: Watermarks for automated (in)activation of this scheme. + * @target_nid: Destination node if @action is "migrate_{hot,cold}". * @filters: Additional set of &struct damos_filter for &action. * @stat: Statistics of this scheme. * @list: List head for siblings. @@ -352,6 +353,10 @@ struct damos_access_pattern { * monitoring context are inactive, DAMON stops monitoring either, and just * repeatedly checks the watermarks. * + * @target_nid is used to set the migration target node for migrate_hot or + * migrate_cold actions, which means it's only meaningful when @action is either + * "migrate_hot" or "migrate_cold". + * * Before applying the &action to a memory region, &struct damon_operations * implementation could check pages of the region and skip &action to respect * &filters @@ -373,6 +378,9 @@ struct damos { /* public: */ struct damos_quota quota; struct damos_watermarks wmarks; + union { + int target_nid; + }; struct list_head filters; struct damos_stat stat; struct list_head list; @@ -677,7 +685,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks); + struct damos_watermarks *wmarks, + int target_nid); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index 5b325749fc12..7ff0259d9fa6 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -316,7 +316,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks) + struct damos_watermarks *wmarks, + int target_nid) { struct damos *scheme; @@ -341,6 +342,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, scheme->wmarks = *wmarks; scheme->wmarks.activated = true; + scheme->target_nid = target_nid; + return scheme; } diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index 7dac24e69e3b..d04fdccfa65b 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -279,7 +279,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(&pattern, action, 0, "a, - &wmarks); + &wmarks, NUMA_NO_NODE); if (!scheme) goto fail; diff --git a/mm/damon/lru_sort.c b/mm/damon/lru_sort.c index 3de2916a65c3..3775f0f2743d 100644 --- a/mm/damon/lru_sort.c +++ b/mm/damon/lru_sort.c @@ -163,7 +163,8 @@ static struct damos *damon_lru_sort_new_scheme( /* under the quota. */ "a, /* (De)activate this according to the watermarks. */ - &damon_lru_sort_wmarks); + &damon_lru_sort_wmarks, + NUMA_NO_NODE); } /* Create a DAMON-based operation scheme for hot memory regions */ diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c index 66e190f0374a..84e6e96b5dcc 100644 --- a/mm/damon/reclaim.c +++ b/mm/damon/reclaim.c @@ -147,7 +147,8 @@ static struct damos *damon_reclaim_new_scheme(void) /* under the quota. */ &damon_reclaim_quota, /* (De)activate this according to the watermarks. */ - &damon_reclaim_wmarks); + &damon_reclaim_wmarks, + NUMA_NO_NODE); } static void damon_reclaim_copy_quota_status(struct damos_quota *dst, diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index ae0f0b314f3a..1a30ea82c890 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -6,6 +6,7 @@ */ #include +#include #include "sysfs-common.h" @@ -1393,6 +1394,7 @@ struct damon_sysfs_scheme { struct damon_sysfs_scheme_filters *filters; struct damon_sysfs_stats *stats; struct damon_sysfs_scheme_regions *tried_regions; + int target_nid; }; /* This should match with enum damos_action */ @@ -1418,6 +1420,7 @@ static struct damon_sysfs_scheme *damon_sysfs_scheme_alloc( scheme->kobj = (struct kobject){}; scheme->action = action; scheme->apply_interval_us = apply_interval_us; + scheme->target_nid = NUMA_NO_NODE; return scheme; } @@ -1640,6 +1643,28 @@ static ssize_t apply_interval_us_store(struct kobject *kobj, return err ? err : count; } +static ssize_t target_nid_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct damon_sysfs_scheme *scheme = container_of(kobj, + struct damon_sysfs_scheme, kobj); + + return sysfs_emit(buf, "%d\n", scheme->target_nid); +} + +static ssize_t target_nid_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + struct damon_sysfs_scheme *scheme = container_of(kobj, + struct damon_sysfs_scheme, kobj); + int err = 0; + + /* TODO: error handling for target_nid range. */ + err = kstrtoint(buf, 0, &scheme->target_nid); + + return err ? err : count; +} + static void damon_sysfs_scheme_release(struct kobject *kobj) { kfree(container_of(kobj, struct damon_sysfs_scheme, kobj)); @@ -1651,9 +1676,13 @@ static struct kobj_attribute damon_sysfs_scheme_action_attr = static struct kobj_attribute damon_sysfs_scheme_apply_interval_us_attr = __ATTR_RW_MODE(apply_interval_us, 0600); +static struct kobj_attribute damon_sysfs_scheme_target_nid_attr = + __ATTR_RW_MODE(target_nid, 0600); + static struct attribute *damon_sysfs_scheme_attrs[] = { &damon_sysfs_scheme_action_attr.attr, &damon_sysfs_scheme_apply_interval_us_attr.attr, + &damon_sysfs_scheme_target_nid_attr.attr, NULL, }; ATTRIBUTE_GROUPS(damon_sysfs_scheme); @@ -1956,7 +1985,8 @@ static struct damos *damon_sysfs_mk_scheme( damos_sysfs_set_quota_score(sysfs_quotas->goals, "a); scheme = damon_new_scheme(&pattern, sysfs_scheme->action, - sysfs_scheme->apply_interval_us, "a, &wmarks); + sysfs_scheme->apply_interval_us, "a, &wmarks, + sysfs_scheme->target_nid); if (!scheme) return NULL; @@ -1987,6 +2017,7 @@ static void damon_sysfs_update_scheme(struct damos *scheme, scheme->action = sysfs_scheme->action; scheme->apply_interval_us = sysfs_scheme->apply_interval_us; + scheme->target_nid = sysfs_scheme->target_nid; scheme->quota.ms = sysfs_quotas->ms; scheme->quota.sz = sysfs_quotas->sz; From patchwork Fri Apr 5 06:08:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618552 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7D0ECD11C2 for ; Fri, 5 Apr 2024 06:09:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9FC766B00F2; Fri, 5 Apr 2024 02:09:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95E2D6B00F3; Fri, 5 Apr 2024 02:09:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 762736B00F4; Fri, 5 Apr 2024 02:09:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 521D66B00F2 for ; Fri, 5 Apr 2024 02:09:15 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 182CD14010F for ; Fri, 5 Apr 2024 06:09:15 +0000 (UTC) X-FDA: 81974450670.08.613867F Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf28.hostedemail.com (Postfix) with ESMTP id 16CA4C000A for ; Fri, 5 Apr 2024 06:09:12 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712297353; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pl1fj+TbSzfW2r8nLoAnsEeUJGcvQxnNOEMN4xvZmkM=; b=C9q3oJgIQrMWX85YYwr3KckxqoXT+d8ppbt3KFLd92p81JTcjNCe1FLzrCo7JFjgASBivB GIHkEwITn00WR+8mR/KK5qDW2DlAgPS7VxyISfsYRSTzif3P2Y6WbpKyC/73UzAEX9Q7YB vR/P/aD5MkSAxnhzSiiQ0erfJixZUr0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712297353; a=rsa-sha256; cv=none; b=s5dxgDdHQtlROWEVYOeATIMbB8EPW/VuiOq28oL76awpC5Wd4Ol81TayFqR9uXWcfsb4Wp bKDohX0lUIYH07S2BfBssp9sOQJhyi5BuBD7ELIHYwUPGDcDz/S2L58rrKMRLjYSfoSuMm ZJlapshgf2sw1uYJStWnuRXFqZfUwJ0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-ac-660f95839088 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 4/7] mm/migrate: add MR_DAMON to migrate_reason Date: Fri, 5 Apr 2024 15:08:53 +0900 Message-ID: <20240405060858.2818-5-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRmVeSWpSXmKPExsXC9ZZnkW7zVP40gwtftS0m9hhYzFm/hs1i 140Qi/sPXrNb/N97jNHiyf/frBYnbjayWXR+X8picXnXHDaLe2v+s1ocWX+WxWLz2TPMFouX q1ns63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95jdxD2WHr6DZvHhiYgsXPWXXaPln23 2D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3yweOx9aerzYPJPRo7f5HZvH501yAXxR XDYpqTmZZalF+nYJXBknrl9kLljGWdH56zdTA+M99i5GTg4JAROJo4s3sMHYS29tBYuzCahJ XHk5iamLkYNDRMBBYtVXhS5GLg5mgf/MEpd7f7GC1AgLuEks6b/ODFLDIqAqcXZXBEiYV8BM 4mvzK0aIkZoSj7f/BBvJKWAuMe/xBjBbCKjm08Fj7BD1ghInZz5hAbGZBeQlmrfOZgbZJSFw jF1iycwdrBCDJCUOrrjBMoGRfxaSnllIehYwMq1iFMrMK8tNzMwx0cuozMus0EvOz93ECIy6 ZbV/oncwfroQfIhRgINRiYfXYy5fmhBrYllxZe4hRgkOZiUR3m4H3jQh3pTEyqrUovz4otKc 1OJDjNIcLErivEbfylOEBNITS1KzU1MLUotgskwcnFINjFPbvzneWznp19+TfxrmB6zesuOw 3I2v99MX+EoF7jfYzx23k31L8c2Pn3nSmfT++TjcefyPIUFHQnaz7+Z3l5fcDhVxZCu/uy5Z eqbEBbPcP2uV51SF9GTc5bu9Mn9S8KKNMsqvn8ZOecE6UfPgo1XptVM+X153m3vTr5lsNzvn P90v57fSrGq+EktxRqKhFnNRcSIA6JiNaLYCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT7d5Kn+awdaNMhYTewws5qxfw2ax 60aIxf0Hr9kt/u89xmjx5P9vVosTNxvZLD4/e81s0fnkO6PF4bknWS06vy9lsbi8aw6bxb01 /1ktjqw/y2Kx+ewZZovFy9UsDl17zmqxr+MBk8Xhr2+YLCZfWsBm8WLKGUaLk7Mms1jMPnqP 3UHcY+npN2weG5qAxM5Zd9k9WvbdYvdYsKnUo+XIW1aPxXteMnlsWtXJ5rHp0yR2jxMzfrN4 7Hxo6fFi80xGj97md2we3257eCx+8YHJ4/MmuQCBKC6blNSczLLUIn27BK6ME9cvMhcs46zo /PWbqYHxHnsXIyeHhICJxNJbW8FsNgE1iSsvJzF1MXJwiAg4SKz6qtDFyMXBLPCfWeJy7y9W kBphATeJJf3XmUFqWARUJc7uigAJ8wqYSXxtfsUIMVJT4vH2n2AjOQXMJeY93gBmCwHVfDp4 jB2iXlDi5MwnLCA2s4C8RPPW2cwTGHlmIUnNQpJawMi0ilEkM68sNzEzx1SvODujMi+zQi85 P3cTIzDSltX+mbiD8ctl90OMAhyMSjy8HnP50oRYE8uKK3MPMUpwMCuJ8HY78KYJ8aYkVlal FuXHF5XmpBYfYpTmYFES5/UKT00QEkhPLEnNTk0tSC2CyTJxcEo1MDpx3F2kWvVkQoFu4dPq S4ucpTqXWJdf37nXh7dbWL3HWsfwxtdei2iTye6vdVXX2wgWVSpe6fUJy7xZoNebH7kpJ587 /fIH0eMTVt+0vZV+1s+dp+3DFfdK4xgOpW1njuc2cJ/WXHH4XeLsubP4Nv/+4L2pvtEo6g6H WnztlUvqBsKsRs++KrEUZyQaajEXFScCAKFMwPWwAgAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 16CA4C000A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: cf97w3cgg8yrcz41sp5kkpefojbo7fc3 X-HE-Tag: 1712297352-558239 X-HE-Meta: U2FsdGVkX1+QggMGywewio3Vef4o3rxz4GlS/kqod27KMlfkdwFvxjjyJy0M9ZAtS+hM5/uLtaRwisT9p/BZtgB6fe0QTJZilwB4yweW7NQXXg8kpZ2au/wYqVgI+f1eChypGTAWbTcK7tybPnxnq4AkHIA5a7mJGNUEa//tgwOgsRTMwdvNHM2pqhoD/qo9d/3mWJUsSGKyw+cNhlMHYZUoq3Ej9ow7zcTNSJIDypOCOog4G9MZ28HMgyTZE3xT4c67yiSEUbwM90L/P+77Yr785pHKmuocvDnlKJHXL0eWiBvZsNXyTn/EM7m68EDXuYiSGLFh/F0DDn5eACrJEDhMPG9crthInxt8TF8su1UIPriLxteueoyikMRw3uPjEbG0dkjCIUom33hcVWPZGMounNOFcaKO+WpBYoU3hWVkZnuPDqSyzuK1hXMf0JheWP5BA1ErkEY5bwTdMyYLGHZmGzcJZrqxMHO+zgJ0RR4aw4K/5pP+yJU6xUfOUeqjnBRB+tXXv0cq1iAETxzusznY3XQyJd163EpypdUo8tga5VK680WVtZuxOT/F0YPgCK0qUhbD2wAfZ4SX4j0nWJZtfS0/ijSuGTKEgsmOHudAztMkioIwz38CvKZiNIZ+XFJJtpc4D0MLKZi60yXphc5aFAhFnnn4el4XSHJJb1F+nECH24enBQw9Z+Ujlj0dfWXs6F2Xcu9zzZ/YEIO6nlpux1LgmNyZtYEb7l7FuSM8aKKW9Qv7LZGeQwU89kELsuDW5239zgqF9lAuXnnKV7o0b3fFd5Tg7TfQqfLjLvGaEyjrTHhtbpdwNexY2oT+7lymqzQ33Ofk7uRXIpY7PKHW+YDW5iYuWj6efao+/q3Fn4x3Mhy5OdGvJw3TJXpoQ249kf714Xgx3yPFhjxBboKQcTfLZ7vklg4HjT0qdQynqFqD0Mb2GtmlmmPeIiNiEZS6k7LQjN/yfK0HyM7 XWdZCuSl qzH7xxph7TV5SliAUaxjmzCj4KRvLoOpA6knBcvbVZ/K+fGMmzCpVcUHNCakuNLY5n20Ah2WVKRx1SGexz8Q03iSIDrc9l2tKXE8e5z50AtkcG20GUwFwGNMcVpGRAjJCusookkbakeSWqWA5RqaoSF9Fiw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The current patch series introduces DAMON based migration across NUMA nodes so it'd be better to have a new migrate_reason in trace events. Signed-off-by: Honggyu Kim Reviewed-by: SeongJae Park --- include/linux/migrate_mode.h | 1 + include/trace/events/migrate.h | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h index f37cc03f9369..cec36b7e7ced 100644 --- a/include/linux/migrate_mode.h +++ b/include/linux/migrate_mode.h @@ -29,6 +29,7 @@ enum migrate_reason { MR_CONTIG_RANGE, MR_LONGTERM_PIN, MR_DEMOTION, + MR_DAMON, MR_TYPES }; diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h index 0190ef725b43..cd01dd7b3640 100644 --- a/include/trace/events/migrate.h +++ b/include/trace/events/migrate.h @@ -22,7 +22,8 @@ EM( MR_NUMA_MISPLACED, "numa_misplaced") \ EM( MR_CONTIG_RANGE, "contig_range") \ EM( MR_LONGTERM_PIN, "longterm_pin") \ - EMe(MR_DEMOTION, "demotion") + EM( MR_DEMOTION, "demotion") \ + EMe(MR_DAMON, "damon") /* * First define the enums in the above macros to be exported to userspace From patchwork Fri Apr 5 06:08:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46AB2CD11C2 for ; Fri, 5 Apr 2024 06:09:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 73ED36B00F3; Fri, 5 Apr 2024 02:09:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6EE476B00F4; Fri, 5 Apr 2024 02:09:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F1996B00F5; Fri, 5 Apr 2024 02:09:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 30F846B00F3 for ; Fri, 5 Apr 2024 02:09:16 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id F40BBA17A3 for ; Fri, 5 Apr 2024 06:09:15 +0000 (UTC) X-FDA: 81974450670.15.E465807 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf07.hostedemail.com (Postfix) with ESMTP id E720F40022 for ; Fri, 5 Apr 2024 06:09:13 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf07.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712297354; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ESIWyjaNk5BwYYiYQCvAAQVOsQgbekkiQzOqtALzHTI=; b=xtYCZ3pYUidzIp27oRptJYD/LQkXWrMKIHkGV1GQLypKzlFGrwAJHP2pdzAgxb67kR+jnw 792U8gYpbBmsagETFYMDUq1noSornxiEnLxBDtOiVJmESYWBsDumgIZQNuKQ/v+rBz/52R 3Q+fBZfwZ6/scbO3qzdthrxkzd//Pnc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf07.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712297354; a=rsa-sha256; cv=none; b=CUP5sq4hiCnvSBV0912XKwNeV6UREklK+LAsYicnAiIwbhqRw4J/iCGC7DuvIrhEei52KG y+G9U28F8DTW329DGashgBsY6CoPXBkLqjkMBpSsZm7EYkj6uC0I7NhhQkGAhSHLhPENVW kLUwn1u0JoNIwatDLqV8/gUgvmwQNBg= X-AuditID: a67dfc5b-d6dff70000001748-bf-660f9584a273 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 5/7] mm/damon/paddr: introduce DAMOS_MIGRATE_COLD action for demotion Date: Fri, 5 Apr 2024 15:08:54 +0900 Message-ID: <20240405060858.2818-6-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrCIsWRmVeSWpSXmKPExsXC9ZZnkW7rVP40g/cv2S0m9hhYzFm/hs1i 140Qi/sPXrNb/N97jNHiyf/frBYnbjayWXR+X8picXnXHDaLe2v+s1ocWX+WxWLz2TPMFouX q1ns63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95jdxD2WHr6DZvHhiYgsXPWXXaPln23 2D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3yweOx9aerzYPJPRo7f5HZvH501yAXxR XDYpqTmZZalF+nYJXBlzt39lKXjmWHFx+wPWBsafJl2MnBwSAiYS1yY8ZIWxv8xYwQRiswmo SVx5OQnI5uAQEXCQWPVVoYuRi4NZ4D+zxOXeX2D1wgIxEltu9bGA2CwCqhLv23+xg9i8AmYS 6w5MYYSYqSnxePtPsDingLnEvMcbwGwhoJpPB49B1QtKnJz5BGwOs4C8RPPW2cwgyyQEjrFL dPcuZ4EYJClxcMUNlgmM/LOQ9MxC0rOAkWkVo1BmXlluYmaOiV5GZV5mhV5yfu4mRmDcLav9 E72D8dOF4EOMAhyMSjy8HnP50oRYE8uKK3MPMUpwMCuJ8HY78KYJ8aYkVlalFuXHF5XmpBYf YpTmYFES5zX6Vp4iJJCeWJKanZpakFoEk2Xi4JRqYGTO1Z7VPcNyR8qcL2IbfwY1nGD8ukzz +eKnGe9WXD7HyMGgt2rulqqdLZvMHSYqXVwqe+C3yd11/tf+fNp1klX585qpDUsnG79RadJT fXK06keZ/Zeea8XyZxRCi++/PGd4V3NX6LmrjzSqmBgmH+lY+Vb+b2CvjcWd/XNPMzpsqb9j Pvuk/9oCJZbijERDLeai4kQAHlV+d7cCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT7dlKn+awYajxhYTewws5qxfw2ax 60aIxf0Hr9kt/u89xmjx5P9vVosTNxvZLD4/e81s0fnkO6PF4bknWS06vy9lsbi8aw6bxb01 /1ktjqw/y2Kx+ewZZovFy9UsDl17zmqxr+MBk8Xhr2+YLCZfWsBm8WLKGUaLk7Mms1jMPnqP 3UHcY+npN2weG5qAxM5Zd9k9WvbdYvdYsKnUo+XIW1aPxXteMnlsWtXJ5rHp0yR2jxMzfrN4 7Hxo6fFi80xGj97md2we3257eCx+8YHJ4/MmuQCBKC6blNSczLLUIn27BK6Mudu/shQ8c6y4 uP0BawPjT5MuRk4OCQETiS8zVjCB2GwCahJXXk4Csjk4RAQcJFZ9Vehi5OJgFvjPLHG59xcr SI2wQIzEllt9LCA2i4CqxPv2X+wgNq+AmcS6A1MYIWZqSjze/hMszilgLjHv8QYwWwio5tPB Y1D1ghInZz4Bm8MsIC/RvHU28wRGnllIUrOQpBYwMq1iFMnMK8tNzMwx1SvOzqjMy6zQS87P 3cQIjLVltX8m7mD8ctn9EKMAB6MSD6/HXL40IdbEsuLK3EOMEhzMSiK83Q68aUK8KYmVValF +fFFpTmpxYcYpTlYlMR5vcJTE4QE0hNLUrNTUwtSi2CyTBycUg2MrXcDg/iWGvI5yvDcEPSw P7yqU3jJ5C2Pvy4L8fm1582vb8cbBHa9dWFsOCph3xcfE7ij53oE2/o5kXf2dcY+e1Nr+oMr 4Hrr5n1XHJdsEYkI9Jt05UHSp92tM6sTGmznMuj59LzSXeVnl+zzf8/UavV1Jhr7TaNcmH7q CmYbMthL8AfIuscosRRnJBpqMRcVJwIAr8ixFrECAAA= X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: E720F40022 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: mtnqam31qk3bk5fq8t77rfrqnku3z9d4 X-HE-Tag: 1712297353-172573 X-HE-Meta: U2FsdGVkX19X8NtJSrKVUXTXEGyiKMjVuvrYMwO9a/J1w5l/EY+QeOa1tsY8Ji6xINU16zBWP4MakGuKpSkWEEmzeUdJdxeI0brfSOi9ocZ4fY21dOgGinsLtg1fttw8mxHXd1Cc4i+ndJ0w1zJ8fynl4PydaBC9b/0iuP1rT6ksNQJXTEccrFBTmSuUOuPTRmOKTOc3tE7kD/c250+TSvavnA3VF1rcOJ2B7SYeLO1sm/j6CG/8TuM8SixcvMCxfBGVembd3wiVk96RULzQBDWy1qP5Ui1FedgK1KCxy0lcd5xci0Ix7Xi8yaNT2b5ZYHO1sr7v6Oxgq7+Y03zFXADQ7q8DD7Im5lFyTmoSP1P/AeG3i1Ax1SRr19YlMKbnWlfD6j3rYaASnPJ9b3+UESW5ZX2sy9INddmhvKpqHPAOkHFddnd9/VpAzWcIKoIX0m0c1dikEQH/7cwlEzicJRO+5V1++Odi1FXxRlA8O9oENi1cbJAQXWLQhwyuitMdT3fRBXSaWkdgNsSMZGNS9YWDvsKLikXL8GpkM7UNil4SCZb1TrWVegu2F1y6Cy2kAjH+I3aJaDOsnlazHA2RSFP8rsCrK3GO49xycaI0Zy4NkwFgURnQsmutIaq3PcE8wLV9MUH0WM5QQvtBblTUOUmtk+JnIMXv4RPFFSCoAV7Yd/2DY4LAVEv8v3Wwu0cMUp8yu9Z0IoRsItr7e/+JGPtGt1II5RBu5FRLz+AYdItdLV2AeJGyWSRsrruUbItMbJbNuSB7Drc4A6haZAQcnRwMPMAc+eOjKT5AcPjEEMOhHO0mlaH4Rp8K7Ub+9+n3i/cJLnZ1gIPMGDn8rl0d67R4jMoUsNfj8IRPhny489vxazK8KxfL+kbrTv9CjXFcyMzN0HG9gvlWT3bBst7VVNyzj/fPJmHEyDHaqxg9Z+w85cq0Oo6YZxzJ5Xh1fGcY/GJHKWjxuVxbFjXb826 l9xC/J7B W5YDWL0EsJuIPqYxPwH6jXACbKtXdWTd05OIFGYamnckMfE6AwDLFakaV96L1CyfaGsC8YGLGD/CkhxWNUhTOsfh+TsHLaupF5jAXaTkPXoej4HAu9QqpP/sL1K2ZLwLvEruKSuAPOie4gzt1W3JEPjLQFg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch introduces DAMOS_MIGRATE_COLD action, which is similar to DAMOS_PAGEOUT, but migrate folios to the given 'target_nid' in the sysfs instead of swapping them out. The 'target_nid' sysfs knob is created by this patch to inform the migration target node ID. Here is one of the example usage of this 'migrate_cold' action. $ cd /sys/kernel/mm/damon/admin/kdamonds/ $ cat contexts//schemes//action migrate_cold $ echo 2 > contexts//schemes//target_nid $ echo commit > state $ numactl -p 0 ./hot_cold 500M 600M & $ numastat -c -p hot_cold Per-node process memory usage (in MBs) PID Node 0 Node 1 Node 2 Total -------------- ------ ------ ------ ----- 701 (hot_cold) 501 0 601 1101 Since there are some common routines with pageout, many functions have similar logics between pageout and migrate cold. damon_pa_migrate_folio_list() is a minimized version of shrink_folio_list(), but it's minified only for demotion. Signed-off-by: Honggyu Kim Signed-off-by: Hyeongtak Ji --- include/linux/damon.h | 2 + mm/damon/paddr.c | 146 ++++++++++++++++++++++++++++++++++++++- mm/damon/sysfs-schemes.c | 4 ++ 3 files changed, 151 insertions(+), 1 deletion(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 24ea33a03d5d..df8671e69a70 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_MIGRATE_COLD: Migrate for the given cold region. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions * @@ -122,6 +123,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_MIGRATE_COLD, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 277a1c4d833c..fe217a26f788 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -12,6 +12,9 @@ #include #include #include +#include +#include +#include #include "../internal.h" #include "ops-common.h" @@ -226,8 +229,137 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) enum migration_mode { MIG_PAGEOUT, + MIG_MIGRATE_COLD, }; +static unsigned int migrate_folio_list(struct list_head *migrate_folios, + struct pglist_data *pgdat, + int target_nid) +{ + unsigned int nr_succeeded; + nodemask_t allowed_mask = NODE_MASK_NONE; + + struct migration_target_control mtc = { + /* + * Allocate from 'node', or fail quickly and quietly. + * When this happens, 'page' will likely just be discarded + * instead of migrated. + */ + .gfp_mask = (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) | __GFP_NOWARN | + __GFP_NOMEMALLOC | GFP_NOWAIT, + .nid = target_nid, + .nmask = &allowed_mask + }; + + if (pgdat->node_id == target_nid || target_nid == NUMA_NO_NODE) + return 0; + + if (list_empty(migrate_folios)) + return 0; + + /* Migration ignores all cpuset and mempolicy settings */ + migrate_pages(migrate_folios, alloc_migrate_folio, NULL, + (unsigned long)&mtc, MIGRATE_ASYNC, MR_DAMON, + &nr_succeeded); + + return nr_succeeded; +} + +static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, + struct pglist_data *pgdat, + enum migration_mode mm, + int target_nid) +{ + unsigned int nr_migrated = 0; + struct folio *folio; + LIST_HEAD(ret_folios); + LIST_HEAD(migrate_folios); + + cond_resched(); + + while (!list_empty(folio_list)) { + struct folio *folio; + + cond_resched(); + + folio = lru_to_folio(folio_list); + list_del(&folio->lru); + + if (!folio_trylock(folio)) + goto keep; + + VM_BUG_ON_FOLIO(folio_test_active(folio), folio); + + /* Relocate its contents to another node. */ + list_add(&folio->lru, &migrate_folios); + folio_unlock(folio); + continue; +keep: + list_add(&folio->lru, &ret_folios); + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + } + /* 'folio_list' is always empty here */ + + /* Migrate folios selected for migration */ + nr_migrated += migrate_folio_list(&migrate_folios, pgdat, target_nid); + /* Folios that could not be migrated are still in @migrate_folios */ + if (!list_empty(&migrate_folios)) { + /* Folios which weren't migrated go back on @folio_list */ + list_splice_init(&migrate_folios, folio_list); + } + + try_to_unmap_flush(); + + list_splice(&ret_folios, folio_list); + + while (!list_empty(folio_list)) { + folio = lru_to_folio(folio_list); + list_del(&folio->lru); + folio_putback_lru(folio); + } + + return nr_migrated; +} + +static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, + enum migration_mode mm, + int target_nid) +{ + int nid; + unsigned int nr_migrated = 0; + LIST_HEAD(node_folio_list); + unsigned int noreclaim_flag; + + if (list_empty(folio_list)) + return nr_migrated; + + noreclaim_flag = memalloc_noreclaim_save(); + + nid = folio_nid(lru_to_folio(folio_list)); + do { + struct folio *folio = lru_to_folio(folio_list); + + if (nid == folio_nid(folio)) { + folio_clear_active(folio); + list_move(&folio->lru, &node_folio_list); + continue; + } + + nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), mm, + target_nid); + nid = folio_nid(lru_to_folio(folio_list)); + } while (!list_empty(folio_list)); + + nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), mm, + target_nid); + + memalloc_noreclaim_restore(noreclaim_flag); + + return nr_migrated; +} + static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, enum migration_mode mm) { @@ -247,7 +379,11 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, folio_test_clear_young(folio); if (!folio_isolate_lru(folio)) goto put_folio; - if (folio_test_unevictable(folio)) + /* + * Since unevictable folios can be demoted or promoted, + * unevictable test is needed only for pageout. + */ + if (mm == MIG_PAGEOUT && folio_test_unevictable(folio)) folio_putback_lru(folio); else list_add(&folio->lru, &folio_list); @@ -258,6 +394,10 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, case MIG_PAGEOUT: applied = reclaim_pages(&folio_list); break; + case MIG_MIGRATE_COLD: + applied = damon_pa_migrate_pages(&folio_list, mm, + s->target_nid); + break; default: /* Unexpected migration mode. */ return 0; @@ -314,6 +454,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_MIGRATE_COLD: + return damon_pa_migrate(r, scheme, MIG_MIGRATE_COLD); case DAMOS_STAT: break; default: @@ -334,6 +476,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_MIGRATE_COLD: + return damon_cold_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 1a30ea82c890..18b7d054c748 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1406,6 +1406,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "nohugepage", "lru_prio", "lru_deprio", + "migrate_cold", "stat", }; @@ -1659,6 +1660,9 @@ static ssize_t target_nid_store(struct kobject *kobj, struct damon_sysfs_scheme, kobj); int err = 0; + if (scheme->action != DAMOS_MIGRATE_COLD) + return -EINVAL; + /* TODO: error handling for target_nid range. */ err = kstrtoint(buf, 0, &scheme->target_nid); From patchwork Fri Apr 5 06:08:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79B4ACD1284 for ; Fri, 5 Apr 2024 06:09:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E6A376B00F4; Fri, 5 Apr 2024 02:09:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DF14C6B00F5; Fri, 5 Apr 2024 02:09:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C44BD6B00F6; Fri, 5 Apr 2024 02:09:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A846A6B00F4 for ; Fri, 5 Apr 2024 02:09:17 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 729481605B6 for ; Fri, 5 Apr 2024 06:09:17 +0000 (UTC) X-FDA: 81974450754.01.276D6AF Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf28.hostedemail.com (Postfix) with ESMTP id 5C4FEC0007 for ; Fri, 5 Apr 2024 06:09:15 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712297355; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GjpUlKwELBdpP06tAlYM7enX6svsxkACaqZOzMm1f0M=; b=OZIXmDbTKRKMkxnett4w3ZIutG1AJdMzRphOGG1LseoDC2hnbrL0eG1836ejkJYWD9ER6e sv4WeP9FV75QT5cAjgv5Lp/mCibcYXUs9TyXXTJ6JHWgpo2WQaefdDgjstJU0BjSHaXsC1 k3ItrvyxCv1xh+yU9JYwhzYMqZ/qo68= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712297355; a=rsa-sha256; cv=none; b=KjvBp2+q4yk6AjXCDmIvOz8S3UwUwMb66wjrTBN7lyavYBDnPi3x2l5Yms0c80xkUj6pZy 2lsweyDzrbQE2H+awz7Iily/T91PPiBjNTZsky/KPnuOw1BzosDjhcfgRrsDLt3//HFoGK CM6PAJk3yNLDw7bxr4KH3D7Z6qv8C9M= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-ce-660f958600fc From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 6/7] mm/damon/paddr: introduce DAMOS_MIGRATE_HOT action for promotion Date: Fri, 5 Apr 2024 15:08:55 +0900 Message-ID: <20240405060858.2818-7-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRmVeSWpSXmKPExsXC9ZZnkW7bVP40g9Mt8hYTewws5qxfw2ax 60aIxf0Hr9kt/u89xmjx5P9vVosTNxvZLDq/L2WxuLxrDpvFvTX/WS2OrD/LYrH57Blmi8XL 1Sz2dTxgsjj89Q2TxeRLC9gsXkw5w2hxctZkFovZR++xOwh7LD39hs1jQxOQ2DnrLrtHy75b 7B4LNpV6tBx5y+qxeM9LJo9NqzrZPDZ9msTucWLGbxaPnQ8tPV5snsno0dv8js3j8ya5AL4o LpuU1JzMstQifbsEroyNM8+xFLxTqph67Cd7A+M0mS5GDg4JAROJ26v9uxg5wcyli64ygdhs AmoSV15OYgIpERFwkFj1VaGLkYuDWeA/s8Tl3l+sIDXCAjESH2aeB7NZBFQlfjxbzAJi8wqY SRw9PpMJYqamxOPtP9lBbE4Bc4l5jzeA2UJANZ8OHmOHqBeUODnzCVgvs4C8RPPW2cwgyyQE jrFLrOjdyAYxSFLi4IobLBMY+Wch6ZmFpGcBI9MqRqHMvLLcxMwcE72MyrzMCr3k/NxNjMCo W1b7J3oH46cLwYcYBTgYlXh4PebypQmxJpYVV+YeYpTgYFYS4e124E0T4k1JrKxKLcqPLyrN SS0+xCjNwaIkzmv0rTxFSCA9sSQ1OzW1ILUIJsvEwSnVwLjet/qB7OnPBmzmEtdW13gl7fzQ fE8npWbXjF1p/bUfO7U8bPxSv3r8iWBgl+bfYtiRfthSR/BMzsLy9y9ZH/1oEVYM8bqpde9h oNTii+dV0nu2hpq9ivT4c/XS+lVx92TvmdWafWDdyfD727TJrsks92M9GZdaPOrXedf19mK4 HN+XfdPcdiixFGckGmoxFxUnAgAvypFstgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT7dtKn+aQd9CQYuJPQYWc9avYbPY dSPE4v6D1+wW//ceY7R48v83q8WJm41sFp+fvWa26HzyndHi8NyTrBad35eyWFzeNYfN4t6a /6wWR9afZbHYfPYMs8Xi5WoWh649Z7XY1/GAyeLw1zdMFpMvLWCzeDHlDKPFyVmTWSxmH73H 7iDusfT0GzaPDU1AYuesu+weLftusXss2FTq0XLkLavH4j0vmTw2repk89j0aRK7x4kZv1k8 dj609HixeSajR2/zOzaPb7c9PBa/+MDk8XmTXIBAFJdNSmpOZllqkb5dAlfGxpnnWAreKVVM PfaTvYFxmkwXIyeHhICJxNJFV5lAbDYBNYkrLycB2RwcIgIOEqu+KnQxcnEwC/xnlrjc+4sV pEZYIEbiw8zzYDaLgKrEj2eLWUBsXgEziaPHZzJBzNSUeLz9JzuIzSlgLjHv8QYwWwio5tPB Y+wQ9YISJ2c+AetlFpCXaN46m3kCI88sJKlZSFILGJlWMYpk5pXlJmbmmOoVZ2dU5mVW6CXn 525iBMbasto/E3cwfrnsfohRgINRiYfXYy5fmhBrYllxZe4hRgkOZiUR3m4H3jQh3pTEyqrU ovz4otKc1OJDjNIcLErivF7hqQlCAumJJanZqakFqUUwWSYOTqkGRuM05lmL5/4X2KG0/vU9 /zehc5peCP9NUohbvKtUgNlj+xYrg3mJyjNlDPVsbDze1gVFuy9207+Yd0de6BmDIcvz6dHL pn0z3h4ancnct2bJ9yUThRcd4D/i1Zx6TfbDwZpFWjP93HiLEzvVvT6U7Hyy47v/5k15i3uX thu2Mz8M2RlzI8XjoRJLcUaioRZzUXEiAIKQFJOxAgAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 5C4FEC0007 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 7nmx71wjy6ea7znw6okxd1zd3zcbqu1c X-HE-Tag: 1712297355-358618 X-HE-Meta: U2FsdGVkX1+tb19pNkXDS2dEIMIQe0CqZASEZsgcSPwxcOCGPoiIVmWcyRqSCpLqicS3s5RX51p8m8ZqW7XxhmPraJEzUInfkTjx9x8Utm3SudXfboqLrKSs7loOKV+nIuK4/ow1jG7m0TaTclS6U+Z1it+ZDmYGunfxpuyJmsGKUKjuOs9mnHe/DTGzfRq6kDP6QLie3h5FSd84vS487X8tfuJKvUVAJtDZrohNHs60L2yu8/3KrGhj/ErAFmvhvVehMS6bJ70OvTp/MbDnzkd40FfT6idTPJ3Y83K3QYGN7R9ySBggc6k8Bz/xzUkjTQkiGGxZouN4DYMyWsmQmNkCUIRaSnSDHb8sNO3Y7ehRmDwhMzq5frWcVf4I5qGtcNy4a0EpmW42BmqFOpIpttPg03RYK6iROjGHPIsJj7x1lHiwTGup5YShlhzUyhC4W5IRv8koWNjQglkWgjf95Y66I9HyNPxtQiFWFJFyMpnr3iCKxGCRONz4jqeDUSD+ZlCK27cPxExe8Xw7rkFb5H24QJhniHtQnaaJSLTwAtcdt5WHFIgUxxuJbnxyaUjEUixGnf9Y2GxzlLToVSxCOJVQbIeQhpVBZsBusvz5t7Bhaj2Mm/5CtkzVs0U1rro5ADiBdBQnWyavdy595GW8ynjTFxAckuVEzan6I8wsA7jN1HusRd3BImXIk+kzxjgvO7YzOB/jTaueoXWRPAbesWVxFh0GMx9YpNfJcIy5Tc+BG9j+PSRbczae/5iMCFqCW484nju8aF95hDuKLivZB2nw98WgGimJtsN8karwaO/mtF1v2PlSabWmE7sIFIqy4xh5+xLXl0e+apT0uLUGFDV5S+we3CcEV7s65TIRsBks3XE1QfAwP9zfapbMBTX7uWhmX3kX9IOgKpG0K3mxpEa8CHOTtzFcs67zLZrD7Fdoz662mnMH/GS4HyIvrjkVRzsHCHbHID6CON7oWp6 HsPpPJTj QeurL846LJGusIn2dQsslmku6ODzO9bIumxsPHjL6SVPk9BS3qP5ImgutgO3FiLIYqk9U4yAfa3tYbohC4p4biQKIm61vcabJetkauwmMnW5mm6GLr5MMGZP6q4xjy0fJP4hhttHcK41nSSE6b7tEMGpQHfKkfw/Qki5FxsrTwsT5JSBoHHo8JGfHsnehiCZCJHuS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeongtak Ji This patch introduces DAMOS_MIGRATE_HOT action, which is similar to DAMOS_MIGRATE_COLD, but it is targeted to migrate hot pages. It migrates pages inside the given region to the 'target_nid' NUMA node in the sysfs. Here is one of the example usage of this 'migrate_hot' action. $ cd /sys/kernel/mm/damon/admin/kdamonds/ $ cat contexts//schemes//action migrate_hot $ echo 0 > contexts//schemes//target_nid $ echo commit > state $ numactl -p 2 ./hot_cold 500M 600M & $ numastat -c -p hot_cold Per-node process memory usage (in MBs) PID Node 0 Node 1 Node 2 Total -------------- ------ ------ ------ ----- 701 (hot_cold) 501 0 601 1101 Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- include/linux/damon.h | 2 ++ mm/damon/paddr.c | 12 ++++++++++-- mm/damon/sysfs-schemes.c | 4 +++- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index df8671e69a70..934c95a7c042 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_MIGRATE_HOT: Migrate for the given hot region. * @DAMOS_MIGRATE_COLD: Migrate for the given cold region. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions @@ -123,6 +124,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_MIGRATE_HOT, DAMOS_MIGRATE_COLD, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index fe217a26f788..fd9d35b5cc83 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -229,6 +229,7 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) enum migration_mode { MIG_PAGEOUT, + MIG_MIGRATE_HOT, MIG_MIGRATE_COLD, }; @@ -375,8 +376,10 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, if (damos_pa_filter_out(s, folio)) goto put_folio; - folio_clear_referenced(folio); - folio_test_clear_young(folio); + if (mm != MIG_MIGRATE_HOT) { + folio_clear_referenced(folio); + folio_test_clear_young(folio); + } if (!folio_isolate_lru(folio)) goto put_folio; /* @@ -394,6 +397,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, case MIG_PAGEOUT: applied = reclaim_pages(&folio_list); break; + case MIG_MIGRATE_HOT: case MIG_MIGRATE_COLD: applied = damon_pa_migrate_pages(&folio_list, mm, s->target_nid); @@ -454,6 +458,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_MIGRATE_HOT: + return damon_pa_migrate(r, scheme, MIG_MIGRATE_HOT); case DAMOS_MIGRATE_COLD: return damon_pa_migrate(r, scheme, MIG_MIGRATE_COLD); case DAMOS_STAT: @@ -476,6 +482,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_MIGRATE_HOT: + return damon_hot_score(context, r, scheme); case DAMOS_MIGRATE_COLD: return damon_cold_score(context, r, scheme); default: diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 18b7d054c748..1d2f62aa79ca 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1406,6 +1406,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "nohugepage", "lru_prio", "lru_deprio", + "migrate_hot", "migrate_cold", "stat", }; @@ -1660,7 +1661,8 @@ static ssize_t target_nid_store(struct kobject *kobj, struct damon_sysfs_scheme, kobj); int err = 0; - if (scheme->action != DAMOS_MIGRATE_COLD) + if (scheme->action != DAMOS_MIGRATE_HOT && + scheme->action != DAMOS_MIGRATE_COLD) return -EINVAL; /* TODO: error handling for target_nid range. */ From patchwork Fri Apr 5 06:08:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13618555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F76FCD1284 for ; Fri, 5 Apr 2024 06:09:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C47D76B0083; Fri, 5 Apr 2024 02:09:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA9126B0085; Fri, 5 Apr 2024 02:09:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9ADC66B00A2; Fri, 5 Apr 2024 02:09:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7EC806B0083 for ; Fri, 5 Apr 2024 02:09:19 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3963F806FF for ; Fri, 5 Apr 2024 06:09:19 +0000 (UTC) X-FDA: 81974450838.20.7F7B234 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf07.hostedemail.com (Postfix) with ESMTP id 428E240003 for ; Fri, 5 Apr 2024 06:09:15 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf07.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712297356; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=maKFdQAUhIGZEbT5bLUApf6fiBu5sCPvbRWSZ1WJmpw=; b=7+tmGd86GRPBcq5w1Ju0LTRg8MvNIvWnsW0hvmpW3CVH/45+rCgVtjyKoXAHp/gSaKpRd2 ccQ1ZrJkqL68gHBxbKXeN0l7AxfkFmkavUktVkkTRcCZHWEqXJIEje1/5LtIjSfqEeeLVq mtWWuyA6Ia7ZdIwUAquGSk1V1l0Pa0k= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf07.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712297356; a=rsa-sha256; cv=none; b=ekoEYlAsKbYzXqKG6MzJH/fccc7odINppNZWf/Bb+rMP/Xzy4QhdBJU9CDaPkLLVvtHX1l bnIfQ91g5OnGcSokalHFYjG2XIbhuZaNuSQ4s/qV/Gi4r4LJQXzYDzFuyBF+QMHMaJne/f Ekj+ZjuoFAdXYJvIlLl7ylYWeZq54nM= X-AuditID: a67dfc5b-d6dff70000001748-e9-660f9587c0c4 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com, art.jeongseob@gmail.com Subject: [RFC PATCH v3 7/7] mm/damon: Add "damon_migrate_{hot,cold}" vmstat Date: Fri, 5 Apr 2024 15:08:56 +0900 Message-ID: <20240405060858.2818-8-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240405060858.2818-1-honggyu.kim@sk.com> References: <20240405060858.2818-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRmVeSWpSXmKPExsXC9ZZnkW77VP40g/v3jS0m9hhYzFm/hs1i 140Qi/sPXrNb/N97jNHiyf/frBYnbjayWXR+X8picXnXHDaLe2v+s1ocWX+WxWLz2TPMFouX q1ns63jAZHH46xsmi8mXFrBZvJhyhtHi5KzJLBazj95jdxD2WHr6DZvHhiYgsXPWXXaPln23 2D0WbCr1aDnyltVj8Z6XTB6bVnWyeWz6NInd48SM3yweOx9aerzYPJPRo7f5HZvH501yAXxR XDYpqTmZZalF+nYJXBknXi5iL+gWqzi65j5TA+NXwS5GTg4JAROJG51nGGHsbfvbmEFsNgE1 iSsvJzF1MXJwiAg4SKz6qtDFyMXBLPCfWeJy7y9WkBphAW+J6xuOMoLUsAioSnxbbwsS5hUw k1iz7wU7xEhNicfbf4LZnALmEvMebwCzhYBqPh08xg5RLyhxcuYTFhCbWUBeonnrbGaQXRIC p9glfm1+BnWbpMTBFTdYJjDyz0LSMwtJzwJGplWMQpl5ZbmJmTkmehmVeZkVesn5uZsYgVG3 rPZP9A7GTxeCDzEKcDAq8fB6zOVLE2JNLCuuzD3EKMHBrCTC2+3AmybEm5JYWZValB9fVJqT WnyIUZqDRUmc1+hbeYqQQHpiSWp2ampBahFMlomDU6qBsVOshzOZ2dzpVu0tg5VW/auevHjv lv74yKOknQeNZLdk2M0SyAxVPnR3b9MiiUMsG4+4qt5cNJt1O1MnY3bv3MNZT/v/X7zGd6t0 xa0yodMLvFNXz3zEe+r2/wB55ddRLDfYdkTkzmyN+znPQa9/4aV4vwcd9SWtFyQPL21K48ru +aJ+/KVqhxJLcUaioRZzUXEiAKaMzZm2AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT7d9Kn+awe31qhYTewws5qxfw2ax 60aIxf0Hr9kt/u89xmjx5P9vVosTNxvZLD4/e81s0fnkO6PF4bknWS06vy9lsbi8aw6bxb01 /1ktjqw/y2Kx+ewZZovFy9UsDl17zmqxr+MBk8Xhr2+YLCZfWsBm8WLKGUaLk7Mms1jMPnqP 3UHcY+npN2weG5qAxM5Zd9k9WvbdYvdYsKnUo+XIW1aPxXteMnlsWtXJ5rHp0yR2jxMzfrN4 7Hxo6fFi80xGj97md2we3257eCx+8YHJ4/MmuQCBKC6blNSczLLUIn27BK6MEy8XsRd0i1Uc XXOfqYHxq2AXIyeHhICJxLb9bcwgNpuAmsSVl5OYuhg5OEQEHCRWfVXoYuTiYBb4zyxxufcX K0iNsIC3xPUNRxlBalgEVCW+rbcFCfMKmEms2feCHWKkpsTj7T/BbE4Bc4l5jzeA2UJANZ8O HmOHqBeUODnzCQuIzSwgL9G8dTbzBEaeWUhSs5CkFjAyrWIUycwry03MzDHVK87OqMzLrNBL zs/dxAiMtGW1fybuYPxy2f0QowAHoxIPr8dcvjQh1sSy4srcQ4wSHMxKIrzdDrxpQrwpiZVV qUX58UWlOanFhxilOViUxHm9wlMThATSE0tSs1NTC1KLYLJMHJxSDYxVlz35HuTVFB8uXW0c wjiDP7l55bRvM1lzQlf5/pHbt+Ju/QXuj0VPxP96evh9mLw/5d0spTs+1asmvjH9eumt7E03 nak8E/sSe2Orte2YEwusOjnfNhlmTrYItvHXaPi0q+z9eh2mhy+EVF0U9eLb9Ez0tl0rMM9/ PfOVi+dvH9Fbn01mGymxFGckGmoxFxUnAgDlDExNsAIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 428E240003 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: nkhxh4zm9jjjcdq97jbz43j366ze7fws X-HE-Tag: 1712297355-396461 X-HE-Meta: U2FsdGVkX187tmJonLmPptudqEPv690amBKvL6vVZDMTWJ87R4FYBiaL7xlX9WXW4Zk8K/TpRt5iVe33/3rBzMHuVq97EqwjiRcKyp1DLUs1tE1FamtvSGjcgYzi1fE1MkGQyuRgk2HLoIW1biDxmvxbfIxeob4PePe73W2zFSf4OJyzaKmbxsXzkstIyvHaAeZF0TWbmoMVusZGGslQce+uXGSAabppoy1CVclpvOb3hVxdusg5+mO6P/HftawfUlkFsfMfOsd8kZ6jE7a03vswWyzVCJGTQilkh1+X8xZLxa0tLmrhy9HvWQCw0K6CMgZDhqFs5DuGl5ipPdjktROKyVBEuH56Oe1SOAfbMr0ZHtJ4D4l6WHl4rWjXp4QCP8vQDsK6+ft4WLe34W+M68cAe2hHx/qfUQamARaXb1d971DldnQEVA8qUYJTjL48Jf9gDRS8AghTrXbAs65qk7ApOJ4JVm0Fshmc66eIsBGEhehBsQPfAxRMjjU6EVobTQOYbuFDTnGadTPr0JsnlfzMfpXabq9syXWtXFLKucXq+yYVqjyYwe6HHBUY55xDfUjml9J8+7RvP6+ztDQ9GHJHt9gx0oxczOFDT+CqC5ivxakIX2CdZM/ma4Iwpd83iaVxViBi8ZMXrtaCnlZZMUNm+EtknSEeF/3o7YBNWPJf9VmPFskMEDYpuJANM9lEzi1vS4GrkLFo6Cd1cczUX/tPVHZU8Y39Y0wCQbMHaDrQ84EwDGyKgQEhtTJjE022YHbcAiRPn3+urpvO5tZyB6mhQhy1OyDIXTHhBRKy4SxiNkxeA3yQi56y4cUXK6TqnYHEzijIXTwfSes621Zl2gAooT7CiviQ13Ob6MMw443nWm5poSD/JcAzyVhL7XlxqKPHlfj7UOvxJJ8eRzfL2OlTBfvPKeTfh+AH87CZbZI+c4ddrLrZSlFf2nFdmoOwx6HCozHkJcmigXDnRcM 10tsyn8U D1D/ZzSQi0VMptkZSmEBV8s2xtyMynpLabn7Zz/piScGy0S2oqVZ/ywM+6iuVLURSUt2NQH13pwqQn+y351ay98b+e+dADrpKim/8bM3tRKN63+swJV/+4NTyq8i7Q8k0q6AhTs7ylUUtmVqdZ1/JOJ6mKw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch adds "damon_migrate_{hot,cold}" under node specific vmstat counters at the following location. /sys/devices/system/node/node*/vmstat The counted values are accumulcated to the global vmstat so it also introduces the same counter at /proc/vmstat as well. Signed-off-by: Honggyu Kim --- include/linux/mmzone.h | 4 ++++ mm/damon/paddr.c | 17 ++++++++++++++++- mm/vmstat.c | 4 ++++ 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index a497f189d988..0005372c5503 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -214,6 +214,10 @@ enum node_stat_item { PGDEMOTE_KSWAPD, PGDEMOTE_DIRECT, PGDEMOTE_KHUGEPAGED, +#ifdef CONFIG_DAMON_PADDR + DAMON_MIGRATE_HOT, + DAMON_MIGRATE_COLD, +#endif NR_VM_NODE_STAT_ITEMS }; diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index fd9d35b5cc83..d559c242d151 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -235,10 +235,23 @@ enum migration_mode { static unsigned int migrate_folio_list(struct list_head *migrate_folios, struct pglist_data *pgdat, + enum migration_mode mm, int target_nid) { unsigned int nr_succeeded; nodemask_t allowed_mask = NODE_MASK_NONE; + enum node_stat_item node_stat; + + switch (mm) { + case MIG_MIGRATE_HOT: + node_stat = DAMON_MIGRATE_HOT; + break; + case MIG_MIGRATE_COLD: + node_stat = DAMON_MIGRATE_COLD; + break; + default: + return 0; + } struct migration_target_control mtc = { /* @@ -263,6 +276,8 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, (unsigned long)&mtc, MIGRATE_ASYNC, MR_DAMON, &nr_succeeded); + mod_node_page_state(pgdat, node_stat, nr_succeeded); + return nr_succeeded; } @@ -302,7 +317,7 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, /* 'folio_list' is always empty here */ /* Migrate folios selected for migration */ - nr_migrated += migrate_folio_list(&migrate_folios, pgdat, target_nid); + nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm, target_nid); /* Folios that could not be migrated are still in @migrate_folios */ if (!list_empty(&migrate_folios)) { /* Folios which weren't migrated go back on @folio_list */ diff --git a/mm/vmstat.c b/mm/vmstat.c index db79935e4a54..be9ba89fede1 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1252,6 +1252,10 @@ const char * const vmstat_text[] = { "pgdemote_kswapd", "pgdemote_direct", "pgdemote_khugepaged", +#ifdef CONFIG_DAMON_PADDR + "damon_migrate_hot", + "damon_migrate_cold", +#endif /* enum writeback_stat_item counters */ "nr_dirty_threshold",