From patchwork Mon Feb 26 14:05:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DB87C48BF6 for ; Mon, 26 Feb 2024 14:06:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1DC96B00CC; Mon, 26 Feb 2024 09:06:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D9B156B00D0; Mon, 26 Feb 2024 09:06:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC6576B00CF; Mon, 26 Feb 2024 09:06:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id AC8EC6B00CC for ; Mon, 26 Feb 2024 09:06:21 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 62415A0689 for ; Mon, 26 Feb 2024 14:06:21 +0000 (UTC) X-FDA: 81834129762.19.6E14A5E Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf09.hostedemail.com (Postfix) with ESMTP id 69139140032 for ; Mon, 26 Feb 2024 14:06:19 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956379; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y9OC4YXYFjc+2FMc0Wyr+afdOtmeZZzGlGXJ/Mb/OPk=; b=hPOOwV2oS9wPSzlCY8STs4UVf+rMa4XNQ1FWA1/olkzZiU8IMwXn21lmMcYtP7522u9tQz 7+D075HVD8PsOtTw6ar4Epk7jz4gKe/bLrTz2FRP83nulaerkoz71kPUtH6dLqVpk9mmwQ qvnEBR2kZkSwYFe6873NJKjv72KZyXM= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956379; a=rsa-sha256; cv=none; b=fs15fRLtV6iI4iFXOgsZQSsRVI4zAUS8uuZ6hoZC9sTax9kcFoBo+axn7GyQ8MzyI2Vk4i Ji4D5PRjWf7/+2PFIrVObSX2gd+f0vwa82SztkGbLH9spINcZvCa0XnQrTOmtesX8WOU5O TM6o58cJLSxlHWmBaEsa+xECptD6mTc= X-AuditID: a67dfc5b-d6dff70000001748-5d-65dc9ad5f17d From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 1/7] mm/damon: refactor DAMOS_PAGEOUT with migration_mode Date: Mon, 26 Feb 2024 23:05:47 +0900 Message-ID: <20240226140555.1615-2-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOIsWRmVeSWpSXmKPExsXC9ZZnoe61WXdSDRb+ZbSY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlzGnYwFzQwlsx78catgbGK1xdjJwcEgImEjfmPmXqYuQAsxdv 1QIJswmoSVx5OQksLCLgILHqq0IXIxcHs0ADi8SPz5dYQGqEBXwk1v67wQ5iswioSvzaNB8s zitgJvHoWC8jxHhNicfbf4LVcAqYS/w/3M4GMlMIqObobWaIckGJkzOfgLUyC8hLNG+dzQyy S0LgFLvE3yvrWSHmSEocXHGDZQIj/ywkPbOQ9CxgZFrFKJSZV5abmJljopdRmZdZoZecn7uJ ERh9y2r/RO9g/HQh+BCjAAejEg+vQ+2dVCHWxLLiytxDjBIczEoivOEyN1OFeFMSK6tSi/Lj i0pzUosPMUpzsCiJ8xp9K08REkhPLEnNTk0tSC2CyTJxcEo1MDK+N4yt27RA9+y/ac7bG3rd Dz+yUDrannj3QV6ted70+a3f/ZKifvgU12UsuB/YVTyxslSkL/lnqMwXP1sPWSGeB3JTFtmG 3z7qLOWYNm9R24NJbss36+yscjzc+K1eUO+lQNXsWZ2JgQd3xe8zfeywek10WvC1nbd1tSsZ rr5e/Ffj2KdFS5RYijMSDbWYi4oTAbUWVdi6AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT/fqrDupBo0HdSwm9hhYzFm/hs1i 140Qi/97jzFaPPn/m9XixM1GNovPz14zW3Q++c5ocXjuSVaLzu9LWSwu75rDZnFvzX9WiyPr z7JYrLsFZG0+e4bZYvFyNYtD156zWuzreMBkcfjrGyaLyZcWsFm8mHKG0eLkrMksFrOP3mN3 EPf4f3ASs8fS02/YPDY0AYmds+6ye7Tsu8XusWBTqUfLkbesHov3vGTy2LSqk81j06dJ7B4n Zvxm8dj50NLjxeaZjB69ze/YPL7d9vBY/OIDU4BgFJdNSmpOZllqkb5dAlfGnIYNzAUtvBXz fqxha2C8wtXFyMEhIWAisXirVhcjJwebgJrElZeTmEDCIgIOEqu+KnQxcnEwCzSwSPz4fIkF pEZYwEdi7b8b7CA2i4CqxK9N88HivAJmEo+O9TKC2BICmhKPt/8Eq+EUMJf4f7idDWSmEFDN 0dvMEOWCEidnPgFrZRaQl2jeOpt5AiPPLCSpWUhSCxiZVjGKZOaV5SZm5pjqFWdnVOZlVugl 5+duYgRG2rLaPxN3MH657H6IUYCDUYmH16H2TqoQa2JZcWXuIUYJDmYlEd5wmZupQrwpiZVV qUX58UWlOanFhxilOViUxHm9wlMThATSE0tSs1NTC1KLYLJMHJxSDYxhHAuf6dzr6NtyNJy7 13qtTeBhUWmln4dEykuTjp9kPrPYvdXukZi588M8jvz/Okuuvs5e+U/k4+zt36xcRG4WSjyW j2Gbe+7GhJ35d3/qb7q6TXVr2Mlrj62Yb2bkL4n3/jzrYsN99qW/1PO3W39Rl/abUabYKMh2 /GZw+UrX89m3ys78F+1XYinOSDTUYi4qTgQATejUCbACAAA= X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 69139140032 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ybsy1bzr185uh78hh6hm8zg8ipj3bzej X-HE-Tag: 1708956379-105089 X-HE-Meta: U2FsdGVkX1+AFzoUO+nyoGweUft8ivB+xFtmr3Fcx4Gdquso9Akej3q47lz1ewfQaNyFRzbh3v+dKH+/P88EDxQTD6MGKnv3f3OsY8tXqUoqV9hbnCpHkcJ7WzXnCkZhYl6yMIYNQXap2q9TYPIQEQ3Be9PuYXqQQ89am04kugKNg32+N6tYPB9XkHAVxOpriiA9Emqu/8v83iMmx4pC4UK+MR1UMA8s7zfG7AY6np5Q0LvCotASow5NWzL6HlpFId2lAuI7QKW3o2dqrWtkmQcKRNtwllZNWmvsc8vkPEgNQFqEoguy8id29wLa+/elaC9oNK0bqdHhkyuV7lKhCwayxofHCRWeXUvVuz3QJJjlHGgiQUvP1X8zyElcBzmq7YFzifQS8uMbZBX7aVHUcICmnQ+5XvYYZ64vhXRQFAv8xDT7lpx9KELBklt6gJCu4VkYUvFmxOvZg2m5lpawb9qUUVcf3OT80E91iUWVKwxWOvx2JryfkrJfIFEMxEMi5Fv6Bdw/vg7FcOTptF2tBWDbvptvk4Ucq7BaIbntwJxKz8ELKO463Iye8EFvIMaoX4rMLUAc/GUz3aMvpNBg38s5KkJ2Us3QUdPv66hP0FAIUh8A8ZE8EVCIoARg/yuYKy55x9AepFKQBZmWE3VD91tBAHBgjYDGl8AHZdAaBMEYViyMdRNrcg8ZaBCTF78/K3Ol4kCvTYzXbqqck5dzYiA4Qun5C/XhkXTgOAXlOcSJgs8/JmIB6O3fahHpUXS1mm/X9891NREj64iGW/iiiUgE3LZS5zqGLeBEqGXavkK8lJOBXigscTwUd4XDpHeuH8oHeIfaicTYhK5MGl82tIr2UR5Bp+rFLG3syHOtwTQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation patch that introduces migration modes. The damon_pa_pageout is renamed to damon_pa_migrate and it receives an extra argument for migration_mode. No functional changes applied. Signed-off-by: Honggyu Kim --- mm/damon/paddr.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 081e2a325778..277a1c4d833c 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -224,7 +224,12 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) return false; } -static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) +enum migration_mode { + MIG_PAGEOUT, +}; + +static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, + enum migration_mode mm) { unsigned long addr, applied; LIST_HEAD(folio_list); @@ -249,7 +254,14 @@ static unsigned long damon_pa_pageout(struct damon_region *r, struct damos *s) put_folio: folio_put(folio); } - applied = reclaim_pages(&folio_list); + switch (mm) { + case MIG_PAGEOUT: + applied = reclaim_pages(&folio_list); + break; + default: + /* Unexpected migration mode. */ + return 0; + } cond_resched(); return applied * PAGE_SIZE; } @@ -297,7 +309,7 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, { switch (scheme->action) { case DAMOS_PAGEOUT: - return damon_pa_pageout(r, scheme); + return damon_pa_migrate(r, scheme, MIG_PAGEOUT); case DAMOS_LRU_PRIO: return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: From patchwork Mon Feb 26 14:05:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572198 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02B31C54E41 for ; Mon, 26 Feb 2024 14:06:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BED86B00D5; Mon, 26 Feb 2024 09:06:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 670486B00DA; Mon, 26 Feb 2024 09:06:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 49BB06B00DC; Mon, 26 Feb 2024 09:06:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 35B276B00D5 for ; Mon, 26 Feb 2024 09:06:25 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 11F4CC0978 for ; Mon, 26 Feb 2024 14:06:25 +0000 (UTC) X-FDA: 81834129930.30.1AB8E96 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf23.hostedemail.com (Postfix) with ESMTP id 1D5D4140012 for ; Mon, 26 Feb 2024 14:06:22 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/O1IDxVh05t8M0nebcpVtrx+/Qgao8Jaq6x7QW3BvpY=; b=qkRChNrliPfBr/zBJ++8zSyPKb6nqZeIh84RX9kbp0H7sf1rO7zy+UmOhrS0h2O/4rOzs6 3bAORypPuwe4VVLBhpnTVQaxQh0uyPuL4og/equuzz+wMctjcWMQTYYTMqLX6CeakbSblc d+C/GR+UhDCSqT9oBtXEbmjIdwElNBo= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956383; a=rsa-sha256; cv=none; b=0DT2SYYawjHgELQjdSsS+6cdqs9NbggV3bpnRXXJLmJKTdhzl7eFdQGEusMCARrtkbOEtM VPYjgTNg/izbmr0JqjLXg4/gV4CIdSRUH/YVYzuAroCVdajoovhOWhWV3b1+PyAQLKbGM2 cX/1+3b78CjNofdvtOvc3y36XEYGJa8= X-AuditID: a67dfc5b-d6dff70000001748-65-65dc9ad7da0c From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 2/7] mm: make alloc_demote_folio externally invokable for migration Date: Mon, 26 Feb 2024 23:05:48 +0900 Message-ID: <20240226140555.1615-3-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOIsWRmVeSWpSXmKPExsXC9ZZnoe71WXdSDU4sELaY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlbHjym73gkGDFtMmn2RsYN/N1MXJySAiYSByY2cnexcgBZl+f JwkSZhNQk7jychITSFhEwEFi1VeFLkYuDmaBBhaJH58vsYDUCAuESdxasBPMZhFQldix4ymY zStgJrF19jZWiPGaEo+3/2QHsTkFzCX+H25nA5kpBFRz9DYzRLmgxMmZT8BamQXkJZq3zmYG 2SUhcIpd4mPLUyaIOZISB1fcYJnAyD8LSc8sJD0LGJlWMQpl5pXlJmbmmOhlVOZlVugl5+du YgRG37LaP9E7GD9dCD7EKMDBqMTD61B7J1WINbGsuDL3EKMEB7OSCG+4zM1UId6UxMqq1KL8 +KLSnNTiQ4zSHCxK4rxG38pThATSE0tSs1NTC1KLYLJMHJxSDYwKC0Iq+iRk+wtOMx6fcP/m 4uKgw+oTP809kbxG4lzdzPaa+Z/lGrYcblm3/HBZ/dRHKVx/8r7tlPv5+yDXAkaGO+s2uXu7 7E1r3zVP6se5c6cVjD2inrp94nrs81HnV0+kW95hG8l3GVY6lhfdF4ao5NYo9RcaZB9vm7p9 RYu/UdcW1ZCPf94rsRRnJBpqMRcVJwIABBv4HboCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT/f6rDupBgfPMltM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6MDU9+sxccEqyY Nvk0ewPjZr4uRg4OCQETievzJLsYOTnYBNQkrrycxAQSFhFwkFj1VaGLkYuDWaCBReLH50ss IDXCAmEStxbsBLNZBFQldux4CmbzCphJbJ29jRXElhDQlHi8/Sc7iM0pYC7x/3A7G8hMIaCa o7eZIcoFJU7OfALWyiwgL9G8dTbzBEaeWUhSs5CkFjAyrWIUycwry03MzDHVK87OqMzLrNBL zs/dxAiMtGW1fybuYPxy2f0QowAHoxIPr0PtnVQh1sSy4srcQ4wSHMxKIrzhMjdThXhTEiur Uovy44tKc1KLDzFKc7AoifN6hacmCAmkJ5akZqemFqQWwWSZODilGhjZNkyuCLp5K9RnIVvv BK/jzJd2fv5YuFJ0kqB47etlR0SNBHy9s7PU5Z+dnLlbueDFzfiM1A/Mc9tWv88t3V9o0+/j FhMptMBwh0blCYtZ65uz/z2Kaj2276LO18eTpxTeaWu8y78lz9Tn4b8TDNJ/9yxo+HR+S7zC pudtX1USPNaWlTpyTvuuxFKckWioxVxUnAgAPK1J3bACAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: etwdgfqm6hnfyjqm7iqjummug86tuhbo X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1D5D4140012 X-HE-Tag: 1708956382-955617 X-HE-Meta: U2FsdGVkX1+0QjZ8ES3LijXpzjZjPGRpDPZSJDhMIpah68pulO1Oxw3wPLh0Ia3qmFn93hDCzzlP7BiVIcEffjqetcIzxKmdvQ3wCnYydHQH6LYyuC7ip1cdJWxOhLdEzKv6/ITjF5ozylYs4vePrwyMpZUnAkBW1LaPEJ6yrv9cY5cfITfgAUp9R9IqPpEcm4ecWxCMmQc/+Y8M9Ll1g14JjtxX37PSeuoYCqQcBL/fY/YGOTPbU47r2UepUXGmzs28JcvVyGkV0KhKwu4/dmo05/Ez5GefOdCZ0aNZtHHjA8B3A/yG/deA0G10X3XxdZgC1EwZGqfl+B+AQIqqls2/jmfYbfDKWC+I5wlmh17JyAgdE5+r9OJpkFAxHul68x5HALkSz6ykRyEejBUeYffXZkcrayvII6vG0Y01e4rkAMdNnbCJnnxIl2BgfPkOu29fKRjEnkK7JRybG7lPZTmVNf7FFxF1kiOvxQ5/5mU2wuB7LPDFuakD2Kdr7KtUbD8zrJh6zd5MIrqDu+Z017KebpCjdgopRxS0UZtCJAlRdOSU6WEx3Ct6DyQdu4MxR2mudLVRfyySk9bz5kVgmFkbhR9Q6KE96V3pUCORRA3kY1JTa5MfmyAwOx/U+FVM7GHwhDOWhc0zmLgC8u7JfXcvttvHxh+Sqh1WAS13O5Jq2MExH/5+5nr8r1S2DcfOWB8TKY6jEplHJyiLF7UXHEUz9+LjYZUjpxS60Pcnc5FpZ5yT245FDEiWAwIjXPOq+njN4yuEqLodYuVKPA3lax2OO+ZvUS8ibO35jkvwphu+cmg6iPNpU/glPVQjalDPqfvbHFjNR02dsCUK+V3a47mQMTaEU39f71mkooYFT+A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The alloc_demote_folio can be used out of vmscan.c so it'd be better to remove static keyword from it. This function can also be used for both demotion and promotion so it'd be better to rename it from alloc_demote_folio to alloc_migrate_folio. Signed-off-by: Honggyu Kim --- mm/internal.h | 1 + mm/vmscan.c | 10 +++++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index b61034bd50f5..61af6641235d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -866,6 +866,7 @@ extern unsigned long __must_check vm_mmap_pgoff(struct file *, unsigned long, unsigned long, unsigned long); extern void set_pageblock_order(void); +struct folio *alloc_migrate_folio(struct folio *src, unsigned long private); unsigned long reclaim_pages(struct list_head *folio_list); unsigned int reclaim_clean_pages_from_list(struct zone *zone, struct list_head *folio_list); diff --git a/mm/vmscan.c b/mm/vmscan.c index bba207f41b14..b8a1a1599e3c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -910,8 +910,7 @@ static void folio_check_dirty_writeback(struct folio *folio, mapping->a_ops->is_dirty_writeback(folio, dirty, writeback); } -static struct folio *alloc_demote_folio(struct folio *src, - unsigned long private) +struct folio *alloc_migrate_folio(struct folio *src, unsigned long private) { struct folio *dst; nodemask_t *allowed_mask; @@ -935,6 +934,11 @@ static struct folio *alloc_demote_folio(struct folio *src, if (dst) return dst; + /* + * Allocation failed from the target node so try to allocate from + * fallback nodes based on allowed_mask. + * See fallback_alloc() at mm/slab.c. + */ mtc->gfp_mask &= ~__GFP_THISNODE; mtc->nmask = allowed_mask; @@ -973,7 +977,7 @@ static unsigned int demote_folio_list(struct list_head *demote_folios, node_get_allowed_targets(pgdat, &allowed_mask); /* Demotion ignores all cpuset and mempolicy settings */ - migrate_pages(demote_folios, alloc_demote_folio, NULL, + migrate_pages(demote_folios, alloc_migrate_folio, NULL, (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, &nr_succeeded); From patchwork Mon Feb 26 14:05:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FD5BC48BF6 for ; Mon, 26 Feb 2024 14:06:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 512066B00DF; Mon, 26 Feb 2024 09:06:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 49BCE6B00E0; Mon, 26 Feb 2024 09:06:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2782A6B00E1; Mon, 26 Feb 2024 09:06:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 123DE6B00DF for ; Mon, 26 Feb 2024 09:06:26 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 87382160856 for ; Mon, 26 Feb 2024 14:06:25 +0000 (UTC) X-FDA: 81834129930.24.DDDF5EE Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf09.hostedemail.com (Postfix) with ESMTP id 6046B140030 for ; Mon, 26 Feb 2024 14:06:23 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IhyZjw+jpLJUwFy4dEx4+1Sb+mkQtPd245NYWtlHCwU=; b=KaA7frqeo7iq8lk91U4SDavUL2Vg7hwpNZky9qOUzZERwpc5omrP7P5gBQYkSkgCee9FRo ZfAszhd7jXaEGpvDGrHsrB+H/ljoMI4dyfQ9AFTSxmbOXX3bUMgn1tXcdONu4bilpcnDgI 3l5rVrh/26k36fyv/hgyKvW1RbcoPPo= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956383; a=rsa-sha256; cv=none; b=fLdOsprBNSphUOEChQwELbYPduL8kKufn0PnYsxESljMJyNnbo40MkNiRiy9HS2DJna6oI d5OsZjbsf5xl6IMvgxWz35Mx/Oed1I8Y86DnCfuw1yz9NZ8l8JCelo3uju5rFGV2JUGKls l5KPMGjy4IHGTexTzvzyAckdNFcPwf0= X-AuditID: a67dfc5b-d6dff70000001748-6b-65dc9ad87a7f From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 3/7] mm/damon: introduce DAMOS_DEMOTE action for demotion Date: Mon, 26 Feb 2024 23:05:49 +0900 Message-ID: <20240226140555.1615-4-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRmVeSWpSXmKPExsXC9ZZnoe6NWXdSDebt0LaY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlvDqzlrlgtU/FtxdbmBsYb9l2MXJySAiYSDy53M4IY8+dcZYN xGYTUJO48nISUxcjB4eIgIPEqq8KXYxcHMwCDSwSPz5fYgGpERbwkeiceYYJxGYRUJW4N/sX M4jNK2Am8afrFjPETE2Jx9t/soPYnALmEv8Pt7OBzBQCqjl6G6pcUOLkzCdgI5kF5CWat85m BtklIXCKXeL/hLOsEHMkJQ6uuMEygZF/FpKeWUh6FjAyrWIUyswry03MzDHRy6jMy6zQS87P 3cQIjL9ltX+idzB+uhB8iFGAg1GJh9eh9k6qEGtiWXFl7iFGCQ5mJRHecJmbqUK8KYmVValF +fFFpTmpxYcYpTlYlMR5jb6VpwgJpCeWpGanphakFsFkmTg4pRoYF79myP4s1c6u8d//s72H 10GOnSfll05a0zc/NefqohmNm5yW8SvZx17mdy5d4DTzk9Mm1nf+tR3xBXFvqrOi3OYKmW9s DQ7f3iNuFle+aPHT9N9+XY8vW2WaL/0S91pgxwcGa/m47XcLrm1R5ew6+rVnd9v/oN37gnb7 mb53yFpdrqO0fw67EktxRqKhFnNRcSIAYaWl7rsCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT/fGrDupBj2n5C0m9hhYzFm/hs1i 140Qi/97jzFaPPn/m9XixM1GNovPz14zW3Q++c5ocXjuSVaLzu9LWSwu75rDZnFvzX9WiyPr z7JYrLsFZG0+e4bZYvFyNYtD156zWuzreMBkcfjrGyaLyZcWsFm8mHKG0eLkrMksFrOP3mN3 EPf4f3ASs8fS02/YPDY0AYmds+6ye7Tsu8XusWBTqUfLkbesHov3vGTy2LSqk81j06dJ7B4n Zvxm8dj50NLjxeaZjB69ze/YPL7d9vBY/OIDU4BgFJdNSmpOZllqkb5dAlfGqzNrmQtW+1R8 e7GFuYHxlm0XIyeHhICJxNwZZ9lAbDYBNYkrLycxdTFycIgIOEis+qrQxcjFwSzQwCLx4/Ml FpAaYQEfic6ZZ5hAbBYBVYl7s38xg9i8AmYSf7puMUPM1JR4vP0nO4jNKWAu8f9wOxvITCGg mqO3ocoFJU7OfAI2kllAXqJ562zmCYw8s5CkZiFJLWBkWsUokplXlpuYmWOqV5ydUZmXWaGX nJ+7iREYa8tq/0zcwfjlsvshRgEORiUeXofaO6lCrIllxZW5hxglOJiVRHjDZW6mCvGmJFZW pRblxxeV5qQWH2KU5mBREuf1Ck9NEBJITyxJzU5NLUgtgskycXBKNTAaWRvZp3m2y1/ukm7Z eDFZ4tL0k58LjnHqsq/6ptO9xqZoWWKmmoSg0L7/OU7f7XUmMPPblv9dYWXUdH7eV6WXC9zO qX+98VfRv0tGwWyLcf3TTyeexJ1nvOPaMjtHOX++sH1woepsPbWN15uPvCx83vNHwCe6/t1K o9VOvCYWMoqJ2u6fVymxFGckGmoxFxUnAgBVpl34sQIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 6046B140030 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 13qemnj8a5gg8t7whj17moej5xwieg87 X-HE-Tag: 1708956383-470458 X-HE-Meta: U2FsdGVkX19ACUFMrTmOPjVZ6LmocCtr5HRrJ8iT5oBmF7LfRCJ/hbP6EUosWWTxoD1lngPAINgV0gwhWulrrK8YknR2pIAbHwOpix58WqceTVpaFmBZp+8NAECilxOS4VI6x8CDKQD1zDhuosUx+ESLvccizrT/ozEdqjmSfbzMItO9J8Iblqrcaz9LNo1tc1tABkGOtztAGu70EzmxONjBVJyezaYGFh8QifnqVCFYSyJdz6BUffRkgzkaevPRcoMH6gSppHDSH6C+daNStbIZMOM7T8vtoUIdeMW9hMYWy+PBqm/yxgazwyWPsQRk1Jis6+R2t1UZcXEbKswITAOhY6LVPdukSfehAJsOZa1VFNaU4Hd3twSa2c0LBFNzExRbi3gf9o0RIYelpr8EHsSre3sJKsTAtci8WjbZLMTX6801YDAVjDsrmzgZDhAyJIflbmXdzhjEwniJr54KvoHOZZhuitCbe1sxjaSct6eFVA8Gg880q8rTbXx30ekPIFAXMchw+yV5OENGrdKjtR82o6bO7nu3+oK3T6Oms3RZdOcWz7ddd5S9wo0eXjC+lnGZxZb2Oxe2F9kDA2/0y+AqpGLR8ssR8E/U1vbMAdo6n044psCeBNkhkoE2lug2pUJ42LLKcris25KmXRBT4mXKjvMd7sX5h+623ZUtXIJdGLOSAzgkyAa57mqCJO0wF9kMF5d6Q8Hxq0C9XbcBTR6ifnZ953zB+GmLuMT2LzuDCvCpKjMtBAnVov9A6f3q4KjenE//KextiQ9DaJsOC0Hy13ryGZbtw3OMDK4TBh2WnOqVlOT7jgfQSU9IJkSqbXurrQaehlzWWFXlNFlHGSNvYHV+lKoCle8TENOMXjOjVbRyhTg/FA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch introduces DAMOS_DEMOTE action, which is similar to DAMOS_PAGEOUT, but demote folios instead of swapping them out. Since there are some common routines with pageout, many functions have similar logics between pageout and demote. damon_pa_migrate_folio_list() is a minimized version of shrink_folio_list(), but it's minified only for demotion. Signed-off-by: Honggyu Kim Signed-off-by: Hyeongtak Ji --- include/linux/damon.h | 2 + mm/damon/paddr.c | 222 ++++++++++++++++++++++++++++++++++++++- mm/damon/sysfs-schemes.c | 1 + 3 files changed, 224 insertions(+), 1 deletion(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index e00ddf1ed39c..86e66772766b 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_DEMOTE: Do demotion for the given region. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions * @@ -122,6 +123,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_DEMOTE, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, }; diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 277a1c4d833c..23e37ce57202 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -12,6 +12,9 @@ #include #include #include +#include +#include +#include #include "../internal.h" #include "ops-common.h" @@ -226,8 +229,214 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) enum migration_mode { MIG_PAGEOUT, + MIG_DEMOTE, }; +/* + * XXX: This is copied from demote_folio_list as renamed as migrate_folio_list. + * Take folios on @migrate_folios and attempt to migrate them to another node. + * Folios which are not migrated are left on @migrate_folios. + */ +static unsigned int migrate_folio_list(struct list_head *migrate_folios, + struct pglist_data *pgdat, + enum migration_mode mm) +{ + int target_nid = next_demotion_node(pgdat->node_id); + unsigned int nr_succeeded; + nodemask_t allowed_mask; + + struct migration_target_control mtc = { + /* + * Allocate from 'node', or fail quickly and quietly. + * When this happens, 'page' will likely just be discarded + * instead of migrated. + */ + .gfp_mask = (GFP_HIGHUSER_MOVABLE & ~__GFP_RECLAIM) | __GFP_NOWARN | + __GFP_NOMEMALLOC | GFP_NOWAIT, + .nid = target_nid, + .nmask = &allowed_mask + }; + + if (pgdat->node_id == target_nid || target_nid == NUMA_NO_NODE) + return 0; + + if (list_empty(migrate_folios)) + return 0; + + node_get_allowed_targets(pgdat, &allowed_mask); + + /* Migration ignores all cpuset and mempolicy settings */ + migrate_pages(migrate_folios, alloc_migrate_folio, NULL, + (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, + &nr_succeeded); + + __count_vm_events(PGDEMOTE_DIRECT, nr_succeeded); + + return nr_succeeded; +} + +enum folio_references { + FOLIOREF_RECLAIM, + FOLIOREF_KEEP, + FOLIOREF_ACTIVATE, +}; + +/* + * XXX: This is just copied and simplified from folio_check_references at + * mm/vmscan.c but without having scan_control. + */ +static enum folio_references folio_check_references(struct folio *folio) +{ + int referenced_ptes, referenced_folio; + unsigned long vm_flags; + + referenced_ptes = folio_referenced(folio, 1, NULL, &vm_flags); + referenced_folio = folio_test_clear_referenced(folio); + + /* rmap lock contention: rotate */ + if (referenced_ptes == -1) + return FOLIOREF_KEEP; + + if (referenced_ptes) { + /* + * All mapped folios start out with page table + * references from the instantiating fault, so we need + * to look twice if a mapped file/anon folio is used more + * than once. + * + * Mark it and spare it for another trip around the + * inactive list. Another page table reference will + * lead to its activation. + * + * Note: the mark is set for activated folios as well + * so that recently deactivated but used folios are + * quickly recovered. + */ + folio_set_referenced(folio); + + if (referenced_folio || referenced_ptes > 1) + return FOLIOREF_ACTIVATE; + + /* + * Activate file-backed executable folios after first usage. + */ + if ((vm_flags & VM_EXEC) && folio_is_file_lru(folio)) + return FOLIOREF_ACTIVATE; + + return FOLIOREF_KEEP; + } + + return FOLIOREF_RECLAIM; +} + +/* + * XXX: This is minimized implmentation based on shrink_folio_list only for + * the demotion calling demote_folio_list. + */ +static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, + struct pglist_data *pgdat, + enum migration_mode mm) +{ + unsigned int nr_migrated = 0; + struct folio *folio; + LIST_HEAD(ret_folios); + LIST_HEAD(migrate_folios); + + cond_resched(); + + while (!list_empty(folio_list)) { + struct folio *folio; + enum folio_references references; + + cond_resched(); + + folio = lru_to_folio(folio_list); + list_del(&folio->lru); + + if (!folio_trylock(folio)) + goto keep; + + VM_BUG_ON_FOLIO(folio_test_active(folio), folio); + + references = folio_check_references(folio); + if (references == FOLIOREF_KEEP) + goto keep_locked; + + /* Relocate its contents to another node. */ + list_add(&folio->lru, &migrate_folios); + folio_unlock(folio); + continue; +keep_locked: + folio_unlock(folio); +keep: + list_add(&folio->lru, &ret_folios); + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + } + /* 'folio_list' is always empty here */ + + /* Migrate folios selected for migration */ + nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm); + /* Folios that could not be migrated are still in @migrate_folios */ + if (!list_empty(&migrate_folios)) { + /* Folios which weren't migrated go back on @folio_list */ + list_splice_init(&migrate_folios, folio_list); + } + + try_to_unmap_flush(); + + list_splice(&ret_folios, folio_list); + + while (!list_empty(folio_list)) { + folio = lru_to_folio(folio_list); + list_del(&folio->lru); + folio_putback_lru(folio); + } + + return nr_migrated; +} + +/* + * XXX: This is almost identical to reclaim_pages() in mm/vmscan.c, but it + * internally calls damon_pa_migrate_folio_list() instead of + * reclaim_folio_list(). We might be better to think if we can have a + * common function for both cases. + */ +static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, + enum migration_mode mm) +{ + int nid; + unsigned int nr_migrated = 0; + LIST_HEAD(node_folio_list); + unsigned int noreclaim_flag; + + if (list_empty(folio_list)) + return nr_migrated; + + noreclaim_flag = memalloc_noreclaim_save(); + + nid = folio_nid(lru_to_folio(folio_list)); + do { + struct folio *folio = lru_to_folio(folio_list); + + if (nid == folio_nid(folio)) { + folio_clear_active(folio); + list_move(&folio->lru, &node_folio_list); + continue; + } + + nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), mm); + nid = folio_nid(lru_to_folio(folio_list)); + } while (!list_empty(folio_list)); + + nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, + NODE_DATA(nid), mm); + + memalloc_noreclaim_restore(noreclaim_flag); + + return nr_migrated; +} + static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, enum migration_mode mm) { @@ -247,7 +456,11 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, folio_test_clear_young(folio); if (!folio_isolate_lru(folio)) goto put_folio; - if (folio_test_unevictable(folio)) + /* + * Since unevictable folios can be demoted or promoted, + * unevictable test is needed only for pageout. + */ + if (mm == MIG_PAGEOUT && folio_test_unevictable(folio)) folio_putback_lru(folio); else list_add(&folio->lru, &folio_list); @@ -258,6 +471,9 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, case MIG_PAGEOUT: applied = reclaim_pages(&folio_list); break; + case MIG_DEMOTE: + applied = damon_pa_migrate_pages(&folio_list, mm); + break; default: /* Unexpected migration mode. */ return 0; @@ -314,6 +530,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_DEMOTE: + return damon_pa_migrate(r, scheme, MIG_DEMOTE); case DAMOS_STAT: break; default: @@ -334,6 +552,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_DEMOTE: + return damon_cold_score(context, r, scheme); default: break; } diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index fe0fe2562000..53e47fad5021 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1186,6 +1186,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "nohugepage", "lru_prio", "lru_deprio", + "demote", "stat", }; From patchwork Mon Feb 26 14:05:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32B30C54E41 for ; Mon, 26 Feb 2024 14:06:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC8C46B00E3; Mon, 26 Feb 2024 09:06:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B519B6B00E7; Mon, 26 Feb 2024 09:06:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A3626B00E9; Mon, 26 Feb 2024 09:06:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 855536B00E3 for ; Mon, 26 Feb 2024 09:06:28 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 53941A0792 for ; Mon, 26 Feb 2024 14:06:28 +0000 (UTC) X-FDA: 81834130056.16.137F068 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf23.hostedemail.com (Postfix) with ESMTP id 8382314000D for ; Mon, 26 Feb 2024 14:06:25 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956385; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w9f3rfKd+jtQPwgacOekgJEcbI06gWAyOzMEzJgXU4U=; b=ao5GrdJKMDc0hk2LMOwVBLvpeLv9xQzbV2SF6idrhD2plLwAFsf0gKVEmgjZExLTJW5WU9 Vy9H8EIpeoBicefXlvrX8wMZisSnq/lgd2E+A1xzub0S8Uhqjc17oc8p+kRFZHZFKx5aLn /868LUSQo5OPoL71doDEh6SYnblWZ4s= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956385; a=rsa-sha256; cv=none; b=jS47p6LX+gycSHddTJ8A5eU1oTR1YSA93xGEANULWLfGuyHRRUKi7T7bLhfrrPJZOCVuJS RSMQ8ZRf1Z4aR0SECBSlErytlXWrulSe9mwljghfokLd21Gy9BHF1aj2RXeCg/HuL7Ff3L 2Em6y4o2PIrsk5O892AwzU9dnunG7Cw= X-AuditID: a67dfc5b-d6dff70000001748-7c-65dc9ad909a7 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 4/7] mm/memory-tiers: add next_promotion_node to find promotion target Date: Mon, 26 Feb 2024 23:05:50 +0900 Message-ID: <20240226140555.1615-5-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRmVeSWpSXmKPExsXC9ZZnoe6tWXdSDb7/YrOY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBl7FvZwFrwSLziz+vJzA2M+4S7GDk5JARMJF6+3sMGY1/cspYJ xGYTUJO48nISkM3BISLgILHqq0IXIxcHs0ADi8SPz5dYQGqEBSIlZkxuYgWxWQRUJVpPXWIE sXkFzCSuP9nHCjFTU+Lx9p/sIDangLnE/8PtbCAzhYBqjt5mhigXlDg58wnYSGYBeYnmrbOZ QXZJCJxil9h0YC/UHEmJgytusExg5J+FpGcWkp4FjEyrGIUy88pyEzNzTPQyKvMyK/SS83M3 MQLjb1ntn+gdjJ8uBB9iFOBgVOLhdai9kyrEmlhWXJl7iFGCg1lJhDdc5maqEG9KYmVValF+ fFFpTmrxIUZpDhYlcV6jb+UpQgLpiSWp2ampBalFMFkmDk6pBsZ1bw5O7DnF3tp751DZCdvF SZc3nJBMjJE8kaukLfLu0KoGno1X57gtVPktNfkX87orou/rHqcrGKrOaozWVr6mMb+o7NPs 1cURJ/UNo17N4WxQLdD9ev+6ZJxgk+HNFgPzlx/3lE1mEQkx3nWkLcLCKbjw085lJwrmT+V7 /0L2VEalS/Y01UQlluKMREMt5qLiRAATmp6xuwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT/fmrDupBj39phYTewws5qxfw2ax 60aIxf+9xxgtnvz/zWpx4mYjm8XnZ6+ZLTqffGe0ODz3JKtF5/elLBaXd81hs7i35j+rxZH1 Z1ks1t0CsjafPcNssXi5msWha89ZLfZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7 iHv8PziJ2WPp6TdsHhuagMTOWXfZPVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8T M36zeOx8aOnxYvNMRo/e5ndsHt9ue3gsfvGBKUAwissmJTUnsyy1SN8ugStj38oG1oJH4hV/ Xk9mbmDcJ9zFyMkhIWAicXHLWiYQm01ATeLKy0lANgeHiICDxKqvCl2MXBzMAg0sEj8+X2IB qREWiJSYMbmJFcRmEVCVaD11iRHE5hUwk7j+ZB8rxExNicfbf7KD2JwC5hL/D7ezgcwUAqo5 epsZolxQ4uTMJ2AjmQXkJZq3zmaewMgzC0lqFpLUAkamVYwimXlluYmZOaZ6xdkZlXmZFXrJ +bmbGIGxtqz2z8QdjF8uux9iFOBgVOLhdai9kyrEmlhWXJl7iFGCg1lJhDdc5maqEG9KYmVV alF+fFFpTmrxIUZpDhYlcV6v8NQEIYH0xJLU7NTUgtQimCwTB6dUA+PaINdoLfZf91+us46d M+uS9dVZPGz5fwVbW9xXNF27aR175kvOuZl+M6Tc2xjtV9hLXT3SNv32Qf/yvKWX7ZYkSpxm ur3k8ve/hVWzpmVtMmFR29aY9PC4cbLtp4U72J6/UF6+XuXSfvaQ7p0GDrz/q7uEtlbM5zYx mm2td+9xbItPYfdB704lluKMREMt5qLiRABxHBSHsQIAAA== X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: ijzfgj5phajtwf6w3rsx8oh64x4qapy1 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 8382314000D X-HE-Tag: 1708956385-688560 X-HE-Meta: U2FsdGVkX1/Xsza3iR0DRLnc+b20NxNmn8lYJMoYiuH0c8HsPscgc98pRH8K4C9nZy5uXY9se5eBwdQa0tl3wPWFSmyGJ+SLe6gBhaHu88nROZIksej6rp7MCMD9PRBYK7VkUpQsjRAwAcvfbEaJiR+UYPorVnBTmtq1SzgybnQWzugd5ei5OeWtpfL1pX6DYcsdCH3krgEFx7d72YN60D2k67BKMMfWxD0Jm4vvpb93YBA4Or56ddqqk/EqeTb5pEdc9lWY/mrKMu8A/SZSO/f25qHnPfKnZvwX50U537n5WPhBsUF4gaP0Oj/9Ive+JtJhfu5sX5FU61PY7h0VuYapLwwYYKCet/+g/YwlCjgwQFNdHIkJHbMIg+YCNUQuxtAFDpCM8GcbGz4QDSMWT8RgBaET2I7CIAaT1Lihgi/9S9BWpe3SiXzQh9/NcyUxRVLBtgYQkSgmv9QHHqdJ5zg319P7LwGslrQtEf9M1av/bhTXl4lGl4qNAyZxa7X+gWx2Jbs2vE3MuzzFzcOXt4q8AZnYa6vMmc6yqex0YFrcCc2gg7EaWM+rDgDNi3JNvXDzkkQ8tbNUuN73pgfvQg8lv1ymp2PqgRPKyO78hlFEA6hfgXsYmhD20S9QrjPOx1mVfgRrBzGGtV8C9WVL0ziO10BXNScPtorE088aaecF4ubNxPTRDZbY+CxnjQaB6BxptzTLCIHs8cJJ7dSMwrQysoQawOifTgRX5ad9JgEV8I0c18Sju4QlNgzmmN/s6szn/QMVpmKGn0ZAf9nbcEQ9KbNgZZYwAMjRfYuFJeNKQINx1J28AjB+CgdyvYi7OzOSEl4qofWnc1tSzLzdJT8WP0ZhSdwowJPAgn4JhFZZfZhQC92lEQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeongtak Ji This patch adds next_promotion_node that can be used to identify the appropriate promotion target based on memory tiers. When multiple promotion target nodes are available, the nearest node is selected based on numa distance. Signed-off-by: Hyeongtak Ji --- include/linux/memory-tiers.h | 11 +++++++++ mm/memory-tiers.c | 43 ++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+) diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h index 1e39d27bee41..0788e435fc50 100644 --- a/include/linux/memory-tiers.h +++ b/include/linux/memory-tiers.h @@ -50,6 +50,7 @@ int mt_set_default_dram_perf(int nid, struct node_hmem_attrs *perf, int mt_perf_to_adistance(struct node_hmem_attrs *perf, int *adist); #ifdef CONFIG_MIGRATION int next_demotion_node(int node); +int next_promotion_node(int node); void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets); bool node_is_toptier(int node); #else @@ -58,6 +59,11 @@ static inline int next_demotion_node(int node) return NUMA_NO_NODE; } +static inline int next_promotion_node(int node) +{ + return NUMA_NO_NODE; +} + static inline void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets) { *targets = NODE_MASK_NONE; @@ -101,6 +107,11 @@ static inline int next_demotion_node(int node) return NUMA_NO_NODE; } +static inline int next_promotion_node(int node) +{ + return NUMA_NO_NODE; +} + static inline void node_get_allowed_targets(pg_data_t *pgdat, nodemask_t *targets) { *targets = NODE_MASK_NONE; diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 8d5291add2bc..0060ee571cf4 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -335,6 +335,49 @@ int next_demotion_node(int node) return target; } +/* + * Select a promotion target that is close to the from node among the given + * two nodes. + * + * TODO: consider other decision policy as node_distance may not be precise. + */ +static int select_promotion_target(int a, int b, int from) +{ + if (node_distance(from, a) < node_distance(from, b)) + return a; + else + return b; +} + +/** + * next_promotion_node() - Get the next node in the promotion path + * @node: The starting node to lookup the next node + * + * Return: node id for next memory node in the promotion path hierarchy + * from @node; NUMA_NO_NODE if @node is the toptier. + */ +int next_promotion_node(int node) +{ + int target = NUMA_NO_NODE; + int nid; + + if (node_is_toptier(node)) + return NUMA_NO_NODE; + + rcu_read_lock(); + for_each_node_state(nid, N_MEMORY) { + if (node_isset(node, node_demotion[nid].preferred)) { + if (target == NUMA_NO_NODE) + target = nid; + else + target = select_promotion_target(nid, target, node); + } + } + rcu_read_unlock(); + + return target; +} + static void disable_all_demotion_targets(void) { struct memory_tier *memtier; From patchwork Mon Feb 26 14:05:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21F80C54E41 for ; Mon, 26 Feb 2024 14:06:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3DA936B00E0; Mon, 26 Feb 2024 09:06:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 363156B00E3; Mon, 26 Feb 2024 09:06:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1DBDB6B00E5; Mon, 26 Feb 2024 09:06:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F1DC16B00E0 for ; Mon, 26 Feb 2024 09:06:27 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AC4361408D9 for ; Mon, 26 Feb 2024 14:06:27 +0000 (UTC) X-FDA: 81834130014.23.D4A49BC Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf09.hostedemail.com (Postfix) with ESMTP id 9386B14000A for ; Mon, 26 Feb 2024 14:06:25 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956386; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WCJbCXrjn/PAdTWQ0hYIswG7MSzXLA6XzCdTSZGM83g=; b=CN1mQGlsawgGVul+1esOV5rvDEUggZ+nZ5lDITTY6vVIasj2fhAX/IeegnWWmhdzmT4FZ1 JSFCiwWatWhpZNUcvEzRmlpTnZrR3772j2ETSP92Yh9n5uSToRBNmRjwkn9WIr7UMEStI0 Re7ozCYtnLykbR5SsdzJLbzju4v3mwk= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956386; a=rsa-sha256; cv=none; b=LvhuZveWIKMznXVcecusfPczscYb24nylTv44Tf8gkmPEqduh2A8vGyStaFCBWgmAeRcQy XgrbVRyXkaBcQFqOJXHxZpsCMVmkUef6Zq9KCSlS7AZdTavS7TNu7TLbakb4P7MkMV4k0i gWPIiWMx9aF7Esl2MbPB3r/cm0Z+aZ4= X-AuditID: a67dfc5b-d6dff70000001748-83-65dc9adb88dd From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 5/7] mm/damon: introduce DAMOS_PROMOTE action for promotion Date: Mon, 26 Feb 2024 23:05:51 +0900 Message-ID: <20240226140555.1615-6-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrKIsWRmVeSWpSXmKPExsXC9ZZnoe7tWXdSDbYckLeY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlTDz7hq1goUXFnL5tjA2Mv3S7GDk4JARMJL4uK+5i5AQzp545 yQxiswmoSVx5OYkJpEREwEFi1VeFLkYuDmaBBhaJH58vsYDEhQX8JDa/LAMxWQRUJW4tEQDp 5BUwk3h5aR4rxERNicfbf7KD2JwC5hL/D7ezgZQLAdUcvc0MUS4ocXLmExYQm1lAXqJ562xm kE0SAqfYJXYu2skEMUdS4uCKGywTGPlnIemZhaRnASPTKkahzLyy3MTMHBO9jMq8zAq95Pzc TYzAyFtW+yd6B+OnC8GHGAU4GJV4eB1q76QKsSaWFVfmHmKU4GBWEuENl7mZKsSbklhZlVqU H19UmpNafIhRmoNFSZzX6Ft5ipBAemJJanZqakFqEUyWiYNTqoExc5uYpGYdU+jLbvYbJc+c C/IN5vrMsZj+vszyRE2yOqvE4RuVi+afWLlBj+PY/aMJf68HfEtMi2UrnTJXfb9HyYybhU27 7QU1a/49rzsus76lnKnRUOTc1QvPN8yKdFG/qOXZelPnfnVxsv7t9TvvPMuTPH1iy2Mu/2vX lxxvjHe9cuvBhjc2SizFGYmGWsxFxYkA+pGG7LgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprDIsWRmVeSWpSXmKPExsXCNUNLT/f2rDupBu9n8FlM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6MiWffsBUstKiY 07eNsYHxl24XIyeHhICJxNQzJ5lBbDYBNYkrLycxdTFycIgIOEis+qrQxcjFwSzQwCLx4/Ml FpC4sICfxOaXZSAmi4CqxK0lAiCdvAJmEi8vzWOFmKgp8Xj7T3YQm1PAXOL/4XY2kHIhoJqj t5khygUlTs58wgJiMwvISzRvnc08gZFnFpLULCSpBYxMqxhFMvPKchMzc0z1irMzKvMyK/SS 83M3MQLjbFntn4k7GL9cdj/EKMDBqMTD61B7J1WINbGsuDL3EKMEB7OSCG+4zM1UId6UxMqq 1KL8+KLSnNTiQ4zSHCxK4rxe4akJQgLpiSWp2ampBalFMFkmDk6pBsaZS5iZNt2Tdbt0unvm iovFUdcX3Wq791p78tTIUrU5c/hXW3OkcVzoSwmaaeu8+/eSDEVN0VZDyT3cux47NJxh37gi KrjL31clSkhh00MFPZb1r0QDVCdOMU741V437+TRA0+1+982M1+fkPPlt3nRZPu23BldO/rM ntjqx0yaLdCxjD/R9ZESS3FGoqEWc1FxIgBxEqWRrwIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 9386B14000A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 5qkhm8uk7ney9dpyuqk1pmige5jm4rfg X-HE-Tag: 1708956385-798702 X-HE-Meta: U2FsdGVkX18Or+ZjbClohpwwECUSXHrXG9VCIsNA84xjvVM8cgygu3naDCPujwaFqHD+7+PJqyy9nzhF8DiJ+lbYZ6FenMR0B+MshUepVzWeaQp2wzzAw5+i/bVOP8iBBYIiuzgTJXGMr0zheYE5wtpl3sX1AvILEgPdmu/XbnBmYFNt20EWY6WstzGlGIutPtetaFTbs/anBTCSzLQVD1iAafr+y3aXIQm9rWxwCsnEZMsICqNFcemtjO9tDRUmqK1gB1VL6Luww8k8fosLYu3vtCuBd8aFkeL3QH6vW+Dyd3th6Rmd6f5jVcCJdTrPLSlpkF0Jy1qcMEsBgpNbBKFxpiktzSoytTo6iHut3S9HioS57OM4tVWPV1Nym2eopXUpNTCPikacCgXkGSF17NBmqy0dvb+A0lzndBv9WzkNlViw9Rc1ighod9d+d7wZLRMe/Bzu6gCwvO/Y+lrLjEagO7MyXu5ljlO3QZWtr8xg+xziKZ4AaiNzKjRtqAR2dnoEh5wQmWN8DqM85Ruc1j+pGlBdzsKX5IW3VmMMnO1NtQL7M1EJIm0jHD2TOymYV8fm9fue/6AncPEzxBuBX58zmhvykGNqjUMCPSzB+hHbkXuN0GHpifPpEnH3RojHEEIWwoss+AAEn9YnmOjDm1LPl6QTx7f8+f2XzeaDwfH7O6V3936eKj460OpShtR4FQ2u2TlO9KNro0+luiRt8D/vY8d2QRbsWFxYSRyYKz8L0kwtInCPewHYdT0/QL4t79jBt6CC4xotDSty+9pM6nlCaUD/Ax3sIet1+x87QrCCAN3xYXXYOxg+NAD0zw0vvdA4IDWTzNoKCNslsh5l+d23GxJg4cTdd/vwyqqBQfYdXsig5H0BIA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeongtak Ji This patch introduces DAMOS_PROMOTE action for paddr mode. It includes renaming alloc_demote_folio to alloc_migrate_folio to use it for promotion as well. Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- include/linux/damon.h | 2 ++ include/linux/migrate_mode.h | 1 + include/linux/vm_event_item.h | 1 + include/trace/events/migrate.h | 3 ++- mm/damon/paddr.c | 45 ++++++++++++++++++++++++++++------ mm/damon/sysfs-schemes.c | 1 + mm/vmstat.c | 1 + 7 files changed, 46 insertions(+), 8 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index 86e66772766b..d7e52d0228b4 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -105,6 +105,7 @@ struct damon_target { * @DAMOS_NOHUGEPAGE: Call ``madvise()`` for the region with MADV_NOHUGEPAGE. * @DAMOS_LRU_PRIO: Prioritize the region on its LRU lists. * @DAMOS_LRU_DEPRIO: Deprioritize the region on its LRU lists. + * @DAMOS_PROMOTE: Do promotion for the given region. * @DAMOS_DEMOTE: Do demotion for the given region. * @DAMOS_STAT: Do nothing but count the stat. * @NR_DAMOS_ACTIONS: Total number of DAMOS actions @@ -123,6 +124,7 @@ enum damos_action { DAMOS_NOHUGEPAGE, DAMOS_LRU_PRIO, DAMOS_LRU_DEPRIO, + DAMOS_PROMOTE, DAMOS_DEMOTE, DAMOS_STAT, /* Do nothing but only record the stat */ NR_DAMOS_ACTIONS, diff --git a/include/linux/migrate_mode.h b/include/linux/migrate_mode.h index f37cc03f9369..63f75eb9abf3 100644 --- a/include/linux/migrate_mode.h +++ b/include/linux/migrate_mode.h @@ -29,6 +29,7 @@ enum migrate_reason { MR_CONTIG_RANGE, MR_LONGTERM_PIN, MR_DEMOTION, + MR_PROMOTION, MR_TYPES }; diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index 8abfa1240040..63cf920afeaa 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -44,6 +44,7 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, PGDEMOTE_KSWAPD, PGDEMOTE_DIRECT, PGDEMOTE_KHUGEPAGED, + PGPROMOTE, PGSCAN_KSWAPD, PGSCAN_DIRECT, PGSCAN_KHUGEPAGED, diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h index 0190ef725b43..f0dd569c1e62 100644 --- a/include/trace/events/migrate.h +++ b/include/trace/events/migrate.h @@ -22,7 +22,8 @@ EM( MR_NUMA_MISPLACED, "numa_misplaced") \ EM( MR_CONTIG_RANGE, "contig_range") \ EM( MR_LONGTERM_PIN, "longterm_pin") \ - EMe(MR_DEMOTION, "demotion") + EM( MR_DEMOTION, "demotion") \ + EMe(MR_PROMOTION, "promotion") /* * First define the enums in the above macros to be exported to userspace diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 23e37ce57202..37a7b34a36dd 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -229,6 +229,7 @@ static bool damos_pa_filter_out(struct damos *scheme, struct folio *folio) enum migration_mode { MIG_PAGEOUT, + MIG_PROMOTE, MIG_DEMOTE, }; @@ -241,9 +242,26 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, struct pglist_data *pgdat, enum migration_mode mm) { - int target_nid = next_demotion_node(pgdat->node_id); + int target_nid; unsigned int nr_succeeded; nodemask_t allowed_mask; + int reason; + enum vm_event_item vm_event; + + switch (mm) { + case MIG_PROMOTE: + target_nid = next_promotion_node(pgdat->node_id); + reason = MR_PROMOTION; + vm_event = PGPROMOTE; + break; + case MIG_DEMOTE: + target_nid = next_demotion_node(pgdat->node_id); + reason = MR_DEMOTION; + vm_event = PGDEMOTE_DIRECT; + break; + default: + return 0; + } struct migration_target_control mtc = { /* @@ -263,14 +281,19 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, if (list_empty(migrate_folios)) return 0; - node_get_allowed_targets(pgdat, &allowed_mask); + if (mm == MIG_DEMOTE) { + node_get_allowed_targets(pgdat, &allowed_mask); + } else if (mm == MIG_PROMOTE) { + /* TODO: Need to add upper_tier_mask at struct memory_tier. */ + allowed_mask = NODE_MASK_NONE; + } /* Migration ignores all cpuset and mempolicy settings */ migrate_pages(migrate_folios, alloc_migrate_folio, NULL, - (unsigned long)&mtc, MIGRATE_ASYNC, MR_DEMOTION, + (unsigned long)&mtc, MIGRATE_ASYNC, reason, &nr_succeeded); - __count_vm_events(PGDEMOTE_DIRECT, nr_succeeded); + __count_vm_events(vm_event, nr_succeeded); return nr_succeeded; } @@ -359,7 +382,8 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, VM_BUG_ON_FOLIO(folio_test_active(folio), folio); references = folio_check_references(folio); - if (references == FOLIOREF_KEEP) + if (references == FOLIOREF_KEEP || + (references == FOLIOREF_RECLAIM && mm == MIG_PROMOTE)) goto keep_locked; /* Relocate its contents to another node. */ @@ -452,8 +476,10 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, if (damos_pa_filter_out(s, folio)) goto put_folio; - folio_clear_referenced(folio); - folio_test_clear_young(folio); + if (mm != MIG_PROMOTE) { + folio_clear_referenced(folio); + folio_test_clear_young(folio); + } if (!folio_isolate_lru(folio)) goto put_folio; /* @@ -471,6 +497,7 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, case MIG_PAGEOUT: applied = reclaim_pages(&folio_list); break; + case MIG_PROMOTE: case MIG_DEMOTE: applied = damon_pa_migrate_pages(&folio_list, mm); break; @@ -530,6 +557,8 @@ static unsigned long damon_pa_apply_scheme(struct damon_ctx *ctx, return damon_pa_mark_accessed(r, scheme); case DAMOS_LRU_DEPRIO: return damon_pa_deactivate_pages(r, scheme); + case DAMOS_PROMOTE: + return damon_pa_migrate(r, scheme, MIG_PROMOTE); case DAMOS_DEMOTE: return damon_pa_migrate(r, scheme, MIG_DEMOTE); case DAMOS_STAT: @@ -552,6 +581,8 @@ static int damon_pa_scheme_score(struct damon_ctx *context, return damon_hot_score(context, r, scheme); case DAMOS_LRU_DEPRIO: return damon_cold_score(context, r, scheme); + case DAMOS_PROMOTE: + return damon_hot_score(context, r, scheme); case DAMOS_DEMOTE: return damon_cold_score(context, r, scheme); default: diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 53e47fad5021..9bc48932eb6c 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -1186,6 +1186,7 @@ static const char * const damon_sysfs_damos_action_strs[] = { "nohugepage", "lru_prio", "lru_deprio", + "promote", "demote", "stat", }; diff --git a/mm/vmstat.c b/mm/vmstat.c index 359460deb377..c703abdb8137 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1282,6 +1282,7 @@ const char * const vmstat_text[] = { "pgdemote_kswapd", "pgdemote_direct", "pgdemote_khugepaged", + "pgpromote", "pgscan_kswapd", "pgscan_direct", "pgscan_khugepaged", From patchwork Mon Feb 26 14:05:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3334C54E55 for ; Mon, 26 Feb 2024 14:06:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 588E46B0131; Mon, 26 Feb 2024 09:06:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 50ED66B0133; Mon, 26 Feb 2024 09:06:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3401F6B0134; Mon, 26 Feb 2024 09:06:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 0A4AC6B0131 for ; Mon, 26 Feb 2024 09:06:30 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id CB29F1408DA for ; Mon, 26 Feb 2024 14:06:29 +0000 (UTC) X-FDA: 81834130098.24.E999089 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf23.hostedemail.com (Postfix) with ESMTP id AC0CC140013 for ; Mon, 26 Feb 2024 14:06:27 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956388; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4NxgZ3WLgt/p9y+eS2UytlMMie6cqo2iGxr0OVp+610=; b=EwECxS5NqByb/69QigUrPW73tYIGTh4vyydEEIdYniBrOrVROhUoYc4w6OEd5ZsfFoE7KX E0VrTjhPxuLe5sMAmqEnwaJc7x6+gBMX4/CsuI9m720yVqLI1J4kDVwMV8cvNQJpWGsEHD hNQnh7r7ZsZIKu/RUZ+jkZFhzAGZJjY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956388; a=rsa-sha256; cv=none; b=40j8GubEbCSiuHx48kNZRCctsg8ihvzHzLklSa9mUWp2lpAh+8ynsOkNs96YrhZvOQFBpD hCtIeHQVnDerTgzogDrKsz9PetBSHb2ZZGsyERgNMo86qKqndCUHHfDq1fZdPWU5h1cxBy Yq0Qw1GWL4fX7pX0hKHQshQeVPgpubs= X-AuditID: a67dfc5b-d6dff70000001748-8d-65dc9adceeac From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 6/7] mm/damon/sysfs-schemes: add target_nid on sysfs-schemes Date: Mon, 26 Feb 2024 23:05:52 +0900 Message-ID: <20240226140555.1615-7-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrOIsWRmVeSWpSXmKPExsXC9ZZnoe6dWXdSDabtMrSY2GNgMWf9GjaL XTdCLP7vPcZo8eT/b1aLEzcb2Sw6vy9lsbi8aw6bxb01/1ktjqw/y2Kx7haQtfnsGWaLxcvV LPZ1PGCyOPz1DZPF5EsL2CxeTDnDaHFy1mQWi9lH77E7CHv8PziJ2WPp6TdsHhuagMTOWXfZ PVr23WL3WLCp1KPlyFtWj8V7XjJ5bFrVyeax6dMkdo8TM36zeOx8aOnxYvNMRo/e5ndsHp83 yQXwR3HZpKTmZJalFunbJXBlXF0wnb3gulXFxwm32RsYlxp0MXJySAiYSBx/+4cFxr59dQcT iM0moCZx5eUkIJuDQ0TAQWLVV4UuRi4OZoEGFokfny+B1QsL+EscnrCcGaSGRUBVom2dEkiY V8BMouvfL0aIkZoSj7f/ZAexOQXMJf4fbmcDKRcCqjl6mxmiXFDi5MwnYBOZBeQlmrfOZgZZ JSFwjF3ie+dLqNMkJQ6uuMEygZF/FpKeWUh6FjAyrWIUyswry03MzDHRy6jMy6zQS87P3cQI jL5ltX+idzB+uhB8iFGAg1GJh9eh9k6qEGtiWXFl7iFGCQ5mJRHecJmbqUK8KYmVValF+fFF pTmpxYcYpTlYlMR5jb6VpwgJpCeWpGanphakFsFkmTg4pRoYQwSEvGbm/NZbP69KaPn8qaoV pU+Tf2QwKiztbJASzsrYlX5OUXf20QNbH7xc7zPpZcaeLdHGvB2nAqbpP+XPOchaqxLnq7bA 8PW+ythV5ncnrV5x8l/J6/0ORQfTP/mZTX3B8km+vNjqX810r3+Ovv9Dti0ve9K+v+CLynrn WO/sbrmp3Gu/KLEUZyQaajEXFScCANrcd+m6AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprLIsWRmVeSWpSXmKPExsXCNUNLT/fOrDupBr+/KFtM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6Mqwumsxdct6r4 OOE2ewPjUoMuRk4OCQETidtXdzCB2GwCahJXXk4Csjk4RAQcJFZ9Vehi5OJgFmhgkfjx+RIL SI2wgL/E4QnLmUFqWARUJdrWKYGEeQXMJLr+/WKEGKkp8Xj7T3YQm1PAXOL/4XY2kHIhoJqj t5khygUlTs58AjaRWUBeonnrbOYJjDyzkKRmIUktYGRaxSiSmVeWm5iZY6pXnJ1RmZdZoZec n7uJERhpy2r/TNzB+OWy+yFGAQ5GJR5eh9o7qUKsiWXFlbmHGCU4mJVEeMNlbqYK8aYkVlal FuXHF5XmpBYfYpTmYFES5/UKT00QEkhPLEnNTk0tSC2CyTJxcEo1MO7dPfsYU8OBR6rquvop 3mKHUifcvyzWssDszQmTtS7CwhFJe1pWvlKt8TB361ObJqC9aNf2ipjZ7hp1uTVrOPNMmzs1 //DzLbBZwFy39a+ClES9U2y8p5RK8X4p7utcq3bt3bygob3Mnq14SfQ+d/23H30nGm/bXXcu n6/TXrHmkl9XuO8lJZbijERDLeai4kQAbAwRP7ACAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Stat-Signature: ynwrj5ha6opsixnnfm73naccia7mob3j X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: AC0CC140013 X-HE-Tag: 1708956387-60744 X-HE-Meta: U2FsdGVkX1/YR87EDPs8hnOpG4B+/7DrO/Wn/jB+U/Zj8o/ZiCGylIq6HBOqcIakRZJbNWrdNTMCyQJz7ZbzO0naG/shgDXl8mn2hpUD4uToScWg7XrnhZCtfwkYEcM+ds0bo/wMh3bt2XzUTNZsv0S2r0I1FSeA8QhX02Hf0YsE3rO3dww+2UEJ6UxQsb8uVwC1s2ZfqP5mAYzxuXzFXA35IzK3eZUBogeOFfpMxU14dKGuIg7ZQVjThZgaI9oMTYZkXg/KWIDlqwQ07CAAuoGeS9TzsyaSaKH2gnfOh437GV2g4XfZfGlJEatQq9MtSDiy4JJeMi4lEKYnM/V+CZtWdQqqrgqDH7cXrGmNksG92f6+ZaswtWMkjmI0u8RvTZvVA4qGp/Rfi5ZhF1gq9FH26/kXGHFxxsgSUcd4/Q7CdZMmG0qxpLA81MbXHtL4eG+nSR/2PXPhx1Zse07YgQr8R4mX1h+VN55lCKt7X1FpVmON4dKNSNXijfQKCZGxzsP2E8p81A0bQuiGEHfNLEM2d1jijPmVJZjF1AqBEYl4M4eVmQoFoh+mEuHEcPTvvwb8VyPTcRKW76RJWp7+rQAzJFyKVRFe7hbTvYmfoTwsDOxBEixMthHUrm0EQzccM7WCeHO7cleySDPy+2MHeBOAk8TtytuCxm+2imht9wVhbYkszdnSh8z1ugwJIuYI1X13S27H9zoyUgepk2wDA+A90hhR04dFdaEdOLyg3T4+yRYMX3oAb8UeQz22SWCarz3UrAxT+BB5MtYKaURRssNdmNjrm1k+j1XlubVNdIapqzGIid55QT/W6l3tHpoliwC17rkCESJg/eP/YoRfPmuq64xQTZj30cyTlOH/asOidg5huwnmgg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeongtak Ji This patch adds target_nid under /sys/kernel/mm/damon/admin/kdamonds//contexts//schemes/ target_nid can be used as the destination node for DAMOS actions such as DAMOS_DEMOTE or DAMOS_PROMOTE in the future. Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- include/linux/damon.h | 11 ++++++++++- mm/damon/core.c | 5 ++++- mm/damon/dbgfs.c | 2 +- mm/damon/lru_sort.c | 3 ++- mm/damon/reclaim.c | 3 ++- mm/damon/sysfs-schemes.c | 37 ++++++++++++++++++++++++++++++++++++- 6 files changed, 55 insertions(+), 6 deletions(-) diff --git a/include/linux/damon.h b/include/linux/damon.h index d7e52d0228b4..4d270956dbd0 100644 --- a/include/linux/damon.h +++ b/include/linux/damon.h @@ -321,6 +321,7 @@ struct damos_access_pattern { * @apply_interval_us: The time between applying the @action. * @quota: Control the aggressiveness of this scheme. * @wmarks: Watermarks for automated (in)activation of this scheme. + * @target_nid: Destination node if @action is "promote" or "demote". * @filters: Additional set of &struct damos_filter for &action. * @stat: Statistics of this scheme. * @list: List head for siblings. @@ -336,6 +337,10 @@ struct damos_access_pattern { * monitoring context are inactive, DAMON stops monitoring either, and just * repeatedly checks the watermarks. * + * @target_nid is used to set the destination node for promote or demote + * actions, which means it's only meaningful when @action is either "promote" or + * "demote". + * * Before applying the &action to a memory region, &struct damon_operations * implementation could check pages of the region and skip &action to respect * &filters @@ -357,6 +362,9 @@ struct damos { /* public: */ struct damos_quota quota; struct damos_watermarks wmarks; + union { + int target_nid; + }; struct list_head filters; struct damos_stat stat; struct list_head list; @@ -661,7 +669,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks); + struct damos_watermarks *wmarks, + int target_nid); void damon_add_scheme(struct damon_ctx *ctx, struct damos *s); void damon_destroy_scheme(struct damos *s); diff --git a/mm/damon/core.c b/mm/damon/core.c index 3a05e71509b9..0c2472818fb9 100644 --- a/mm/damon/core.c +++ b/mm/damon/core.c @@ -316,7 +316,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, enum damos_action action, unsigned long apply_interval_us, struct damos_quota *quota, - struct damos_watermarks *wmarks) + struct damos_watermarks *wmarks, + int target_nid) { struct damos *scheme; @@ -341,6 +342,8 @@ struct damos *damon_new_scheme(struct damos_access_pattern *pattern, scheme->wmarks = *wmarks; scheme->wmarks.activated = true; + scheme->target_nid = target_nid; + return scheme; } diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c index dc0ea1fc30ca..29b427dd1186 100644 --- a/mm/damon/dbgfs.c +++ b/mm/damon/dbgfs.c @@ -279,7 +279,7 @@ static struct damos **str_to_schemes(const char *str, ssize_t len, pos += parsed; scheme = damon_new_scheme(&pattern, action, 0, "a, - &wmarks); + &wmarks, NUMA_NO_NODE); if (!scheme) goto fail; diff --git a/mm/damon/lru_sort.c b/mm/damon/lru_sort.c index f2e5f9431892..fd0492637fce 100644 --- a/mm/damon/lru_sort.c +++ b/mm/damon/lru_sort.c @@ -163,7 +163,8 @@ static struct damos *damon_lru_sort_new_scheme( /* under the quota. */ "a, /* (De)activate this according to the watermarks. */ - &damon_lru_sort_wmarks); + &damon_lru_sort_wmarks, + NUMA_NO_NODE); } /* Create a DAMON-based operation scheme for hot memory regions */ diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c index ab974e477d2f..973ac5df84eb 100644 --- a/mm/damon/reclaim.c +++ b/mm/damon/reclaim.c @@ -147,7 +147,8 @@ static struct damos *damon_reclaim_new_scheme(void) /* under the quota. */ &damon_reclaim_quota, /* (De)activate this according to the watermarks. */ - &damon_reclaim_wmarks); + &damon_reclaim_wmarks, + NUMA_NO_NODE); } static int damon_reclaim_apply_parameters(void) diff --git a/mm/damon/sysfs-schemes.c b/mm/damon/sysfs-schemes.c index 9bc48932eb6c..8bf5aa98d916 100644 --- a/mm/damon/sysfs-schemes.c +++ b/mm/damon/sysfs-schemes.c @@ -6,6 +6,7 @@ */ #include +#include #include "sysfs-common.h" @@ -1175,6 +1176,7 @@ struct damon_sysfs_scheme { struct damon_sysfs_scheme_filters *filters; struct damon_sysfs_stats *stats; struct damon_sysfs_scheme_regions *tried_regions; + int target_nid; }; /* This should match with enum damos_action */ @@ -1202,6 +1204,7 @@ static struct damon_sysfs_scheme *damon_sysfs_scheme_alloc( scheme->kobj = (struct kobject){}; scheme->action = action; scheme->apply_interval_us = apply_interval_us; + scheme->target_nid = NUMA_NO_NODE; return scheme; } @@ -1424,6 +1427,32 @@ static ssize_t apply_interval_us_store(struct kobject *kobj, return err ? err : count; } +static ssize_t target_nid_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct damon_sysfs_scheme *scheme = container_of(kobj, + struct damon_sysfs_scheme, kobj); + + return sysfs_emit(buf, "%d\n", scheme->target_nid); +} + +static ssize_t target_nid_store(struct kobject *kobj, + struct kobj_attribute *attr, const char *buf, size_t count) +{ + struct damon_sysfs_scheme *scheme = container_of(kobj, + struct damon_sysfs_scheme, kobj); + int err = 0; + + if (scheme->action != DAMOS_DEMOTE && + scheme->action != DAMOS_PROMOTE) + return -EINVAL; + + /* TODO: error handling for target_nid range. */ + err = kstrtoint(buf, 0, &scheme->target_nid); + + return err ? err : count; +} + static void damon_sysfs_scheme_release(struct kobject *kobj) { kfree(container_of(kobj, struct damon_sysfs_scheme, kobj)); @@ -1435,9 +1464,13 @@ static struct kobj_attribute damon_sysfs_scheme_action_attr = static struct kobj_attribute damon_sysfs_scheme_apply_interval_us_attr = __ATTR_RW_MODE(apply_interval_us, 0600); +static struct kobj_attribute damon_sysfs_scheme_target_nid_attr = + __ATTR_RW_MODE(target_nid, 0600); + static struct attribute *damon_sysfs_scheme_attrs[] = { &damon_sysfs_scheme_action_attr.attr, &damon_sysfs_scheme_apply_interval_us_attr.attr, + &damon_sysfs_scheme_target_nid_attr.attr, NULL, }; ATTRIBUTE_GROUPS(damon_sysfs_scheme); @@ -1690,7 +1723,8 @@ static struct damos *damon_sysfs_mk_scheme( }; scheme = damon_new_scheme(&pattern, sysfs_scheme->action, - sysfs_scheme->apply_interval_us, "a, &wmarks); + sysfs_scheme->apply_interval_us, "a, &wmarks, + sysfs_scheme->target_nid); if (!scheme) return NULL; @@ -1721,6 +1755,7 @@ static void damon_sysfs_update_scheme(struct damos *scheme, scheme->action = sysfs_scheme->action; scheme->apply_interval_us = sysfs_scheme->apply_interval_us; + scheme->target_nid = sysfs_scheme->target_nid; scheme->quota.ms = sysfs_quotas->ms; scheme->quota.sz = sysfs_quotas->sz; From patchwork Mon Feb 26 14:05:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honggyu Kim X-Patchwork-Id: 13572203 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C9DEC54E4A for ; Mon, 26 Feb 2024 14:06:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 863CB6B0132; Mon, 26 Feb 2024 09:06:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 78C1F440147; Mon, 26 Feb 2024 09:06:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 424716B0132; Mon, 26 Feb 2024 09:06:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 2977D6B0133 for ; Mon, 26 Feb 2024 09:06:30 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C1DEE1207F2 for ; Mon, 26 Feb 2024 14:06:29 +0000 (UTC) X-FDA: 81834130098.20.58B7AA3 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf09.hostedemail.com (Postfix) with ESMTP id E18AF140030 for ; Mon, 26 Feb 2024 14:06:27 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708956388; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zXbLP2GSUHeXTzjP624Gn+oPXDlC4J4rQeiJ29BCBCE=; b=YWQ9qSjynX3+/4qNAOueLIVjm0efCwLSA1pB1wwvqA3dCe7wYyFZazanphfg4ix8O0KI9u 00lRJ/kCJp4P7mx/TmFWtxOqHLv0FZf0hBC6ngT/XunD53JIWFIythSLXWgIq2p8qI5xxV TcIh7Q3eZKTP/1rSN1WbAio02qbmYT4= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf09.hostedemail.com: domain of honggyu.kim@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=honggyu.kim@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708956388; a=rsa-sha256; cv=none; b=sBAf3LD3N1JRm8cMBpTCcTeYnrYlYVNAWouBv5LecbMXVU0U3H/JlGwKUFtQf7hD2MFyPR qHYHQ7bht54+eDagGFEGfId5RctmTMOxnuTiHmMifPDjdQD9axqrfis/Soa+eaNIbAkEZp fL41sh3goZ9AmomU9z/uEZHAwCEilRE= X-AuditID: a67dfc5b-d6dff70000001748-93-65dc9ade7648 From: Honggyu Kim To: sj@kernel.org, damon@lists.linux.dev, linux-mm@kvack.org Cc: akpm@linux-foundation.org, apopple@nvidia.com, baolin.wang@linux.alibaba.com, dave.jiang@intel.com, honggyu.kim@sk.com, hyeongtak.ji@sk.com, kernel_team@skhynix.com, linmiaohe@huawei.com, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, lizhijian@cn.fujitsu.com, mathieu.desnoyers@efficios.com, mhiramat@kernel.org, rakie.kim@sk.com, rostedt@goodmis.org, surenb@google.com, yangx.jy@fujitsu.com, ying.huang@intel.com, ziy@nvidia.com, 42.hyeyoo@gmail.com Subject: [PATCH v2 7/7] mm/damon/sysfs-schemes: apply target_nid for promote and demote actions Date: Mon, 26 Feb 2024 23:05:53 +0900 Message-ID: <20240226140555.1615-8-honggyu.kim@sk.com> X-Mailer: git-send-email 2.43.0.windows.1 In-Reply-To: <20240226140555.1615-1-honggyu.kim@sk.com> References: <20240226140555.1615-1-honggyu.kim@sk.com> MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBIsWRmVeSWpSXmKPExsXC9ZZnoe69WXdSDZa1cVtM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBad35eyWFzeNYfN4t6a/6wWR9afZbFYdwvI2nz2DLPF4uVq Fvs6HjBZHP76hsli8qUFbBYvppxhtDg5azKLxeyj99gdhD3+H5zE7LH09Bs2jw1NQGLnrLvs Hi37brF7LNhU6tFy5C2rx+I9L5k8Nq3qZPPY9GkSu8eJGb9ZPHY+tPR4sXkmo0dv8zs2j8+b 5AL4o7hsUlJzMstSi/TtErgyPl+/wVTwQ76iqe8rWwNjr2QXIyeHhICJxLEtr1hh7L2rzzCC 2GwCahJXXk5i6mLk4BARcJBY9VWhi5GLg1mggUXix+dLLCA1wgLxEiu27wPrZRFQlTg77RqY zStgJrH9dhcjxExNicfbf7KD2JwC5hL/D7ezgcwUAqo5epsZolxQ4uTMJ2AjmQXkJZq3zmYG 2SUhcIpd4tSzU+wQcyQlDq64wTKBkX8Wkp5ZSHoWMDKtYhTKzCvLTczMMdHLqMzLrNBLzs/d xAiMv2W1f6J3MH66EHyIUYCDUYmH16H2TqoQa2JZcWXuIUYJDmYlEd5wmZupQrwpiZVVqUX5 8UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhgDSu/zu586uPSX6Afb/49c H9mJSL3S3Oza2bzPtvHlHn6HZ4efrk3U2mXIuNGh8s5Jmd4db1Y+nJi7XHX+Ufut0qt7jJb9 esMlvcxL6ajI/jlM4QdWZr8N3MNz/v+t4Cus5efixB2shdX+MbnsCYqb/UjEXDJ7ltbRPvNL joKHVk16Hv13+9JSJZbijERDLeai4kQAoSBEXbsCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprHIsWRmVeSWpSXmKPExsXCNUNLT/furDupBrPvWFlM7DGwmLN+DZvF rhshFv/3HmO0ePL/N6vFiZuNbBafn71mtuh88p3R4vDck6wWnd+Xslhc3jWHzeLemv+sFkfW n2WxWHcLyNp89gyzxeLlahaHrj1ntdjX8YDJ4vDXN0wWky8tYLN4MeUMo8XJWZNZLGYfvcfu IO7x/+AkZo+lp9+weWxoAhI7Z91l92jZd4vdY8GmUo+WI29ZPRbvecnksWlVJ5vHpk+T2D1O zPjN4rHzoaXHi80zGT16m9+xeXy77eGx+MUHpgDBKC6blNSczLLUIn27BK6Mz9dvMBX8kK9o 6vvK1sDYK9nFyMkhIWAisXf1GUYQm01ATeLKy0lMXYwcHCICDhKrvip0MXJxMAs0sEj8+HyJ BaRGWCBeYsX2fawgNouAqsTZadfAbF4BM4ntt7sYIWZqSjze/pMdxOYUMJf4f7idDWSmEFDN 0dvMEOWCEidnPgEbySwgL9G8dTbzBEaeWUhSs5CkFjAyrWIUycwry03MzDHVK87OqMzLrNBL zs/dxAiMtWW1fybuYPxy2f0QowAHoxIPr0PtnVQh1sSy4srcQ4wSHMxKIrzhMjdThXhTEiur Uovy44tKc1KLDzFKc7AoifN6hacmCAmkJ5akZqemFqQWwWSZODilGhjNfZR5/H8w5K5/oMaV Y7nh34Ppcn3CBydPUVNkPLBLYOv6U00bsl53xitqxr+L+F4TvfNi5YUdIfFxIqxS28oFOnQP JGv0b3lRYO4jc03guYyn96xV6R6z1vlvqTzA5jKD+9xvw00WryXCAn49CPO/LvWttOIS143F y+IzlzhrTombvf9R9SQlluKMREMt5qLiRACjXRGRsQIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: E18AF140030 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: qcc67f7yjpe1pxzx39tqe543f7mr16m4 X-HE-Tag: 1708956387-711883 X-HE-Meta: U2FsdGVkX1/+8OzBzYmfPbugzwrDhjbfjs2fi35abMnH0X+khix9r3mPq+KqeeDYbAiZwSmKLBYBZwuIUmp0vQtD8NNfaF5xlhQjeCVsrkErRCbJ3h0r1PZRy1hP07RlC8sYfR6H+mPKmn7a2Pf1z9raLOQrVg/FinAMZ4FXRbsrBfLGQXanojNOw+T3+oRJB+E1duzJ+ltP9rlIGEg4WB7hpon+OY6ho7Z3IcLnPbv9Cmv623cPvUnMkr5BRhYgxrhTbchnBIbVyC+dSI5j3TEC6OAf9ZhkS579bTVe6a/ytO3IveP7uv7uU3eanHxlNT13xxVxiGQbOwebuy+hP8c5tGMNd3fb9Ma6H6NJy44S/P14J+zbd+I40l5v/aX3afbOr2M9BEz0voXW5RuXPYyyKscQH0iNMCQuA70wxXgfIs6stpCaaB9qfJ1ilpnzTvOD7x9h+7JTTZH0gC/SL34i/FG86FdZG3dhX5RGUI+IzdNZ1DAJLtNdXxKcYSMd7hoIY0aVNbX5dh45eJoC/wH7nECPVq6TYE6CK9GW/0xcu1bHeIF1rkw1nDHrQBlc4NVWT48nEWL97uNKwvzwVECQGaucgF4ZAZ++gWmf+MHwO3/pXdCsG48NEb5Fp7VyANSTW4fNx3ZfMxnt2nix35tGzTLxC2vjUI2pGBXFdwzA4wnBCpFhhJrZY9q9CZPlqXZvXc+zB++mGuVcXr3SFvfIhIhEwDMGXbQbt5Dpf2lJhC+I0DSxHNEbMcMB+AXTYYuW38Ag4td9NAMBaT/bYxg/tV6hHgDyWmXcv00hCvZi3ceUsQcnA4vAJJkFJRGcDvlzWWxDMncf6fN7g9usIqz+1d8QwPFrjWAj8fEwjkQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000012, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Hyeongtak Ji This patch changes DAMOS_PROMOTE and DAMOS_DEMOTE to use target_nid of sysfs as the destination NUMA node of migration. This has been tested on qemu as follows: $ cd /sys/kernel/mm/damon/admin/kdamonds/ $ cat contexts//schemes//action promote $ echo 1 > contexts//schemes//target_nid $ echo commit > state $ numactl -p 2 ./hot_cold 500M 600M & $ numastat -c -p hot_cold Per-node process memory usage (in MBs) PID Node 0 Node 1 Node 2 Total -------------- ------ ------ ------ ----- 701 (hot_cold) 0 501 601 1101 Signed-off-by: Hyeongtak Ji Signed-off-by: Honggyu Kim --- mm/damon/paddr.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 37a7b34a36dd..5e057a69464f 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -240,9 +240,9 @@ enum migration_mode { */ static unsigned int migrate_folio_list(struct list_head *migrate_folios, struct pglist_data *pgdat, - enum migration_mode mm) + enum migration_mode mm, + int target_nid) { - int target_nid; unsigned int nr_succeeded; nodemask_t allowed_mask; int reason; @@ -250,12 +250,14 @@ static unsigned int migrate_folio_list(struct list_head *migrate_folios, switch (mm) { case MIG_PROMOTE: - target_nid = next_promotion_node(pgdat->node_id); + if (target_nid == NUMA_NO_NODE) + target_nid = next_promotion_node(pgdat->node_id); reason = MR_PROMOTION; vm_event = PGPROMOTE; break; case MIG_DEMOTE: - target_nid = next_demotion_node(pgdat->node_id); + if (target_nid == NUMA_NO_NODE) + target_nid = next_demotion_node(pgdat->node_id); reason = MR_DEMOTION; vm_event = PGDEMOTE_DIRECT; break; @@ -358,7 +360,8 @@ static enum folio_references folio_check_references(struct folio *folio) */ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, struct pglist_data *pgdat, - enum migration_mode mm) + enum migration_mode mm, + int target_nid) { unsigned int nr_migrated = 0; struct folio *folio; @@ -399,7 +402,7 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, /* 'folio_list' is always empty here */ /* Migrate folios selected for migration */ - nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm); + nr_migrated += migrate_folio_list(&migrate_folios, pgdat, mm, target_nid); /* Folios that could not be migrated are still in @migrate_folios */ if (!list_empty(&migrate_folios)) { /* Folios which weren't migrated go back on @folio_list */ @@ -426,7 +429,8 @@ static unsigned int damon_pa_migrate_folio_list(struct list_head *folio_list, * common function for both cases. */ static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, - enum migration_mode mm) + enum migration_mode mm, + int target_nid) { int nid; unsigned int nr_migrated = 0; @@ -449,12 +453,14 @@ static unsigned long damon_pa_migrate_pages(struct list_head *folio_list, } nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, - NODE_DATA(nid), mm); + NODE_DATA(nid), mm, + target_nid); nid = folio_nid(lru_to_folio(folio_list)); } while (!list_empty(folio_list)); nr_migrated += damon_pa_migrate_folio_list(&node_folio_list, - NODE_DATA(nid), mm); + NODE_DATA(nid), mm, + target_nid); memalloc_noreclaim_restore(noreclaim_flag); @@ -499,7 +505,8 @@ static unsigned long damon_pa_migrate(struct damon_region *r, struct damos *s, break; case MIG_PROMOTE: case MIG_DEMOTE: - applied = damon_pa_migrate_pages(&folio_list, mm); + applied = damon_pa_migrate_pages(&folio_list, mm, + s->target_nid); break; default: /* Unexpected migration mode. */