From patchwork Wed Nov 8 06:58:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13449621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 676ECC4332F for ; Wed, 8 Nov 2023 06:59:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 046FC8D009C; Wed, 8 Nov 2023 01:59:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F39DB8D0058; Wed, 8 Nov 2023 01:59:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD9C78D009C; Wed, 8 Nov 2023 01:59:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C904B8D0058 for ; Wed, 8 Nov 2023 01:59:22 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A32CDC0BE7 for ; Wed, 8 Nov 2023 06:59:22 +0000 (UTC) X-FDA: 81433885764.09.EDE7BE4 Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2092.outbound.protection.outlook.com [40.107.255.92]) by imf30.hostedemail.com (Postfix) with ESMTP id B606080011 for ; Wed, 8 Nov 2023 06:59:19 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=clmJ5Fma; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=vivo.com; spf=pass (imf30.hostedemail.com: domain of link@vivo.com designates 40.107.255.92 as permitted sender) smtp.mailfrom=link@vivo.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1699426760; a=rsa-sha256; cv=pass; b=fjNy/jjJ6GL0iAQr0HwjCy/54gzF+QiZ9eUNO10hidSL6sIhmAlW3BfIzkvKkKe3KWqy34 0ZiRxBilj/NJwv69a9tUkwQoroeZz0IzJ51LnFvjRYG+mpEUlU6Gj8rO83KYslSYj6kOIH aPkK0NdKLXGySeo+V3UKeDXUknI+350= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=clmJ5Fma; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=vivo.com; spf=pass (imf30.hostedemail.com: domain of link@vivo.com designates 40.107.255.92 as permitted sender) smtp.mailfrom=link@vivo.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699426760; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IY+4KZ/gKYHUwky0cc5tBRLchnjCs2TVZRgBQ1lHr2g=; b=6ugfKpxKfAf42Cv5itwCj6kybgUjVuo72cR/vtBJao+46psCJ4pc4ZvNYDmU8TRB4yaanH ZvbcDlApfCUV0dH1Cwpe9PdPDa6K8YM/XnRRcoPPpz3JsVLjrYFb3DZhlniAYKSmJr5JSp MjGKUst3x+9NRswx6Y+LiS949w7EkoA= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=og/7ipoEpj7QhCnqrKQBMKHou8aurJLOkJO6KmdLxw6jfrcLykOQdGSUd566prgNn+jaeJ+MjZHoagHO3OLgnJjSPaZ1q9OdiXxbbtxLh0S7qvjTx2EHS3X3aTweo4d1oOefXPzurd/ASNbcdQhjZxbkKo3QX88+DaoJMtGIoVsDDBqvy297qB1d4MrLXVwr1HAd4waH+hBdHiN7pw5DDE9NsE0WpfCDxyL7NfTQEiW3GufaP33XWzwm0aDxNrG0Xu9EBMCeDxjlhicXvD6AA8j5pD0Fofr4OuojRYj9F5wnf4tDzzoKxwLGdPAPi03cOXeFoC5BgCsXPsHmY5cDmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=IY+4KZ/gKYHUwky0cc5tBRLchnjCs2TVZRgBQ1lHr2g=; b=UNTcquFMuWT1UNOXlPSym+DDlhbkUgIJ4+6zaIZvSo+FTEBPa29BJUcwKfz0t2B4qzN1skD/23iC8bZfc3A4sG1SbcylzBI3r8xo9oPNrXOOU23yr1GD4MGiNMKeLeOhiUQ+Y7jMpgP2upG/zjBUxJ4LYBUrTCVqpLpCu4bgwnQ9Cu8DEdinhyMUdmCrnBs0WussfWwJOGn0/B42smt7dX76DyT39Di7Pth7lZWn78phtp0rWwl+XDyilCtNI4mxY6dSYDDiBKGkREEmlLlUrq6VoSlJHAY9XkIGd53Zx2t3IGZs6ZHGrrUYr7Pufblt9yPvPZqQqtj74aL6tuiEoA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IY+4KZ/gKYHUwky0cc5tBRLchnjCs2TVZRgBQ1lHr2g=; b=clmJ5FmaB1ehw+6p9k6Y5/LZ7fFqPom8LAzeyK6h+b9sgyRs3bXqFpALvIEoHUEWiOUGC8lCDEJjzCSPztl3epdzaOVoVct8GZvzoXEA+BvSlEhjbrCu7X4iHNMe2Fjh+fqXkvbNDxJhKI1jzZoYT8L+K2Qi4w22qpJIDZnKj+h0BmfAg5MAnG0PRemm8pgS4SEs91AVHpjTrFyyUQtGcUk7XekLhz50rMw2XXHr1ghx/0HeBD5DtLRbXPFs/S8jKHORK91aohbwA/kVbhamtv6J50x/xOe9byLNu1AzrhAiLe/s8tL+KqXO9uP1G0sOJjT+z5P7/WKIyPMxnI7mtg== Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by TYZPR06MB3983.apcprd06.prod.outlook.com (2603:1096:400:2a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6954.31; Wed, 8 Nov 2023 06:59:16 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::d754:7b3:dc4c:6b48]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::d754:7b3:dc4c:6b48%6]) with mapi id 15.20.6954.027; Wed, 8 Nov 2023 06:59:16 +0000 From: Huan Yang To: Tejun Heo , Zefan Li , Johannes Weiner , Jonathan Corbet , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Huang Ying , Yosry Ahmed , Kefeng Wang , Peter Xu , "Vishal Moola (Oracle)" , Liu Shixin , Yue Zhao , Hugh Dickins , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: opensource.kernel@vivo.com, Huan Yang Subject: [PATCH 1/4] mm: vmscan: LRU unbalance cgroup reclaim Date: Wed, 8 Nov 2023 14:58:12 +0800 Message-Id: <20231108065818.19932-2-link@vivo.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231108065818.19932-1-link@vivo.com> References: <20231108065818.19932-1-link@vivo.com> X-ClientProxiedBy: SG2PR04CA0187.apcprd04.prod.outlook.com (2603:1096:4:14::25) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|TYZPR06MB3983:EE_ X-MS-Office365-Filtering-Correlation-Id: d59434f4-ddf3-471d-2eac-08dbe02838d3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OC10UwMzJq9lwG1MKCtZWBAljF2GRqDt06D4WRCPcyNK2NP38ig6PbFIu6Zicy+0fUuvyK+/srDyzxeEsJycynhKBxyMF7vO6ZBGenDkp27Va+t+xQyxR/QkkRXZ+A7+vKJjfy77/0pgmPjTYWasUdoUwWW770danrERLoXzLXBLjievpgxTX7f7gaQJrVrpZmd3mFoghjsjTfECUFx2ftKtXcP9XjngtLjZ9/rWKdONOe7yFWZSzsjjM8cvCL25ylsQWFZz3k++QRjPYNq121+63y67Gu53mf8Yi40GRrbyGFfLNFayAOZRnAMLG0lidtE66gHidjTeU0UMgCUbzLpqtQzCwDgtaX3rhGOi55VlcBb0TxDVD4MxEwBBDACos2sd6vA7PCyWY/vfst9pyOelt9B1hjgUzP13ru5x8HR2zWpWasNTaqMAqcMkanrLHnHD50oQBogI2Rr+y0fuq5UMC0XnGhphY3TvDTwwPCsDfOa0sFRv0jA1OoosdTWGPLFxz9bRMupKFk4dNIUSB/YMcuFTVkmLQmdbbepMIchZd9h690iJ21V+Vovr00nl0u8246AS7n85BUO1ItSJtTwJJ80zNcVbURxh03C0AEY5z4gGbgWgnDfbgKESucCanQ9nb7cBaiRUxLjWv3Xjjg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(376002)(346002)(136003)(396003)(39850400004)(230922051799003)(64100799003)(186009)(1800799009)(451199024)(6486002)(6512007)(5660300002)(6506007)(478600001)(52116002)(6666004)(8936002)(8676002)(4326008)(7416002)(316002)(2906002)(26005)(110136005)(66556008)(66476007)(66946007)(86362001)(41300700001)(38100700002)(2616005)(107886003)(1076003)(38350700005)(921008)(36756003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3TtqxLh1Yc89LV/rI0is1nEgwXsPy5ho3SmJM0eSUXP7+gChf4WCxSDLCdQJwmVur1d2/sXmbl0n8ip2n8aBL7Omt45nyO7wkLdOCdNYurvuoRUmpGCwJrRp/h+PjzYRQPIQr19wGUQQD08fTszr4yqTIRabD7GY3jpkCTKOpCpr2/Yq0kQCkVcG2bbBrWpLBL6i4hxVp/PFrYevrBABVEd4STO0WdANJkyP7lwzXEeiThYr+NlCZFai648S+WFzNIEarxmEW12XNp7bP/6NpuC/FUUMBeO2An8qcxUCHkaQfZuAuzpWTAKXIss0o7gwgopUFiVTNWwraNAGfSlp++GuAuQJOX2ybhoph/Ea4Hl9OvXzflFbcvGAhzy2Bo41wWaNXN8pPWXvG+88qAFiwZfygWl6aBn6e4jm1FZ0OpWimXz1Ay6reteZjquf43HC9TptMvqjsPXawNOCAbEXPOj8DgGCFc1w5JtcBn3PGlnKOirPmNkqz1/ADnzq+kwi7LtewbweLx0n+OWK1bgSYWehRa01iXgk95xlZCWApKwpIwno7CHsdPW7UnC6kzIghXM22+OyyAafLuG47ymkzoRLgeY/63u5xupX6FZWRBqj5BcgCT4ErFdxSrgkzzTCE4b3z3TqJgalkvhjDdWSd5vSa4Gs1XuZUFWFO8fiL2vmp2j0blU8aVKG1uwKMdYoITfLXNfFrrQaIwEisjYG8CCmX4xSJif6A0WE6xUMRbHtozvv/vk8rN98vZNlOPLuXEZUYizHC/EbeLc1/QYCSdx2YdGfna8Ki0V4GndAnB9EMnOxXiUn2Yc0mUk/BAKT6GcJxldlmWjVweG8OQyK7oCYKhVP5vXHj/nS+gV2Jv9MwNeMiZ/RH0v7L4Gwe/cSV1zuNkCamaF2DGOvOYEUjnzI/1IIMrC6uTeHi3VAFKVOlLWViVGi6mPYMzfpK2Trfjy1y1fzGDVUoDbvT1AS4OqH9Vu2sMh0JIYZcZDDWJtZtqwljjRhj/v095i7NSf1+JfrdN5dvPPkIboPLK00M0OXcrpFZBtgBBI7STCb0gW6Ets0Evp4qTkP0BQHIAUkftyFw3JqirRFnjzGLqriJnWowi4cBZkLmgScQDsbOYPGrELWktLzneapz2CxIWQ1MP9I2qOy7vwDC9bJOdR6eWRJrYcrZ/JEcOD8us8NawNitKmbBd7V5Gc1ubHQFxzzb9rrp4W9uvoV+H0Gj43D0SIcRA+CDkd1QmDxaiHfghBZMafHuSYH7ME4LipKWW5EnHFOOl/oZKatbnb/MXDcgCRvxNb+M+DMpLhAAxf7zIpGl2jr1MbZj+2tLBiYY/v81KixcHSYpy4VmvEGytEVQ2zTT+fWVJ1FRmTjW60+r9d/TPqu17bius7eS0Fxb6BOTa1pzEWPVLqAQCHnUsVzrsjTonM3c4fe2xPfPhF7qfZycEfJAPvIiY0VPfRLLE1sFDVgy55O3to+0+y6tm5TwVNlTgaGgTP5D+W6qrG9UhHQqSzqV87j3SENAERexbuWYadgSiBKudOg9vZDphZyxySnf7fAXsUTD4ECigluFipHWQ7ul5ZO+M4irktOqsJC X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: d59434f4-ddf3-471d-2eac-08dbe02838d3 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2023 06:59:16.1763 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GCEiU6qf7qwij7S3RtyHN3QzsWYOHjWYQwDwoZtVVNdvMDAJ0zfPOsZJlbdNkCnUMN1JesXLoV6R73rGgioC4g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZPR06MB3983 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B606080011 X-Stat-Signature: yjudfp7uhtumascf9yag8iqa6op7rh6g X-HE-Tag: 1699426759-210847 X-HE-Meta: U2FsdGVkX18+94Z8znk67KCTCfv04hBMkn1FoWrRHxzLHoCaxDj5sGTdXPC1sR49xg6y6GZt+hGMQpvh+kr2X/+U0FIpkHX2BahRfwZ04tUs0uEWw4Esk0hkpiU86zJJaWAuIg6Ucg51JfR6hAqxBAbCkDTqbIVuM5+iCmOexOorBVDE3nivMKcv4+K2Y78xgRUYNJRLqf5W11C+zVQIo0jhe96rqXMc+iej0GUqlspgQSTWjje7FK2SDn+k7TqErJ+RjmE/D81nI5L6KJn7nyJ22pWcIDI2u5ca1Mjy0V8dzAELNP0aIpGIfKEtX8yb//TIx5iWkYavc9xDKnlSmjsASxjiF3ukEf8P3tmks7QhePBWctL9Pym+g+24UsgOrov1jedVwweeLPC66SR/LftmX4cwjT9hg/swW8oTlBXTxr8hDbwTE7eJY7HG+MUpPsqaMewPm5z1PTxulBl51tANws1IlsneHAD9izl/Fn/rwuLCxBrSPV+guOnhehycD/xgPCIWqdCe4BT7fdpuNJsY0332lYzXrUocc/FMDSKPp2HeqPyKYh1fmbb+oXId9aiP+ArXe3WsenHp+eEEbkoUmTO+mLWQUXY13dJ9HiiwGaGfChsHHe7Y2nMeryhkTqgygY/z49J+pObmNBC70li4kJZ86lJZgobywVSqpkUk6UX9NoSRN5+C1sTFr0vuT/vAFZ1CKXjuZLvCGptPNF+XmFwDIIcOcniQJdO2gG2/LJMXUgF9gBJVBJJ6HWrp28o29bdbekmJwZxOJ/HNIA2GgWWFpFZp5sX0Jj26lO6dpyaQP+poQF1dCWuIteGBxwhHf1EDCqZGNpigzjutYW7F+3fFsBpjUxTIWGwChpDdCglsv2cYHbYNgPNykS2is5E0+bImD3vkyYkX1Zh2AwGXPvm9CeQpngOBd7ql8N5W2iVpCd60uzFoMdZvQERePHp2VnbW5H9UCWXNQ2D HhT4Gmq2 phUEe5aYtPMQlv6zFQgIgsP4Lu/9jppJv5UaFtv8t/B+K3IqyAZcjC5U9oISma7lQxzgdAkRVy8O77/6hk8orZ6pK/ahxMcLyQE98/RkcgzG4EHw8+EJ8fDpl7aMThlRHkfuC6pYEdBtx4xMG1Q70s5ozukj0O5imdNw/At1L48QZvncELOP5/mgBhzvDFU+FgDcxxTZoRzwJuTra4r5nuUOzJpkR0Khd9VGH8DpoNnizI2cV6A6IEdsxSuqQ2yiVtPDVG+ybnTWktnGmxRqTxRc8Nshl/vapBUorRhBMbkJfCW8kUZL1B8Cg6fmjZkwoyhob0prhWclDkOVQCPReW7g+p4nECJgv6wf1AS1go+S8tzmLgUJB9S36uTTqm0rtG4477EvQBA11THVSkzckn0hDkVXsAdhFYTBa2CMC9AqNcXwGFiK+2cMvaDN4nZoRIDe57xDTWSgfoOU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch add unbalance flag into sc, and just allow cgroup reclaim trigger it. If flag set, LRU reclaim will only touch target type folios. Signed-off-by: Huan Yang --- mm/vmscan.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index 2cc0cb41fb32..6ed06e73143a 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -111,6 +111,10 @@ struct scan_control { /* Proactive reclaim invoked by userspace through memory.reclaim */ unsigned int proactive:1; + /* Unbalance reclaim pages. */ + unsigned int unbalance_anon:1; + unsigned int unbalance_file:1; + /* * Cgroup memory below memory.low is protected as long as we * don't threaten to OOM. If any cgroup is reclaimed at @@ -2338,6 +2342,18 @@ static void prepare_scan_control(pg_data_t *pgdat, struct scan_control *sc) } } +static __always_inline bool unbalance_anon_reclaim(struct scan_control *sc, + int swappiness) +{ + return sc->unbalance_anon && cgroup_reclaim(sc); +} + +static __always_inline bool unbalance_file_reclaim(struct scan_control *sc, + int swappiness) +{ + return sc->unbalance_file && cgroup_reclaim(sc); +} + /* * Determine how aggressively the anon and file LRU lists should be * scanned. @@ -2358,6 +2374,16 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, unsigned long ap, fp; enum lru_list lru; + if (unlikely(unbalance_file_reclaim(sc, swappiness))) { + scan_balance = SCAN_FILE; + goto out; + } + + if (unlikely(unbalance_anon_reclaim(sc, swappiness))) { + scan_balance = SCAN_ANON; + goto out; + } + /* If we have no swap space, do not bother scanning anon folios. */ if (!sc->may_swap || !can_reclaim_anon_pages(memcg, pgdat->node_id, sc)) { scan_balance = SCAN_FILE;