From patchwork Tue Jul 2 08:44:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Huan Yang X-Patchwork-Id: 13719117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4766FC3064D for ; Tue, 2 Jul 2024 08:49:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D73776B0085; Tue, 2 Jul 2024 04:49:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D22DC6B008C; Tue, 2 Jul 2024 04:49:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B28986B0093; Tue, 2 Jul 2024 04:49:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8B4FC6B008C for ; Tue, 2 Jul 2024 04:49:14 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 1BF251A202D for ; Tue, 2 Jul 2024 08:49:14 +0000 (UTC) X-FDA: 82294188228.16.1D3BBCA Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2082.outbound.protection.outlook.com [40.107.117.82]) by imf21.hostedemail.com (Postfix) with ESMTP id 106381C001A for ; Tue, 2 Jul 2024 08:49:09 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=EXvsFRi1; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf21.hostedemail.com: domain of link@vivo.com designates 40.107.117.82 as permitted sender) smtp.mailfrom=link@vivo.com; dmarc=pass (policy=quarantine) header.from=vivo.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719910128; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JHncJhtTb2CVCgR6KyprnXFeC/NMYgOPEuy1IeHa92A=; b=rhGGNjNKOS9iR5QrlhsdDEFRdrjnkvbCEVBsn4my4/BrCmO4T7/nVivd8yqjr/UxutHl8w li4/zkN6sY+ywyQMDKE37sw0HL7O0vaW0QlrBxO9a3q76Qic9m6PtmzVKWwTcMDcuNE0qW w4lTCOTrxsEXLB8mKBfVREfslZV2cZ0= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1719910128; a=rsa-sha256; cv=pass; b=VFgrzKP2P+w64N2WlARr4HpZHP4Qr3TD9T4dductZVTHXy1yy8mYxlGou4VmdBmz0l38u7 qwPz3e0eiCgb8OQKkFNAQUMBFJvBKdaNt5fA5ikPx3/o4SKylDsP2Vf4d/NBIpvi6Bfk2l PEluud8obH0qAvZCoYJXwUopVqNHIv0= ARC-Authentication-Results: i=2; imf21.hostedemail.com; dkim=pass header.d=vivo.com header.s=selector2 header.b=EXvsFRi1; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf21.hostedemail.com: domain of link@vivo.com designates 40.107.117.82 as permitted sender) smtp.mailfrom=link@vivo.com; dmarc=pass (policy=quarantine) header.from=vivo.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NG0QT6WaQyDMRNhuKpwKm4IjyN6ujj4mv9OjwCybe07stJvxBXnODj5XQo4R6+aGR1LFplJoRWF1eSM9nuazxdi0oO8QlzchB14CR7t2xPm8Mp3d5V0rZKwLRQRQ+bBvt/LgrMcNN+9nBYscp92Z14c43nA2ch78kyODkZaqEt97YZ8ELfc3yPESpAglE55NNWGg4L0XfFPH2fJJKJSeZ8UzX956EDiqglbXuS09osPj7Av5qsrAPPJhhHTJz5KQseffF2SRUvbCu1IUxwp6yKb7ejrZoex+9guTWCpCKr0Xg/SflpwsWufTnNppbV5w+YUYQ7jZ75vxIeNOglCbiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JHncJhtTb2CVCgR6KyprnXFeC/NMYgOPEuy1IeHa92A=; b=gWM7HRpHHNti1miCfZWpwf4Er+XP5/Z0vnPPDJqfzCyDiBeGajzkFZS+FanDXCUTBSmZR5WacOMC8LVwcX+2AYBVlTbTC2kiIvEK9l3hT1xIGQo4nb78M92FLRVBcJRsvFtMCRaZajwQYFUVI49Dcb6IjMxTQA0ItjGyqqh1C8nCxkRL/7eou/icHfrOYugVadL3udXI6kEacox4OFTDcRM7orZt6dXhvEf4nUBuMQqRBBlCpQxJEW5maMVDGvJxv9+X0gcmfd86dPSsRiSP+MNeoBKNxX3ZTPLEaj4LbtFJDi0EPKmbRrGP18f3IUGQ9/zV6+XcSbFp/Il6K6P8Ig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=vivo.com; dmarc=pass action=none header.from=vivo.com; dkim=pass header.d=vivo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vivo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JHncJhtTb2CVCgR6KyprnXFeC/NMYgOPEuy1IeHa92A=; b=EXvsFRi1MNmtpSI2shGOY8GWVHvzvVZNozT6or9XUJHiM5GIYo/fKOl8hRUt7nW9s/hJWbln3yw+OikN5mC7fp9Q0Ttbm//1MIIQJ8qzRB9V0omtK5CEVi/TumwHze603NHGVAqB34zT41OlcZ4qog+xAJkN3tI8AFhNDedKbXo/8WELMroCD+hP2emNMVfGbT4SNBCsIDJ8E7jE+eyR6Fqpuwx7kwgTRe23qn4iGoT8vs6Z/6oBcDm/PXWJJESM9Vrm24EBY3WwmG71+qafy7n/WzZVVZ0kln54An/ha8rR6aJMybtp7u8MxkgldIRTkeTsVfUvp68wuuaMtbHSMA== Received: from PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) by SEYPR06MB5790.apcprd06.prod.outlook.com (2603:1096:101:b9::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.23; Tue, 2 Jul 2024 08:46:15 +0000 Received: from PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f]) by PUZPR06MB5676.apcprd06.prod.outlook.com ([fe80::a00b:f422:ac44:636f%6]) with mapi id 15.20.7719.028; Tue, 2 Jul 2024 08:46:15 +0000 From: Huan Yang To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , Ryan Roberts , Chris Li , Dan Schatzberg , Huan Yang , Kairui Song , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christian Brauner Cc: opensource.kernel@vivo.com Subject: [RFC PATCH 3/4] mm: memcg: pmc: support reaper Date: Tue, 2 Jul 2024 16:44:06 +0800 Message-ID: <20240702084423.1717904-4-link@vivo.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240702084423.1717904-1-link@vivo.com> References: <20240702084423.1717904-1-link@vivo.com> X-ClientProxiedBy: SG2PR01CA0148.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::28) To PUZPR06MB5676.apcprd06.prod.outlook.com (2603:1096:301:f8::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PUZPR06MB5676:EE_|SEYPR06MB5790:EE_ X-MS-Office365-Filtering-Correlation-Id: e9c9a2ec-00b6-4829-04f8-08dc9a736ef7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|52116014|7416014|376014|366016|921020|38350700014; X-Microsoft-Antispam-Message-Info: R+ALswNfFrf7uwYXx6CNHD6PVJXXWXMzyIzOMBrmf6WIpIEbzYtLvmBLFD9v+0eCUNthFpaiSC9Gigy4S+GSUyxKpyAiXRw28OV9k2w9LTGmwnk1CFz5Ld0+rUn3AZpMsR1/j79+pp9Ok87mEW206l2K0A0RuGyuoG3lqkKix9T/Fk7lEQXmHwJzMOF9kakAEpI3BAGFqCiaIf80+hlTIBdekNoQvxQNACAyyv8V18ypMD0hY93bTS/0+xhnS8kAm9vRSQT8x5VX+DZNGVLcwZBBfiqqOdjOyG4lOgIGSxXWV4ig7ivvBfbxU0dylMuRZLE7yE9kt/ejp08mcsu98zZBJTCWYfyl3akI2vfYGHtaYEYWl1SjiIoBeDwI9ZK3jt4vkMlXwxqQ9Vy/WFAoiuUqnVV0WosJwHgttx5o3UDJmXZW5QJCzsXhWcDc2MDvk4oYkaGK7CnaWWnfWuxU5foTowNGMiB4FBpLyKa2q6wYFVJMpPUcKuGSYByIb2WkIS2RKEBDM15V2IJdhn/KwS2a8AhUygXngxxlwdrFTUnXjRrkaY9DUTETw3c6OKw3SBsXPBqMytLefR/t/skwpNy1gWa2RX++wL5Xk+32XbRdgZupSE73qJJbcwGfLSFvRwMXMuBD20veYtypoQobT3sn/5SXcjdB/z6oOd45TNIaa5o2xXg9TkqlYrZ0v3dARE5WKAlWvX5dPaKRU3EMhtzhOkv1NBX0KXW6+q2xp0MvdWhanKrP9alMN/MmqkFqyXZ5Gw33V55Cz0dbZgYRrewqEjfyn8eKWqTxnM/NkmfngMcwnNvV51bmbt+SxF8Tb7aq+PWSpQSPXVTFFIyXuKm7LQUAvDe+Ve6aXmm88W4cdWSToovYtP7NtMq23yOvOiGpu7I+0unOMaoju6LfO869Z0FImcy61AYt/vnQpI/sQQW+lxGqu4QEsVLJIV0lil/cUwIj3FrkNq2bLaPQqB5HGTd+uhtIV2Weja5B96WX3DCck5J0+Ci2MIoCEtqh/OOh393KNHbqyOHxKRr3nXoz00P6lBu6cP+t+y4Etn1Ujd6F8Vty64kxqJlmSebZwQT87VXm205iwdpfqJZ4d/chUJP6sOlE15zsVbBGfEiYhEUkzXNkpLThqXPdfxLBqPoOrsxxDGh807r4LU9cmdPu02Xk8mJize1FWKHZ3ch3lYczMHyIR9Dsn4CoiMybHIpkxas2U8PICwR0+q+CGAJRpRAdyPLrlWVXU39nmPFa2CcJ7HiGrWn/ExEZUz9+v/qpLWJj2mEAZwbRytA8s8lA4vRn7oxBXKwtL1xBsmGAljwHbzQQogMHb3+vBBFQeeoGbiuezULJMeRQkeVlsTtteihdbpG7jq/OtSaL3J6RTaDd6fsntly+K54Igcz+ia5eJy9C4iBEdtxE7uOEUr1hkmAkyH9Uq2Ag3N1cgqo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PUZPR06MB5676.apcprd06.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(52116014)(7416014)(376014)(366016)(921020)(38350700014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: r45KEwGVvi1egnDNkWTzmCZ0Dq3RfYHlV9fz3IMCmCE+tjCO1/SRF+yW+Yf+sU9C3Fd1wfn/vBpIDkVCzhkmIGkO49J+N6mgsjrye7e06+izDoLlaFFHVXGSWJ7eMG9Q+RPa1U7Pq9FAhf+mzu02y5L5VMMqPxu1llfbLijAPzQVnVRv1lys2shyonsFRvQ1NmX2OgWU6QYGoymV96VWZGjNX27xTyGUYCOTXnTYFBXVb3CR/FzV3QNn5DIhouVPs/q1sB9nWLUlK6CrFd3itd3lwnW95HJOAt5ANZOPIEb6qgnzCRVLzsvWhVj9n3OgK86HO7p/tOycorKw+0svgBOhJiyPqDpik/q48ZQ/H1cjguNRb/J0RtNYI9i1LGerUTPXjxXkEEGzXIkF8B2LYecxB4Yo0bcwfMrA7SKCceqVtOBWm6wjOA0YYs8lIowhPs78sVJcss8pM9TwbEEs4foE9HjKfFs0giJuVxezPIzlD9ZLJvmy2pgR7rxNwD4vp0BWLhAkJygx0bqxA7edaY+l0MFES3H1K9uRr5fT4fjHVKa7a4SVkDVUXxfTwvk0EWQgPJlG1DL9JtgBDjHkE+vmCAnJUkAstvYUpKD/xbMo7Zm1luGWdfgrOJt+kjP6s1Z21i5IAVpXnd6JyJMKsuRxYcwZwUyqjAqoPl8kFphfotD2NNa+3DU42kOU7a0muEjvUbTYN/jTittf5gQyZ5SN3P8KhyVnYvbnX5pjcQB5e/mZiXyRUhfylKFhprJvSH7PawEFbU88kbIv8XXpcPvEMiVEYM2vuxyn3y2LYnOwNY7m08gjqAUjR8ftDpp3Qg1hWQZOAfwbZ6ToLJArYQEKPr4VjIQ/8KM6LwiPhnYk4iEitCNDKE6qOB4yM3ZAVmszc9cZnsDM632zftROOtJRgYFv9Q5bvREee6MIZ01e7JUlSpMf4WjT3TIllWYMSxFCIncKT8fJAUbQhjwBlyEXrLW4bbMQlCSCXTrIr1S9bO09sqBCtHZUd4fTGsjKMYG65gqyt86BUjLqB9eVeSUlw44FK2hZcUxfdDoh0tR3Tpn9Gqd2EWXLcPR+zMf0KQ7n5AWynaWyZeKrb5ylqSzkG8tnvZMvKl6OjCl65y73BQ/YlOwpXYJFvzJXYS5Gi1LT8VIC94zybgxXB5NLx4TUiZ0K8VmZI6nmQtpwX7qnt/0YlXCCkXbgT+w9PZZqTiBjVG8QMUFn4HGhmWAfJTKYgGRab86PtrcohDzMshXkDAfhQlh2m21ebf9OBhSj9NnEqaAAk4UwvmWzXrJonR17O3zzb1ZUynJFiVJ0XfA2Qhx7hFLIwhoOVym5cgIpUFRx0RsGF9Q8DzIDrNUnNVrwXxM8fbO6ybL0PVW7gANQTIAO/M6eL0g/ZQWFnKRL8TQMT7fNYuG6Tq6T4fgeDFmDdvZSnSSTiQ+gvV5kO/kmZR9xiZNTWgGyj8hqGLEhFcxVZZZgu5vNVDOpKveYmpos60LFaMhtFMtj9zHCfaRJRLg+ZskpO1uIxBmkpWUB9jEqwFGlAiUggtRtThI0vlqBj3CQz9HWTaLUk2akv3MQkIuKEY7Vy5tdr4IOoCUS X-OriginatorOrg: vivo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e9c9a2ec-00b6-4829-04f8-08dc9a736ef7 X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB5676.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jul 2024 08:46:15.4736 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 923e42dc-48d5-4cbe-b582-1a797a6412ed X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IQlY2azfKDb1kg7oQIMdIERcJVFFW+6CsslydbbvgCpB/EyythP7qZTREJ9qZsJ2abYPA7x8j4htXKNs8EJ+2w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SEYPR06MB5790 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 106381C001A X-Stat-Signature: u73okzh5fmeaphugsjjcbejf3m88xbjd X-HE-Tag: 1719910149-130406 X-HE-Meta: U2FsdGVkX18zcySt1Nqs87jYF/nPmuPqN2VvD6rXi9jxTPFE7+f6a31ic6ULgdhtmV2p6VGu93wIH6sAc+2RFSouws1lofb17ud97BRK/PvcDdT2tz9o4HhqXdy3qhWkToTJAOsbtMIlWShOg/GSkQ20dlMCGZmYubCCvz3FKL8vGBL8OAyRZSlhgD2zjU1BelYi/e/BB2TtBqYjfnK2mY8kmUvMUrNJ3qjwFWB3PRHm1WGGCv9hpW1xAX9Nv5MmtFYqz84xpB4rDO6sTjJ60+BL0QxYR7slpPxlTTDOVyk4HXH7G2bNLSc022vZRPWIXj4FhL3az+U2mbR28BYqsKfm1NGYCgrS7EbEWdm/yVnCaawfR4+czSoDFhe3ldEbBuL5NsghAhIZ0QMPslGlCQ5OZ37aervb/5f6MPGBa665SMnvConolvnJtswS7yhZ1LntWS8r0nKps5HV8OPalYyVslLs3L9uroYZXjsvtBk8VIJ7vbAVY5gwsFMCw4n5TmRbToMs6vh0XuGToAxdh2Qzwmscsvvfp/p625i+3HQLwiMTu0oDxuIOe4G94ObkU8bJ4X0QraXtnXhN0nfKiQL80D3WLYfgJWqUnMXsHlp/6pV2+XGg1pQU4vs+pc50fJCTanqL7RiFJ+oEaJTZyJASIgZkYhDJNrVQ7adICq/7Wu62DWnF4FyX7JFcPoGpZ9rDjIvweUIQgZR91jDjNyAfQnP6TjQPO8yhh0A+uxl37lPskDAQCR3dNnwkARb1cYZUA74JMa6la1BoyFWgh3OHBhAqvKD8X1AVUbovf5HyXSVmDWluLLYNtt0qE6JADZjoZWR1KqkS+N73j+GQe39Qxy7+xNdWamE1gSqgglzYRSNGi7TP45mA47mTU0TEAJ988j/pZgeSZDQSJ/uYRtMSpFVwTAEcWa3A/ANGKjACbf6omXFVHMXYrkm2uIpCvNpV1mKODxxEtarHNhs 7cLMJ9yS SVUKZz/XjMMB0hVjuoxkeDYbHejjn9fQSQxfD/RSYOwfoOezeeL6j0fah5E0+VqiJXi0wE2bc2ZCkoVcXlOVivZS+CjzN5w7+8ewD41UZYozaviHWjf701iQIMCvTd5mVGPsEvdBBUnSpaIlW22vUR6zJ9uwGH99HRh9fXCnwAEkkOeAX3IJ1REuZnk0wWaIDvIzl4PSC6ruXZbu0MEg34KJb2EuQFh/aJGgq9dg9cmM6iasOW9DDj28De0FOZ6J/K3IFNa6Gm2qZH9maQAnsE8fpZAM6W7jlrF2m/gVAojsh4+1Pe9jFgxr6PyZ0+nEsc/BAvqa82whTkT8F7FvAXZ3YeAYge3IFuQzLUomd6G+PUHPRxvh+Z7sINw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If memcg enables pmc, it will cache some pages. However, if all processes in memcg exit and there are some remaining pages in the cache, those pages will not be used unless memcg is deleted. To avoid this situation, a periodic reaping job has been added to each memcg when pmc enabled, which will reclaim all the cached memory in the memcg at regular intervals(default 5s). User also can change reaper interval time like below: echo keys=reaper_time=8000000 > memory.cache This memcg will reaper cache each 8s(type unit is us) Signed-off-by: Huan Yang --- include/linux/mmzone.h | 6 ++++ mm/memcontrol.c | 77 ++++++++++++++++++++++++++++++++++++++---- mm/page_alloc.c | 1 + 3 files changed, 77 insertions(+), 7 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 773b89e214c9..b56dd462232b 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -608,6 +608,7 @@ struct mem_cgroup_zone_cache { struct list_head pages; spinlock_t pages_lock; atomic_t nr_pages; + atomic_t nr_reapered; atomic_t nr_alloced; }; @@ -616,6 +617,11 @@ struct mem_cgroup_per_node_cache { struct mem_cgroup_zone_cache zone_cachep[MAX_NR_ZONES]; struct mem_cgroup *memcg; + /* cycle cache reclaim time unit, us, default 5s, 0 means disable reaper */ +#define DEFAULT_PMC_REAPER_TIME ((5 * 1000 * 1000)) + unsigned int reaper_wait; + struct delayed_work reaper_work; + /* max number to hold page, unit page, default 100MB */ #define DEFAULT_PMC_HOLD_LIMIX ((100 << 20) >> PAGE_SHIFT) unsigned int hold_limit; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 9db5bbe63b34..ae6917de91cc 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -7101,6 +7101,39 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, return nbytes; } +/** + * This function use to reaper all cache pages by cycling scan. + * The scan interval time depends on @reaper_wait which can set by `keys` nest + * key. + * Default, each memcg which enabled cache will be reapered every 5s. + */ +static void pmc_reaper(struct work_struct *worker) +{ + struct mem_cgroup_per_node_cache *node_cachep = container_of( + to_delayed_work(worker), struct mem_cgroup_per_node_cache, + reaper_work); + struct mem_cgroup *memcg; + int num; + + if (!READ_ONCE(node_cachep->reaper_wait)) + return; + + memcg = node_cachep->memcg; + rcu_read_lock(); + if (!css_tryget(&memcg->css)) { + rcu_read_unlock(); + return; + } + rcu_read_unlock(); + + num = mem_cgroup_release_cache(node_cachep); + + css_put(&memcg->css); + + schedule_delayed_work(&node_cachep->reaper_work, + usecs_to_jiffies(node_cachep->reaper_wait)); +} + static int __enable_mem_cgroup_cache(struct mem_cgroup *memcg) { int nid, idx; @@ -7141,8 +7174,13 @@ static int __enable_mem_cgroup_cache(struct mem_cgroup *memcg) p->memcg = memcg; p->hold_limit = DEFAULT_PMC_HOLD_LIMIX; p->allow_watermark = DEFAULT_PMC_GAP_WATERMARK; + p->reaper_wait = DEFAULT_PMC_REAPER_TIME; atomic_inc(&pmc_nr_enabled); + + INIT_DELAYED_WORK(&p->reaper_work, pmc_reaper); + schedule_delayed_work(&p->reaper_work, + usecs_to_jiffies(p->reaper_wait)); } if (static_branch_likely(&pmc_key)) @@ -7184,6 +7222,7 @@ static int __disable_mem_cgroup_cache(struct mem_cgroup *memcg) p = nodeinfo->cachep; + cancel_delayed_work_sync(&p->reaper_work); mem_cgroup_release_cache(p); kfree(p); @@ -7207,7 +7246,8 @@ static int mem_cgroup_cache_show(struct seq_file *m, void *v) if (!READ_ONCE(memcg->cache_enabled)) return -EINVAL; - seq_printf(m, "%4s %16s %16s\n", "NODE", "WATERMARK", "HOLD_LIMIT"); + seq_printf(m, "%4s %16s %16s %16s\n", "NODE", "WATERMARK", + "HOLD_LIMIT", "REAPER_TIME"); for_each_online_node(nid) { struct mem_cgroup_per_node *nodeinfo = memcg->nodeinfo[nid]; struct mem_cgroup_per_node_cache *p; @@ -7216,13 +7256,15 @@ static int mem_cgroup_cache_show(struct seq_file *m, void *v) if (!p) continue; - seq_printf(m, "%4d %14uKB %14uKB\n", nid, + seq_printf(m, "%4d %14uKB %14uKB %16u\n", nid, (READ_ONCE(p->allow_watermark) << (PAGE_SHIFT - 10)), - (READ_ONCE(p->hold_limit) << (PAGE_SHIFT - 10))); + (READ_ONCE(p->hold_limit) << (PAGE_SHIFT - 10)), + READ_ONCE(p->reaper_wait)); } seq_puts(m, "===========\n"); - seq_printf(m, "%4s %16s %16s %16s\n", "NODE", "ZONE", "CACHE", "HIT"); + seq_printf(m, "%4s %16s %16s %16s %16s\n", "NODE", "ZONE", "CACHE", + "REAPER", "HIT"); for_each_online_node(nid) { struct mem_cgroup_per_node *nodeinfo = memcg->nodeinfo[nid]; @@ -7242,9 +7284,12 @@ static int mem_cgroup_cache_show(struct seq_file *m, void *v) continue; zc = &p->zone_cachep[idx]; - seq_printf(m, "%4d %16s %14dKB %14dKB\n", nid, z->name, + seq_printf(m, "%4d %16s %14dKB %14dKB %14dKB\n", nid, + z->name, (atomic_read(&zc->nr_pages) << (PAGE_SHIFT - 10)), + (atomic_read(&zc->nr_reapered) + << (PAGE_SHIFT - 10)), (atomic_read(&zc->nr_alloced) << (PAGE_SHIFT - 10))); } @@ -7257,6 +7302,7 @@ static int mem_cgroup_cache_show(struct seq_file *m, void *v) enum { OPT_KEY_NID, OPT_KEY_WATERMARK, + OPT_KEY_REAPER_TIME, OPT_KEY_HOLD_LIMIT, OPT_KEY_ERR, NR_PMC_KEY_OPTS = OPT_KEY_ERR @@ -7265,6 +7311,7 @@ enum { static const match_table_t fc_tokens = { { OPT_KEY_NID, "nid=%d" }, { OPT_KEY_WATERMARK, "watermark=%u" }, + { OPT_KEY_REAPER_TIME, "reaper_time=%u" }, { OPT_KEY_HOLD_LIMIT, "limit=%u" }, { OPT_KEY_ERR, NULL} }; @@ -7282,6 +7329,12 @@ __apply_status_for_mem_cgroup_cache(struct mem_cgroup_per_node_cache *p, STATUS_UNSET_DEFAULT_VALUE) p->allow_watermark = opts[OPT_KEY_WATERMARK]; break; + case OPT_KEY_REAPER_TIME: + if (opts[OPT_KEY_REAPER_TIME] != + STATUS_UNSET_DEFAULT_VALUE) + WRITE_ONCE(p->reaper_wait, + opts[OPT_KEY_REAPER_TIME]); + break; case OPT_KEY_HOLD_LIMIT: if (opts[OPT_KEY_HOLD_LIMIT] != STATUS_UNSET_DEFAULT_VALUE) @@ -7319,7 +7372,7 @@ mem_cgroup_apply_cache_status(struct mem_cgroup *memcg, } /** - * Support nid=x,watermark=bytes,limit=bytes args + * Support nid=x,watermark=bytes,limit=bytes,reaper=us args */ static int __mem_cgroup_cache_control_key(char *buf, struct mem_cgroup *memcg) @@ -7361,6 +7414,14 @@ static int __mem_cgroup_cache_control_key(char *buf, return -EINVAL; opts[OPT_KEY_WATERMARK] = v; break; + case OPT_KEY_REAPER_TIME: + if (match_uint(&args[0], &v)) + return -EINVAL; +#define MAX_REAPER_TIME ((10 * 1000 * 1000)) + if (v > MAX_REAPER_TIME) + return -EINVAL; + opts[OPT_KEY_REAPER_TIME] = v; + break; case OPT_KEY_HOLD_LIMIT: if (match_uint(&args[0], &v)) return -EINVAL; @@ -7402,7 +7463,9 @@ static const match_table_t ctrl_tokens = { * 1. nid=x, if input, will only change target NODE's cache status. Else, all. * 2. watermark=bytes, change cache hold behavior, only zone free pages above * high watermark+watermark, can hold. - * 3. limit=bytes, change max pages can cache. Max can change to 500MB + * 3. reaper_time=us, change reaper time, default is 10s. Set 0 can disable, + * max can change to 10s + * 4. limit=bytes, change max pages can cache. Max can change to 500MB * Enable and keys can both input, split by space, so can set args after enable, * if cache not enable, can't set keys. */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 54c4d00c2506..1fe02f4f3b33 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1310,6 +1310,7 @@ int mem_cgroup_release_cache(struct mem_cgroup_per_node_cache *nodep) } num += i; + atomic_add(i, &zc->nr_reapered); atomic_sub(i, &zc->nr_pages); }