From patchwork Fri Jun 7 13:48:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesper Dangaard Brouer X-Patchwork-Id: 13689996 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A1E8C27C53 for ; Fri, 7 Jun 2024 13:48:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDF206B0095; Fri, 7 Jun 2024 09:48:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D8EB66B009B; Fri, 7 Jun 2024 09:48:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7DD36B009E; Fri, 7 Jun 2024 09:48:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AC77E6B0095 for ; Fri, 7 Jun 2024 09:48:19 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1E51D1C2CFA for ; Fri, 7 Jun 2024 13:48:19 +0000 (UTC) X-FDA: 82204221918.21.C071EEB Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf18.hostedemail.com (Postfix) with ESMTP id CF2181C001C for ; Fri, 7 Jun 2024 13:48:15 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jxUbegs5; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717768096; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=0Py7yS1GdONXUvy0PHE4b0fU1vvN0R7GKzjpi1aB/3M=; b=Axs9Vsx7a4CpPCdusEcUFsZ+YedwD7yP/XlcF+TSsVawVTMLStcWyRvFkHaLWrHAPatun7 bbbGtINXiSLGzyWOYc4dLJwcXCvPLGU03rBfsczzVhCTp5JwvfgO7TnAEEUF4hAbpUdFDd oZnTK4mHYqSV+RMbcuAXWRMonyavFdU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717768096; a=rsa-sha256; cv=none; b=TF4/pHG/ulguTvtPEzasaCoUVz980DMwmA8z7UwrLSK3lnMTHJnu+WU+52pP06VjVxRR+2 l5zRbwO6CNePZXsQ9yCjPZ+vp34Meh/q6FrxfwWtol/ArQtFirr5M+3zWRpXlH/KNUuaXn 5k20hic0crV/JF75MrP48hF2tU4Qs6Q= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=jxUbegs5; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf18.hostedemail.com: domain of hawk@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=hawk@kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id CDF38CE1D8C; Fri, 7 Jun 2024 13:48:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A4B10C32782; Fri, 7 Jun 2024 13:48:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717768091; bh=TQyKi7DdPGzpHrY7zAT+50KqIFo9l+0hqUZDJ1H9PuA=; h=Subject:From:To:Cc:Date:From; b=jxUbegs50WXk2Yd4F4sXnmPaVFzxACNse6F2pABiZWF8OZDrk1Sx0xEbZR9w2VuWi bf/sJ8HBjGMEqnyj+7U4yHbljoQVJZGcfmBsnlvmiD7NxHCv9RrUiobIA5WsUec0Gg YQCY5UCpAwyo6gS5JXEr2QiIr8aDnst6qXkj7bNVP+HYWV2MVzNDJs2u8OXNCcwzXE kxC7eFcF3DR38BlqtcZS7hqsvP8MJnaurmOfg92rgO/UCG5yF9C9UjPh7BKXR6HAx0 K3C5ISy9fHOpK1I7Rg2ueH5yACJ44rWz8RntfKEA9y8apUhZivpvsRNMOoSuOwXnk2 KuxTGt73+csaQ== Subject: [PATCH 6.6.y] mm: ratelimit stat flush from workingset shrinker From: Jesper Dangaard Brouer To: stable@vger.kernel.org, yosryahmed@google.com, shakeel.butt@linux.dev Cc: Jesper Dangaard Brouer , tj@kernel.org, hannes@cmpxchg.org, lizefan.x@bytedance.com, cgroups@vger.kernel.org, longman@redhat.com, linux-mm@kvack.org, kernel-team@cloudflare.com Date: Fri, 07 Jun 2024 15:48:06 +0200 Message-ID: <171776806121.384105.7980809581420394573.stgit@firesoul> User-Agent: StGit/1.5 MIME-Version: 1.0 X-Stat-Signature: 6raqmyqrtuuegzu71415kmbnjib8usze X-Rspamd-Queue-Id: CF2181C001C X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1717768095-243245 X-HE-Meta: U2FsdGVkX1+yqS4JsxWtjCLZRhKLV0IhrKVyvgr5H+flawTbQrmfe0Ev3hAQ25AsWjPqT4S5CI+Z5g0E6koueL6DNQjVYc91ovqlUlF1x/ss8J6I6NkA4YasmfsksKnP7sn286F1ZwYqHhD5rcx38Tmang5EViSphtR7OeK1es/n99ik97CSQsHUH5lvVb6dwx4pVAmUxTGtWDI7s0cUFSycoJOkOlUKFSH2tmM8z2AtqonvLtnqPeKFC7uyOcKbAkKPQr8L7ZJJ3ZnAJwc433D4sJ0Gx5pf2nOyq5OfSu+8+7NUYbvglTWHoE9qgDR5YYEWZUcSsEnERwBIT8EPAz1BkyzF2YfvVG8PJ6CXrGL92XKyC/JoAQb5Wyb5Guihl9jWukWfT3QiZLvpSHBHPiHpAAfZ9Si/EbPDp1+t5R4zgFdj8Vnw/AAPsqZZwajvgBjC7DQFXMyyrJrRygfUxABS9C7LMZpW/9fMZdbYclowcKmJ+7T09JADOSOFijCThuYapaEjyRzoO3a6xVQO8IdCbyMP/EciUgKzO3FATHBDB5tSHnVULev477soUbIqGAUPngpEkrGBf8hPOw3H66THvgPlKagtPEhUoSwKmw0LOdHFrTuwPOJiYdNVFeTmfv4M/2piY2lP/c7qtlIJn92RxQ7MY4Ly1JzKHAyuKS6H27v4SVPwpYQH76WRxi8NT7rWByvyfqy1v230mXXt+eEpwmX6qUzenMfQ5x0DOSMwxy9HNtovQNbBEYI8rvIL0dQZpjK1ryfsbSiy81X6WO4JoAqkfQikcZoXS9fxj1Bb8h4TnuipWTR6H0GR4cm/ntFuicRCpeDM14T4JIdMKTq0Xlt8KgIifhJOsH9djF/vptzkh/W7QjY8fdVlklBlfMbPRKHDlSt6REpIxxAeptVAmV3hOKuflSOJ7jdNERsgUuhAI55hT60sLOm1NSuD0IDtSiwXXyQbXZ7CbOX +0BUFWMI YqwymgHqBTTmi/rPggJoFCGOeamNS2Z/2V7vetTN0VcMrm9mBqbpJF45rYgAB1eRFN8FZmL/xmhM8IpLNe6kKubjJFZDEY3d4dJB3NOZpkfxe0Hu7D1YZMbYgnqD1XcRaCFiMdDguQV6kb0qdrp9vXvHCHZRjluy2f12LlraV7DDX1C5i3CSHT/b02X1osYIejRu1QmV4J2a6SRLNmSxQ5B+bAzmiKIv2ciBEpIALJaDuqxabG9lffSURABgfJGdLOHPm8c0amA8gx7dyM1Xo5g+gbO0gw8YThNT1tJXraOqY4kEIZzlYoFRENsUv4vyC2tLhY7MgK9Okz7pKVhANY9QIW8SMpH0ywOJj8GyNZieOu/BP85f0Gavp4lhMpR9rQrgLnDnt8AQj3dFX2eeGdPxx77fwpTkXjX+MjWA1OUv4lk0LDx/Z28uidRWyrGN5RHiQwJxscnv1bMTcJgIMxJCxf/Ef9LH58abgTPyWZwZ/fK3jph0whiZaGJcQ60x1HgC5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Shakeel Butt commit d4a5b369ad6d8aae552752ff438dddde653a72ec upstream. One of our workloads (Postgres 14 + sysbench OLTP) regressed on newer upstream kernel and on further investigation, it seems like the cause is the always synchronous rstat flush in the count_shadow_nodes() added by the commit f82e6bf9bb9b ("mm: memcg: use rstat for non-hierarchical stats"). On further inspection it seems like we don't really need accurate stats in this function as it was already approximating the amount of appropriate shadow entries to keep for maintaining the refault information. Since there is already 2 sec periodic rstat flush, we don't need exact stats here. Let's ratelimit the rstat flush in this code path. Link: https://lkml.kernel.org/r/20231228073055.4046430-1-shakeelb@google.com Fixes: f82e6bf9bb9b ("mm: memcg: use rstat for non-hierarchical stats") Signed-off-by: Shakeel Butt Cc: Johannes Weiner Cc: Yosry Ahmed Cc: Yu Zhao Cc: Michal Hocko Cc: Roman Gushchin Cc: Muchun Song Signed-off-by: Andrew Morton Signed-off-by: Jesper Dangaard Brouer --- On production with kernel v6.6 we are observing issues with excessive cgroup rstat flushing due to the extra call to mem_cgroup_flush_stats() in count_shadow_nodes() introduced in commit f82e6bf9bb9b ("mm: memcg: use rstat for non-hierarchical stats") that commit is part of v6.6. We request backport of commit d4a5b369ad6d ("mm: ratelimit stat flush from workingset shrinker") as it have a fixes tag for this commit. IMHO it is worth explaining call path that makes count_shadow_nodes() cause excessive cgroup rstat flushing calls. Function shrink_node() calls mem_cgroup_flush_stats() on its own first, and then invokes shrink_node_memcgs(). Function shrink_node_memcgs() iterates over cgroups via mem_cgroup_iter() for each calling shrink_slab(). The shrink_slab() calls do_shrink_slab() that via shrinker->count_objects() invoke count_shadow_nodes(), and count_shadow_nodes() does a mem_cgroup_flush_stats() call, that seems unnecessary. Backport differs slightly due to v6.6.32 doesn't contain commit 7d7ef0a4686a ("mm: memcg: restore subtree stats flushing") from v6.8. --- mm/workingset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index 2559a1f2fc1c..9110957bec5b 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -664,7 +664,7 @@ static unsigned long count_shadow_nodes(struct shrinker *shrinker, struct lruvec *lruvec; int i; - mem_cgroup_flush_stats(); + mem_cgroup_flush_stats_ratelimited(); lruvec = mem_cgroup_lruvec(sc->memcg, NODE_DATA(sc->nid)); for (pages = 0, i = 0; i < NR_LRU_LISTS; i++) pages += lruvec_page_state_local(lruvec,