From patchwork Wed Sep 13 07:38:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13382560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0380CA5500 for ; Wed, 13 Sep 2023 07:44:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 42DFB6B015C; Wed, 13 Sep 2023 03:44:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3DE916B015D; Wed, 13 Sep 2023 03:44:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A6006B015E; Wed, 13 Sep 2023 03:44:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 1A8D66B015C for ; Wed, 13 Sep 2023 03:44:31 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D932E140380 for ; Wed, 13 Sep 2023 07:44:30 +0000 (UTC) X-FDA: 81230786700.10.8B705E0 Received: from mail-oi1-f202.google.com (mail-oi1-f202.google.com [209.85.167.202]) by imf20.hostedemail.com (Postfix) with ESMTP id 2B3571C0032 for ; Wed, 13 Sep 2023 07:44:28 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="EOI/nyLR"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3DmcBZQoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com designates 209.85.167.202 as permitted sender) smtp.mailfrom=3DmcBZQoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694591069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3zT0hHZy8cis/iWPUQS68SvJSs9nRfkwgZ1RrbYh+5w=; b=x2LszrXNJV9+zEPMDIdT9tvgUjBMlSbLEhaHNvlPRm/iT6Jx2IMt/fnHmSD1Sk5uUXw9e9 4QY/cM1Hf473KdImLqVpXTw68JW/OnBzQFAJ1U9FYa/3wOuWpQg5zL4l7QYWy9Gu3tS/cr 2SG9GAqmGrKREo7tzbNai41PArbnZP4= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="EOI/nyLR"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3DmcBZQoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com designates 209.85.167.202 as permitted sender) smtp.mailfrom=3DmcBZQoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694591069; a=rsa-sha256; cv=none; b=A+zno57iACf2ZXMRbBN5gbg+DP09LwZik3gp4a8Hf6d/k3ptspLmy9h0e2l1WewY1cdBBL n6vft0Blrtyn/I5fYSTY0bF/I/EcnomKmnNC3ze+tnBkNqaj1AbH3spZc6hsXHC1/mxiEZ 3r/4AN81qDuG1/Sfk96aGTd41U4oukA= Received: by mail-oi1-f202.google.com with SMTP id 5614622812f47-3ab7edcfb4cso768616b6e.0 for ; Wed, 13 Sep 2023 00:44:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1694591068; x=1695195868; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=3zT0hHZy8cis/iWPUQS68SvJSs9nRfkwgZ1RrbYh+5w=; b=EOI/nyLRIKGgVx7KJ5doPpqCbBbLrTl6KLQq54rNd+oAAXcG1MBbRblzB7D0d0kl9o Y59GRxn24/HIA6xZjNwpIN6dTguWoHdK2dgLQP6La1ggfgK8hNaIFMePy99WRVF3r/Eu 7LkeMejdFJ5/L0GBJnJwfy+IvGyD/FlXj6hBCy3uPxTKrjQvOXEkel+ZGYS6Bluw2qI1 wvGRqDGSfghQX7/iHhNOf+O4ryzJupKi+bASM0us1ttJDykLWXB9UDrG4So/yq+8KFdN TIKVV1R8Vx3nBd+bdIqeMLCqkgWnQIu2NsCeSHU5PqYanFE+6F3IZyXajr4nQ6elqBP1 BTOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694591068; x=1695195868; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=3zT0hHZy8cis/iWPUQS68SvJSs9nRfkwgZ1RrbYh+5w=; b=cNPtSprBMyp+hQImlUgEbIc65z/HkKN2RGa0JC8QPzKu/d+kZKAr+2eg2iAvtJPpzz c7XdwR+4FBvzecrzSNYS2sipQy284Jvyu2xeiXRZzsypXbesiKIZXkbmj3UdYbcKyzLo mjxxRIE7tNK7gmY0VSx2eVTewfxvd8udXnJnoAUMoc7SuNIDlex73wHfTDrXgki3JmmF UtXxxICY/a69pCZTiLOeskvLyGxWBPuS57UY4nno7c5vYaMEx7JEGFAPnpUx8cZQIKWh bdHORSF3LwPs0Hs9TzxZsn/PO4x9kK6MlFJaB811bEdIjodM2nwT3kVfJTHkJQ3Tf7vA 3zkw== X-Gm-Message-State: AOJu0Yz1qN6zgLnDUqs3nFXpTdO8HlGDyqNeatyUVg31z2hkw+tZNEmJ z4zcJyhIJ04tvppKJj2tL6cEdWfmS1/5K75q X-Google-Smtp-Source: AGHT+IG3xAxVjeVugRkb9w85c4h3IHyzBTJ6cMwknrJlaNgvG0fGdTwmuzoswAYft+iKkPbWy/xXuohXtxPiB5qL X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a25:d2d5:0:b0:d78:28d0:15bc with SMTP id j204-20020a25d2d5000000b00d7828d015bcmr54189ybg.4.1694590734274; Wed, 13 Sep 2023 00:38:54 -0700 (PDT) Date: Wed, 13 Sep 2023 07:38:45 +0000 In-Reply-To: <20230913073846.1528938-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230913073846.1528938-1-yosryahmed@google.com> X-Mailer: git-send-email 2.42.0.283.g2d96d420d3-goog Message-ID: <20230913073846.1528938-3-yosryahmed@google.com> Subject: [PATCH 2/3] mm: memcg: rename stats_flush_threshold to stats_updates_order From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Waiman Long , kernel-team@cloudflare.com, Wei Xu , Greg Thelen , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 2B3571C0032 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: ndqfgh5yer5nny85t8a7kabk8gr8wxrz X-HE-Tag: 1694591068-588870 X-HE-Meta: U2FsdGVkX18qsHnxOKDEEf7urj9IRlPwkiCNoMzEizWH98eEXbsGU4HWiRIdgIgqpSlnzfQu7NQKIfRwMQkHPi5TXUZmOjPAmmFjjvF7YmJryYvoSvXK2K4oI5m8nbtgUUcLWBMsLAnxxqeIt+C72DuPAO4Jx7a+JIydyXJ7erjw/TRXkg4/5DzZ+bYQ+ia5UVzTbg8HzfU2c65P/zQAVe0axsJzx+fnBruUSSawukV56VcI7agvjrlcuFJuD/nDy01v+JoKihSeBabjmY/YseWIk4J6FhL+cm4SZsM8VtyA+HGTN36o7DGx8mzris5/1PdbgCbj72U+wOEVinJN3V0K5y1ZjslwWWqncgYewaLdXRb8w81AeamzpwBwKx0Cqdfpy4wct7iRTJIoLWxJlXjLE58lcBOCrwzbnebMu5x1hbSn82ZnYqAek4j7kCL/BzK2YGjtW+AVixOJ7WggmJCCvdk0NjpZE93ZFvLZvWqYVZDBPGKnrBjOxgqNYc+4txNcoJIca7bqwmHJJOISEMIHbRXHwWYQc7pA2SDzZbkgE1vv40UQAbAQvBqEQMylZ55X4aYmo6/06ylYi4lgfcn4Uy4KPj9zbz7R8V4rIUjkAthqTxHxgQSyruJdfRgicI7xhUIgKfWHQI5NDpA//GvkoQQTKXnmao7KLt/ASKgzktwzplokJod6P+kQVgmr82lnGRhGjFsYRdR9CYjg5ihbrcGnr/sJKl7rEAfabHXGzi9psltB+AMQ83L8wJuN6hvZj5p592y6V2SkdLCW/pXOKv1mq5zhBNofJZy0NW0dn4KzLit0AVJcXwiW/z1mtyroH4UX50sjdOlm5mvcYKKC7giQGIdcRKKdjIpxkz7CFZE17sC7BWKdcrLgfli1oLu+9WylcO1PBlATf49izj7O6LuoyxPIReSnuojAE5QIRl8LuH9goTDILpczgMwdHKa17YWPlHNVaVUeYzy 6SINFiSy eHJyucMyEME/dVNImR5M1h6vDutVF8hfjb/vstyLPkc0jDHuHJkNsXkFLNmg89TBYYsNjTS9zCKNQ+yEla9TeCV6vNerylG2oa9c6ykrxPBwUBWFmLW5Gh68LbFRIq0LYFn8QGW3ZqB5gMqEWKP/GyOc3C+Fj2akqXhWJurORiKGAWbls9QNK3SOgetq2R+V7XiMM4KE+ackoQVoaUFHVnXgsQPENeUsp7zY1sVhZryrrAuS5wTbB97fR4FlOajcv5scKT5mYFA4xdTp5R7bSMbPBs9Q19VZ9AfN0jxQpsv9FhKgler4mUNVnBNyeFFRMXHFu9kC1ZQQvW0FSt8tOUrQ5EY7xonqdyzhoiPUpgd/JI9JNxfVpZdQeedpJ91OlHQwDAXEe5vrlmtLOxI1tAYrz5S9WiMpciR/0QVO1rZfPDqgIDL7h2sYVjjn8IYENhM34Y55FzGV5Y5USsE1hK9ChuY0RQWl1EUT4IroiZ2ShaIlnG+V5G6Yb5TAjHCcQoOYcaMdYQueA416EBSmh+xUZcWelLthfNPEJK+4cO9Byutq7GTJtZzR2ABs7guLvxUHrQbKz4+uC4AtnacvxzNNqsUIc0DyVu6qkFexbTOJjdqqIIKLtS8x8Usma6OaGdLWmO4D/RcMygud4wlnuNqVPH3mNCSCO523u4JEikzbUYmPlS0ClHZUUPw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: stats_flush_threshold is a misnomer. It is not actually a threshold, but rather a number that represents the amount of updates that we have. It is counted in multiples of MEMCG_CHARGE_BATCH. When this value reaches num_online_cpus(), we flush the stats. Hence, num_online_cpus() is the actual threshold, and stats_flush_threshold is actually an order of the stats updates magnitude. Rename stats_flush_threshold to stats_updates_order, and define a STATS_FLUSH_THRESHOLD constant that resolves to num_online_cpus(). No functional change intended. Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 35a9c013d755..d729870505f1 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -589,10 +589,12 @@ static void flush_memcg_stats_dwork(struct work_struct *w); static DECLARE_DEFERRABLE_WORK(stats_flush_dwork, flush_memcg_stats_dwork); static DEFINE_PER_CPU(unsigned int, stats_updates); static atomic_t stats_flush_ongoing = ATOMIC_INIT(0); -static atomic_t stats_flush_threshold = ATOMIC_INIT(0); +/* stats_updates_order is in multiples of MEMCG_CHARGE_BATCH */ +static atomic_t stats_updates_order = ATOMIC_INIT(0); static u64 flush_last_time; #define FLUSH_TIME (2UL*HZ) +#define STATS_FLUSH_THRESHOLD num_online_cpus() /* * Accessors to ensure that preemption is disabled on PREEMPT_RT because it can @@ -628,13 +630,11 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) x = __this_cpu_add_return(stats_updates, abs(val)); if (x > MEMCG_CHARGE_BATCH) { /* - * If stats_flush_threshold exceeds the threshold - * (>num_online_cpus()), cgroup stats update will be triggered - * in __mem_cgroup_flush_stats(). Increasing this var further - * is redundant and simply adds overhead in atomic update. + * Incrementing stats_updates_order beyond the threshold is + * redundant. Avoid the overhead of the atomic update. */ - if (atomic_read(&stats_flush_threshold) <= num_online_cpus()) - atomic_add(x / MEMCG_CHARGE_BATCH, &stats_flush_threshold); + if (atomic_read(&stats_updates_order) <= STATS_FLUSH_THRESHOLD) + atomic_add(x / MEMCG_CHARGE_BATCH, &stats_updates_order); __this_cpu_write(stats_updates, 0); } } @@ -654,13 +654,13 @@ static void do_flush_stats(void) cgroup_rstat_flush(root_mem_cgroup->css.cgroup); - atomic_set(&stats_flush_threshold, 0); + atomic_set(&stats_updates_order, 0); atomic_set(&stats_flush_ongoing, 0); } void mem_cgroup_flush_stats(void) { - if (atomic_read(&stats_flush_threshold) > num_online_cpus()) + if (atomic_read(&stats_updates_order) > STATS_FLUSH_THRESHOLD) do_flush_stats(); } @@ -674,8 +674,8 @@ void mem_cgroup_flush_stats_ratelimited(void) static void flush_memcg_stats_dwork(struct work_struct *w) { /* - * Always flush here so that flushing in latency-sensitive paths is - * as cheap as possible. + * Deliberately ignore stats_updates_order here so that flushing in + * latency-sensitive paths is as cheap as possible. */ do_flush_stats(); queue_delayed_work(system_unbound_wq, &stats_flush_dwork, FLUSH_TIME);