From patchwork Tue Mar 28 06:16:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93072C76195 for ; Tue, 28 Mar 2023 06:16:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F69C6B0072; Tue, 28 Mar 2023 02:16:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A67A900003; Tue, 28 Mar 2023 02:16:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06EE66B0075; Tue, 28 Mar 2023 02:16:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id EE0FD6B0072 for ; Tue, 28 Mar 2023 02:16:46 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BB0A2120121 for ; Tue, 28 Mar 2023 06:16:46 +0000 (UTC) X-FDA: 80617298412.12.0113D44 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf03.hostedemail.com (Postfix) with ESMTP id 051FC2000F for ; Tue, 28 Mar 2023 06:16:44 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=IrEj06HW; spf=pass (imf03.hostedemail.com: domain of 3S4YiZAoKCAo8y218krwonqyyqvo.mywvsx47-wwu5kmu.y1q@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3S4YiZAoKCAo8y218krwonqyyqvo.mywvsx47-wwu5kmu.y1q@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984205; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Cs9g98LfB/5B5fgCHg0khEoDDLXk/FbX9zQPYHFGol8=; b=a2oIiNa74DJe0wyo+OhQioXAgcWPgxRtQbSaJBFPuuOcEBKt4W0esnnNaJyJp84ZnMAfXh wgzy+xstW2MuRIfLqQ64eseDsEkThXeRBZUF/xggAf/pAZ7z/eORa51ZBYj2QUb+UFrk0c OvtI0lzEkMdJpZhcjspxZ4e4VbOMoU4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=IrEj06HW; spf=pass (imf03.hostedemail.com: domain of 3S4YiZAoKCAo8y218krwonqyyqvo.mywvsx47-wwu5kmu.y1q@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3S4YiZAoKCAo8y218krwonqyyqvo.mywvsx47-wwu5kmu.y1q@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984205; a=rsa-sha256; cv=none; b=HrjPjc/ihqzAMXSQIhYfnBgmJCUHxFpgZZJ16Ob4KffvazDTX+LioNE2jyFnox5lDTHwH2 muSBu2h707zm8Odr26M5RZ0sFig4svbOgb4DZBEMQiKEn+U19KWHUTF6oAFBRZ7qcJM9IR POA5eIZkwPbsgDQPnHldtdrczdj20JU= Received: by mail-pl1-f202.google.com with SMTP id s11-20020a170902a50b00b001a1f8fc0d2cso7139079plq.15 for ; Mon, 27 Mar 2023 23:16:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984204; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Cs9g98LfB/5B5fgCHg0khEoDDLXk/FbX9zQPYHFGol8=; b=IrEj06HWrJaiEe47kle+4mjwycO8VkK4UYyVi89hOr/BT4ba3YoJaIkMxMmDu91Azt OCiIdpmqw/gfkGEUiinzv01MOm68VDd/zLRgX2gIU8f7UybpdpXtpixpCNAld7gzowZR +gyt1jsqLawCKRD+ojRnZ6c35iauByhdICeE6tA9/AW2kV9h46wsE+qITfJzQSmauG1W ySDyn3mwIkhUXYNihBaZeA4rjBJGjlP0HSdTCoy9A7k1vdMTk4m51J+idABUhgX1bKO7 He48ZTeiLxeRfpruo08A01IDO1BpVxjEsw+pmwGtbL4byIuTDU8yveD/qiON77jThzUh Vo3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984204; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Cs9g98LfB/5B5fgCHg0khEoDDLXk/FbX9zQPYHFGol8=; b=pk/H6wwt95xjwnVIVuQnEdwLr6n3TjmYG+t0Sif0GVVnVMEB/Dl7W1HoYPR0f6spb0 7rT85qoPNVpjv9Wc1cpvb7KpGtLEEf6vPr2UJ7RyQbDnc0zhAVMZn8PwCM4rZKmTLISF kSCEnXvdG5ChAratrjvJKH0Su2+pfsFqc0FfLNK+75HNrEnjXtNz7yaookESUzc3xpSm Am5Dm88OxKsCHqMRzZFAjv8OCBIP/64yQWz39l0huxzNmS/85+4ZYasTwnX5xJi/GGlY MO4U0CTSYEZsVihFR50YotQXCKKsHRCRvS1TdOMKf7M7XqnPSxlzKF8fSA/aT6ivgeGP hsXw== X-Gm-Message-State: AAQBX9fdPVlPqp0BWUAefyL20dztJAIhfwXXAPXOI3EEO4tKQsg4Auha 8JzTF85ZMaU/pDFFvowMGlLEFvMN83UEZN62 X-Google-Smtp-Source: AKy350aYOPbMPoVcZ2xVfirHkGMy5US/bJka4xE6zLbD6emP+rv0j2dGtP2N8z3/3tcuS7Hx271FErAXLO/AbXkX X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:24c4:b0:629:f9a2:64e6 with SMTP id d4-20020a056a0024c400b00629f9a264e6mr7565323pfv.4.1679984203700; Mon, 27 Mar 2023 23:16:43 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:30 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-2-yosryahmed@google.com> Subject: [PATCH v1 1/9] cgroup: rename cgroup_rstat_flush_"irqsafe" to "atomic" From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: heij338xccugibzofoyuu3c76zams8d6 X-Rspamd-Queue-Id: 051FC2000F X-HE-Tag: 1679984204-78330 X-HE-Meta: U2FsdGVkX18CWF3qGGLzCblqD/W/giclpWHv6RdhTRQc5Q8WNOnrHqcNcC0ypSMfTPHgd6Hc2x5YZH3MCetEpwGYm/WgmNgwsYRyHS3kWhi7nM4KP1PGcBCjStW4mr+S5okdK4J+dJ1cYDneXuW9m11HZ5to8rQR28ntYJoPXVtXJOTBHK0LDWhkkSzmD9sBXWcTauUd2zfLztoHKM3ZEzzUICkBlg4vowQrXA3dAIUv+PGxHvnNHL6TGFUylMTUxsJ2x/h08aSR3Dy8S1evJayGNollYIR5e6MlFV6H2gtrboSdR+lofxrtzJx22wB64nQxUZ49xeZw6pFURX76HwebR/Wow/jk1HwLQ0VYKFYitQ4UIxH7LjNNb9Oez3FhBkBTNqX5V1KX+26jvruzcawSQSp5usil8f1a07rxRBryTWbvfHPqEqOmMSyDH0f5DDyTCBrszA67EcWlgrYJ8mtws4M/V0Ez3s5yj7pzc+qYNZSNbehnZ59owtRdZUT/rizL2VBrXaduhrj0p2hemSPe7cTu0U29LTlRk7K5+vyAvLQgV9oJKWUG82aIY8AU1TBpC7X4e1NMoxH8iJa4y2t2ibbucmGtOmlDPy1i7cIqKF8Upig/JX6prs2zONI3PFRqYgV2uih/cxvB/n0gjGwp72+oxkUgIKsvzpid2WxL6+enyrn9NyKsgIpFPwgehbC1KE0X3/y0MB7QTnGp0yNbQ+ZO7XHwE2+YsLyC8t2Wp79wnFwUYutN/iQGEFMibA3wXonJy+cnNu9Clu7WL4wSJoV6h8vIT34nA9C0MAwNUCqjfWRmfVU0e/abLHan1n4mFNI6az98EQe/Zwzje3p4xGeW+ks4BdP4dzonLZWr9yEhy/r8Vxxdb7eTFeWl/wCrbmbAGEvtsM9TrixbKPHKvYiqVSyr/BcIcXdtTCmXZRxP61j5PtJx0nhwsPHuKUKcReWolMiUbv3MCOI yg6493Xg DudVEG0CM0cTKhbyer7BB8hw2WyyghadGY9ezxXvFErTQRfVANeqCW5DBJBjprrXgnqF3K6EyYlu5mnaEJYeynbg4oVQCD+8HbQOPCpEYF/97Z4TOVuVGIDD6Ee56pgOIhx+gKFahZdXUut4yU5frcl1DxxYdGj2BSxVq/pDSorbt3roJgw7jcUMsI9W/2Q0NETzASSR9p76Nu3mRkxjxVUwC9Jx650hZNzdwnOUHr+3kIoBTClj84J5Lpzbn/QZDSSjZtBq0zbhSTHcd2lglOS7S5TblFtfD7RwRnAKp09+35GPlOyN9pPGWZ/XSFHNcwEQjYS8sBzIqvePVPye61T7FFEjXWlXnc0nxaf8zqv6vQlm3JCZJ2oHFSJG1JEz6q3h+CoSR7hlFG8eiKKJWxipYjwj7YGY91on6g2Qui3I3OHbHbjahLICsDoJb/BehhKzFdExC4hR2MiPBIeGcIuls1a2wu8NIY/kLw7mSPRqTMH7d4WdXHE3CVOV7ky79deLmvnB2bMV1aE/D97Mm1NEDgjInIei3T4g6/FDJESk687eA3CNGHTY6wf9Ht93TmKRSRED6tv9VymPBIEd85osYn72kE2t6zEiboAub9qgx2FHZpNw/sDoFpt7m8BZyUHgwBt0l62DKC36I6BzwDLSNF4OOKcbF+quueGYXJ41XIPo81liesIYkTME2H1u9MHTANgMt9JbIuT2dhJurle+45bRzUBeMsEMkyI0P1STsK5az7ey7+EUSlTDToyBsEgW5RkPvVAigm70= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: cgroup_rstat_flush_irqsafe() can be a confusing name. It may read as "irqs are disabled throughout", which is what the current implementation does (currently under discussion [1]), but is not the intention. The intention is that this function is safe to call from atomic contexts. Name it as such. Suggested-by: Johannes Weiner Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Johannes Weiner --- include/linux/cgroup.h | 2 +- kernel/cgroup/rstat.c | 4 ++-- mm/memcontrol.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/cgroup.h b/include/linux/cgroup.h index 3410aecffdb4..885f5395fcd0 100644 --- a/include/linux/cgroup.h +++ b/include/linux/cgroup.h @@ -692,7 +692,7 @@ static inline void cgroup_path_from_kernfs_id(u64 id, char *buf, size_t buflen) */ void cgroup_rstat_updated(struct cgroup *cgrp, int cpu); void cgroup_rstat_flush(struct cgroup *cgrp); -void cgroup_rstat_flush_irqsafe(struct cgroup *cgrp); +void cgroup_rstat_flush_atomic(struct cgroup *cgrp); void cgroup_rstat_flush_hold(struct cgroup *cgrp); void cgroup_rstat_flush_release(void); diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index 831f1f472bb8..d3252b0416b6 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -241,12 +241,12 @@ __bpf_kfunc void cgroup_rstat_flush(struct cgroup *cgrp) } /** - * cgroup_rstat_flush_irqsafe - irqsafe version of cgroup_rstat_flush() + * cgroup_rstat_flush_atomic- atomic version of cgroup_rstat_flush() * @cgrp: target cgroup * * This function can be called from any context. */ -void cgroup_rstat_flush_irqsafe(struct cgroup *cgrp) +void cgroup_rstat_flush_atomic(struct cgroup *cgrp) { unsigned long flags; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 5abffe6f8389..0205e58ea430 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -642,7 +642,7 @@ static void __mem_cgroup_flush_stats(void) return; flush_next_time = jiffies_64 + 2*FLUSH_TIME; - cgroup_rstat_flush_irqsafe(root_mem_cgroup->css.cgroup); + cgroup_rstat_flush_atomic(root_mem_cgroup->css.cgroup); atomic_set(&stats_flush_threshold, 0); spin_unlock_irqrestore(&stats_flush_lock, flag); } From patchwork Tue Mar 28 06:16:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C49DCC76196 for ; Tue, 28 Mar 2023 06:16:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06F8B900004; Tue, 28 Mar 2023 02:16:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F3CE9900003; Tue, 28 Mar 2023 02:16:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D197E900004; Tue, 28 Mar 2023 02:16:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C10DB900003 for ; Tue, 28 Mar 2023 02:16:48 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 953FA160832 for ; Tue, 28 Mar 2023 06:16:48 +0000 (UTC) X-FDA: 80617298496.25.652662F Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf03.hostedemail.com (Postfix) with ESMTP id C90252000F for ; Tue, 28 Mar 2023 06:16:46 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ML25Z5sJ; spf=pass (imf03.hostedemail.com: domain of 3TYYiZAoKCAwA043Amtyqps00sxq.o0yxuz69-yyw7mow.03s@flex--yosryahmed.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3TYYiZAoKCAwA043Amtyqps00sxq.o0yxuz69-yyw7mow.03s@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984206; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wKfUWPDFjTfCFAmAu+PDSXoVaLfaEZa4F2xpA6M98WI=; b=4AMlDaWGx+Yk1w2hqLkjWNBj8p04pQ0pnhyBkru5ic6BIdaRcnnK202kc75a+5H3/cpe4F mh+FdBEjkQYZv0xo509D8LbJuJ5xPyLU7w+bTXjIjT0W9g5R+/nk7qjaIZkNhdnDy0/wdx mP3/FXN71le5emkUJq42rGCMRPUc/xk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ML25Z5sJ; spf=pass (imf03.hostedemail.com: domain of 3TYYiZAoKCAwA043Amtyqps00sxq.o0yxuz69-yyw7mow.03s@flex--yosryahmed.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3TYYiZAoKCAwA043Amtyqps00sxq.o0yxuz69-yyw7mow.03s@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984206; a=rsa-sha256; cv=none; b=ZiJ6Mrn1X0B1ZUOur70aWXjidUdOkiDLhP19IleK8RdxsCoFvCHGr+HLo502fFTEDcfOQr 6yGgFJ7kCRGuPnJqNYZNYL5oJadjzqU1sj847YrkdKrJ/FWuPWfy6wInMK/AM01BoGjRLZ VeJtw2dwloZm9dx+gLpdN3cKH/hMKHg= Received: by mail-pj1-f73.google.com with SMTP id q99-20020a17090a1b6c00b0023f0c6c6b3dso2971533pjq.1 for ; Mon, 27 Mar 2023 23:16:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984205; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=wKfUWPDFjTfCFAmAu+PDSXoVaLfaEZa4F2xpA6M98WI=; b=ML25Z5sJR8k5t8kSQzNLjGMjEKmdcebxHFtYwKZcrTLZX2wK0gMrUstJ28IAfa/l/B dmJfAqHqeUNGDMqpjHMcUn/3rl6ZtTwoLtJc4kM6PKx7oTSnQD1aK8fu6wcMuvNpddNP JXz/aPGcRNjn7CdP1wNYEAXnpd0Uj2w+rzBilII9bcn8vsNoKap2/emH97XB9yYmQX0j zWlTQgN/LRjTLn7LSTnUMs5Zi8YiSMdP4rhUt8H9KINdPVTgc9z/VX9FnrviFmsMUsqK yY+aBb6I7/+gAXbDQAsJSzcOoGzk7Nj4yOSGJy0Va4z4/Vp/gmfRCs4LrcZySZjB+aj0 I08g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984205; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wKfUWPDFjTfCFAmAu+PDSXoVaLfaEZa4F2xpA6M98WI=; b=rTMRUAKGKTEVgA7yFF937Hulw7alhS1wpeY6XfyaIN4Syfc4md+lYhG91QLlCTRJS7 /J0nLkWV0o+MSkrXcKEf3uf+vfz84bAIHRPPnBCQBeNZKA5juohBr5NPJsa0KomLkGyk sPkOEZReJeY98lOXnXKT+InuETRRQu0ZNZz1u4xFYIH7BFiW5c3Q8scjWEONTYZv7ggX AXU8rWmMurpa3kJpYpFBnO+OtnML10fwyLlFQczEkNxbn5M3RYOZX8IUzkin9xNOlRGf 7xjSk247ET0k2aYtMZohyeCHH+ZaeoW0WfyUnF/ExJBo+6zOHluee1K5LzzA3JlzSY9t Q45Q== X-Gm-Message-State: AAQBX9cDJgqgQDhBmapNowuAHJnAS/G0LI8Z16qaeJ/QczJt9hI6VfLB ERhjHd22vcX40XE0001ER52etRnfnLIk9LZK X-Google-Smtp-Source: AKy350Z/+j+6P1wMetl7E14Y1prBzO4hFyvwBGRqSlF4uLs1fV1I04aur9Q5rKjp70R1U9RxRmWGSymD4DlPFw5z X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:7088:b0:19a:5953:e85c with SMTP id z8-20020a170902708800b0019a5953e85cmr5289230plk.1.1679984205498; Mon, 27 Mar 2023 23:16:45 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:31 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-3-yosryahmed@google.com> Subject: [PATCH v1 2/9] memcg: rename mem_cgroup_flush_stats_"delayed" to "ratelimited" From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C90252000F X-Rspam-User: X-Stat-Signature: tboidrn6zwkqctbnzji9jp5ebdhe5tne X-HE-Tag: 1679984206-764095 X-HE-Meta: U2FsdGVkX18qL7xKUuLx6BESReB7ZnAOfmKMiDHhs2Rm+oASY262rZTb0J81qjq27dBk+A1ZKE4+RDP1CSHLjZM5JtqzX3k4n/MQAZI9+AXkAYABNWMRvwPgXzf74MTIjZdtgeeXMPezBqX5MG14pvqqsmv5JQm92UpTrzK38ijidLPEm3i1MTS6QJb5Aeb7W++qq9df4qpT5B1GRBFJYovKJGXG9/JGu+mFxTxtlaqk/MT5hRT8lnolRBFpLEVda6+K8eJu4OkDh1vMXef+x3zxfaw/xVK+jNaF7LR5Fj/hkjQ/RpoqHxZV2TU0hOJY3KB+jtgOZn3QSpPp681SVTQdKTYrpF0iqJbqltGh8bLYmhlALwx2JhRZPujQMY8ahVVDWsekjjHj4FVHzL9KB6gAgqTRzXytbYafPutCafcG0Rss5FrLuFkuKiPJ5w9d7g5/KoktKLyTTi5s9lCXA7xvDVmf5zHaistAWnnXjHsf64d3fJPNeGNxLsa1+IRtxe5v+/VveTCHkYqRol8rX/TJSGZEvdmQUDvd8s8/6C2Ia27g0WpnTVO7supO6z9DV8qprVLFcTQ/v9VPgjg0ET/wSM1uP+rUMmxStFC0ho6SRu6br+2mMWVDu9Pv6pNN3HvU2bpBVnG4fUxDgquMLSixeagk2FhKk284xpHzhabOb/0/bhxc9kanO+LUOmkLDQXsdBP/ETGS2cQMTYvGVNfOKisETXj3ZwHj19wBog88vAtkrGdXLmgHTJUcmsZzLhC5tdQdublMRQ98WocnMlTdBLMRM+U+AlgAuaMGtJ/JuZlFxKJ+fqoZVw5iE9Cbuejxb6izK0T1vym0cBCzSMzaUZbNmeL6OZr6cQVp6ZmYubLlHS7/yg9HLzqKiwIVBkrBgFBZGdvX070jeF1KqEP5ao0iklvrYaJcbHuS546AaQdVuqwBUjFuLOidj5Gg+dvfgdh7iu5+56kH+Lg fHuYbr0Q OTox7kd7RB1ly0kvVq6PCJvgQpcUCbi0vN783sdfQjthgJ4m1cCXKzyO67aXPQv+iwEX3PsIkU9ugKIQ8F5LJmBh4pn7Aubtxan3XFru1EDEmk+tJv5NlCCTM2wn4m8ltvsdmNILUdk1CZfmjnBO4M5Ipt87TFjlq/6g6DUim/0qxl33F+1r1VJ2ElolwQM/qwI6uEqwJ6VncCrIqAsWb1BZSMn1dE4PxOEEpg5ovAOcVe5k4Pd2mbJzof4oHJAi6wP00lywcxuoVLsbFUZWcr4OVmE25nQ9HyqwfCnrMf+U7sWGywqkzhDJLkkRjVlsItVavSXDMIUaGwez3Ud1I8HnmraKz0FaWOjXTqrH34H7F39ttsIGffDRO8lAKYJe/wiaVf45cubSOfxTnQHbI32TfyPfmvWLB1D02apheKQlA2kbiHUZiO5zMCe8TctlKyA2k86hGce7fnjGCF3OE0QtjltmbKIkB7iRz9nyiqjfYEMvKY4T29u8n09gGXmL2syAvGkHOW6Eybya8G3JJxKJPK/XAEjaJjqsUhGCK+fHxkOnIWjxGQMXXpld4IvUjTSiSFppMEYrRoUHQxddeIMtFJmvjnVjDBS8RbP4H9xkgy1z5a5vcUNP0h3fOvFnc7PMQezHiG6HYqyQMpJYzZovueqDN/cOBigCQI/kHLWBNtJAlAUGINtW+lXJcXCQUXTCbcMfyts04Z2wZWyTSuTnacI2Ija4So6Fwtemay1CWB+jEDOSxgYQGqH4p+42IvJy1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000020, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: mem_cgroup_flush_stats_delayed() suggests his is using a delayed_work, but this is actually sometimes flushing directly from the callsite. What it's doing is ratelimited calls. A better name would be mem_cgroup_flush_stats_ratelimited(). Suggested-by: Johannes Weiner Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Johannes Weiner --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 2 +- mm/workingset.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index b6eda2ab205d..ac3f3b3a45e2 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1037,7 +1037,7 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, } void mem_cgroup_flush_stats(void); -void mem_cgroup_flush_stats_delayed(void); +void mem_cgroup_flush_stats_ratelimited(void); void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, int val); @@ -1535,7 +1535,7 @@ static inline void mem_cgroup_flush_stats(void) { } -static inline void mem_cgroup_flush_stats_delayed(void) +static inline void mem_cgroup_flush_stats_ratelimited(void) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 0205e58ea430..c3b6aae78901 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -653,7 +653,7 @@ void mem_cgroup_flush_stats(void) __mem_cgroup_flush_stats(); } -void mem_cgroup_flush_stats_delayed(void) +void mem_cgroup_flush_stats_ratelimited(void) { if (time_after64(jiffies_64, flush_next_time)) mem_cgroup_flush_stats(); diff --git a/mm/workingset.c b/mm/workingset.c index 00c6f4d9d9be..af862c6738c3 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -462,7 +462,7 @@ void workingset_refault(struct folio *folio, void *shadow) mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); - mem_cgroup_flush_stats_delayed(); + mem_cgroup_flush_stats_ratelimited(); /* * Compare the distance to the existing workingset size. We * don't activate pages that couldn't stay resident even if From patchwork Tue Mar 28 06:16:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190556 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D2D5C77B61 for ; Tue, 28 Mar 2023 06:16:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8872E900005; Tue, 28 Mar 2023 02:16:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8375A900003; Tue, 28 Mar 2023 02:16:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 661DF900005; Tue, 28 Mar 2023 02:16:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 576E1900003 for ; Tue, 28 Mar 2023 02:16:50 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2E5161606F5 for ; Tue, 28 Mar 2023 06:16:50 +0000 (UTC) X-FDA: 80617298580.12.7FEE79D Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf24.hostedemail.com (Postfix) with ESMTP id 6954418000F for ; Tue, 28 Mar 2023 06:16:48 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sq4WxgrX; spf=pass (imf24.hostedemail.com: domain of 3T4YiZAoKCA4C265Cov0sru22uzs.q20zw18B-00y9oqy.25u@flex--yosryahmed.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3T4YiZAoKCA4C265Cov0sru22uzs.q20zw18B-00y9oqy.25u@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984208; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wILXj9LqWwom+6PQC1+rRSpOfUrkR+GzdkCDGSSBhGY=; b=iWkDdRVioO5F0tLLLd7qXrFCwSRcZlrO7rxDodr9Zos4SVig0XQuxRzl2A0Thnsd9kEglA boo+vr2BE9CCLBcUNfOcxko9nKxWycD8eqp9K/ZxuzZm13wa6FWALsgHQAiabLw2rfMO5l 7qrC2umO6sipg/kUimfK6coA3Q2Y7/g= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sq4WxgrX; spf=pass (imf24.hostedemail.com: domain of 3T4YiZAoKCA4C265Cov0sru22uzs.q20zw18B-00y9oqy.25u@flex--yosryahmed.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3T4YiZAoKCA4C265Cov0sru22uzs.q20zw18B-00y9oqy.25u@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984208; a=rsa-sha256; cv=none; b=r76yvcE0q62N5diIb30B8mJJXMsGYUDHvuxMS9oIgR8hYRGQBp+q9aiFHoKaIQEFQk/lDi CoFZR2n2/rua4SQXytbyFzilH8YwkVPh9aCOOuyAXT01fsOyT8kHXtiR2ZwJXZ3mxKqPzA TLgsSfTzC1nb2npSEPLMnd9Eez4QmrM= Received: by mail-pj1-f73.google.com with SMTP id d13-20020a17090ad98d00b00240922fdb7cso1384477pjv.6 for ; Mon, 27 Mar 2023 23:16:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984207; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=wILXj9LqWwom+6PQC1+rRSpOfUrkR+GzdkCDGSSBhGY=; b=sq4WxgrXs35T1kNaqttQxLQHBBfHPzw9iKcGHtZRmWn7KoDZlSxKdNufMOgLyKUFr+ TMiE/wqtjNlMTixUEQeY/Vrw6MeR14OOP90t7VeDCccykNrGNGWLRshNs7Tr5dzPpQxM EqNAZipGy1I27uXZW4pV5ZH3WWGxC9s0suXWrK0/Qi+Fx4TfEA0shsUjup5r4I0T4jcO otnYwmVT7RvIIuwqMJTULG/AsEi9P5/3lltCZIuxMi9242jZnrgGHko3vAN7pXgBxOSv 6bmFPpVjK2j81kXxaYSwcWty18LJNFnzIpVeFZG9+e6GZQd6MIhYJNlu+ckzRRvaCz8L Rlqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984207; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wILXj9LqWwom+6PQC1+rRSpOfUrkR+GzdkCDGSSBhGY=; b=U9CwoM1UJadpMsc2pRCbLSPakfSQZKQ2b32shGd0lvIs3YPCU0EIRiwM8dtdhCEN1G PbC9Im3bh+97ma7zkD0Qvvc3pYrmRIl4HY7XWYO8exn6uA07U2oJw5F/W8xR61L4IVSf HqLaDjqCT+XnnsIU4oGD4H8GlGVF4l5jNIavuSNYhoabB+0kxX4EsVXKJV2IvmY9VvuC ozcwtk4d02+nC31BXJIFRbHdYWZnBU1kKU7/IK/ayWQ+fVesPDzrpxKz1yqrA4xib3N0 pDV+aUM1cMWiDVEp88RZDXmlxmmzbHmTmfKBpEU0VjnZutz2p8w6Gft3L2uXrV8inm04 vTdw== X-Gm-Message-State: AO0yUKVby/DoKqXLC+RENZ4HPjAZUFLMbWbdKBfoqmUvp61N3edd58lq 1pn0iBNnkJQjahm1EsnBxGxOH1Bkgxt1h2Fc X-Google-Smtp-Source: AK7set/NXJ9x9Sdrim9yKl61I9Pd5pvCD8XFm/ZA+2N5HaWD82pwRjDqKXOdg8RZFiQsg1c0bhIGvLI1ByyxVbFu X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:4510:b0:626:1710:9b7d with SMTP id cw16-20020a056a00451000b0062617109b7dmr8924786pfb.0.1679984207461; Mon, 27 Mar 2023 23:16:47 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:32 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-4-yosryahmed@google.com> Subject: [PATCH v1 3/9] memcg: do not flush stats in irq context From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Stat-Signature: 4b88gkoiezhq86ihxha8koazwby57gdh X-Rspam-User: X-Rspamd-Queue-Id: 6954418000F X-Rspamd-Server: rspam06 X-HE-Tag: 1679984208-850795 X-HE-Meta: U2FsdGVkX18/Nr6qaCT5ndIFv81bGjIkm43qLXWbM8CJ9G/cyWioq1Vkg8D/fuACl89QjQwN0VIfdf7+FZ0hUxDKJ+SSHiu6CVJeqetfp5Pdaia6LNywBSZGhp5IuhID5rm7FY4Z63KYpzZazD5otOIKadeaASKh88Ry4nb+XtBo2wqkE0VVj/JhcfeyDkIaoZ9+Q7Lxk7A5OZ/6zXlG2zrUzBL8P2g9ygKy+LiLEIL81Yrj97ULxfjhWlBx8MVRdirBRzkM7K7V2l2wYzSIYqcpkvUptRy9+9uLT2EjBN/kue7c7UiFTggYCUrEp28Q0aAZWOQiWBJmBJP0dmes5FzpyDnBrLX3SsW5XuXwjD4tt+A6cAm8UQkUyv6LyIxNJfL6zWVsnvMTzv0O4eE5QIHMqVsmNI633gXzxgzpqqUcVfGzK3PuQwGLwqKRHyszxkFowMx2rhQDnBQ5wg9gCFPbiXp33UmrllU/1v2mquW7lx1vytef2QxuRzzBgqfTkTEtryGRdRKmG4wuL8AnYnoSg4lfYidpPrqXJ5Hbzdoyhr053ZYrsp13sC62zd28+6buKERi+rOiS3bzxhfVD79uZRTSi+Gx8/+xxgaXlvObKMuir4kUEs5Y1KGWhFi6g7zfW4QURhI4GvbfyuNnzPMgZRL68viZ7LwLhU4wk2dPxl4mNSRHuNzeQmoGXa/tegSLQT+Fy9gu8uBrOZK7pX3crVlFWulYglg6v00u4lqClMo8wGSnww94guDmSyUi2xXowGTgPecZOZYoeV5+e9wMp8ld6NrJY1t56PJ/O9aulpahhZCXsbZCcZuS7mLyLKA8vr0Jpk9zdQnJclLDVEcW3fwpmU5Fhaq0PfLXf/WyG68Eo5ARKvbp+cu8PFa9EvjQBp5VbhJWKlhamqVYlxWopKcPNqMvguJ5G3ncB2RDFFvWgnKhL3E3Qa2PBcd1Xok4X3UwuNM//xYSIVg KBIAh9/B q3pEX2uFmtXBAOCWwToSecAvKG11hlwRiP3pNFK6cKkMA261uZgGBRY/hxWMqieagbOV5eh02Js3I8URX//wb5K2l6PHJjxysXVCxYjFaXOvb+OAJilHDPUCmM+Rfx/MJcdKGEa3OnJgXlVXx9hUh8kW4jjH47PjR5nfvEAep9082emQDF4MUmdVAurFlKHbjnRmuQkxLCXeEOwezh5aLcxvlxb+lfx7E5uEwyM4t4vMg76SH3VurzVYIRNAZRL6vC69ujJbDJANHZq81koz97KCScQMDTYN2s8/Itf5VSCEKxQgn0vgV19Qbv6r8qLaIZQ5GuAUIoBP/ywNWaLmwGT3cRxry739AeDNjuUNZ79tB8yixf+8fStGOABm1exyspKwvaSwjiv5+wPq39u0Oh8pTk9/zSMkXxUFq5mO6BLKBayZ9eV8wm6ygViGrtmUM97IA0KRLewA+DB7dtZ3nT3n7aQVSOuEulQE5zOanyRsH5v7gFcoM4WhPrBbKWMxN6T+jurlv4t3WZ0pj1EXxIcRyM/EJDSFOoguOG2H2zpnQ6q1xnkBAKGJ9TjFXdTnayjD+kYaPR39IzzPe0YNhuH0dRsQ+qQVhDX65Wz1zT70EQhoA7GfljJKYmB3G9IeJPIj44CPBEfHpH15PNR5LVngirWhB5naxPKKeLGkWrW27Pq79DYLFxjxhqts9t9DHtsmAQSQYAfr/DWuqkfsdpHtznw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, the only context in which we can invoke an rstat flush from irq context is through mem_cgroup_usage() on the root memcg when called from memcg_check_events(). An rstat flush is an expensive operation that should not be done in irq context, so do not flush stats and use the stale stats in this case. Arguably, usage threshold events are not reliable on the root memcg anyway since its usage is ill-defined. Suggested-by: Johannes Weiner Suggested-by: Shakeel Butt Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Johannes Weiner --- mm/memcontrol.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c3b6aae78901..ff39f78f962e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3669,7 +3669,21 @@ static unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) unsigned long val; if (mem_cgroup_is_root(memcg)) { - mem_cgroup_flush_stats(); + /* + * We can reach here from irq context through: + * uncharge_batch() + * |--memcg_check_events() + * |--mem_cgroup_threshold() + * |--__mem_cgroup_threshold() + * |--mem_cgroup_usage + * + * rstat flushing is an expensive operation that should not be + * done from irq context; use stale stats in this case. + * Arguably, usage threshold events are not reliable on the root + * memcg anyway since its usage is ill-defined. + */ + if (in_task()) + mem_cgroup_flush_stats(); val = memcg_page_state(memcg, NR_FILE_PAGES) + memcg_page_state(memcg, NR_ANON_MAPPED); if (swap) From patchwork Tue Mar 28 06:16:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6FC5C761A6 for ; Tue, 28 Mar 2023 06:16:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8873A900003; Tue, 28 Mar 2023 02:16:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 83678280001; Tue, 28 Mar 2023 02:16:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 614D9900006; Tue, 28 Mar 2023 02:16:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 540F9900003 for ; Tue, 28 Mar 2023 02:16:52 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 1DC7AC06BE for ; Tue, 28 Mar 2023 06:16:52 +0000 (UTC) X-FDA: 80617298664.27.E2A78F5 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf30.hostedemail.com (Postfix) with ESMTP id 496788000B for ; Tue, 28 Mar 2023 06:16:50 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=FtVms547; spf=pass (imf30.hostedemail.com: domain of 3UYYiZAoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3UYYiZAoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+vNiceMoZ2PdzrNG/xVeRzEo7O+ovCZNHcQMp7VClk4=; b=cJBfkeAoJVcSxcfbR2oxIfeSMejKbXVxE4+Z7OpbDpw2R21x0UTE8KkLK/IK5BkPqVJcJq cfb3Zz87Gr2geg1mQWNpXvRHy+xkeufIA7KEioQg2fbAAlvFqtQBn5g3xVdiVpTZJw+Eap F+bgKuUa1iq3ZMjMp4Go0HZykojA6kI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=FtVms547; spf=pass (imf30.hostedemail.com: domain of 3UYYiZAoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3UYYiZAoKCBAE487Eqx2utw44w1u.s421y3AD-220Bqs0.47w@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984210; a=rsa-sha256; cv=none; b=iI6G+AbXJExc/mdfgXIHfnE0NHDIp1TJg38H+balIzEUuEN4jh9oezmmzWjxytvcOOqfsm U7QEkd3BA1bZ/TA7OF8yWSt5RibUcLY8x9/0b8n0YhGO9gJQYKXpZdfcw7cRZvwjIJ+wQY 1emBQIQhvMz8MC8Mk5sFAS40OIAW2i8= Received: by mail-pj1-f74.google.com with SMTP id ie7-20020a17090b400700b0023f06808981so2970010pjb.8 for ; Mon, 27 Mar 2023 23:16:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984209; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=+vNiceMoZ2PdzrNG/xVeRzEo7O+ovCZNHcQMp7VClk4=; b=FtVms547YGVdf9XAqjguoo0WST2Bomcq4pMeCW3qlg0ksZV5swPoVz0GmrxFu8vfrh HBzgJptVGGKy7geKpBPvC2PsWo0HmjT2F27i9dbdVA6zCF4EYemeZRNrh89DbSPqLObX h82tbSvZ/WCeLjLtii1Meb+k3BLVneGl+TdsHqmfTXT/vAFZIC+/S3Aw9k3ElSrpUy7X 9xgxQRUTlE19S9rQfATi59HRnf65TEpEpviTmJbLID2U9/qvSayVfcGFQGEl03zvUEu5 uDhGnK3XWBy1kJngfOB04Odcd1UeEO5WA33WjKyVDhleLQy8k50zMQSY0RXklX82o1Jn CmFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984209; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+vNiceMoZ2PdzrNG/xVeRzEo7O+ovCZNHcQMp7VClk4=; b=0H6C4qQvoAytTZUPcy9PFmHYVmcYp4TUOkG3/hhBkxv32GVpWbdAUvH7jTyDkguksk +u6/WmMmBAHnzkBZ87htn1lh2Ujl0gTG8La0AT5juWgZhdyRb5PXuuhq0YQ0woNApjKP Rlo3kd0XcGKHNkQzFmMc6grjVziA4EkDamG46l6Fe9miWhAAZJkwuYqhs3uj1Bn3j73K v+wvr7c08+WP1PbVRPeGj5SEnagTcHAFvwVB3b55GHkbJpcE0JrNli8BZcmrnVhDo2Hz xPeWwSJT1MfQI1FJqJkrPJyWSvUfaB76rcSIt45Afj3/9M4l+JRergKLLNhneNfICBAP U01g== X-Gm-Message-State: AAQBX9fBNq4kYjjQ0Pd+f0JikTAjaH5JDb9H0TCSdrBpO0YKlubFpHde f2TABakJPfnnjvUPcnCJDJoQnv8+0RDf6zLO X-Google-Smtp-Source: AKy350az2ZQxKTBjGyFUiNza+YDGvWpTHjavWcHAugAuQk4rmOEm1wQoqfQkURL2u01jT9Cm3t+xr5muiADnxiOT X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:1a46:b0:625:c7de:48c1 with SMTP id h6-20020a056a001a4600b00625c7de48c1mr7269994pfv.4.1679984209151; Mon, 27 Mar 2023 23:16:49 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:33 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-5-yosryahmed@google.com> Subject: [PATCH v1 4/9] cgroup: rstat: add WARN_ON_ONCE() if flushing outside task context From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: mpcpf5hoiyos4dhcjgof34usgoyw7ehh X-Rspamd-Queue-Id: 496788000B X-HE-Tag: 1679984210-355937 X-HE-Meta: U2FsdGVkX195XSQw41vXBNMOP8+532TDd/IFZwUeulpKFeYHAkeBI5FcQO5a+nnQmsVeuuRqiyIj80u045yH40ajDDg6dYRuEdaDHasp7S8H1VMtGgLlsZ/dS1Q6mQUZYJx9b28AmZpOxjRZ+zDDxMacfySZ59zZenyGOZxIbERJITIy90xUjY6vHj8EkPaSNN1uSUMsFY163rzeE66ZtTeahOey2T7upfEjy+szHQRJ4va+Di3YyQLn6fmQBT2eMSBfYuQTh3MF5c7i4z5xpeo6sXHbeTxWqcdnFCOeA5N/bQ1EkVj8sx8krG2PCZfYLyBQ8u23vFVqFOUGBidt3HjPj2SKqBdmFGNJ7KYVisbxlNj5DDwNs3ohr60lkVcNvEWXw8GKI+P7UQIpVnDN2vorxMFInuNOc5FrYePT3aN84Y9EfL72LCrKfK7O/ie4K9T95lmmjWmE4t6llHaVqKdN+DPn9/0WUwL/2W28KjhANi8D1XvO9leeGkcoqsc/+RBj1vQHLxFaPLUJt5+T3Oql+lKZjk6O13wvbzhf1spaBsSPI5wb9ygekXHjNPGG2HoQ+COjxbCHStqBLWlUq0rffyLQEnxdR7iEt1tBCZWaKJlXLvvMS4RSkk6K/u32lJEeqiwVnjpSlpvd7Y9BzSQwIk+MLPdAek415Z5zu+arkPoi03/rn8W+atkhgzyGWzHMZQxLblh/IvnvI1ZuuoYQ/kMc9TAdpR7QqVjJtazAl80cZhEZvcbLoCs/48M+CkGKWMK9NU0GebelL3e0kzGWuFCd92G8FTSdygvj4EOiAXWjohFfdPVhlXWEBTjWVWKVbe5LtKnYNnYsDv9wsDdIK24TCM7EH+NdkH9UAhMzNe9w4H2cDMjBKypzA+/3syQd+IEWbS+N9TBFUWEo9HE/5BLi16ADEtyWVgdRsPApgJ+zl9El+CjXFsRyoBeP16mALTqh6D1rp9IgLv/ S9EkOt7v 5oq3TXR5efLJ56A41ureKqyjUuE3jp6uSegGGnLz2WXp1d+vbzP6JIYWv9RCdDcvlRkD6TZbPyvrHqkvpUk0WH1wzdqE7VVfLmQSQBmRyc5BHJDqd1Avf9Fu5xk+Yc2FdlbLUXR8Q26s/mmXozr5VKtyfajy40mPlXnEVXJONqYXNsJ6EDPuLdILsGPaYw80T7DvJEYe6+yw2IkCE4Zj3oVrW5tJ5B2aEqgjqmnwje7Gp4EgHmURAfE7XL78knt7mjLd8iLeKvNIxgqAOTQF9/LYSSex9jwa+rjwF9DEsGKCLlHBGd5WEzWR0mbq87NOpwAxwpSbN4nhD9nKrLknt/pJlSLSsEf2VKAFfoK2mGk/mvA8LiRfQ1wb2/1wnI6gc6oY6YUkGSY5I1DGgc1jOmX1Rlt5Zx4VpdNdV9kics+ZbF6QnzZ8+dFbNeWNQfRPB+Mgj8y9fGURzg6ZHG5sfTJ2OFuXbaNNx72uLXrL+Xt6GHTkzOpNjOQ1H332ujTM4b6hdUAN5ujVUgKJiky1ZZcFkLIyb8RmpUeRNyDYoAsvBFy6Io267S5XkVMuHRsC8G+UfsCp3OXbnjSFKwK0lVJmzLk908f7sRk6eudX7BMTeQY73OQAOP5qqJAfbJxg6Os0ZL9JME7xZPu2gGeop2dA4xyaNpHoYXpaaVioXpKrau3rL1fOvBRAqO0/Tu4Ml50wzwi7+X7q9wS7aggSasNQm4Ib3Vs0VGaKs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: rstat flushing is too expensive to perform in irq context. The previous patch removed the only context that may invoke an rstat flush from irq context, add a WARN_ON_ONCE() to detect future violations, or those that we are not aware of. Signed-off-by: Yosry Ahmed Reviewed-by: Shakeel Butt --- kernel/cgroup/rstat.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/cgroup/rstat.c b/kernel/cgroup/rstat.c index d3252b0416b6..c2571939139f 100644 --- a/kernel/cgroup/rstat.c +++ b/kernel/cgroup/rstat.c @@ -176,6 +176,8 @@ static void cgroup_rstat_flush_locked(struct cgroup *cgrp, bool may_sleep) { int cpu; + /* rstat flushing is too expensive for irq context */ + WARN_ON_ONCE(!in_task()); lockdep_assert_held(&cgroup_rstat_lock); for_each_possible_cpu(cpu) { From patchwork Tue Mar 28 06:16:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63F8AC76196 for ; Tue, 28 Mar 2023 06:16:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3D7F280002; Tue, 28 Mar 2023 02:16:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F21F8280001; Tue, 28 Mar 2023 02:16:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6686280002; Tue, 28 Mar 2023 02:16:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C9A4F280001 for ; Tue, 28 Mar 2023 02:16:53 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9A83E1A011B for ; Tue, 28 Mar 2023 06:16:53 +0000 (UTC) X-FDA: 80617298706.18.4B21C83 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf14.hostedemail.com (Postfix) with ESMTP id D16BA10000D for ; Tue, 28 Mar 2023 06:16:51 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=dxPzci0v; spf=pass (imf14.hostedemail.com: domain of 3UoYiZAoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3UoYiZAoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984211; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IdlOdGF0YCCYf+h2+Wx+Pr2285UVOLm52gCW9Jb/OEo=; b=Du1RYOtgxZl7RMoQ44s8el/HXUln3NlQttKc7Jyr+hJBqoPGs/o93fIOTqRwWsZu1nr76N ZD+RghXHxJ57RQHGPYpTaMOUqS8btBETke2Fhrs/KmCz7NkwJDXZHPPcdf2HWPULdaP/ix 5Qpj/ts+6c0KLeqVM2emFr+Wkiyddx4= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=dxPzci0v; spf=pass (imf14.hostedemail.com: domain of 3UoYiZAoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3UoYiZAoKCBEF598Fry3vux55x2v.t532z4BE-331Crt1.58x@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984211; a=rsa-sha256; cv=none; b=iSPSJlAeXzfNY4M4JVyxHSvTC/R3+qhaHbcJRHmvJTzG85KQ1bukTenMr10WtJceUq3Uow DoOrfkrLMZkcIER7DrF6f9/GXGzxA6c53zHdTzoHAD4PMwBcjKMNjIVztL3Vh+ir3qnmW+ klV1EOMWb4rg8U125DkYgqCehyjtYmw= Received: by mail-pl1-f201.google.com with SMTP id x4-20020a170902ec8400b001a1a5f6f272so7068002plg.1 for ; Mon, 27 Mar 2023 23:16:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984211; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IdlOdGF0YCCYf+h2+Wx+Pr2285UVOLm52gCW9Jb/OEo=; b=dxPzci0vIh1xyJxlLBjWkEhAjm9YbPSdsUTO5u3B5j0mjvT8KDBeosrn5UiPm3Ksna ECFLKks3+i/JLwnVARkeiq9XCne6PDhOU+plEpcmqroZ0FIo5b3stpDp4azRz7RDEUXr fg0RX+xHWkbr/97SE9yMggMcSWIL9SDqbA2byVm27sTtHVM/RcaUTUFVwNm3hT6OyTW8 KGm1B3oJJOu2mLGqIrSENlsxO5AJatCTwMfnNnO9IVILzvLEURQL9lNGe4Pge9n4mWvC 01ODxHoNCu+DMqEmep2UjWrEts9KjVkOV7sWpi8aLYcEgH/6hlGK54jM+qhbvTAI+Y6D 5qIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984211; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IdlOdGF0YCCYf+h2+Wx+Pr2285UVOLm52gCW9Jb/OEo=; b=hkELiDIM7IY8S7L2MxyyF90s2O6I9mv+dSb323wZD0c5UpTEtup/iCXNBRb12tBlUj 6D7Tyw+ur+VPRxw9oY0SvdE2lhKBEJs8tq2pTv80jHHofKln/wI9K/4UjrU7hnwA8I1O syCiJh3f4gpukzuU4lTdjkz9Ic2MOUqR7r5MbVmHrUf85t9N7FjHPeCjXU4DM6qk5tGR yZvU+VFGA51YLcIduIo5xBG3OnrkZhQjwAUO1mQqrVYrsoxd24frrJaX914KCrNAmATj 87W6FM/DHZUGqNyJ56mLR0KuycphSFWW4iYfUg7Y6NqftO0qpFWzRJOc4KrBjnBNdZmo G7mg== X-Gm-Message-State: AAQBX9ds3fa6/J/JGYP75/tix3eMQN1PvwkwaTCfe5CFFqRdfSxpRDv0 TuXubSCCh+NTNWHDyUsP2SAvLVRfduwj8VkA X-Google-Smtp-Source: AKy350ZiMUeqTdYquDazwFQP3GVswROLJyitcbpH2br9NmUFXZZud9qfmMVPueO4yaGFykMkMD2PV8f94RgXrI2o X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90a:74c9:b0:23f:1105:4295 with SMTP id p9-20020a17090a74c900b0023f11054295mr4298168pjl.7.1679984210825; Mon, 27 Mar 2023 23:16:50 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:34 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-6-yosryahmed@google.com> Subject: [PATCH v1 5/9] memcg: replace stats_flush_lock with an atomic From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D16BA10000D X-Rspam-User: X-Stat-Signature: uyksggcihr8ofj6wpn4c1g4zju19qzu9 X-HE-Tag: 1679984211-988156 X-HE-Meta: U2FsdGVkX1+diptex1VatIeXxEZTlxLZMwfIQR9GxRSwD+LRu+4ZV00/9L+4noxy5yWbjcmwcwHJkQlCeWQfPvtwid/1WbSM49zxidy0iseWE3rXdCINiL3qQwnJWIGAc11pETXuQGVYUX5xOysV+InnKVYYwLqfUKj7FAeG6yfJZx+r2GWkhLDjBdb9G+aGBjSxY3EQ1JQlAVKjjMbrf4Zu+nhIsyDp4l01YCHI7rQWw5izWravnIdgxXXLd1cF+//UZc9Pq76yNCagNgxjgeGtIZDt8oKyYHtD+Mw7DbfqJ6EQR2t41q5CZNyDgsT+ahq7ZcE0NXGV+bCWaS3/cmpkWoufwut6LKUgFxZ4qXbQpS8TPT8mO4g705EBc4NAhYAffzq154lCfVengqX3xhS0mGl+xAaSDvKLKuNjtgjLttT9tbeaOE5gNCdOdetZveWYc07gGmYWsq/VxA/7WNAL1CPr9z6o+OlAf1YE0sJoaPoK+4vNhLy0ka7GLRmjUTeSx5ORU2CgDB9ctZezaVq+/noRh9vJpIPVOZe36SIN4vqg9hxsAym5joGMC/qw7/wGDEBh9qgRcDV08GIy/qtlGxjXzN+D5HJznCNQySRXjmdat/S+fnAaYAi3WFCpaWpB+SmPJMq08mKb/0DOK4+Sr7W8ashD23ytnDQ8I1NXMtt1D7t3qWMS4/5PyINxhNe5QCFPX61SpTpFoTzAz84mThslA4Tz6/y++4K900irZgjnq5NQDPdMu9+oZd0gWATiCBlM4zVgYY9A7QER9i7WtxbvCWGMqJOSuQ+ayEqPUOF27GdXWlLk6d979OM9P436r9GLY6JdvSvORC77dyCKBysdG+umuVZOwaoxC3M59kS7gyrhqX+Cl0fDBWBSzrMNPf4A69hdA+Z5jcKTn4MKdbBWC31aYYs1FxN6B3aICBt07Gs8fWsjPrt4XOV9a+GGT4R1Ly5AoA6eolp /mi//Qih UfyRdrN2xKDmE+SrIHDQhVOKEcyeK9tECLPGBEa0IkwrhXiJKN0xqZYS7QU4ByZLM0op5OtMwlDet76bUwQA++gcz/4rVBBwBKU8TxKy/BdJoKD+XjtjQeiTa4Y3DBkQsTrUmj9ckwsC8Wbp7MsGw5aNltrndYjRbjmzcXlhFdBvh4k27KrqATfjnrGX91xiBxUK6eLPwuAAE3xCRsovksG+QfHMSzYjnrH8sGIZSQWt5mc1Kac3VVaWHfcDIoliXlFHMXnmF5MDJqJBkoiL/TzCXHbTDNnSCHoZwbNSmvnwPYx7MyAQQugTGGxyXwyBfGryHbP2IFHK/6VgKb3g0/u9h4pBvKLri4+oS0wFRbfeYuNR0sb8DRUrOFOYG9Lea8n7hDlJH/PjWuRbg0kmqYcwcqSgwqDowuT/taun4+Z3+vTUk1DquhY0dQKUcheliK48ym8BZWUW3dHiBzCQWRM1UtTR5Rv/uY1B3oZAB6BlDy9aItNEz8O8vJYW4J1Wzx1JT3LWDcfExn79LaKh4c7kDTWwx2GpBBv96zNZvoONAgYEHDu1yGmfmxAuh9+p2cxOxNw8wqjJRceeohR3kdzh6Hh/YMsHSPx9c7oTOzSMgkVH+4b+cdoDAsPxJxscnv538N2Pk/OkAN+hkbYzVJmdyS/0NVPwTEfNq3Rq5csCZk/pk8mgmfiOLKKRhIxL3WKxH7opZbrb5KJZJMKDxKMTiDAPw9+ojTAXA+PRl1ukxr58sQ2fvp+EDCT69qve9BObr9I3b57RoqNIGW4C7DGs7m7EzjQwP12Nv9rfOaaqDq7q0DW8vbVO68525EGUPS7yx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: As Johannes notes in [1], stats_flush_lock is currently used to: (a) Protect updated to stats_flush_threshold. (b) Protect updates to flush_next_time. (c) Serializes calls to cgroup_rstat_flush() based on those ratelimits. However: 1. stats_flush_threshold is already an atomic 2. flush_next_time is not atomic. The writer is locked, but the reader is lockless. If the reader races with a flush, you could see this: if (time_after(jiffies, flush_next_time)) spin_trylock() flush_next_time = now + delay flush() spin_unlock() spin_trylock() flush_next_time = now + delay flush() spin_unlock() which means we already can get flushes at a higher frequency than FLUSH_TIME during races. But it isn't really a problem. The reader could also see garbled partial updates, so it needs at least READ_ONCE and WRITE_ONCE protection. 3. Serializing cgroup_rstat_flush() calls against the ratelimit factors is currently broken because of the race in 2. But the race is actually harmless, all we might get is the occasional earlier flush. If there is no delta, the flush won't do much. And if there is, the flush is justified. So the lock can be removed all together. However, the lock also served the purpose of preventing a thundering herd problem for concurrent flushers, see [2]. Use an atomic instead to serve the purpose of unifying concurrent flushers. [1]https://lore.kernel.org/lkml/20230323172732.GE739026@cmpxchg.org/ [2]https://lore.kernel.org/lkml/20210716212137.1391164-2-shakeelb@google.com/ Signed-off-by: Yosry Ahmed Acked-by: Johannes Weiner --- mm/memcontrol.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ff39f78f962e..64ff33e02c96 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -585,8 +585,8 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) */ static void flush_memcg_stats_dwork(struct work_struct *w); static DECLARE_DEFERRABLE_WORK(stats_flush_dwork, flush_memcg_stats_dwork); -static DEFINE_SPINLOCK(stats_flush_lock); static DEFINE_PER_CPU(unsigned int, stats_updates); +static atomic_t stats_flush_ongoing = ATOMIC_INIT(0); static atomic_t stats_flush_threshold = ATOMIC_INIT(0); static u64 flush_next_time; @@ -636,15 +636,18 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) static void __mem_cgroup_flush_stats(void) { - unsigned long flag; - - if (!spin_trylock_irqsave(&stats_flush_lock, flag)) + /* + * We always flush the entire tree, so concurrent flushers can just + * skip. This avoids a thundering herd problem on the rstat global lock + * from memcg flushers (e.g. reclaim, refault, etc). + */ + if (atomic_xchg(&stats_flush_ongoing, 1)) return; - flush_next_time = jiffies_64 + 2*FLUSH_TIME; + WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); cgroup_rstat_flush_atomic(root_mem_cgroup->css.cgroup); atomic_set(&stats_flush_threshold, 0); - spin_unlock_irqrestore(&stats_flush_lock, flag); + atomic_set(&stats_flush_ongoing, 0); } void mem_cgroup_flush_stats(void) @@ -655,7 +658,7 @@ void mem_cgroup_flush_stats(void) void mem_cgroup_flush_stats_ratelimited(void) { - if (time_after64(jiffies_64, flush_next_time)) + if (time_after64(jiffies_64, READ_ONCE(flush_next_time))) mem_cgroup_flush_stats(); } From patchwork Tue Mar 28 06:16:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6431DC77B60 for ; Tue, 28 Mar 2023 06:16:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3E44280003; Tue, 28 Mar 2023 02:16:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA0A6280001; Tue, 28 Mar 2023 02:16:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D3FD9280003; Tue, 28 Mar 2023 02:16:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BE771280001 for ; Tue, 28 Mar 2023 02:16:55 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 87EF980849 for ; Tue, 28 Mar 2023 06:16:55 +0000 (UTC) X-FDA: 80617298790.03.09FDEC6 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf14.hostedemail.com (Postfix) with ESMTP id BD9FB100011 for ; Tue, 28 Mar 2023 06:16:53 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ntB4QsMV; spf=pass (imf14.hostedemail.com: domain of 3VIYiZAoKCBMH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3VIYiZAoKCBMH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984213; a=rsa-sha256; cv=none; b=3/dwhwHHACLjhYq5wDzMqp0sjmA2k7C0nQnHaIMFTB4RH0IOZOmJ38Cuyf/XCAWh7EUpLQ BL8sDiMi3zwwDPGwl7+Q/1c3En5EhJ1n9m78YmigYom55bMeBF10N0aEfOlRnxlfnmkQbm xBOFUWmlyhaK9MvSYOPTXbuamURsEBY= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ntB4QsMV; spf=pass (imf14.hostedemail.com: domain of 3VIYiZAoKCBMH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3VIYiZAoKCBMH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984213; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9IOPAG+bqxWxKt0wdRgDKHhdCXWwpmJ9UigZO44sKyE=; b=5DpXTXJgLD+WqcKY1bVX7o53aybEDmgQIIlKeVkEVyRjudM1ENhJzxmGFpLObzIpM5UHcb yYho8jx3GASH8YxcZObLy9Q4tOu+9SU9rWvWDAR4f2hzkxZbl+LjxpG/5eWUECXb4zWf/Z 21g2PMtvJ7vRC5gqS/hNPGwP/IQbjoQ= Received: by mail-pg1-f201.google.com with SMTP id k1-20020a632401000000b0050beb8972bfso2919994pgk.7 for ; Mon, 27 Mar 2023 23:16:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984213; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=9IOPAG+bqxWxKt0wdRgDKHhdCXWwpmJ9UigZO44sKyE=; b=ntB4QsMVT8Fqv0//N2SyNJC8TO+SF5Nd//5MMCXPuMIlVWZztrbNiDvPnYTm36PsVJ 0K9UDAdVpl/jKPNVqGESnlXBby6gGh6KP9olFVGSWL7LZjX9hTDyTd2rcdVRYaALTq3d Svdw5D9xcS6f+R64pTOq4hP3DTwLDxc7s6WmPqIXIzObjUGQ7BCksYYcDbsn8CMNCe4l XYGLULvlYDvGRGQwaFkUUkpNiN3oSuW1tCmNSMxILm7ttbjB9+T2xkgSS2wr2ydIjDg8 ulb0ThWq/NbS6X5lqaGmn6zu6IKbfwWSX3vLp0eLAeAr3Ue3Dif4Fen7adboVvdM5kPM ZMpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984213; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9IOPAG+bqxWxKt0wdRgDKHhdCXWwpmJ9UigZO44sKyE=; b=Hi2Ec82poc7IDPcEUbShnrI1gCZSpatfoX1gbXoQEDZsptW35C0XlEeCfdEsnNq0k5 N4FOQwCnB0XlVzSWakdHP4LOSDGBbi9b9Rqu3yDnzhfGMYKv9JdFMymGa5Ho7I6Rry1d ktC+oOjbLZFnvD9w2kZbsu1WDjk1CoVxHArfbcES5a6z0YYrTGJzkfUOy4lSYdSc9Wum nxinF+cdd2Lg85Zak1mLTVoI5/7BT5Rt2oY60IQ2FpzHC+9JQ8FbLdqSTrwPxZT8lP8o 3Vgr+FzIFhoBJrNd7u15XNVNonl0GkMEUT1ZUbut0bO1MTSYNG51pVgDDKCu1g3Sh9cm 180g== X-Gm-Message-State: AAQBX9fCdq6udg91u4jI1pf4i43pHGVOToci0YkiH3B37ZezKlsEhGLA UL3XDs6Pn5Tj+ftxYXKngLMXPu1XEq+s7eWR X-Google-Smtp-Source: AKy350aQyFDG3vc2bblz86MnnpZXVqNeZCl/Rb1kLJ1qs2lbXDpCUugeQMSOiQHvyL07sX6z7bmDfgTHunmuYJ4D X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:4a41:0:b0:507:46cb:f45b with SMTP id j1-20020a634a41000000b0050746cbf45bmr3969874pgl.1.1679984212630; Mon, 27 Mar 2023 23:16:52 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:35 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-7-yosryahmed@google.com> Subject: [PATCH v1 6/9] memcg: sleep during flushing stats in safe contexts From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Queue-Id: BD9FB100011 X-Rspamd-Server: rspam01 X-Stat-Signature: e1t77jcck9sw7cded4i4tt5nzdfp3ooz X-HE-Tag: 1679984213-767051 X-HE-Meta: U2FsdGVkX1+YvFrJRI/W7McarTxDTka+IRUK5khqF3Hxk3I4rM5anl22xhcjG7szLANnqWwG83Y7Z8DqDzrU6E9HxW3DzyHdRif3b2J+pgCjwMvfQQnPgrGh2d+KlXi+8pw32MiHkT/PZEX6taH2dvkWq7/aAps2MuIQWLo05bYoThjOvp72qn9zN3uVxOKD2s2CAgk8Oz9LEFaPiipOURDEej87fvHOxzw49v7Vm/T+qFA/ivaQxBZVnfWTm1SRuD71vaBMC9Rt5+JWe0+VdBEVEJPY6fEQf8nEsBpXJxfIdkt03dMNMT2AZ/ICB47uDpyxvf77IlDwY9pxk5gtptbit5E8TgH/k/YVpLTK5tkIC4C72htCvgQZbUYAJbekk6V9hQWPnAB2rdAl+B913Dp0UgRyby4DeEaVfpitMatnfHMHJpfb0uKGm3YJgHI+4syzPUKuU8v/waZPf0WwqHORmxCBp+66ux9+vMSlQZGrcGIVlh8AbJicYIy167XYeZ5w1clQRYHqtVTz5bc1PkNhdNSP+zZj9xter/0vJ9rDojJVXkbJlCerRggMd0HGwHOZPLQy27ZZR5vb9IsF3ld3K+i6B+j2vZncVsUrpGaorlYzGaU5EqEF0VoBWzvjjHnF/vBviEsKQZFw5r2E0NaLYfDVwcrB0hrvu/aPzL1OaR8dXAZ1S9e1oPXzp4daexFSd5xvBWiGR2ETXOUDYGMrDYJH2197e8jaJdFUGON/jnuE5TnM1JoPXgIstf1fqx2Y2bGemoUMZC+dR6H2wln2QlIJmQSt12Bxxv2g3ZYk1Nmws4Zno4BJ0upF/NmUMzyFFPCe2h+nsOACFZdKiolSmA2E/1cIhFjyhwaXWQireitWEzt/W5JdsPqkasmM+icRDRHzxA+KsTKYU5AwQeCl5Vr4g8GL9c9je277ZeFKshugd9ef02pURCXpxvmCbY9HxkOLHLvOjr3wVM7 el3mdFwv N7OxAvLyJkGnCBu2HJ3msf6o1B+03dSO11N4+r7HMBaxHMj4s5EFNPwfQ9uQtvXJqh0J1DvGgL9p5MT1C6TlDHYS/94kCPiYvdQmNheltdXJUtM0LVSY2Xvrza4RNniGSl2IGSil5bidhG3NgZAVlO9aoF2MTkLmiU6S2vjOgWfcfHHUjoO8pA2c766oLpQgZ1dHFFZm09GurFPWnW9OwQy2nDQeRa9XimgQNisfRzM4r8K2kf1ebpYIkWVkopzw48gSPrdBxLPg0vIuuTfwdG5xwb+Bc4OIN023xiOW8DvRueFGEqPnE6HuHMmojfmXcy2dz7YMZIqqpB7XN8Z8Z0XESOzUYm6nuZVDQMCD9FS/GEuHDDPa/L5AiNly0FtN696H1WMd1DMj/jJNjfg7l25NEngTIdvoV5vnrm5H+rzWqL2t674AFKJBCnFgKLMdobGndQesNcS3OeLfGRRUzK5ScGt8e4vZWx3r3/Cxuz9wSn2q+dQ5HxVJPN8hFUZKPyvFdMOWgVxk2aYMFn2RLlQseLIpA/2o8wib43cqKCCwhu6msgI1KlJ40vmXDQAgqYgEEVh08CVklbWvyex2W2JI63ynk5czJJaGKgHLSJOfoYHtf9zOuHL3zxjDLG+jLuwq7DcjpKT6EJG4vp95ZRlVh/g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, all contexts that flush memcg stats do so with sleeping not allowed. Some of these contexts are perfectly safe to sleep in, such as reading cgroup files from userspace or the background periodic flusher. Refactor the code to make mem_cgroup_flush_stats() non-atomic (aka sleepable), and provide a separate atomic version. The atomic version is used in reclaim, refault, writeback, and in mem_cgroup_usage(). All other code paths are left to use the non-atomic version. This includes callbacks for userspace reads and the periodic flusher. Since refault is the only caller of mem_cgroup_flush_stats_ratelimited(), this function is changed to call the atomic version of mem_cgroup_flush_stats(). Reclaim and refault code paths are modified to do non-atomic flushing in separate later patches -- so mem_cgroup_flush_stats_ratelimited() will eventually become non-atomic. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- include/linux/memcontrol.h | 5 ++++ mm/memcontrol.c | 58 ++++++++++++++++++++++++++++++++------ mm/vmscan.c | 2 +- 3 files changed, 55 insertions(+), 10 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index ac3f3b3a45e2..a4bc3910a2eb 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -1037,6 +1037,7 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, } void mem_cgroup_flush_stats(void); +void mem_cgroup_flush_stats_atomic(void); void mem_cgroup_flush_stats_ratelimited(void); void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, @@ -1535,6 +1536,10 @@ static inline void mem_cgroup_flush_stats(void) { } +static inline void mem_cgroup_flush_stats_atomic(void) +{ +} + static inline void mem_cgroup_flush_stats_ratelimited(void) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 64ff33e02c96..57e8cbf701f3 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -634,7 +634,7 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) } } -static void __mem_cgroup_flush_stats(void) +static bool mem_cgroup_pre_stats_flush(void) { /* * We always flush the entire tree, so concurrent flushers can just @@ -642,24 +642,57 @@ static void __mem_cgroup_flush_stats(void) * from memcg flushers (e.g. reclaim, refault, etc). */ if (atomic_xchg(&stats_flush_ongoing, 1)) - return; + return false; WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); - cgroup_rstat_flush_atomic(root_mem_cgroup->css.cgroup); + return true; +} + +static void mem_cgroup_post_stats_flush(void) +{ atomic_set(&stats_flush_threshold, 0); atomic_set(&stats_flush_ongoing, 0); } -void mem_cgroup_flush_stats(void) +static bool mem_cgroup_should_flush_stats(void) { - if (atomic_read(&stats_flush_threshold) > num_online_cpus()) - __mem_cgroup_flush_stats(); + return atomic_read(&stats_flush_threshold) > num_online_cpus(); +} + +/* atomic functions, safe to call from any context */ +static void __mem_cgroup_flush_stats_atomic(void) +{ + if (mem_cgroup_pre_stats_flush()) { + cgroup_rstat_flush_atomic(root_mem_cgroup->css.cgroup); + mem_cgroup_post_stats_flush(); + } +} + +void mem_cgroup_flush_stats_atomic(void) +{ + if (mem_cgroup_should_flush_stats()) + __mem_cgroup_flush_stats_atomic(); } void mem_cgroup_flush_stats_ratelimited(void) { if (time_after64(jiffies_64, READ_ONCE(flush_next_time))) - mem_cgroup_flush_stats(); + mem_cgroup_flush_stats_atomic(); +} + +/* non-atomic functions, only safe from sleepable contexts */ +static void __mem_cgroup_flush_stats(void) +{ + if (mem_cgroup_pre_stats_flush()) { + cgroup_rstat_flush(root_mem_cgroup->css.cgroup); + mem_cgroup_post_stats_flush(); + } +} + +void mem_cgroup_flush_stats(void) +{ + if (mem_cgroup_should_flush_stats()) + __mem_cgroup_flush_stats(); } static void flush_memcg_stats_dwork(struct work_struct *w) @@ -3684,9 +3717,12 @@ static unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) * done from irq context; use stale stats in this case. * Arguably, usage threshold events are not reliable on the root * memcg anyway since its usage is ill-defined. + * + * Additionally, other call paths through memcg_check_events() + * disable irqs, so make sure we are flushing stats atomically. */ if (in_task()) - mem_cgroup_flush_stats(); + mem_cgroup_flush_stats_atomic(); val = memcg_page_state(memcg, NR_FILE_PAGES) + memcg_page_state(memcg, NR_ANON_MAPPED); if (swap) @@ -4609,7 +4645,11 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages, struct mem_cgroup *memcg = mem_cgroup_from_css(wb->memcg_css); struct mem_cgroup *parent; - mem_cgroup_flush_stats(); + /* + * wb_writeback() takes a spinlock and calls + * wb_over_bg_thresh()->mem_cgroup_wb_stats(). Do not sleep. + */ + mem_cgroup_flush_stats_atomic(); *pdirty = memcg_page_state(memcg, NR_FILE_DIRTY); *pwriteback = memcg_page_state(memcg, NR_WRITEBACK); diff --git a/mm/vmscan.c b/mm/vmscan.c index 9c1c5e8b24b8..a9511ccb936f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2845,7 +2845,7 @@ static void prepare_scan_count(pg_data_t *pgdat, struct scan_control *sc) * Flush the memory cgroup stats, so that we read accurate per-memcg * lruvec stats for heuristics. */ - mem_cgroup_flush_stats(); + mem_cgroup_flush_stats_atomic(); /* * Determine the scan balance between anon and file LRUs. From patchwork Tue Mar 28 06:16:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B2C4C76195 for ; Tue, 28 Mar 2023 06:16:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3142280004; Tue, 28 Mar 2023 02:16:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B44E3280001; Tue, 28 Mar 2023 02:16:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 948A9280004; Tue, 28 Mar 2023 02:16:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 86EF5280001 for ; Tue, 28 Mar 2023 02:16:57 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4F34D1A0126 for ; Tue, 28 Mar 2023 06:16:57 +0000 (UTC) X-FDA: 80617298874.19.734A78D Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf16.hostedemail.com (Postfix) with ESMTP id 8CF29180017 for ; Tue, 28 Mar 2023 06:16:55 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qIxeFfAv; spf=pass (imf16.hostedemail.com: domain of 3VoYiZAoKCBUJ9DCJv27zy19916z.x97638FI-775Gvx5.9C1@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3VoYiZAoKCBUJ9DCJv27zy19916z.x97638FI-775Gvx5.9C1@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984215; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0xrFtgulASYXrMO3GaE6CpeYp9Puj0hiviKaUT94oIg=; b=M2m2HPLwMoGiea4FrF09YTfHf6+iku1imTM1aEJvbVci/LKkivQp3Q8GdCi1uBs+Qx+Rlo Bxgmeg5S3bS+g0nVtQzdS84YVC+SPOAY0hQ+q1fHFEP7dOUa+mW7LPF4WaJgkLzVV8PYAz GGWOuFAghbuLjYO/pQX5jlRZAMTgKws= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qIxeFfAv; spf=pass (imf16.hostedemail.com: domain of 3VoYiZAoKCBUJ9DCJv27zy19916z.x97638FI-775Gvx5.9C1@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3VoYiZAoKCBUJ9DCJv27zy19916z.x97638FI-775Gvx5.9C1@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984215; a=rsa-sha256; cv=none; b=OvuaZy3fK+bQymHthh0yJ8yFoyNrPDzyASrc8xKEKuR8V2SannjioLNpLyi3WNPmEXpjiQ s+1UjYmLAD+CokSwK1BLX0Vy2x7zkFHd/jYmyRENC4k0+uawFh0EFOMgxlK3epeiku7uLQ x2QSF2B8lpc56/Y/97qrxU8ZIuw3YdA= Received: by mail-pl1-f202.google.com with SMTP id w14-20020a170902e88e00b001a238a5946cso4524582plg.11 for ; Mon, 27 Mar 2023 23:16:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984214; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=0xrFtgulASYXrMO3GaE6CpeYp9Puj0hiviKaUT94oIg=; b=qIxeFfAv7LZhUaMY9U0TvTtrAPX4wRqNbnIeY0Yrn/RnFRGntBTlu0hfuIAsMfAy2n 1P28EpdkOB3njh6q599A6aRPcbQ+/6Ngp2xMzs6rBLHlr6vd8s4xgEvIn8RMUD6Mze8g o6DwJQ//Mohpbdf8YD2EiOxbMyBh8HdrF+/Yzlad7E0D+Fg0j+P6NKtF5T845D/HLRrp rP0WF5ztka030C2opL/oGdjy+y9vRWrHiXRiQXH/e0QgpKRtMDaGNueTHyNdIDcH2xFM Q3P8IwnLdzShQrK/qG/mnGtQghgE6XGNQRvgt1djyelN/qv5GiJdyRld4l0gXlsYj6Rl orEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984214; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=0xrFtgulASYXrMO3GaE6CpeYp9Puj0hiviKaUT94oIg=; b=PUXgYn7FfgUFYHnFl71tO7borZmFe6lelqpmlE+Cahj/RKNipwdE+itovaYpVLAGEB +V0sGtjcqGvCIZH/0jo+34qq44PKnWvIitVg6ilLlTrXXq8AmCGWGUq7gSqktYcysoxU j4ujazA6P61VrRKxkyjsBmpeumgL6G7KwGFGDrQ8o8n9LZ9X8tfavNjfvIxymSwTQIC7 UcL9qZQ8ZsMlGpXqeaM5pG10/lKqXQKorA9azhOsSsQZHNT1XyNpCjzbAG093i2tAb9r nfKJ2II71/+egr6YhEFilzfqzakuMp7IV3T084pFr2YTPJjXp2iMepvxMuxR9txL9qgA J+Jg== X-Gm-Message-State: AAQBX9dFyHCGnxmtPqcG7Z3p4014a1mHTwXCAQzSaV4+GOc/qs/2hK3A jJ+GW783UkYxEPm5tF0Fv40p3NetmMcbr+C2 X-Google-Smtp-Source: AKy350aBuKbfjhOAuCJYm2iwk4HKu8S/jfcCuw8uP63TY3FEEFp0rTq/+r6BOwoVOwDyCp4xkVINZeVu5jFTXH9F X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:115a:0:b0:50b:dfd4:b56f with SMTP id 26-20020a63115a000000b0050bdfd4b56fmr3747909pgr.5.1679984214584; Mon, 27 Mar 2023 23:16:54 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:36 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-8-yosryahmed@google.com> Subject: [PATCH v1 7/9] workingset: memcg: sleep when flushing stats in workingset_refault() From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 8CF29180017 X-Stat-Signature: pk1neh9e7gosku5z6b3hjnoc7u4j1z5k X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1679984215-625486 X-HE-Meta: U2FsdGVkX19RmZfo9QkG0KTzR/WbUrx14Z1fEpNro0DoFP/N2M/It5jxW/DJytRpM+OvDhG/neMIABpcpNysVOzBRbMA+hL0GU/Oqxv2962q5Ql5cbCfIYyE0Ey+q9fqRQTksQ1wRhSvvT2Fbrq55A1EQt/yqmD3zmwiVaJxHUpISTWkWD0gbC4/iUnDqrPCmY5V43Uy0ZqjPt25EGMgSVa5L1HIbSXj4wO9/snTdoUA+SCQ2+82n1pryDtcW7mcm/xJP0VaDzkZSlfUiwdF4LMqhXYZnh+7XFsprvmwEfR5jqNPmOKxTvyf3TR07a5FoXHcIjmVVuGd4fcLxgrjyc/4sAYbLzCEuLi59ihNdTTC+qcob105UsG+ANcnLlFw3U2ao8FcN4cCM7vL7JDId5mPWHcjAoyolmfeSAVfk4eKFfsGdd7ynwgjhLEmODVqAxRRpzh6D6TI2NmqvuanbbDbfD4YWTv3RYcaYZLTase4k157x98ko4VX5pU6PLJyabQe3TgyH3GmE8Amb0tiOVm8VE87Ws+PXIjuP6tkqq209x01JYf7LvwGTAAjwotlWMFSvA2EfLn5W6YaQ8dqc4YcI2/qVhEd87oObBBH8RNlOglUaOX8zZq9j/fRuDc+/aoKFwrg6Ye+676Wt+XWnFb1JDDWgbo7g/XqGM/neqzJWUbUP9It9XRX1HGMYurSpdVbN7Xb2vpXIQkB8+Nkx+i2F/11YpyjDFEoMW5NAgKHYfmzE11EYvmqCgG35PMo4VarttVSeHUtZ1BN8FZ9sED0QPX+gKhO500EagUbQznmYHarDtADjFtch9ALgkTlNWW6i2X5BqTmMVjQHQjbeFOqDfK2cmt1OVzaiCm9PS7SjF0sOoJu2O466F9M7D2IBGQaUg81UXPHR36vxBJeWtSVdx8baWL1Klxr8QiVZAe8sTaHx5OjXQF8Q7387+YYzGcrO0d82o5mcpYRAhP JKW8oTyi Zlw8ovXobsHIOMDjsTPEJkK9RF5QOIEOV5z+jPIDMfmXCEDvBGzWeIVeZVRFN5HaT7u69qFhzE3FqS1NV9KFSgRbkk8BcWFS0keISAi0A5/3pINfjtO8n+VdJcyfjcFYrPfW8twhAIFzlQ4H5RadHMUTxk5zPxPNwwe2IXqSehIZbjkzI+UiVpsE0UaAyXl8WXG4T32ZvmYeLjqeawhEqP9nNZZKkYiiii3pl+sWflT1MeLP4zxWJRoQ9fGqrz2Xf56imsDLF/FtjGxuvrORpeaLqySfNusDugJF+NZq2fSeMLcxcXEIgvrzPzZW35HMNKqhZYX9cdplDFQ+MaZajHT78/UGqC7LWvtnNvfsBGIwrfc5uD0owYAbFct6Tm0UwVLPGz88j+pXxbg+ClWUvA8s9/elSULln5bV4IMq5X8lZtsyyDMkBTKZKM4gg3k6a46BIzZAP5PbDDxgcgloxXu8n+agnzUE6XxIgCEfuEuUqaNVopTIgcIoh620yKZDY68BiypWAfcPNNpmuBtJ16asVB0/v7UNQ2RUkfS8PwyvSeyLiInoreZjI6LNrudoWOZQpaIcA55cEf+joseNIhfOYSPcOpGh4zQMcoMIQGIbOBGxUSzyhn6GLmHpzGIhovaoZU3o7kEEWm7nVggquO+USpRrqimxh1ni4jnRUACqpgCPjLM0+rA0zLmdm8e/ayl8h6Ic0kIu+7es= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In workingset_refault(), we call mem_cgroup_flush_stats_ratelimited() to flush stats within an RCU read section and with sleeping disallowed. Move the call to mem_cgroup_flush_stats_ratelimited() above the RCU read section and allow sleeping to avoid unnecessarily performing a lot of work without sleeping. Since workingset_refault() is the only caller of mem_cgroup_flush_stats_ratelimited(), just make it call the non-atomic mem_cgroup_flush_stats(). Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Johannes Weiner --- mm/memcontrol.c | 12 ++++++------ mm/workingset.c | 4 ++-- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 57e8cbf701f3..0c0e74188e90 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -674,12 +674,6 @@ void mem_cgroup_flush_stats_atomic(void) __mem_cgroup_flush_stats_atomic(); } -void mem_cgroup_flush_stats_ratelimited(void) -{ - if (time_after64(jiffies_64, READ_ONCE(flush_next_time))) - mem_cgroup_flush_stats_atomic(); -} - /* non-atomic functions, only safe from sleepable contexts */ static void __mem_cgroup_flush_stats(void) { @@ -695,6 +689,12 @@ void mem_cgroup_flush_stats(void) __mem_cgroup_flush_stats(); } +void mem_cgroup_flush_stats_ratelimited(void) +{ + if (time_after64(jiffies_64, READ_ONCE(flush_next_time))) + mem_cgroup_flush_stats(); +} + static void flush_memcg_stats_dwork(struct work_struct *w) { __mem_cgroup_flush_stats(); diff --git a/mm/workingset.c b/mm/workingset.c index af862c6738c3..7d7ecc46521c 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -406,6 +406,8 @@ void workingset_refault(struct folio *folio, void *shadow) unpack_shadow(shadow, &memcgid, &pgdat, &eviction, &workingset); eviction <<= bucket_order; + /* Flush stats (and potentially sleep) before holding RCU read lock */ + mem_cgroup_flush_stats_ratelimited(); rcu_read_lock(); /* * Look up the memcg associated with the stored ID. It might @@ -461,8 +463,6 @@ void workingset_refault(struct folio *folio, void *shadow) lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); - - mem_cgroup_flush_stats_ratelimited(); /* * Compare the distance to the existing workingset size. We * don't activate pages that couldn't stay resident even if From patchwork Tue Mar 28 06:16:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A64A8C761A6 for ; Tue, 28 Mar 2023 06:16:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4685F280005; Tue, 28 Mar 2023 02:16:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43F31280001; Tue, 28 Mar 2023 02:16:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BAAC280005; Tue, 28 Mar 2023 02:16:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 18617280001 for ; Tue, 28 Mar 2023 02:16:59 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E9CF880495 for ; Tue, 28 Mar 2023 06:16:58 +0000 (UTC) X-FDA: 80617298916.08.F30F791 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf06.hostedemail.com (Postfix) with ESMTP id 31CD0180017 for ; Tue, 28 Mar 2023 06:16:56 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=BGZjnL6U; spf=pass (imf06.hostedemail.com: domain of 3WIYiZAoKCBcLBFELx49103BB381.zB985AHK-997Ixz7.BE3@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3WIYiZAoKCBcLBFELx49103BB381.zB985AHK-997Ixz7.BE3@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984217; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=x7IFy8THM3GBUAgd4d/GEKRMIpmDkqCRA5AxR/3TFCM=; b=XKGzvPT8miH9f2bmU7HW+REbntpyqDV8vm2B2DSd13Ilc0MSOPBTn57mkvcB9Hsx+ih4dA 63JUsXeHsxTbjvNZ19v6pxpwST2DfOAAj92x6ug1kwvZJjZvXKTSl8llDdx0z1O41QYpRG ivt9O5mUxSISUQitM4iSllbiXTMieEE= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=BGZjnL6U; spf=pass (imf06.hostedemail.com: domain of 3WIYiZAoKCBcLBFELx49103BB381.zB985AHK-997Ixz7.BE3@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3WIYiZAoKCBcLBFELx49103BB381.zB985AHK-997Ixz7.BE3@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984217; a=rsa-sha256; cv=none; b=Uw/6mStdSu2EoO7u5KY3ESCZFNbIQ5+39HXnEHoPS6L4GIN2BjJ2nyMx5vO/yp717/NyJM JelRJ4Xnq9hcAJ1H+J4I2ChFB/OGvC2n7R1PzPmELkjqgy0mC+Vtg2+Dk4zvjywQ5F4SQr 8prWJFjzALGqnQCIVb6RGViusBCxR8I= Received: by mail-pj1-f74.google.com with SMTP id nm11-20020a17090b19cb00b0023d0c90d851so2939752pjb.1 for ; Mon, 27 Mar 2023 23:16:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984216; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=x7IFy8THM3GBUAgd4d/GEKRMIpmDkqCRA5AxR/3TFCM=; b=BGZjnL6U27C3nu3nfjJxe/VV851RC/VP1GsLD606AgPd6FshXe6WNQk0I659Owa2wU gVJoC9Crt60CfuYM+ql+MgbEx44waD/ankuLiQjCR+6oc5mr8Ex9gHFcY/MTbuN2MESv WG+o2vmHU2skqPZDZWhEEvumwbMUFH8XB/2P7lCWIxY9j69P+XyQb7YmlC5lJf3dl83N R/+G9MzJdyv2aInyc3KBZTsCKswAeBzT3OW0sicqNfQnZRl6LhPxfTDtAjqxejZWrmX4 kekpZffHaXAsR+R6/xnXuXgj0ngR8GdA98eO9lk+Gv9Mx4rtmu3ZFp7UhRacTBWLo1ks cDtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984216; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=x7IFy8THM3GBUAgd4d/GEKRMIpmDkqCRA5AxR/3TFCM=; b=DkaOauf7BRSlMGR1dCgKhyWS0LDhXcnFDSLn55qpG4f75EaQC6N9M2oWNNxeXvjq39 5hxoNBXs5uamnMQ9I4/0wpJnjhUrIqyUvc+CNAyo/lp7uszgZ6kxt0bNUDyWsU5kb/bS qb1KZy9zDevLM2fWbA2j+5/9iqDOJLaxgAV/p3KAcroZh3rg5a8fAOuKqPy8mygo9BvV NsgKAR8zX3jvPBpKtwrE/LAQcJ/yqDggmAn7UH8pKJsc8lPpzk/AYhC3AY3p9WlygsOa gaHRNUEr3At4TyEumR5j6AWamDJ72/oebZC2iRAuc++5yevgEKj1kSl+FdgcYMF5/n95 qTgQ== X-Gm-Message-State: AAQBX9fr7+JjW0W++T9L1oqJ3bRqZHswzXT5n8ju10sdUDMZX5GVjAG6 jQFEuzjvpVdcBimExJJFbNsL9w3wD0YWJ8Kh X-Google-Smtp-Source: AKy350ZXKLWX99WV75Zn5bhPto8WUfX5S0Ha1cMIMK8Z1sFTdLjrJB9JMxzEwgWixGxokt8xM3tEMiwbJzx2bThN X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:c49:0:b0:502:fd71:d58c with SMTP id 9-20020a630c49000000b00502fd71d58cmr3919025pgm.9.1679984216134; Mon, 27 Mar 2023 23:16:56 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:37 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-9-yosryahmed@google.com> Subject: [PATCH v1 8/9] vmscan: memcg: sleep when flushing stats during reclaim From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 31CD0180017 X-Stat-Signature: zhodqpkxm66tc3rhh13wndmc1os1pya6 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1679984216-581932 X-HE-Meta: U2FsdGVkX1+1eSPj1y4BLQ29M3BC0TbHxmW7UlptEYUc72g8y0TPuT7GcbjRDcfS4YTSqmaKVq8x8X+jy39QD/TwvPYib0KaX51Qf0k7wBndW9yshKhVjriGH8xGGkPM77E6a5OZ+5mCeSRqMDw9/OwD/0SB9VmR/oURls2JMvq0NnU3hZR8gyaor5yoolkzu0JwpxH8kuMJ867kYU/ixJN+Rw8QFTu3AYT7RYRY9RyCM3Uw/KMQW19eazueps8C1ZpgpCCLTj9CMTt6swOdKCajAlmO4fQ8o8ZkExCwgZ2Cgi9zt9KAoz0dbTDR2uJsWiXRmPE4+snOsxWes00bgN0gE1/YPWSm4rdk7vBILmmqhiGo6nFdDW9rkI3T17P4aDqgi1bD2aOIwYZYv2XZj4PYNfdWaNRylsoLfCqLsk/gCkLNn3hitgJ19dC25peac3O4UP11G+wk0PbOkt3DkB+TJxDRzZTlICbPGujT6aAjfe2tM01067FV8q32b7sJd9tLmPgLzorDXOaazUzix+b+4hw6bCg75Z5ltPzj3uwQ/3KSPvH79bw9XqieTnexj+1cGHRVGS1HrTI54XM6nBtgRJ/fQOY88PCp4sDwGP6U28NrTZvoBOymN2kg0naz9DImV1wQ9mABARDYU/gkb/zrGsex/A6Zbp+mQ7JazGMQgB5ArlLDVtO0lkt+PQyOjPD7GuvFXpfddNAlutmpbZMX1mZ9GolkgggXwSM/VYTByO6FQ6vkarz3aGiWJ6dlL3Rjd8lLNrqiNY5BKsd9BDPev7T1hfBZ/500SANxNvJRlMM814TEbxeLWr7SBDRr/XIZ69hwiMNtC3N1L/VhgbS5WbA8k/1ZExUjyKgqkxPld3JLKtotfraqKWoSDT/YVdDI7CGntQKyM4tuF6lHfvmS0IEtkh1Yf4Y5ywUVv+V3Ja4lkH+tY6zLQpVmJPMe49c74J9zgQVJpLMfigy k/HTZwmG PAVgkCM4OiNSMIaSzUcwk7Cgn0e/wTOGDYnqrrQdhpAKqzVbNSAaa2dagX1NcdGCin6sZfqoF6ZHPb7Sj25UjWudk84X0IlwDh8rogIStyXtbiTeWGpJTlItUVN9tfMSigdIiPqN9OsiMt4bP5InCjNOt6qCKqRhqod1DZUO8Uv2JCYBLeUh+QtMmgAHbanhxEMP9CfP/DaexW/BJk8Plxdc4JItlzDuKJ6fhv4QMfDNVfC04o4x8BWPkTLAFr9/pqGTi73wYNnzDC7+c4sis9t2t/VZTzDJEgwluZRaQ5oxEDK47TxK1iMOJkE6Zi3fW/HmwJ92EEtHwLakfsUZ6f9TEV/vnJq+QPIyjHyjrEUm2SdLMXdD66keY6a0StDsoov/OuQC2vdQEZ5ReJUg2FKoIgHRR9puAuQ5MW5EYjimSqyOLOndpKagxI6iHmEqCiuyKtxufPMHP+mIDuDqEGpzKGfyW/jfT+vjIEyXr8FS4wFmAVUbQWe6pFZxtHo5Z43HY0M6PKvBDUh6jRfotSAT/04FePE616nDvCZZhVgmFIxh1sixNi52sv8FnRi5BftY1PEHFg5UtWHazM+8PG6J/LWqzFsmzUZdYdhWrV/OYMcFnqwq6rJB6aP+5UEiNSozx670H8zzmMWdsoaff4xqZ8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Memory reclaim is a sleepable context. Allow sleeping when flushing memcg stats to avoid unnecessarily performing a lot of work without sleeping. This can slow down reclaim code if flushing stats is taking too long, but there is already multiple cond_resched()'s in reclaim code. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Johannes Weiner --- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index a9511ccb936f..9c1c5e8b24b8 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2845,7 +2845,7 @@ static void prepare_scan_count(pg_data_t *pgdat, struct scan_control *sc) * Flush the memory cgroup stats, so that we read accurate per-memcg * lruvec stats for heuristics. */ - mem_cgroup_flush_stats_atomic(); + mem_cgroup_flush_stats(); /* * Determine the scan balance between anon and file LRUs. From patchwork Tue Mar 28 06:16:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13190562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 398FEC76196 for ; Tue, 28 Mar 2023 06:17:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4092280006; Tue, 28 Mar 2023 02:17:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF115280001; Tue, 28 Mar 2023 02:17:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BDFD0280006; Tue, 28 Mar 2023 02:17:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id AB3FF280001 for ; Tue, 28 Mar 2023 02:17:01 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 06CB9ABAA5 for ; Tue, 28 Mar 2023 06:17:01 +0000 (UTC) X-FDA: 80617299042.27.09B4FB7 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf13.hostedemail.com (Postfix) with ESMTP id 28A3C20003 for ; Tue, 28 Mar 2023 06:16:58 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=G3C3U8FV; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3WoYiZAoKCBkNDHGNz6B325DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3WoYiZAoKCBkNDHGNz6B325DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679984219; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vy8wz9BoJKucmcMBXvIGPOuITeThi///HtllaZbgtCo=; b=okbC1wTf0/TrLgl3F47IiWE5ZEmZrPLA5qnCYQw7qRRF8E9KbPrpDQeiu+a126RA7x+tJz svAcNUnqY+Vz3yqGidrDJRiFTL9oAc0zti6nECvSptPJPs7RRUzvzdpeh/RP2AkpclIoUR Dje7oClsAXjYAtjAMJAxbjfcdTrY9T4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=G3C3U8FV; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3WoYiZAoKCBkNDHGNz6B325DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3WoYiZAoKCBkNDHGNz6B325DD5A3.1DBA7CJM-BB9Kz19.DG5@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679984219; a=rsa-sha256; cv=none; b=yI6x1lWc0MUF4OtvaQwLNGkNVfyeEBmECisDRXz+PDnp5I4t4o2WHEQA5bpGp9ayTikkW+ F6mHVxhWsFlE4twMVPF3b2CJTkiJphkQwBnN80HdKQ7tdk+BlY77X6f313XFjM3tXnUnsJ p+fOBVUJ9lrAlgUFxBK/7K9Bd3ZGQZw= Received: by mail-pl1-f202.google.com with SMTP id k17-20020a170902d59100b0019abcf45d75so6954802plh.8 for ; Mon, 27 Mar 2023 23:16:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679984218; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Vy8wz9BoJKucmcMBXvIGPOuITeThi///HtllaZbgtCo=; b=G3C3U8FV5xO4vsplclhU9+4PkBH686vIIIrQFFquICu8//oGn5xL/Tmt4VNVE1HnMt hORJK4mB81itHhGs1+plTWPpHn6PXb4UvAXp2In5M38sRtIYXUydb86gW9UCcwU8NUMK AKSJULzbSmJ/Zh8xR8z20CA4EuuFhupmK6yUb7XM0dWX+orGZyC4roBLz+bFSVl/SrGz jDFcau8INkey+e5jkuXnym7vXagjhy1slym0NeO+iUEtBeJGIjBlnt29eYQhaZCdCsb9 TItPIolnMlslNIl39r9v4yrb91hLywelng+1GUeMlXmGbk+u0V1YHrp34uG87/cMCuf4 DKLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679984218; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Vy8wz9BoJKucmcMBXvIGPOuITeThi///HtllaZbgtCo=; b=6FPmaP4y7pD0GqsdDl9D6B0/bc9irJanv3ygAmFU26IeHTC2bpXqsS61KUPEhyP+Pm H95q1xEq1OaiEeMo6qxG9zJDy1IZ9ZGUzSQFeYklKG7xcO8s/efQAPBXrVmA1KClLKq9 4nPUZ2FYz4kQq0zEUPhzB+1vXrq+IU9jv1uuxQ8tccvx2QpU9fdq73cIFYv/LpJ/ywlb kHSTuxdv/g6BXv8r/80YXRoS8RYzn9U9TI25gxTou8/4uk4905uFlb9m4AKoAdCKYSEx NUjAaCJY0+/YJ7AN6/VMCYamScfcAmlyuGR7XXrPQPEXPh7N1Zyblt8JVD1B8sJ4Phqp 4+fA== X-Gm-Message-State: AAQBX9cC4V2OoHHA6SFgW8YYl8R2B5shWgxcK8y93TJ09+wgqE7FuKCD s5lIlU4c0VqU5YrYkqvTipRkWCcKnPc+myPZ X-Google-Smtp-Source: AKy350aVn7i51pwbUpBHdDCfopSzB0y7+YNrEoZI1+8jV5mnjDSATrQRcelVSU3ikfIyno7DP54WO5q/0DmrmRgv X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:1781:b0:593:fcfb:208b with SMTP id s1-20020a056a00178100b00593fcfb208bmr7364360pfg.3.1679984218020; Mon, 27 Mar 2023 23:16:58 -0700 (PDT) Date: Tue, 28 Mar 2023 06:16:38 +0000 In-Reply-To: <20230328061638.203420-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230328061638.203420-10-yosryahmed@google.com> Subject: [PATCH v1 9/9] memcg: do not modify rstat tree for zero updates From: Yosry Ahmed To: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , " =?utf-8?q?Michal_Koutn=C3=BD?= " Cc: Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: 28A3C20003 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: ps91dhh8jpa7fxnxytq78tbq6gawkkig X-HE-Tag: 1679984218-65429 X-HE-Meta: U2FsdGVkX1+Qy47AqLBPoXMnWoE35RZsWXzXpk+1hgXLT0Vynm8elO4itpvNxPC2xK656Hv1hQx7lPWnDIDrJGNlzYfr/8dEGU3J+OWNie8Xk3oHomDMDpyZgt5M2PWn9wJR4yVf4ZHqPhfT0T2h+KDzMeo2AfK0dvyil9JSy2BKjBfsRU3aYB+v3QOuSGPC6eZfGly6MZNwS1nX+CuIxRv+QkrMPWkz+tAEb/ZqNvlWR1npUS6NqmBWbCbwPlBMdmlLuKc8E1+n87O1YLk1g9WEs9xLU7gjd98oBCK73C2la0CdmhS3t23zrk5dpFsxm/8A5AmI/j45gCsWiIV/q7XzYqekLZzwOSpQb6DckoGfInT2m1dx8XklpHK8HEzS6pr0oxqKa3W1CqnnsP+t4Kwxk29xMYldmh3HYdL84XpWb+wSyWtsAB8bWH39E4Xi5GhRY88QwryT3Mm0Ac33yPeVXx7RIVzJK1B3or/Gz/QgT2r/iLjjaUGM+qvB2ZGGkmKtLMnsVdzyG/R+5bdxoy6XWx2H+nFoXZ+FrnOYPUt5XqvH7xhl8kn9aXxnSXOkYd+FMkY3UYeH1g59rxxggHy7li+VjqMUwPu9QwqTB1SDTND4CUcIputooalc+RHy6g4/qV9qGADvNH1tp39seiPRmT59W2QSxLeogW99/m1gUco2EoQF5dJto9XVsx7mWtSuMrJ6keUlEAtN5eHW4smnq4VH6QZbNlpNsACEXMY4EuY/Wbk0VIsNhOg8mapj+oJduMJtMSjgReLwiIfZIrG2W1ua56q8I3Vh/A+PKDeB8cEketF6BKokOn7e3wIGVUM0ArpL9oIKH8E5K+uxDmCAuERFX8XKfqWNp+CPR3vjpDSLj8SRDuyhiuv7+GF+47SSXtzWZMDD8NHhFVkmY9SYw3nISGbeT/sg74+RSeJ6RkX0m0Ke2JvGy08pXh72PRBHoXfDJHK0KztAf4f 1rfbAScl HMLvEZ0/nB93XypvvSZtIA3bzpqWLY10xy76ndm7QEJjTyQWd59pqpZ1Htoai1V2sY2z4/FTMYt54/32aLGbCE+sZWHr3+2ImmAIJ+OW/2IJGf0vr41/NxD1zJq/i6oo9q+/0RRbxARr3k/iJCI/85kw8C76ds/da95IC+mYJotnU7J5w7vgXVYbIzmMSO4Xkr8xg8DotFpppKPztTUAoB1lYO0fbd2vArWKmc91DJFs1VCIrjvr7ZAuWGRlsbqDP3DpF31nyLnfo9TewM8KNCUcUasp8934AIVX35jlBFmnu5dE8VMGhPFgXxwXSf5PhCWvMcRi1b8RElgx6zpnvglF2jEoTMbZbQCW1y+bYnilVBgZHZI5uBHt5yGUZDHWFZjJ5U3biAoF2qlmbCbKcJULC/PSvoL92m9r8pZPqPy3KQ09d53LmzoJ0ZRzoPKbIEzS6E2rAbRZzPZmNXqNlSHch7BN7Ov6GcaP2bcJ4CPs+NpsbKMLLqCede3yVnt3qa6pw14wdXpXTRmKXzwskPHH73DPdk9g1g7yg7o2CU01jWPNEAMlycjC2fVbV8FtqFAA+uPaAysYkQIIWDkolqWGgeLZwk9T4lCWorkVMbpP5/y+/ugzZWwehl/trIQP/nKe/l7yLYNsRO72y923kG4Z5/Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In some situations, we may end up calling memcg_rstat_updated() with a value of 0, which means the stat was not actually updated. An example is if we fail to reclaim any pages in shrink_folio_list(). Do not add the cgroup to the rstat updated tree in this case, to avoid unnecessarily flushing it. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt Acked-by: Johannes Weiner --- mm/memcontrol.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 0c0e74188e90..828e670e721a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -618,6 +618,9 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) { unsigned int x; + if (!val) + return; + cgroup_rstat_updated(memcg->css.cgroup, smp_processor_id()); x = __this_cpu_add_return(stats_updates, abs(val));