From patchwork Wed Jul 24 19:02:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13741272 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C91FC3DA61 for ; Wed, 24 Jul 2024 19:02:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8A22F6B0083; Wed, 24 Jul 2024 15:02:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 853496B0088; Wed, 24 Jul 2024 15:02:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F4A86B0089; Wed, 24 Jul 2024 15:02:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 4EBE96B0083 for ; Wed, 24 Jul 2024 15:02:24 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 03A451C1E74 for ; Wed, 24 Jul 2024 19:02:23 +0000 (UTC) X-FDA: 82375567008.08.2701A23 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf06.hostedemail.com (Postfix) with ESMTP id 33ED718002C for ; Wed, 24 Jul 2024 19:02:21 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Ey8W4nYk; spf=pass (imf06.hostedemail.com: domain of 3u0-hZggKCBwCAFK6Q9G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3u0-hZggKCBwCAFK6Q9G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721847718; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=iLDZhQ0id4uDXJarhBRTOmxWLKOk3mwU62F8HwEfdow=; b=vFroBQxNYcyjRkWoV48667F8l5osJESFrxobE2RyCwu68FQK0VsgSzGKind1PMFKdAUCIj ER5NKi99jHAJBiqUebQ+US523VyPMA0bfd17jB70ByKArMc5JfDfw3OM0G1oyhRsNbpID2 dGtT7RACGK2mDta+wbzdU3lSe43Bex4= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Ey8W4nYk; spf=pass (imf06.hostedemail.com: domain of 3u0-hZggKCBwCAFK6Q9G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3u0-hZggKCBwCAFK6Q9G8GG8D6.4GEDAFMP-EECN24C.GJ8@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721847718; a=rsa-sha256; cv=none; b=O+sx1sZXPdAdPVnS5a3hftXXztJXFpGhwGKDs3BXPT7VNZ0eYjxQpYZX5thvKqXSumkKbH hxxtQ6J3Tc7gcIu9sbSKjEWQnL0zOHXD0+5l6qo/2G3+Oif1mhgPRyONI6LRnDp2GNy6Gg FngQZO1aZml/9yoPVQZph9c7kPgBQB4= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7a2a04c79b6so1202542a12.0 for ; Wed, 24 Jul 2024 12:02:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721847741; x=1722452541; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=iLDZhQ0id4uDXJarhBRTOmxWLKOk3mwU62F8HwEfdow=; b=Ey8W4nYkXtJp0EkUPnG1PmUKGFiCukMrhWzmJnTdupgCu1tAcVE5cVMdFqXRgBA517 YcgLDOtzDN9mWGkDJt8KKUCnjD/p1j1D3/0qmQORZGV4CzkVNwMM3SG85B/9B7Ht/DZQ nN+1zq2sED7vsMKG2Qgl7cp8BA1xVSsSVJyViP5kG/OMq8UbtnhQgniumktq5/xSGk29 6CXP+0YbHjKK2ZM4WVdr1GtLLjmbrjdUr13PqMDxH594v27Tf8NBsR8qlgKuiKVbW5Au aEV1CEwlnmRCx7tdf47qPTWnukG4i0erhFyfbFbtmmh/FxOeaPSkDiuuEcx43MrnlInB 3ckw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721847741; x=1722452541; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=iLDZhQ0id4uDXJarhBRTOmxWLKOk3mwU62F8HwEfdow=; b=bVZgKyl3vxE/GG+rSIFzc3CykcRQ3EaKuNzsBoUDzhiq4YFQ1AJ6o/+4baT7s9Zij4 Qmue1sulXnzWgCyWN2fptKs9Qy1FTtSaNsIjEmYUGJd7eAzyctcZoGw8K8SzYcT7wkaS 4BL3IWF+5uWwXE2IMz/p1IaDzhL6M51OhzuS8g9futTDYPepPbIphIe7E1veQpFLWtYj XcekesjfuwEoNn2QLAEkOxK7VHKa13bBx56IUGtq8A+Wlg3dF5NMy9Bn/OfTuziNDwaS n/kJ0eyQJ4ytGhXJc97UDa7fnV5l5fGuOREf7qknXSOfhlitgNtMEkSElgArgmV8QEba /yFg== X-Gm-Message-State: AOJu0YzcPUwVd8Gu/ehE4d4oM4SCAmRQlsp+8wte5Gvc0acb39JUcows xaJJil2QiNe5N6zbdC/NsCoiSU3rbFSH3Tmg46aImwBBJnl15OkPtE+5ayg+VSuNHOommyTCRqj /JJYS8QZixQ== X-Google-Smtp-Source: AGHT+IE2wliyUgg36eRbrdQG0eW2t+LBhnewXuB2szeGvK9iDIHirGHs2LzRSu2WcnyoQhiliA2iavEfWfM/Kg== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a17:90a:6b0a:b0:2c8:632:7efe with SMTP id 98e67ed59e1d1-2cdb9661b65mr32367a91.4.1721847739514; Wed, 24 Jul 2024 12:02:19 -0700 (PDT) Date: Wed, 24 Jul 2024 19:02:11 +0000 In-Reply-To: <20240724190214.1108049-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240724190214.1108049-1-kinseyho@google.com> X-Mailer: git-send-email 2.45.2.1089.g2a221341d9-goog Message-ID: <20240724190214.1108049-2-kinseyho@google.com> Subject: [PATCH mm-unstable v1 1/4] mm: don't hold css->refcnt during traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Kinsey Ho X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 33ED718002C X-Stat-Signature: ysrng7hyeb96yamoxbwa61tiz188wxqe X-Rspam-User: X-HE-Tag: 1721847741-204683 X-HE-Meta: U2FsdGVkX19M0ournU9p1KvkJFTks3x5id9JT1YqZoxvuAveGyT8FD+SaG+zO8Q9VWj1wUsweRCnHDjVqT971AnQEJqsGsRESzRTVGk4Hqq7EM04zQ52xUOlH8dNTWkMancQGU9GqmhVlMIdlenZZjLei5tXJWlD/w2lUJv4eSiYjMM6QkAV7kbTrl00ltvvDQZuoXzlxQs7RncwZ190yWX7tAW8Id0u6uXrcUrGhaXRkTgBZOjMReLxzGtqyYNfFv/R++NP4nktbQpKS4Hk65FEqGKHs3RMKKQ5i/CD9jyIPnzaBeHqRrTMWF5ATU2cSU2e1/m4+Fgv79qYffZTw1BgY+sKG4mcnEsQuL1wfGNOHymkhInJza8T6uku2sZGljL1CBBoeIp/fl3mMAKQWUI1tEd+jiazDvu2MbsMkVnXySIb28qBCEqt4EP7LfhItVBNTF64mCJW0rzMvLSdbRz5CpmvAvpffvv4AVfxvjSyfDpCi5dGOpxWDNLD9R3C05yDGaysyrgHnM7kJ5PHlhFu7QlChZUFp3S8OVRGWtgUsv38kHReXkmdOfLCljc+XcZFRlVliIMhgrzhPzNhmJGQiKB93mgkog9i5QlULNFxLT/BfELfJDUvoLERoqq3mCRBp5Y/2qWCKEC4si4XJdyw7mdr+XvebU//8SAgoDwgcCSDfQ6xdW8Z6ij9SlWM6aA8F5W73n2W1mABmegje1uBRjyFO2t2Gwj8/vJsoxjPb4EopwR4jQVH13UfCxkyJ/BtObpeJ6FgUOc0l3Se/Rv66eeJyvEvgI0/FuFEznZgjKqEauQUtd1OvNPdIfsz+/f5C0f1t/iA69H1VGW9zDAZ++PSdtSyxAiWDNr6yzdPD9Q0TV8CyIXdrOTcO319+IzfDo4zBKLqT89xwB9dC+5cRQ21Chmyp2nld4h6AFLS1YKzEWM2xkqB8t2PRtbhF/MBaWAXOREgbuV1+1X mLMA1u3v UDYZ/Db2L1q3nloMl/4uyPxvWt8gITE9xK6gypGhc7d/nNS6MvwdLZuZO/H5fnjvEXRGhyGG5YgGYNgw+pv9ReVkpcHvihp3wZpHu9IlMHcIDGqcanX9ld28n3gbzJDxIGgUEFIR+neEwKrnUr408HncgXWUrJsrDanrAHdVWEq3hsE0gwTsX+GgYnB5IRgsyRg303+c2TN1n76oPvSpK2TzOTBPR6IzZMuQoj0hssrVgmpFvdDTqsdQ2EwEIRnDxUSVjG4376ExY4TER7WdZlXE0ZB8IyFxFBRygyV1hpzQBWCCb0mtTTIFj+joVS1lRMG+B37qWMkU0PK4zYf/ayFgncR4qKscfOD0ceZjFLPDyGa8iUYFtNM+8oec5tM+fCb+8bNK7eMeEjQKrn3DKPUjL8S+WTRe1lV5gCwoSllxCYo48YWKmdkyrrzgNid9osomF9JuwGrPINHJTqcjdW5BJQah87NZhorlOdEmmGkKT6+aQaBgKU+ahZMNsX8ZxnLg3yLHVKyiOZ3NPnHFct7cIOQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000009, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To obtain the pointer to the saved memcg position, mem_cgroup_iter() currently holds css->refcnt during memcg traversal only to put css->refcnt at the end of the routine. This isn't necessary as an rcu_read_lock is already held throughout the function. Remove css->refcnt usage during traversal by leveraging RCU. Signed-off-by: Kinsey Ho --- include/linux/memcontrol.h | 2 +- mm/memcontrol.c | 18 +----------------- 2 files changed, 2 insertions(+), 18 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 7e2eb091049a..4cbab85e2e56 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -75,7 +75,7 @@ struct lruvec_stats_percpu; struct lruvec_stats; struct mem_cgroup_reclaim_iter { - struct mem_cgroup *position; + struct mem_cgroup __rcu *position; /* scan generation, increased every round-trip */ unsigned int generation; }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 960371788687..062bfeee799c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1019,20 +1019,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, else if (reclaim->generation != iter->generation) goto out_unlock; - while (1) { - pos = READ_ONCE(iter->position); - if (!pos || css_tryget(&pos->css)) - break; - /* - * css reference reached zero, so iter->position will - * be cleared by ->css_released. However, we should not - * rely on this happening soon, because ->css_released - * is called from a work queue, and by busy-waiting we - * might block it. So we clear iter->position right - * away. - */ - (void)cmpxchg(&iter->position, pos, NULL); - } + pos = rcu_dereference(iter->position); } else if (prev) { pos = prev; } @@ -1073,9 +1060,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (pos) - css_put(&pos->css); - if (!memcg) iter->generation++; } From patchwork Wed Jul 24 19:02:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13741273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84950C3DA63 for ; Wed, 24 Jul 2024 19:02:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 177AA6B0088; Wed, 24 Jul 2024 15:02:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D9E76B0089; Wed, 24 Jul 2024 15:02:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EE35D6B008A; Wed, 24 Jul 2024 15:02:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C6B7E6B0088 for ; Wed, 24 Jul 2024 15:02:26 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7D39EA4487 for ; Wed, 24 Jul 2024 19:02:26 +0000 (UTC) X-FDA: 82375567092.16.69621BB Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf30.hostedemail.com (Postfix) with ESMTP id 187478000E for ; Wed, 24 Jul 2024 19:02:23 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=El0g2WUQ; spf=pass (imf30.hostedemail.com: domain of 3vk-hZggKCB8FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3vk-hZggKCB8FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721847720; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AmGz1OUSAYzLu3Mkr9WfTC/ktWFGonkMAhMj+nbyydo=; b=kXAEA5KkgJxpZ39RiIJbY3BzSY+Rc68Y6tm80+Ga49zQUsUFQGXiQLc8vk+rS5j2KNQrx0 YUr4o0JJl7PGei8ydmkk9WoyjLPUHG1IkCvqfxgoR4TzL8t40UQJvKFi0sOsdHrih7wgLJ GFuYQjkGvY/yf+9WfuTWXttomwkX2Io= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=El0g2WUQ; spf=pass (imf30.hostedemail.com: domain of 3vk-hZggKCB8FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3vk-hZggKCB8FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721847720; a=rsa-sha256; cv=none; b=XdcMgL41iLyz7ua0xJY/aYMZGTWunn3Cr+dq593T42sPS2YJw5pIv42XADffUcHN2BolbZ lP/CJMaJYowABccloMQWA17KCtNKnweNMv32adLpHcKYijKfO0UTwjN5Ifi8Gu6IDfQjkX pxx/cr0BhNZVHHMjTTd0BJxjz2pRON4= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7a994e332a8so102090a12.3 for ; Wed, 24 Jul 2024 12:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721847743; x=1722452543; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AmGz1OUSAYzLu3Mkr9WfTC/ktWFGonkMAhMj+nbyydo=; b=El0g2WUQJWUcPlZ/36alMQmUqxM0ckAcXzZbAIZ+se3ILxkSkGEi4Vf+pYYIlpVkKo H4XOrx37WAGRHBmBN+OycWCQkD/QtN9BtlonjhYfeNBTv01fakCBiLul0doicVqR0QH+ Y04QIr8PgUC4YIpfijVKL/stQJYeLwg7JcC+1LPkGjrrlQkj6+CEyWzIfxYWz0KR9spf oKsafRquzus5pAu+UcGVWcr5QXsg0q5sta5wyQiG8rtkjZS0nVKPm1uP3AXu28zyndOh ZZ9RlSOqdb9aaGmqkcZDcKG/nUHFUIaIZkwByd6I3LBTb2e51NgWkZkAh2mpQKbj7WjP u5zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721847743; x=1722452543; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AmGz1OUSAYzLu3Mkr9WfTC/ktWFGonkMAhMj+nbyydo=; b=kAHp5iszNjxi5hxNUPTcorfDVCCH+5be7SQfsQ7fxTBJj+14rq2vtvAIgHpSkMoaWW AuH3cYQkRfxs7rDYLmd+Fe8gWkck1DxheEzTLRoxquUL0f8MTaxJSIdV2IiGUTkN/wce siRBBT5NptGNlNZgqBUrKxqC4bDC9fD47qkgSXUhoKs+e1HiiqfLxezZzFhdGXpH/2p9 4JcdE0JpV/sShLiThGhVYPhQNQy5vSnaj+S/p/eFFveSkiWuvUxrJE9ISyi89CtlIgY/ 0CTrBaBvj22KKCl+h8uEGlWJ8J0VdBHN5SHvwxlS9ELTY1Sbvq3L79TTlxnIXf2JS2gb Fz/w== X-Gm-Message-State: AOJu0YzgIXrTJvSIxz0h5q40CdX/GYpA2tO6KuEGmY/ezKnd/LWxMR1U Xuh8Bv0FFdFLzPfHNjSuemFbSitXM86eoMEzuuwE32K1uB52/DD8aDVLacPexc6v0McwvC35wWa s2un2e0Vhng== X-Google-Smtp-Source: AGHT+IGLvyDW6X3i6jqfX6m7Tz2BzQtjRTiuAHBzI1w6qDTToawljkbGNllzBAuC3KsBrh2qsLZDEguaiP99GA== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a17:903:1105:b0:1fb:27bd:82e2 with SMTP id d9443c01a7336-1fed38ccc31mr9895ad.8.1721847742363; Wed, 24 Jul 2024 12:02:22 -0700 (PDT) Date: Wed, 24 Jul 2024 19:02:12 +0000 In-Reply-To: <20240724190214.1108049-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240724190214.1108049-1-kinseyho@google.com> X-Mailer: git-send-email 2.45.2.1089.g2a221341d9-goog Message-ID: <20240724190214.1108049-3-kinseyho@google.com> Subject: [PATCH mm-unstable v1 2/4] mm: increment gen # before restarting traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Kinsey Ho X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 187478000E X-Stat-Signature: bi3pc1oqkqoumr17eanc6fyrur3spd6u X-HE-Tag: 1721847743-529909 X-HE-Meta: U2FsdGVkX1/CbzMuO7XxYIPoc6k6SBTULN5uQU44la1ISw2O2wuZbTp0qbqY341T+fD0XDtivjMFzBUZgFAScqIhCoSPsTbvSrnZZo65c4P50LGFaKjA3bLDlMC0ewWDQMXvE9coLQLbU/u4MXCMOsa3rAyFyXmeO3gpimYcS5IxqKMx8cpxzB3Cy2gEpCRlZexyZJF/YtwqX/BBH4ZaEFUBu+O9wzRa1vTnPWiMWuUFUpGlvRsNdZvJ6g5EwQxoJhAchCiU8kndG8o57suQBs2pN1D2GVJnfv4lemV+uiFw1J8XHE9HVQviiCmHFxAPG+b4flZ8QuCFvrrcbHZ0xE4w77fbCHz+qQB3sjIVLhbj+59PYc2d9tFvxkxAlJ55D1iq/wRfK3gcb0rcsfv3OzXMbwdWKM+T4utVQ19UhAUFzuwLJ/HO7H5KmgV4i7qSzH1Rbh1IdpyclkZClH3RH1wJqqV7arZr4LhBrbUxsBrzIc0O790/uO41DG8y/E0xK0rjPANIRQeMCChg8BHtLaGAjMpdSNSjcCaMvY4TGR/ijvjBSy7hS18oUqGJyrvL/3nzCdC7cXyDKce/qXK1c1jDlSejZefTdTBcQVFuZ6WutiaUpxyCJw265nzl2XbFK8v1mhTuUn/4BxxbjjgZp7t0tSm9IEn7ykdSMjab338dvrj2SU7IHmsyUPWfLi9oRNqIqhz1BHU+CCkUk47BkkIOz052ZqWXeNRMuD3fw6Qx/VPFMzChRLXKxVW+LFv+5CppFBZaFhX60iMC+eAeCaWJP0V7MSqs9dad0MxsEiZsudjRGI8S33sZ2EuKQfc1jKHK1zonPn5JHlVPLBrLRI4A7URJ2xRTCNOzY9fBmYQuh9qAw9z6C1xcnX6OJhem87zzBGcNvma/ihKosoFZhuR5lueC3n3+p0ANrayG9z35L3P7FQjygocABSFFgWRv5HqnERunxM2S1L4SCzh pLCQlBVE AVEjAB3jkyaPb8O9OHxe79j4yLoMdL9/bMlmBnxOhfSUayEN71S1Aq3qtansrE/3Fwhfs7P4ykIWdME7ef7L2Cy4N52IEFobicbPb2zAUpKTXDa8guKxaPrdETJxCkar0Dp5F9vbY48rNAj+2hCME+nxtD4GSipDEUedYI64ZaifYeZX5xT/MhDuSUs5o4QOEHlqc/lxzTFOJZ2M8SFiwuI7kY1LLznYTBrjZTVVusD9qO95vV1ZwOJL9YPIIrXMzGnOrokZj71JUKfMFcO2QLexf/Pq2PW/7djuSMdx5bTuNyTZXYmbQxB3VzfmPbJ3biGe8E4qNOoTbpdifSwf3jCJFIL7XlrmtTWqslGyBHI5e7ax9vx1HL3uc86qOZBUbMHBDLph1n0uBCIqw/TTG4S7BzAEnUVviYqb3XqmMZHBkaB0YQA+5iunooVlGhLQPhlM+PW1AcPhxxPVibivnTp8a7DEdY6oCnSa+fRYkAvRobxI0FqMCkE0WG4X4hP0wZ0ZIg/X6Q+Q727FGJk5XIP24Xg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generation number in struct mem_cgroup_reclaim_iter should be incremented on every round-trip. Currently, it is possible for a concurrent reclaimer to jump in at the end of the hierarchy, causing a traversal restart (resetting the iteration position) without incrementing the generation number. Move the traversal restart such that the generation number is incremented before the restart. Signed-off-by: Kinsey Ho Acked-by: Johannes Weiner --- mm/memcontrol.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 062bfeee799c..f672bc47c6b5 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1003,7 +1003,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, root = root_mem_cgroup; rcu_read_lock(); - +restart: if (reclaim) { struct mem_cgroup_per_node *mz; @@ -1030,14 +1030,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, for (;;) { css = css_next_descendant_pre(css, &root->css); if (!css) { - /* - * Reclaimers share the hierarchy walk, and a - * new one might jump in right at the end of - * the hierarchy - make sure they see at least - * one group and restart from the beginning. - */ - if (!prev) - continue; break; } @@ -1060,8 +1052,18 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (!memcg) + if (!memcg) { iter->generation++; + + /* + * Reclaimers share the hierarchy walk, and a + * new one might jump in right at the end of + * the hierarchy - make sure they see at least + * one group and restart from the beginning. + */ + if (!prev) + goto restart; + } } out_unlock: From patchwork Wed Jul 24 19:02:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13741274 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07471C3DA61 for ; Wed, 24 Jul 2024 19:02:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E9186B008A; Wed, 24 Jul 2024 15:02:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 371756B008C; Wed, 24 Jul 2024 15:02:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1777D6B0092; Wed, 24 Jul 2024 15:02:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id EB3D66B008A for ; Wed, 24 Jul 2024 15:02:27 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9C2E6807A6 for ; Wed, 24 Jul 2024 19:02:27 +0000 (UTC) X-FDA: 82375567134.05.93C4671 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf25.hostedemail.com (Postfix) with ESMTP id BB2D4A0027 for ; Wed, 24 Jul 2024 19:02:25 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="op/YRiLR"; spf=pass (imf25.hostedemail.com: domain of 3wE-hZggKCCEHFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3wE-hZggKCCEHFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721847721; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/TQiG1UXUGUFW+lVBdf0Q/W1aoqYwGoLbWZS7cahHzw=; b=G6zu1MGTqGSXYWdJMrE547iGq/zeJg5xeHUM18eX2007zhW/kWaoJv0s5fVDgzRxX9Qtvt /WdgqM3XMiPst4cUa/HSQlWN+syqdLD+A+oRaFUeIDBxwqiB78LmTZKOdfUlAL9aOmlSgq CQhriZmwi+qtSVH+BpTDF0KlWk7JQmQ= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="op/YRiLR"; spf=pass (imf25.hostedemail.com: domain of 3wE-hZggKCCEHFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3wE-hZggKCCEHFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721847721; a=rsa-sha256; cv=none; b=3hYZmNVML/3rSPVnepBIcTFkAQ85LaConHHGwAL4gt6gHZi7OqemrKGR4U2dg0BvaaVzfZ 0dCjDxECl0IJ9/zY3qoJd/1jgksqJXr/wx15kJ6DFetxGAn1RvmpOXsCCMVkunUgeKAVql MC4/IZ5Pj1jsW4Ej6W67oC1eGjno4Pk= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-1fc5e651bcdso815605ad.3 for ; Wed, 24 Jul 2024 12:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721847744; x=1722452544; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/TQiG1UXUGUFW+lVBdf0Q/W1aoqYwGoLbWZS7cahHzw=; b=op/YRiLRGAahUToKNvxkZDLgwLaxhndYx6THbByF5Wc64YglkEq2p1TmnvnUGFPFOa rUACb5tHV6xXlztiPkQOdisBw8jeRRe/wTolqe59O2qI0R9osyVEW1zi00dS/UasbSyB CgujoCnfiyh7Sft9bDgvtENWH56R5qJhB6grQoK/v9liHyNpijhtVXp5I/hIh8+mlOgk 9V6s7L9giOG/r8lb+ntgCSqC4hw23g6/MX3ZKRhDSR/EQaLBVR6yHDB++IQNkqk2qeM+ 7XtcFIoalQ/6eFZYuQ4jFf0KCW+hJgSA487nabefPQK02BSRV83E6W0Xkn74Pypy0UvU cilw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721847744; x=1722452544; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/TQiG1UXUGUFW+lVBdf0Q/W1aoqYwGoLbWZS7cahHzw=; b=pA9f63Ji8Y2dmya123T6sfufdP7VwYfn+rvCsy2zloelDGMT5SIqZgnrYNngVzO1Fc dXsWuBQa2Uw4fDBOnxNLofk1IJePESZzYMAUXuiRVzl4ovK7V1nyU1/8cxhkxx+zIjdt BdFoXbPKGxn0jWQx6tXkhoM01W81gdYjXKzPEoAx5Zuaa2fAvlLjsu9gaL8EEHw/ytMp qWa+ez57c/nCBLDjhQX/NfuW4EE1Ho+PuWCxwZYm0jNzP4X5IL3tQL9Rkr4TFwCOXMGn GE+4Gbof5SAC2Ds8on+k/KcuDRrFbONKyhBW8LGLrhfZn74mxC3Lb9SwHEQ4vyJpllZ0 xkXA== X-Gm-Message-State: AOJu0YxMDoiERu9ZkHwS1JFTdbsdfHL1xuMjLN3T3ZrSYEw+EGdjBBxD 9LQh+Cx/cIMq/3bEOIdafNkvejEt11RkfFXMSwQ/9vfGQOQ5zhExhQRuu2I43hlLzJQ4+oiaS3N D2BM+KgNRRQ== X-Google-Smtp-Source: AGHT+IE+tnQU31Bbj4dJVy+3PQYbdu9GD4nWCJk68x0wq+h9pms1Ho5g0zih9cGcyJ9eyRiAWRuy7HbM8eN7gg== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a17:903:24d:b0:1fc:369b:c1b5 with SMTP id d9443c01a7336-1fed3531a98mr17415ad.5.1721847744302; Wed, 24 Jul 2024 12:02:24 -0700 (PDT) Date: Wed, 24 Jul 2024 19:02:13 +0000 In-Reply-To: <20240724190214.1108049-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240724190214.1108049-1-kinseyho@google.com> X-Mailer: git-send-email 2.45.2.1089.g2a221341d9-goog Message-ID: <20240724190214.1108049-4-kinseyho@google.com> Subject: [PATCH mm-unstable v1 3/4] mm: restart if multiple traversals raced From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Kinsey Ho X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: BB2D4A0027 X-Stat-Signature: yw7tkptro46hr5cfejng1tb6e534ig1a X-Rspam-User: X-HE-Tag: 1721847745-363923 X-HE-Meta: U2FsdGVkX1/r3BNMJNN08emhNkHKXNX1EnOoGWG6IMYY8fRfPPZEv0MZv7wowh0i3PE6HTRmCRt+xj28OAuz/DIhVIdF5ymjX0sMsOatGh8f8wz6C4cx14lUgscUyB831esi4hHqTpvBIh4hiK8OkqO66Idj43+0UrzeUumTZTac3X4p/2B+BvSEBY1E+vg8wdK/etAcEUGZCQIAwLyICAIBvw8lEb2smRffOTz2PurWWC2q8Vt9laqxzyVTEGy+3WYSL5o1ZYGf1n9JsZLVFGOd8lJCZJiuXfjMs6EMVTV5knRkp5T9Dqz41Wl5Cptxnlf2z1Av5yHHvb8vUK+4SqLN5ucd7nvUlPs1+eQSl5TbFHP9QhlNuZdZrZW9fiXeb1vIqon6DvVQtBi4MQB/aL3UOciKUauJD0qw6IORzTlotpascFJY+3LgzzRsVWntrfxB2twT7qzX9igXka7V1Zt3RzK+U4qdCY0aqBPIewErL69+aFTEzca5mxBkajXwi6vVMCebiD8ZMEXeGRGjJtT7itEHzEX1q3QP4Bb0D99as/K8uVyCzem8BwLR6vZ+elM4quAdlnl33jyzNn3khxsg1xp3JgO+6OhL8H7IM0+EZ2e3SjISwejZ3ZyeZTt3dbjrwbv9b+LELTYv0dTm3zrF0ramgbQsZKwJ4M9Ulc8+OPvujcDCwFUoUkcEdgxUUc2xZohvK3+/DSPXTdNimzM8oFzoDPWbhHSHs46WAtIXOiDE7A3RWSSAOv/Wl9CILteU/VFUdE94ufZeR/3AGxx6dZxNOjKt8x4SQPLDdsmhJHIR+bhjbljPIQdkFpJNUXGeXYTdY3VwPs++xs7Uf/rCCYcFPw5xS5HiUTtSKJdNlkDNu22mVWkD7XVbAsPq8LBq6cnhYz2X6uH+RUOayVBnV1ZA7w+dk9YcIp5c1UqCkjKGBNmfdKXUMgWZnFBXd0t5sPcILJ4IQugsXwT UI4hLq/6 pa/Pom/hpBnydswKNx+O157iWky2Ggd8+4+VEu3EYT7Sr9qnWNpNM4Pmet0rpoliFG3sDr+SxmlMflG/pfbzuFQ07v/GzuHTDjYULfCEVqwD7YLRQw2M0ThJniyUq11Ert18KwLjdJ5ESqvQsQEhuY6pbM4hgcK/ar5KMZzPkh0tdBGjYP05fYAsia1Q6NKvchD5v2oA1uB7j6lXvlUHdBo6PqRf9ZyGFYu5uYg7QXNLSHGbFUCEY8bQsoU9WNbbrGCokIScZIDKj3UrWY1yfFH9zYZg3+khXyRnl5j00mf0q4vNgoyn/b3NcmFNygl5/eCS9iy5iE8PXpczuTemQmVfAAL4neEpiIegcgIjcaYLXAFdkOkgaGyvjiz8sXJxE6Zcqhk+JtQQCNZp4AYXk5VRvxxdkZwQWmkKzI8amV7402D7p8aIOpZgoTRgZD08j7ncTpCDR7a/6KjPQ3t3jrbG0wg5SqagXsLxPq1CUY8VZdHKt5EWBOchk9ek6thuxRnxbPGBed6grZyM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, if multiple reclaimers raced on the same position, the reclaimers which detect the race will still reclaim from the same memcg. Instead, the reclaimers which detect the race should move on to the next memcg in the hierarchy. So, in the case where multiple traversals race, jump back to the start of the mem_cgroup_iter() function to find the next memcg in the hierarchy to reclaim from. Signed-off-by: Kinsey Ho --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 14 ++++++++++---- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 4cbab85e2e56..2b354abe6d48 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -57,7 +57,7 @@ enum memcg_memory_event { struct mem_cgroup_reclaim_cookie { pg_data_t *pgdat; - unsigned int generation; + int generation; }; #ifdef CONFIG_MEMCG @@ -77,7 +77,7 @@ struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup __rcu *position; /* scan generation, increased every round-trip */ - unsigned int generation; + atomic_t generation; }; /* diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f672bc47c6b5..4314a2b8848d 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1005,18 +1005,20 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, rcu_read_lock(); restart: if (reclaim) { + int gen; struct mem_cgroup_per_node *mz; mz = root->nodeinfo[reclaim->pgdat->node_id]; iter = &mz->iter; + gen = atomic_read(&iter->generation); /* * On start, join the current reclaim iteration cycle. * Exit when a concurrent walker completes it. */ if (!prev) - reclaim->generation = iter->generation; - else if (reclaim->generation != iter->generation) + reclaim->generation = gen; + else if (reclaim->generation != gen) goto out_unlock; pos = rcu_dereference(iter->position); @@ -1050,10 +1052,14 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - (void)cmpxchg(&iter->position, pos, memcg); + if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (css && css != &root->css) + css_put(css); + goto restart; + } if (!memcg) { - iter->generation++; + atomic_inc(&iter->generation); /* * Reclaimers share the hierarchy walk, and a From patchwork Wed Jul 24 19:02:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13741275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DD83C3DA61 for ; Wed, 24 Jul 2024 19:02:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1039E6B0092; Wed, 24 Jul 2024 15:02:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B3116B0095; Wed, 24 Jul 2024 15:02:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E93A46B0093; Wed, 24 Jul 2024 15:02:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C3EE86B008C for ; Wed, 24 Jul 2024 15:02:30 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8ADB1160B35 for ; Wed, 24 Jul 2024 19:02:30 +0000 (UTC) X-FDA: 82375567260.16.C079FEE Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf26.hostedemail.com (Postfix) with ESMTP id 9D50114002C for ; Wed, 24 Jul 2024 19:02:27 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=j5PmW+K8; spf=pass (imf26.hostedemail.com: domain of 3wk-hZggKCCMJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3wk-hZggKCCMJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721847699; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ys2craRirgSkjM83RG964eoUiETMfTD1cND24tGgc4M=; b=lhQbuU4BaY8betAJRJuBMWA7ahv7Fa1UvWzmEYVAPEvE+dn9P5f5Qn+AcQIRDLXl7Sj5KU +kEiNPKTz+vP2MGbOKPPlRQucGMuy7nsgMBUN6Mfd2ZE/SEdFSV/Z2M34XDTa19bgGakzW sOrYriEW3swFdaXguC68UT6zn+LhiUI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721847699; a=rsa-sha256; cv=none; b=lQqco1LKtj6cdoKOjCmnfKQkurFpXvrJaqh0UMuU7Z+C0WxDgAL1e+RCM2U2rSM0jeT7J1 cUrKA7JF7e4OZFUqcsKW8zJT5rKQPUre1f//G21DiRktMikaQLTybSEYRO/CHRS1znH8Yl tzuvArioUBuTrKmAgQSo4e7S0B7ot4A= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=j5PmW+K8; spf=pass (imf26.hostedemail.com: domain of 3wk-hZggKCCMJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3wk-hZggKCCMJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2cb656e4d97so168423a91.2 for ; Wed, 24 Jul 2024 12:02:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721847746; x=1722452546; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Ys2craRirgSkjM83RG964eoUiETMfTD1cND24tGgc4M=; b=j5PmW+K8cw7alSLrvllyu7IsJuzhsaJ9SF/mViDIcouDxMLMfisvJ/BeyRx4BEQ9P+ 2WIlXGqgglYrwZlWcGl92Jz0gP3ThAWeJA16HRBj3q6xYb1+ueAY6hN66t7F5N1PVASf k0IWes604TCbicIgC1NPKoAZog/bTtQm0X/zPJtAhP9nzynK7vNlw9BLI/kiynxJhyt6 KzZJKmWBAsvfAF3BKmrKxoM78LEcpSLDyjNaNh1n6KHC+WUxHccAXJuK3v/bpP07c8J6 4xzqv+cPMbqYPhoDs2JyUhMIJb61GuViWoumNBdTmIpg6bNaIvZpVX4zjor3XK2d0pM6 3beQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721847746; x=1722452546; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ys2craRirgSkjM83RG964eoUiETMfTD1cND24tGgc4M=; b=Npm+5QoERUth/OrOWMxRguD911nB+VzZrIenqd64iahd1KJz2qT7z6Y4ttqgkONtLL uaeZWjsAgwa+raikeAbR2BrblWhsY6Qpq1EFVf+yET8IMfKAgAxHtNe7lf9FETfDEBUC SbGDnWNNsJVHA3B9gHsEW83tjR12n6YpI+1MZonomcpjf0TQvKFXl+VgRh2e2/OIYFv5 a8v7lzyGpbUXGNUdXZWu4ifoYh6JDVq/r/MSnryRFT8ws/puCLBw8dsuduckdnMBfn+K NvIDAUdJhIdsRUc7IAzXG3bgH+E2YJbbqhMceX5cSLc6Y7W5Do3ne36fIm09u1qqwUZG cHag== X-Gm-Message-State: AOJu0Yzf6GSa6AOMNRMDRoZNuphNTXIPmWSJSvYlsDPhZLS+Gn+SJEPt doj+7RbD8fNZO4OtF2iSff5HLSMipxZhXhDQZYzLFRrRp0rOq6+tE1ItRQAJBWJVs0XGQtp4TS5 jk9XD3V2YDg== X-Google-Smtp-Source: AGHT+IGYuWhGCshfz/1OfOrlFgcnBDPUyftLtmcQ4FX3/fi/oAJTe3cbfgo0sG0FCKIUMQtWQ+YJ7xLut+dAxw== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a17:90a:17ec:b0:2c9:9c25:7584 with SMTP id 98e67ed59e1d1-2cf23e1b73bmr1597a91.5.1721847746082; Wed, 24 Jul 2024 12:02:26 -0700 (PDT) Date: Wed, 24 Jul 2024 19:02:14 +0000 In-Reply-To: <20240724190214.1108049-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240724190214.1108049-1-kinseyho@google.com> X-Mailer: git-send-email 2.45.2.1089.g2a221341d9-goog Message-ID: <20240724190214.1108049-5-kinseyho@google.com> Subject: [PATCH mm-unstable v1 4/4] mm: clean up mem_cgroup_iter() From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Kinsey Ho X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 9D50114002C X-Stat-Signature: tmwidgsqd8houg9tgh9zbpwizt69didy X-HE-Tag: 1721847747-846920 X-HE-Meta: U2FsdGVkX1/4AmtCryG4Q9R5nYHrPIZwfu3evkDixhKwcw29NjIvHBil27RTrGaG4flVEbK9r6Wgjl7FLRnQnLh/J+t2g8pQV+CbkmXnekhChHKpNT1JrEpMNQALBqNnkhkUB5CyW1MdfSzA0it88qxujimYBKffmJmP3QqkifwqB5eqzPM1c0LiA49JnErTnLDmAdXn5jpsokx1MoJnEQBFTR2MoJ910MnN/m9uKszEFyCcksIA20t+BAr8nYXeA/lpxbccdIwhsoTsY0bu4n7US3UQ+JFrRgGrz29bfEVQzU0RilzXfiMYU3cMNiBRhZQ+Xh7ErgrBmGUVhySULIy4jZDns6RrhT+MKcstAShK/taAkJMCIGDLUSBrbBTtYotsHxHwr0DzdA5cBzAqli0LD9dt22wcQyUaqc2Xk8onOXsAVan49vEOQyERdrjL4UJypvoUNtyR1oOkmx5zcH8FfWjXL6k4mGeJjkten82CcPR/HG8f3xuaaKzzx09zw7peRggqeeEykpJ7NJwpmSlFzmGMfoQtSJ/YfLo9ysNqZOcc1tCguYQ64nrEdPzMxD34IOLj7rKoKvDr+CZAiCwD8JlaJFiMjX7QiufI3orBeAvlHXJ3NBLrZ7yWH7zhYJfkntSDDIN93aBhPfsn8sgUQOO6u8hQ6iMNUM7y2c09wph8mmzczGgBhNzR3WvE/TU8HiKNRb8KKJTX5HmiDtCoZZIPhixwJW6rto0zNNmoe1AUWOHRtjxugngShBxmn4/gn958dhb24MEvjh1/zKanYGJHYRyBaVVwLbl01x/LGFYVbrD8clB+D+AOUOn1uBGL1lci0KKMj0TDtnw4PYl/EAOZclh8A4AyPqkvqksWrUyZAcQ2X1/fdQRjkH9+FxggggPyyVCa7H5/LOCZkFcmcvVvky3sz8BlfTeqPs3FJq3N+2FQypzxA580KWXyjytQXLND383YP7gMQks lD0P7P3j jJsm68hVTlGxbbptiFZKgDBldoACPSAeE+qjH0yy8PY9ySPw0Mk3EwRO5QS+bb6Mf0WgmxBI925fPRVMRm6nA2RwD2AIqaOCX0AiBZjOsHr/QeZIGGEFahdM2viY3inTwfRJ57tIkG8jH6mjbbRWKju/m97O7/jgqwRiveYCzKHN5qu8F6YkDzsYJvyGxm0JxVABXepRUNH/c+KJXBi8WWb8xIDZAPqBDD1NrX/Xckr39UK4DdTh9xxrkAAZTAxqTJcWWu5BHseQBaisOeRwOV6oSRK4xocQjsnFkDDy0UOMg2IooWdfbeeDQQTTMMYIIwXdVbTFZgBIq83O8xw6dUPX+pTpqymua83V+IbdbPkvMIoPPXKC3zAgadGJvUnDkZMNL1rCVmzGbkMihG6N7KPBa2Dgv0naqG+nwsp8DVF7jsj9ZEKpIahMatH79FzPL6jCvgerCDbRkIEMoHnq+lYhh5aCAVbNccnamlvkSKmokEvYLm/M6CnbfE2slSPpQWi3gfuUy/xX0Xa5xzfk5tailAo5SEPC4cxqSbk8qQ33bk6yPiGHYCJe2WUtGCRH2jTpoM/1Q8pPPOdA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000006, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A clean up to make variable names more clear and to improve code readability. No functional change. Signed-off-by: Kinsey Ho --- mm/memcontrol.c | 36 ++++++++++++++---------------------- 1 file changed, 14 insertions(+), 22 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4314a2b8848d..7e3e95c62122 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -992,9 +992,9 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup_reclaim_cookie *reclaim) { struct mem_cgroup_reclaim_iter *iter; - struct cgroup_subsys_state *css = NULL; - struct mem_cgroup *memcg = NULL; - struct mem_cgroup *pos = NULL; + struct cgroup_subsys_state *css; + struct mem_cgroup *pos; + struct mem_cgroup *next = NULL; if (mem_cgroup_disabled()) return NULL; @@ -1006,10 +1006,9 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, restart: if (reclaim) { int gen; - struct mem_cgroup_per_node *mz; + int nid = reclaim->pgdat->node_id; - mz = root->nodeinfo[reclaim->pgdat->node_id]; - iter = &mz->iter; + iter = &root->nodeinfo[nid]->iter; gen = atomic_read(&iter->generation); /* @@ -1022,43 +1021,36 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, goto out_unlock; pos = rcu_dereference(iter->position); - } else if (prev) { + } else pos = prev; - } - if (pos) - css = &pos->css; - - for (;;) { - css = css_next_descendant_pre(css, &root->css); - if (!css) { - break; - } + css = pos ? &pos->css : NULL; + while ((css = css_next_descendant_pre(css, &root->css))) { /* * Verify the css and acquire a reference. The root * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { - memcg = mem_cgroup_from_css(css); + if (css == &root->css || css_tryget(css)) break; - } } + next = mem_cgroup_from_css(css); + if (reclaim) { /* * The position could have already been updated by a competing * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (cmpxchg(&iter->position, pos, next) != pos) { if (css && css != &root->css) css_put(css); goto restart; } - if (!memcg) { + if (!next) { atomic_inc(&iter->generation); /* @@ -1077,7 +1069,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, if (prev && prev != root) css_put(&prev->css); - return memcg; + return next; } /**