From patchwork Tue Aug 13 20:47:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13762480 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C00BCC52D7C for ; Tue, 13 Aug 2024 20:47:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 544B16B0088; Tue, 13 Aug 2024 16:47:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CE426B0089; Tue, 13 Aug 2024 16:47:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 36EA96B008A; Tue, 13 Aug 2024 16:47:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1B0536B0088 for ; Tue, 13 Aug 2024 16:47:58 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CFC3080AA5 for ; Tue, 13 Aug 2024 20:47:57 +0000 (UTC) X-FDA: 82448408994.08.0D4BF28 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf25.hostedemail.com (Postfix) with ESMTP id 18C6DA0002 for ; Tue, 13 Aug 2024 20:47:55 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KY0qMkMM; spf=pass (imf25.hostedemail.com: domain of 3esa7ZggKCDMZXchTnWdVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3esa7ZggKCDMZXchTnWdVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723582040; a=rsa-sha256; cv=none; b=votLucSxZDV64eFrqeYV13Vk34GErBFjYiF6uzNfiE5LiLJeEoccjymxyY2XxqgkBDWmiP AYWGAsKlE5lRFEGG+HmDqVEqjGqltiMThjVdcdUXQu5NB7iexQopoS6ylET93X8Q0H7V+Y Q2l5q01A5GYzxJj74Wyc+/lMMt+5Oqk= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=KY0qMkMM; spf=pass (imf25.hostedemail.com: domain of 3esa7ZggKCDMZXchTnWdVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3esa7ZggKCDMZXchTnWdVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723582040; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BYWJhwEE6NYCkgyDLBhCsOHKcazNm9LYuzYyiRCtiIM=; b=6RthOxdi4l0PbSc9Um3fpSphKiuwextS8PHxlhNJy3xoKIBPI2rSIV6vo0IWrIRE87WEaw CTZ2afUDA+Q/hcETF1Erjtg4kWUMxw6xU8n0c8AlelkyaTtfq/Sk15GUxBIQAN3uTmaJVa DjMNIS0M3MtP5P5kWzm+poV8GNL/mx0= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7163489149fso5671087a12.3 for ; Tue, 13 Aug 2024 13:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723582075; x=1724186875; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=BYWJhwEE6NYCkgyDLBhCsOHKcazNm9LYuzYyiRCtiIM=; b=KY0qMkMM74trKa4IbhOc6/gMRi1YFaCX9McZxjn4mz+Wvg8WUJxqETzJhlQrXLGGxr ZvpcGoXZMJY6oPqf8Z85eJquMU2MwdIMjSSvTVZbIpgFTRG+oCB79w0dGshtgdXnXg6/ K0ogped9qx7wlosOniisfkR+bpvYbnOoj4IgmB+9P/E2ndvbmAINfenyDm1si/gxwnCG wQa/k3g0Zi1SXtm+odkyjDiSFun4LAld7VDRdwu/oOQM76VNV2mwzI1sAXJ41w+JwJJC eeIEViAXsitRzwzbHYQhGVMRXmn6UIQp+rTCGlafGQrj5M8Q7YMGylAOE4pABVc4hOgF Vy+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723582075; x=1724186875; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=BYWJhwEE6NYCkgyDLBhCsOHKcazNm9LYuzYyiRCtiIM=; b=qSfigpbTMX0TvyksR7GKUOPyaeNA4cdf59zltu8cftMTMSJOgxhsxwD/Vp8FiDoCqc 0xAc5zbpcdlOdGUEc+CsaPT41M12VTIsY2qNJoXzrH1G59viYx2AjWLc9CT3MLavJBf4 WUJQYK28s8r9daTnQpBpZP+5JXdPFOmS3YqZl+WRhLL7Oea/nbNcke05XoFDHRI+Wswu uyXlZ1lp9s8SIsAl8vpC5fd1LTauBBw7xiC7ffHgndH9Gma9/lxg7rjiYv38CEP6xGtc ikAFbPtzVe60CxAkD68h6rIpBxeTu0HTXuf4inH4NPDKP2bMH9L8UA6tALvo4gUNoNV/ 6JsQ== X-Gm-Message-State: AOJu0YxhegXTY8AX2rCklB77GZVqDL5TWWOXwfr/1xct69oT6u4rbQJp Mb9jJ+OPVJeYt6BmcQmBJPfsGJsqaMd812ZiPUoCLIXhNYRNkMn4B2jI8QyyFYFObHSvlqBHQwz 5BLJVLjSMVw== X-Google-Smtp-Source: AGHT+IHxMBB1WIpSpQKruSLzaZb5mFQ79yQ49km4F0bFBxJKf7BgUc4XswgMUAKJQxmwnNey7FUrSuzdOv6Aag== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:114a:0:b0:694:4311:6eb4 with SMTP id 41be03b00d2f7-7c6a587fd3fmr1059a12.8.1723582074503; Tue, 13 Aug 2024 13:47:54 -0700 (PDT) Date: Tue, 13 Aug 2024 20:47:11 +0000 In-Reply-To: <20240813204716.842811-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240813204716.842811-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240813204716.842811-2-kinseyho@google.com> Subject: [PATCH mm-unstable v2 1/5] cgroup: clarify css sibling linkage is protected by cgroup_mutex or RCU From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Stat-Signature: 8qeat1dawfiq7z46zic83u55qabu6cns X-Rspamd-Queue-Id: 18C6DA0002 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1723582075-442775 X-HE-Meta: U2FsdGVkX19YnT9lu8eRfdBF+FdFR9fyiR9uIfUl+EduA27chLdIG4ZkyS8LoLietaKipP58NtnpUUy2A0EL+ZpDHNUjfRz6yIZIgEZCn9f6Ceh00lW7/JHuUnfCBJMTGwOHvaGNt6QDUrC/Pa44owU8BR47Ith4suPbDZCMzormXYlmZorkLP6u/OcB4eL8riVBrLxKASivUwxgtWSXg07sB46VHVgGQ5yJlWjYnxEYbPfWSoCtTBAQktlxdsRPO6tZlgq0TeG10K4kV+LOXn2F7a5/2GlYq2+NOPGW8bsS4mSEwbexvdcATQUjShUCFRX7poW5rMQVfPFNYfy5GMpbk5WmjKXp6eYyRe0hdVavRA3nk7ntA1dqy6aTjXSMiO0tH2uN+8yAlZUjG9vTrHenxlK8F5mtou1unG6z7806e7c31vyBwkNSdSer2ANolgPHJYinXQSNiq4I/me0GnG5oB4kFbPrDcRs9UD2GyBMt92O7akhDaW6yZQscTEjq5KhYFGJzoq3u4WqksBSBiQUTqygvq3QEl0GfBv6lL/h7WE5yAKoe5Vek+mJKsBhFTUYPdIkVbEFekh8TGkF8tII71EsMgw16QDOujptIxmlqJ5OHYyRm6nEOK3FmgWnC2n6YCCBk4r0m5GpZPA6+tUvP4mDMNPylidP2YKD13rceYP6kEfWFRytfgnAzZVph/ixlkNu0dPWlNVqionvNF2vSlINNRU/u6BjMvuK1TvqII/oXUlQv+/psOb91bGnvgu+t5mXuBVQt7MidfZgWBJp3zv71Js+qb99yvXUpTLAQeWKbGN7Im3szD5qbrQiowmWJ3J6vbORf84OXgNxNU6fS7WqBbedyEFevYji8/YcbvVT5bLDVPFSu7IFOuCTQNtZlGPjYA4M4FyDZNfGE3tBtZtwLKTjl4x9vhg1nTJHBDao57+1KkBr+J+guBhUBYZDUCoZi3QXLT7SlRi oYJwpgaw hlKaAkLSYI6vp9/FyKKl95CKKFmf5leEUaVkZes4aQnQinYgSn3lFgmheWvF92KoTy43v02qCw1/TbhTRnDd9B1z8apAnv6DnMNoUFGSur2idLijLTYm/LjikCW2ynbCkoDeYGH0WuU5greAfhqZmUmGOkCVe6SjdfHTUgCjDugpYDuRBr/Pc9NnGwZbPqIJGpjxiocRsgwZh/wGTFKu9pVPXmgOoWV55yLo87h1Bmgqo1rfdvnlP2yMawqA2IWd+B1QpZetD8aNF34tHCM+rymuFR4p2i+Cw9DldZuVl8Y+Z4zv2BOXpo1al0PQ4HFIajxGpY9HvfGMFWPwqjt7mwzbLAntkATdjKFz3PMCBXWrXKE5rOxV7xyuMeZdyME/BLD6DYGHTAKV+Wim0aph/BB4Sx61nT52jEdZm2VnFreW+vTZ4jgL0HJAt3+FOP+QJJ28aW+g8sy+9I9W/8p0ftlYjQ1ICFQnK7R2DzAmD9CL9P8eimfdn2l88dyKEgBq/iPL+xKxP72tS0Hm7VRlyaZCfRhr6+I+k+iZ6Gmn9KqqVm4TZYgkfYoWONvbDgA9JqHuh9yHSCmUsZTg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Explicitly document that css sibling/descendant linkage is protected by cgroup_mutex or RCU. Also, document in css_next_descendant_pre() and similar functions that is isn't necessary to hold a ref on @pos. The following changes in this patchset rely on this clarification for simplification in memcg iteration code. Suggested-By: Yosry Ahmed Signed-Off-By: Kinsey Ho Reviewed-by: Michal Koutný --- include/linux/cgroup-defs.h | 6 +++++- kernel/cgroup/cgroup.c | 16 +++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 7fc2d0195f56..6862243bd1c2 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -172,7 +172,11 @@ struct cgroup_subsys_state { /* reference count - access via css_[try]get() and css_put() */ struct percpu_ref refcnt; - /* siblings list anchored at the parent's ->children */ + /* + * siblings list anchored at the parent's ->children + * + * linkage is protected by cgroup_mutex or RCU + */ struct list_head sibling; struct list_head children; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 0a97cb2ef124..ece2316e2bca 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4602,8 +4602,9 @@ struct cgroup_subsys_state *css_next_child(struct cgroup_subsys_state *pos, * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @root are accessible and @pos is a descendant of @root. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct next descendant as long as both @pos + * and @root are accessible and @pos is a descendant of @root. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the @@ -4651,8 +4652,9 @@ EXPORT_SYMBOL_GPL(css_next_descendant_pre); * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct rightmost descendant as - * long as @pos is accessible. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct rightmost descendant as long as @pos + * is accessible. */ struct cgroup_subsys_state * css_rightmost_descendant(struct cgroup_subsys_state *pos) @@ -4696,9 +4698,9 @@ css_leftmost_descendant(struct cgroup_subsys_state *pos) * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @cgroup are accessible and @pos is a descendant of - * @cgroup. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct next descendant as long as both @pos + * and @cgroup are accessible and @pos is a descendant of @cgroup. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the From patchwork Tue Aug 13 20:47:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13762481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E039C52D7B for ; Tue, 13 Aug 2024 20:48:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98CD86B008A; Tue, 13 Aug 2024 16:47:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 914B76B0092; Tue, 13 Aug 2024 16:47:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B7F96B0093; Tue, 13 Aug 2024 16:47:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 56C996B008A for ; Tue, 13 Aug 2024 16:47:59 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 000A01C406D for ; Tue, 13 Aug 2024 20:47:58 +0000 (UTC) X-FDA: 82448409078.08.CFB6E1F Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf04.hostedemail.com (Postfix) with ESMTP id 4BA5940011 for ; Tue, 13 Aug 2024 20:47:57 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AuNXGrO8; spf=pass (imf04.hostedemail.com: domain of 3fMa7ZggKCDUbZejVpYfXffXcV.TfdcZelo-ddbmRTb.fiX@flex--kinseyho.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3fMa7ZggKCDUbZejVpYfXffXcV.TfdcZelo-ddbmRTb.fiX@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723581983; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hmvEk3aWaELcFNHaifCCcbwnogY+IyR2g5JVb5or6kw=; b=LPU2Sqtp2YtrKhdy8CfM9hF5musq+J03YFPl8YDrI24MArj945TyIE2DIcQPhGVFuEse7K Oofbu3ALawp8ukdrmK+3gUd7ZSTIWeOUi/wF+by5DDv/3/SyZizFMnyk7mcfYrNYWrCSHW YsrJnCKtzpl9KxEyrh2tv1pducH3xPI= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AuNXGrO8; spf=pass (imf04.hostedemail.com: domain of 3fMa7ZggKCDUbZejVpYfXffXcV.TfdcZelo-ddbmRTb.fiX@flex--kinseyho.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3fMa7ZggKCDUbZejVpYfXffXcV.TfdcZelo-ddbmRTb.fiX@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723581983; a=rsa-sha256; cv=none; b=NwftUoAFmcw1qVwjfMQfCkYiumTW1qbqUrjPrj1fksUjBTEggdX/dQZ6Qv9nv8Wt/cT3H0 APGsxNap7uxQwv0dVkzpct9Ym/2qlF09EvXpQKeWCiJiNJXRmFrrOWBw3DF9xqSCuKkQ5D tcGhQQAsiyOwiWuutSLJoT+M6FF+GaA= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-69a0536b23aso129522217b3.3 for ; Tue, 13 Aug 2024 13:47:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723582076; x=1724186876; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hmvEk3aWaELcFNHaifCCcbwnogY+IyR2g5JVb5or6kw=; b=AuNXGrO8cQ5eShkAgFv8GVsTn4f7oaqa1W4oqSSY2pU2IKBQJQ/+WWBAITI2dPr5is n/rl1F4jvQKit5yNZo/mVWf8kcRb8bo3RL1iUmBpn//bZrE6Bdv78LTqMpAPHIavYLis JCNkutTPwIYj3q31ZLEy9VhktHeP24NxmrLjRjXlgvboU4QEQrXY2iD30m+77zdqOtmu 96Ac4wA8jx+Uthx+G2YFl1kWwyO11y7kHogcdYy1FS6+bCNXtVwTgbAwUplUQzOK2C67 vBM7XWDO1tj+BWIIwohzlblM4mX1DZoIQqqxqrat3tuVzUIrDNHuC9Yoqr3oHppkdHrC UOOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723582076; x=1724186876; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hmvEk3aWaELcFNHaifCCcbwnogY+IyR2g5JVb5or6kw=; b=c7eP+rGtCc6kkkV2TsfcOhs/lM/w533WoQ9g4h0KTDk0JjBA5lw/686G2+gIX8w7PG FnxZ1fy+jpTrQFyjo1W6+zUjrtitxuhcOSZ2srPrd8CwPph+zLFeOwnBQwmDbhgy/Tmb pvx+UhNlt6j0t+HMpM2eASuFL0eozCFZ/VQpkoSG61DHLgegeKYx1gqGe9BqOR/un6Jx rQzRXSJFfo1AwSukJcgyDUXE7OF1BU3J3DvrGXwl2wiPkZIjNcMhywuciKq877G48kDe sAMi+dIQrX5VA0c3L/zYbEAey8NWdqLQkrdpN8+Auyf3oboqAa+L/HP6vDY52BovIi5d dx2A== X-Gm-Message-State: AOJu0YzpR5mA7Chak9mzKztqAdIWbclR8U0Lo/2qFhLa/Xz+eJmn4e05 jqwHzdyF8d14udSHkIx5UUEucbWkiXblwJ/Hjd4EADGurrSd2xFgh2HaQYau7DzuZ11T4gxUQdb YfzcDnQZ+SA== X-Google-Smtp-Source: AGHT+IHiDHCjFRbPR5+mkcQhIrf9lDLp4M5EcHeImCSP6xD1TY/GSvpgSst1fxEwRAEkFihKnMepRjSKepm4bA== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a25:c788:0:b0:e03:62dc:63de with SMTP id 3f1490d57ef6-e1155ada202mr39205276.6.1723582076373; Tue, 13 Aug 2024 13:47:56 -0700 (PDT) Date: Tue, 13 Aug 2024 20:47:12 +0000 In-Reply-To: <20240813204716.842811-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240813204716.842811-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240813204716.842811-3-kinseyho@google.com> Subject: [PATCH mm-unstable v2 2/5] mm: don't hold css->refcnt during traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4BA5940011 X-Stat-Signature: pgpcjdy8eqzbds9waf75reeorihnxrim X-Rspam-User: X-HE-Tag: 1723582077-306441 X-HE-Meta: U2FsdGVkX1/ASkTHlUlIZj326KjYZT7WOEXm5UyD28BfA4R8D2Sp5hxnSTiUJ7MrVx2yZbVRGt2dnRAKl/xloFCx5g76dYRcBoPfjJ9cvoE2kcGb6Bc0W90RPr+1fv8LUGSp9/rpCg7NrRLG0tQkH80/E73imMVhSsSih0g8L3fanq4c+8BkUfbxOPf3CweHizdAUoXXkvHLMLKnl/FYsqhp/ZJ94QJDTnQQyiTWOYubdCTEjCglIt6GZakKNvPTDMD3Tzc0GIc5FA4rfEtHsmg0h6QiEjTWWLCxPLaeKpZ2yFuIL5cMJtaw85FR1/uL9HPFE6bv0wmYPUrMMFQFkl7nbM0B0CdK6X83wS0/L/IEWNB5mCrwd1uvA/gNItYNTWHAqFIu4tGC3ZWuZVlbAekzKexNCZ2aLEyrGthbWkkNI+8V07Z8t/Iz5ZMy00cQgIMbApMG8+3fQoYk349tGw1cMknhe9sHC6HLOyW24CBuciwRi9GrillfhgGnF2Gft1sUfExEenkE1A9j55LTfZ5sDRN3AJMeP82tJHzqmcyjxhh0sUtT3KQr3LBd7dZa+2tp5elkOSljuU7sNp8bPBnV/rmIg5fsXggbDL9nqtkip/jI0JuqQN11fkpea2355kNZGcwCYuBhtRxSoMbIbESothdc+VDHsNz/j/YYuKBfMNqaG794cRp20mIn8fABpYeyxmfHSDliZrCf3bOAvvjQ1j2XgHpdiNsDmatRa1yNlwnfop6rGS8c2MuDQ+m5fNXAN1pWXCJcewYtGw0uR8Smaz1BrISklzAV19SbU7AZALF8mxQRb7mC1gOkst+wLOQDrciCP9UZ6+iuOUb0kkUwbcDk2g+2I+5ac1OZWrD1/MWAazTN/CurUAjnld/LA3cPd1MAkr+2n6DAEkN6a9y5ZI77yqJLZVtAwrghcXncBzUwzCo/En/eR8oOJZVr5KaXXRG03adj7teC97V a2sUnXAp cviKmDtgefMvccBD5vrDKuZraow4q6YZ7pYgs1jXiCfHCBT9YiI217x68rdzTy5M4EEPLO0CIYZzxyj3NXpnm2Jq1HJBfppv/QvTuRF0CMrOx8ku5+1W7nHujU2EjBxcF/dtwxJyUuHMqIXyXZnRP2aPrfEc/KlSz/xXReghJJOPjr3cf6OsAJGlpUk6j3/10Y2LfRKHeyW4AXsXO3iyueOkdhzq+Def/4ZMdQMv9tj9R4fNq22CQc0bOptgG1j4G4ymy5WZUcJeo6SxOFOqgUE82xDmHyby5cYj/uDw0K3BrGRhzsXQl6euYoYNjceBEfcXdg3yQf67n6iFshHWAj9K+52nzYB3o/cTE+lcYrxgPb5lxytvwXWtLAyJ/LFT9H1uN81ZmF8W2o9DasRhAwt5yBupZT50QSaazZxJvmMB2qfE+Wj73h68PSmTO7d1YIJ4NmfRa28Uj8wPS8+/SS4AaM+gD9dZFwFryMOp0a9UqoW1salZNJtGvUw7o5XyfmIwOlv7TX3gwBieY1MuEPQDl/A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To obtain the pointer to the next memcg position, mem_cgroup_iter() currently holds css->refcnt during memcg traversal only to put css->refcnt at the end of the routine. This isn't necessary as an rcu_read_lock is already held throughout the function. The use of the RCU read lock with css_next_descendant_pre() guarantees that sibling linkage is safe without holding a ref on the passed-in @css. Remove css->refcnt usage during traversal by leveraging RCU. Signed-off-by: Kinsey Ho --- include/linux/memcontrol.h | 2 +- mm/memcontrol.c | 18 +----------------- 2 files changed, 2 insertions(+), 18 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 90ecd2dbca06..1aaed2f1f6ae 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -75,7 +75,7 @@ struct lruvec_stats_percpu; struct lruvec_stats; struct mem_cgroup_reclaim_iter { - struct mem_cgroup *position; + struct mem_cgroup __rcu *position; /* scan generation, increased every round-trip */ unsigned int generation; }; diff --git a/mm/memcontrol.c b/mm/memcontrol.c index dacf4fec4541..1688aae3b1b4 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1052,20 +1052,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, else if (reclaim->generation != iter->generation) goto out_unlock; - while (1) { - pos = READ_ONCE(iter->position); - if (!pos || css_tryget(&pos->css)) - break; - /* - * css reference reached zero, so iter->position will - * be cleared by ->css_released. However, we should not - * rely on this happening soon, because ->css_released - * is called from a work queue, and by busy-waiting we - * might block it. So we clear iter->position right - * away. - */ - (void)cmpxchg(&iter->position, pos, NULL); - } + pos = rcu_dereference(iter->position); } else if (prev) { pos = prev; } @@ -1106,9 +1093,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (pos) - css_put(&pos->css); - if (!memcg) iter->generation++; } From patchwork Tue Aug 13 20:47:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13762482 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A59B1C52D7C for ; Tue, 13 Aug 2024 20:48:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E9EC6B0096; Tue, 13 Aug 2024 16:48:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 124BB6B0095; Tue, 13 Aug 2024 16:48:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F08A66B0096; Tue, 13 Aug 2024 16:48:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CE7496B0093 for ; Tue, 13 Aug 2024 16:48:01 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8DC73A0880 for ; Tue, 13 Aug 2024 20:48:01 +0000 (UTC) X-FDA: 82448409162.22.10A3199 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf20.hostedemail.com (Postfix) with ESMTP id CD4ED1C000C for ; Tue, 13 Aug 2024 20:47:59 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=LES5v7eB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3fsa7ZggKCDcdbglXrahZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3fsa7ZggKCDcdbglXrahZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kinseyho.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723582067; a=rsa-sha256; cv=none; b=vFXN2djx6IRKJlTsgqVQxdxibjDxyx9lEyjO9BzqQBQyzVRQScilyPUZ8/i1SaJ/zMWW12 Z408QNHkcbfo8ywjY4+x6iNKp3NJ2+1uREVn8WEG/TPYZ1lEfdUgHzsWymSvARbkuN6rtb 0lgIUhROAyoOJ4N7n8G1u2eQc0/hiQo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=LES5v7eB; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf20.hostedemail.com: domain of 3fsa7ZggKCDcdbglXrahZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3fsa7ZggKCDcdbglXrahZhhZeX.Vhfebgnq-ffdoTVd.hkZ@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723582067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7PkbmuKxOROTjGI4E3wXgUmmHtkXl9USrDaeWbBH7v4=; b=I7sN4wt/HAa0FkVrzEyvlmWlrhLcsik1syAv/LyObxua2+y3mqUP4ntLeJizeGlA6sUoFK BQ4qcSMtEDZMVF+n6rkLXo+vCt9i53AQKSKFJKjWPZuNFmbl2LN5twBGXZQzTihJEHCOd3 NUALkEoPDzKT0MYtqF7izkLM2IQQT+Y= Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7bbe0ab18caso5679826a12.0 for ; Tue, 13 Aug 2024 13:47:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723582078; x=1724186878; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=7PkbmuKxOROTjGI4E3wXgUmmHtkXl9USrDaeWbBH7v4=; b=LES5v7eBGr+UisWSeCLiIJ8U3A8s+zL2KsMJd/mSh8qQrCXgjXlBScQ9UxgdoGHjLk l4C31rvSLyHojIvtaIAphWwIYzw7KUza0l3rbjSsWF4NYoy3T5qO+Rdo5sIGLD7OGeLI KrcTVjJurMiZs0njXDINz5f6WlHSdjXwfO+wxtAgkAnXYVN+lyIUknBI0hj1NUbCkLVQ BThN5mkp88OYrIdZR1wCxkyvnBFbU10tKtqLjPWOAXcO+pY7C5H4WP1QcvfeMHr4p5TI a9QqcjSCmLqfhqfYElbLiXLnRh1A9MrzxUtAhsReF9/NlHSZ6tQAMCsCA9ZaXFvq0tcE T12A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723582078; x=1724186878; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=7PkbmuKxOROTjGI4E3wXgUmmHtkXl9USrDaeWbBH7v4=; b=Q8oBEJEC4ySNade5R8VGwCdUk1FS2Xk27TxqWtetf25oqY7akcy0uLo3Ww9VfvOGus siaXpcUs98Aj9qoYXgl3QU+E1k4LeYE9y+oeUxxJvwo4aULIlaiPPRHLTeC/SiiQr2yx Kt09p1oP48OFYQ0c7H9L/Tent+GNlvboyAtmDuGIyadq/ID9L0hevuieMLjODFO+lzHa CgrJkSGP9Lg40nk+GxWuA1RFto22MNNyP6tz+uXPUtXtFtt4Aqnr0DYcDvXOosPqRnWU cNLlmEWQWDja7UIsNQVKRZFqv2PwCwyyQzdD9y0SVVT2kjg/JRKKu4OtUIroZXR9IIvT mVQA== X-Gm-Message-State: AOJu0YwmFODeQulBijNrrFJUhfjZ6t/3FXSnwGBaJWRylDE20fblSAFC ti2jYQFtcwxZIT6g237CQ+IbMSRqzJLZQjBLzvDC/IuSJkWNYLKj3iifMoO3MAXgFw4MlCSGIbP IG6Z7ysUcQw== X-Google-Smtp-Source: AGHT+IHAciPzqd2lPyRlBVSYjYEEfNedI8L6N+K0qntudc/ZA3AgPVxsZZlT71exmwJpvjWHZeV3opJ32aRJGw== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6a02:58d:b0:6ea:a4f3:14a3 with SMTP id 41be03b00d2f7-7c6a56d0242mr754a12.4.1723582078235; Tue, 13 Aug 2024 13:47:58 -0700 (PDT) Date: Tue, 13 Aug 2024 20:47:13 +0000 In-Reply-To: <20240813204716.842811-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240813204716.842811-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240813204716.842811-4-kinseyho@google.com> Subject: [PATCH mm-unstable v2 3/5] mm: increment gen # before restarting traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspam-User: X-Rspamd-Queue-Id: CD4ED1C000C X-Rspamd-Server: rspam01 X-Stat-Signature: ffms6sxyxukzhcmjtcwzr7iyxu8juhnx X-HE-Tag: 1723582079-626707 X-HE-Meta: U2FsdGVkX1+1Ai3XbrOiE3Wr2EciBf416u4CaJfv1z7FE8HkiOf1CNvHeYoKpXAUFQVj3BVlgQXslGa4WcjDAs2bzw/BB0HEdSYBXxX59byjlmtcVX3c+FkaYUD6y7m6yZ8FjvRyYJ2yZCOMWcV8B2qfF94KMdCfPMGQuE0p9FzVIaZHPfPLHb5qJ4oSD9sQ+zDnMejD0VjRgPBlJ+u5Dc8rnNDZ2AGLrRzCQsq6qwS1fKMOKPVxQ7wQWygRm9S5KSyBnaigLQCjNR2wlB04Ak/xJm/HjbFoJS07EnsF/VQCXJb/HYLmqAHo/Bi21nf/jeFneq3ZdkHijGtQqiT/wMjAYi/CMbWXZDxJsk4E9zC3ByjxHEiBXaPtSi3OQniRx60k9EJRy5C73PyShCNJ4ZTqEb+Gcwffi29HjNvOtHZ4XWABmBjvvHSsYo7v6mEix1AGshYSn70kC/admrVlKMNRMQ5dxHED33Lu18aLx4AGgcGVeayOKzFlWqgFG1xc6mliWYj9OGpstcUb40SMCFksceGRADVwKAGPkyCBlIp91iuBlKIWray0QSJLw3Wmnwl2kd36ojpwZraPCiWynvIBbLMjIH5YjgimALHdr3/7/xDfzq6AiyZYh7RqbLw/km67QM8g/tJzyDcBLns5vyKrD/3QkdVsnOTjDyq/kKfgfSrMeCEYugKuktsAECqgEqTndfep8zRh2L00O0abpxjWIc3t6S8gK7YTH0hislvQ2ZmJkOFhZSLKdoqwxEfAtoTtl2J0+7tHcxp1k33eY1wyAsA+jkPMJwnFtY+V4oKwd21rdJn4x19372GOMj/iLzfL8Ydffve8Id97eoLMp9vtd3HLXfKt2IKFZSCfo3svaM5GclwEpTLt8TUi16HN0zHAqXlKO3llajnUPFpckarnt8yKhiOD3iJkyCztEcYN01Pp7nwIIdtrwghroBzfl7YvtSCCppukPrTA61S 61CnCDEK 3AK+pMAZvuwMyFWrXxFOjTLPR8jRYMcMNYAFmtQXit64nUj5UUMxY7yxsHgREsNvOhcdMiDSiLgaMdkLQ/DRuZAKTYnzxKAg9cm5ZFrhRqF461uJXcz7VSqIKO4iw0/Ss1Ka6GR/Mib++67EzBZKU0DA3qbJf0nEpcaZV3rnwsKtGn1UI3AgME+QJlVwDtD8RCyIwCCqAOzpQeUAOqgfZRaV8HBkM6I73fCVNjKaY5uB9zsgjmPs+mUOtS1Hkr98OkFV39i1l93ijII2y2bpmWoBXq0buGv8W19aQn9igpw4Jw9rl7g3F5YaeHriCG4Q+GkJKcwNFBnFO9IPyvqjF2R/jGVUQc3DewPsNaESx1KNOtBgDQ3WsJubDE9aMv4ngfhdH8DROUIVoMpoBTivnfuhMpM2yqG8NDXSJd2t1wOnmq/hkvPZ9AQ5ENhuStEK7B7ti798sHqcg50p2G5beV+AeSS0cVH/o2+jmd8fo4U9pG24ZK31m4AjpyiWPRbKFezFvIgWsbXAcPdqyfRJnncy6iA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generation number in struct mem_cgroup_reclaim_iter should be incremented on every round-trip. Currently, it is possible for a concurrent reclaimer to jump in at the end of the hierarchy, causing a traversal restart (resetting the iteration position) without incrementing the generation number. By resetting the position without incrementing the generation, it's possible for another ongoing mem_cgroup_iter() thread to walk the tree twice. Move the traversal restart such that the generation number is incremented before the restart. Signed-off-by: Kinsey Ho --- mm/memcontrol.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1688aae3b1b4..937b7efc41ca 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1036,7 +1036,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, root = root_mem_cgroup; rcu_read_lock(); - +restart: if (reclaim) { struct mem_cgroup_per_node *mz; @@ -1063,14 +1063,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, for (;;) { css = css_next_descendant_pre(css, &root->css); if (!css) { - /* - * Reclaimers share the hierarchy walk, and a - * new one might jump in right at the end of - * the hierarchy - make sure they see at least - * one group and restart from the beginning. - */ - if (!prev) - continue; break; } @@ -1093,8 +1085,18 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (!memcg) + if (!memcg) { iter->generation++; + + /* + * Reclaimers share the hierarchy walk, and a + * new one might jump in right at the end of + * the hierarchy - make sure they see at least + * one group and restart from the beginning. + */ + if (!prev) + goto restart; + } } out_unlock: From patchwork Tue Aug 13 20:47:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13762483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44911C52D7B for ; Tue, 13 Aug 2024 20:48:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 08E8D6B0095; Tue, 13 Aug 2024 16:48:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 018C96B0098; Tue, 13 Aug 2024 16:48:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D312B6B0099; Tue, 13 Aug 2024 16:48:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B52BA6B0095 for ; Tue, 13 Aug 2024 16:48:03 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 757DCA0A92 for ; Tue, 13 Aug 2024 20:48:03 +0000 (UTC) X-FDA: 82448409246.08.42D08EE Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf22.hostedemail.com (Postfix) with ESMTP id ADBB5C0027 for ; Tue, 13 Aug 2024 20:48:01 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4STg+RO4; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3gMa7ZggKCDkfdinZtcjbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3gMa7ZggKCDkfdinZtcjbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--kinseyho.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723582069; a=rsa-sha256; cv=none; b=3FKi0lmmuNxMup7yiZqUcrTrs/72ChRW6nf4RY3zjMz89eMvtZoiMMgg+qgthtXH2uQ3hi WoJV93P03UIxBl9TiczCPRRJ+4KNeTz7ljaLG5DeburttKqYgm/64yKknLETJ1DIUkoYRS 8OleKRb9KU/MMPaXztCs8f0PfCjkiWo= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4STg+RO4; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of 3gMa7ZggKCDkfdinZtcjbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3gMa7ZggKCDkfdinZtcjbjjbgZ.Xjhgdips-hhfqVXf.jmb@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723582069; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AL9cOogrL47hO93mBXXtaHX7JJ+ZXjoDFTtox3tfTp8=; b=j2rVRCWAnDu8bfj7171tQSYD0jxuUgXVeVhHAfsYjcSdJ4u+H1aVVIwyxAOKNv3Al+n8CH nCavvaFROKIdJmO1SrENQLXqOyT8MhTyGhA9gyHf0s0T9rrIFPJifjnZS3Dug3i6yRyjbe Aj2CXjFooJy/eqCM/i7mpnRyr0qxR6k= Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7c3da1ac936so1439107a12.2 for ; Tue, 13 Aug 2024 13:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723582080; x=1724186880; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=AL9cOogrL47hO93mBXXtaHX7JJ+ZXjoDFTtox3tfTp8=; b=4STg+RO4NKn8gSuBe2QmAqxeAGrXceQmKKRK0X5XzlfVCTmgpG7wtmYPVa0i2NnKcy CLXupu4GQNCV8h+A15PlNfmx+0LMaGa/EvgGeONmoYuSJwpdaynAYJVgRkHVESd0JGZx yDBz9MEGgfOfIH5M8G1NBEDisLHhDR8+yDySdK5Fd7hnhCLxjfoDeigoRGq2sQBUh14G zHEmnAc9Wrqn3Mj3ZjcHuHwIeIcIvFL0i4nEA/kaxSyMF7VGafcz/gTin5MGIuVyeVn7 nfdQXg4gRux3av8wagLbYXKJ6oFWtThIuGw0bOG3yDgx9aJqDd3M1IyPrkkX1yeXpPpC R45A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723582080; x=1724186880; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=AL9cOogrL47hO93mBXXtaHX7JJ+ZXjoDFTtox3tfTp8=; b=qfncWwFRTQclbuNL/N6nhl4x44U69pFdBdH3F1y3hMtutvMkUxdla3YYJeFntq9Ytp 0LM9tbuEDfQuRQZTsp1nwMYh3Ae99TNRZQqUEoboUAimmESd45n//R22+UymHH7GWWhG xlRSQgji/OShpxFeZxSYeOG9ejfcxB7f5f3twXmeWD1CUyTG/BTiDnElxNdNSaNJrNXI 4w9cH5niy19qJXzu83KC6Wp2QPKxvLKOoEFYR4SXG9XLG6Jf62keUAiq955qTPpVCFeI ILET+TibH1iqX3fhpTTjByEbu2cxYm/gZYqk21mQ6sFsfI9nxWjvdYHxGySuphsAxaDk KWpg== X-Gm-Message-State: AOJu0Yzfsip6yvey1JvkNlXRJk54wDeAqfjDeGQpKgi6znNx5SCleE+T aypb8gCS29UH/OidZaVgJx12heg/hYNqLtuNWbeoRifOQ6Bc7W6XryvMZwW+DjuFdsG7yDJImj7 wpr2Vz/e3Mg== X-Google-Smtp-Source: AGHT+IFYKAB9rOjPeQV1OvEWBj+/G0SON71UCsBpqeijGE33YYArHBwlllAogxZUus4RJ93ox/3eY4q8R0h4Eg== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:4045:0:b0:77d:336c:a857 with SMTP id 41be03b00d2f7-7c6a588a314mr766a12.11.1723582080284; Tue, 13 Aug 2024 13:48:00 -0700 (PDT) Date: Tue, 13 Aug 2024 20:47:14 +0000 In-Reply-To: <20240813204716.842811-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240813204716.842811-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240813204716.842811-5-kinseyho@google.com> Subject: [PATCH mm-unstable v2 4/5] mm: restart if multiple traversals raced From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspam-User: X-Rspamd-Queue-Id: ADBB5C0027 X-Rspamd-Server: rspam01 X-Stat-Signature: uangn4krh4rx8x433mhdmeqcdhriht5z X-HE-Tag: 1723582081-818238 X-HE-Meta: U2FsdGVkX18PhKz52ae6DG2AbXrBWjSp5+8V4Ne8E++B49wgO4I13aIwRjLOZSuFzsDcc2R9Ba16T6No92kKi+SdJYLcG4j+MnqFQNET8FjbSdKT0LPqGdXjPc/KTB/jqoV5W+Q1q+FSCjop52KBziDkIsb3sGYO6bn9HmiuQVE8kSfphhvKT15Ef3IpJHNVDGDNSU1n9L3AbkSZoNrUziRvOf1KwtvCnyFLC1vU65tHQHAEGJGfE3ijzK9UTfg7hsze3EeUqj/zRHi9Ivd5W17KxIziH6M2+JlILOHhdUIzUlLNmPBGE0gz/kTpnknMj0VqOZ5GscW4/Fdo8qZ7YEdpCxgBn4Ge3nVD8o4EBa5YuJaczv+pSZz+1dgcVUlvVHotTa6w+wiH42Gn2N0hrXzFe9f1sfIhYxcxnOgq9Fmy+aF1tSsSlOmGYXRRArRQaJNRlVbWvNSlL3JzVeLW61pog2tXm1RW2q/zupBXCsp09Yrwq0w7LEDwDlWYj8yyAfrgueTnp+ZKM/r4QO7CQzbBH7Nbed9RKGKDob7rW28tpZ3nw3pFHNK0fuAkmHhh1PwdalxsjCTcB+UdwytiXPwjwLdexVkUOSx2jOXT5SBCHAgphvaQ4tXRgVLGDP1ikg8svsYi685VO0FHYuXifuahTmCYWU7AufATon25rbuvQ0iQgt/eZEYcaDaoB9KvZAUrz8Sjj3lKhSaPdQNl06JkuNGotJ4OYuq4B6KL97qJv8DONk9yOgiD2coqPtf2MT/3QsmQ7iabeCw3GCdYCOHr4q5L7B8f2GxFhJsmSBcRCCb6ZRba4BGg8IW2VKbYP1WmReQIdf+bQRSNTzZb+Nqubq9EHHrKORe7gfkkBauE+Eg+mOkBqcVdpUNDMzIjCx4Ng2+Z6imA9lc7hVZfaxlormISWebyNTQBUiup1NxNGHhONh5TwBBfNP61oA56Xr2u/TJbA5DfkObWcrK WqiQpDTg aUxryiK9vzaDL65FUwovd6C+ZADMtKutqC/RdAaOYsDWAH9H9g7P57Ztms4u3Xpn+1hU6cK2g8VKHhv7dj4n68bNsAoO8YhLMtNzIhi76rqw0l/LJhLuZZMp0/pl5eWjYmaGlhCshhKBMgRCQozqKhwTkx6AOfrw4S1eskBltNGxmSjFJCukyr7pARfIEP5be3M20E4py+Rx+qaHnCfsoZeUQvjm7KHvz8cpDB1qHSJ2htPw0aomUD8jZL9w73wQRka7pJHwn4FHmCfKBL3Er3GFkUWDDL2yxhS0hUwcK6tkB0QHqyPdPXS2dWmLMJe5+QoA4bo0vFup5vkWScZAC+Vxwu6d+pCAUcgG/q9clw3UPQ1hN0yu+cO1iB3HLY5yUIfHfdFQ3TdXh+pQZiDRpmAZ8CBWSYNviljZr+5AKjMJILn4qwxxzyD0+PuoE+NFA9Hc0BFXzx34/GS1vnoohFkg575FsmxyzRbtKwoqDEASA6a4HEd6q/Uhi3q7B269bkD5ACXOGbVQKKrzGpCb7mb5bkg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, if multiple reclaimers raced on the same position, the reclaimers which detect the race will still reclaim from the same memcg. Instead, the reclaimers which detect the race should move on to the next memcg in the hierarchy. So, in the case where multiple traversals race, jump back to the start of the mem_cgroup_iter() function to find the next memcg in the hierarchy to reclaim from. Signed-off-by: Kinsey Ho --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 22 ++++++++++++++-------- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 1aaed2f1f6ae..aada9ef3ca44 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -57,7 +57,7 @@ enum memcg_memory_event { struct mem_cgroup_reclaim_cookie { pg_data_t *pgdat; - unsigned int generation; + int generation; }; #ifdef CONFIG_MEMCG @@ -77,7 +77,7 @@ struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup __rcu *position; /* scan generation, increased every round-trip */ - unsigned int generation; + atomic_t generation; }; /* diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 937b7efc41ca..84de46ece9a9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1025,7 +1025,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup_reclaim_cookie *reclaim) { struct mem_cgroup_reclaim_iter *iter; - struct cgroup_subsys_state *css = NULL; + struct cgroup_subsys_state *css; struct mem_cgroup *memcg = NULL; struct mem_cgroup *pos = NULL; @@ -1038,18 +1038,20 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, rcu_read_lock(); restart: if (reclaim) { + int gen; struct mem_cgroup_per_node *mz; mz = root->nodeinfo[reclaim->pgdat->node_id]; iter = &mz->iter; + gen = atomic_read(&iter->generation); /* * On start, join the current reclaim iteration cycle. * Exit when a concurrent walker completes it. */ if (!prev) - reclaim->generation = iter->generation; - else if (reclaim->generation != iter->generation) + reclaim->generation = gen; + else if (reclaim->generation != gen) goto out_unlock; pos = rcu_dereference(iter->position); @@ -1057,8 +1059,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, pos = prev; } - if (pos) - css = &pos->css; + css = pos ? &pos->css : NULL; for (;;) { css = css_next_descendant_pre(css, &root->css); @@ -1072,21 +1073,26 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * and kicking, and don't take an extra reference. */ if (css == &root->css || css_tryget(css)) { - memcg = mem_cgroup_from_css(css); break; } } + memcg = mem_cgroup_from_css(css); + if (reclaim) { /* * The position could have already been updated by a competing * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - (void)cmpxchg(&iter->position, pos, memcg); + if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (css && css != &root->css) + css_put(css); + goto restart; + } if (!memcg) { - iter->generation++; + atomic_inc(&iter->generation); /* * Reclaimers share the hierarchy walk, and a From patchwork Tue Aug 13 20:47:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13762484 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31740C52D7D for ; Tue, 13 Aug 2024 20:48:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 397CB6B0098; Tue, 13 Aug 2024 16:48:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 31FEB6B0099; Tue, 13 Aug 2024 16:48:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14EBE6B009A; Tue, 13 Aug 2024 16:48:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E1B996B0098 for ; Tue, 13 Aug 2024 16:48:04 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 91A9D140AAF for ; Tue, 13 Aug 2024 20:48:04 +0000 (UTC) X-FDA: 82448409288.08.53D052B Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf17.hostedemail.com (Postfix) with ESMTP id BFBF74000D for ; Tue, 13 Aug 2024 20:48:02 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="FttFr/2r"; spf=pass (imf17.hostedemail.com: domain of 3gca7ZggKCDogejoaudkckkcha.Ykihejqt-iigrWYg.knc@flex--kinseyho.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3gca7ZggKCDogejoaudkckkcha.Ykihejqt-iigrWYg.knc@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1723582011; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IUGrqgCsQ93s1mn5jRCuzR3kYh6lZBiPS4o7+rn6SUc=; b=oRUxutSKILTi1Tx8yWEcTgXtWLbqPc9HROSl0XrKE21FmZIqk4WNxeTZYYxcTCBKkzQUaU ujqGsjR1uoEjN8YsadaPY/fCqVPLjPtirnj2Bg7HgWkopQS1HWp1nXkYUkh84qAJE90mu7 SI0HM4dNGuOpqESmXCTaPaPRKGcSIhk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1723582011; a=rsa-sha256; cv=none; b=jVv10PTJwXDbFjT1VWHg50pDuNUFGgg2wQYjZNrTa+t31pJ4ZfIKWabSEzDSKXn8MhLMJz pUFY+Ul1bQDS4d5otkXHtfeZ+N76E6WRA+BdC21ovlnVtcY7WQoyHIr1Qvn3IFaNu1aYC4 uXh1GQCpkbc/MHn/gyCzJZT1dsvMUlY= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="FttFr/2r"; spf=pass (imf17.hostedemail.com: domain of 3gca7ZggKCDogejoaudkckkcha.Ykihejqt-iigrWYg.knc@flex--kinseyho.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3gca7ZggKCDogejoaudkckkcha.Ykihejqt-iigrWYg.knc@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-664aa55c690so135083437b3.2 for ; Tue, 13 Aug 2024 13:48:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1723582082; x=1724186882; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=IUGrqgCsQ93s1mn5jRCuzR3kYh6lZBiPS4o7+rn6SUc=; b=FttFr/2rlFddbtg4qj1Arhp5lyL7U1kEa+RDWLLK43NcazJfoK7Z9XTx7BouJtjAVj NJgfZaZkZUoHDiXA/bLivcoXYn0sdmVWTGcTAc0wxSTrcBKcAuQITob5q2twCo0PoXfw vqPmR2aFZXpJabGOJWcmSDaKtZbyU6tBmaAwhHhXDlx1Rr+peNRLP2UfT3emlWC9TVl1 Tqw6871RTgDUgFoD3fGSB/OC2/XOZAzm6wED1RZriwOHB4IHlsogUE7C0UEDBnpYD2Sm G7xKYlKWT/AuyNNiXtvMW6vx0cP3WK2nl0vi28h+/i+A7cYn8nB1KXyjvW+X77bn9uSP 7hFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1723582082; x=1724186882; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IUGrqgCsQ93s1mn5jRCuzR3kYh6lZBiPS4o7+rn6SUc=; b=QBC/GmZmeSpPWKlsGFRMNe2q2Aw6UsdgF5bsO71y5jmxryiZ9rg5MtbcvmsZ997/HX Hl71ve8gYYBcWI8zoSqQZGFcQvzJw0xG3/NBsngIOB6trLuse61GjqSKiA3/NaGfgig5 x8wgQmuGOeFxp0AN1gIrdvK4XPrI/dJKbYtE/+ZMLjzl4ZIcSazlGiiz1V4uswhL4v5d KIdlri60GG/mxkt3QbJDPiHvB6Y1i5EpaToATEmYdcWa3UDeLmZWgRidaTr+xoy9bSDL S2KczL8Wb8mYxDy/DFA3jTviTVjqW/BMFBuw7OkdU0/3ASkjpLVydYneNt35s2CGjMLs JVlA== X-Gm-Message-State: AOJu0YyYcSfaesc8gVufnF6XwRfxgIGrlQj8N6bpGPhI+zshEiFZPGIB NJfcQ49PXzR1CfMKa3jKsJzJUcLxxbZSagwie5BBSzkoovlhVqlIRxhtQmlZw4Q/+4qCmqgoFMI gOvEkVW6odg== X-Google-Smtp-Source: AGHT+IFKq4mle76EHVxRvn3g1KvqjdbIlRdYAcckM9mm8INZMPhwqmIEIFFgUqblYHs8iR9k3Dh891L/+5f5YA== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6902:18d:b0:e03:589b:cbe1 with SMTP id 3f1490d57ef6-e1155b67c75mr35668276.7.1723582081844; Tue, 13 Aug 2024 13:48:01 -0700 (PDT) Date: Tue, 13 Aug 2024 20:47:15 +0000 In-Reply-To: <20240813204716.842811-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240813204716.842811-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.76.ge559c4bf1a-goog Message-ID: <20240813204716.842811-6-kinseyho@google.com> Subject: [PATCH mm-unstable v2 5/5] mm: clean up mem_cgroup_iter() From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Stat-Signature: 3bgwq4smzf8fkwybn6im7abuh3qf87dd X-Rspamd-Queue-Id: BFBF74000D X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1723582082-59884 X-HE-Meta: U2FsdGVkX1+AsnlOGf7zj9mf2OvdLdPmhOKQJgUcgcxdfHIZaxjdVrKLMg/4WV8jDdVHU6YJX7WpD9NTeENH10Lktrc8YmL4l1uUIC+7ndvo6yAISIzI19+ddq04WnGtnD+vfRwd6uBNfC1vwV1PvI5eP/6jeCjXf99ewGC8ahbjph7U+eGwmaKtg9ungAam7pXGBwbsTlfNyvyO6hTpbYKaZaYoD6pdc6+RX+aI4DKiu4+JnCYNvlUCa6Ior3FFzfOaaPi8+jyU8pbgKsX4i1GYxeRQ7D9QrEqBmxq1ZZ/dxKEuoa12Pqxpwm9EqlzZw0fNrSS61zxG4cpbM6NDnxDNdiOFkHWQ2BtJNNJ1Atyv88Jd9itTQ39OWxV0Xr+XY2w7WFLTeDrT3Xx6e/FS4tpc0AG2Q5RPOTCWXfpWRbgn22VxgYs7m9vTZf7ZwnOfmyYLa4fIiB7ftj0UWAQlDSYeQfjZn2v0h8TzU/fcUrR//2JRxdfkOQy87EHXY+i5x3o9jVHFb95txxtE407ffcLe8yenw8EZE3j2zmfELyn+QYglHnlDTz5pVvbnXGQXfcz54MdDdsYFkAznSK5w9NrAHoLTm/VDjpWL6KhDwLZVc0ROokVqI5px2ceFYtZDy7Ecy3yDyaiMAMYf888Zsnw6QkyXYdA+B0+WH1ElJ9BpkQmZP1Dg357tg2UTH3/jzbAyeD6ULqrTmJIPjFTBAuOqiW1qABJR540Ag1aCybLmGMNE9CEOpn3yn6BryrWvsPQYpwNlrCAce5cCBLl1PY3QY48zM8qe2FD2P4rUBxyJLwpWUiq68g2ljMOESgivL3I3EyHhTILiz68PiGfZmFJ4SGSxkAIpSUtqQHC9YlMuXO05IKMJZFCHp0Y7kPJAr9/XCgR6pm2+j15HbyqGsSVAH7oOLeODPmc4icZ4CyxN1dIiri7UCge1+/0+7vKQRIDUYm+WznimdQwPx7q p481a/1u gM/4sGQIuZ1vniFVY9okEoOmMsfZD/G1gAjVUCHLhyJg5FX+cIdTnHyDebgK1jaxtBp3bLh4V+9PBy17kKhRnVHjBD7KdfOtGVaP+0kpI1ZgXGriNoPfDwPP9TOTNDTLPKUnWOFwPMvLjkF7CS4vuiW/OB2mAuZUW6Kp6Y8++QS7KIAy+WbqXj/Mw24YH8kyEIRdWMNv0juT2r3GFJjx4KcnK6rNe6dUjlDZ4pntGcn3pIt/lByCDKYorUxlsxbHVrb6S62iEJKgPfDvTOQbYShYooQWbJFGKIWjfRb61qUARVeEWFfiYPvZzp/KuS+bYkVnrh0e02UdBko7Hw+yV5p6AsRIUxyACXPPcjuIKQ47w3UZaH/2/aplH0MDlki9WNkIkW0z9v9cxfvmw1jxFK/Gly32faVx/o08m5ZlYENzcm5xxQLAVdk/QKS1VZ7c92wOK8a+DgtdpJhh2b+8fl+H2vMNtspodlMEK0VXtTtIT8efBAAHFIJwrYa7Yrw9iQTjRrRtR/vWlgcWShXCzk3h+fTPnUejh34CKbJgNYur2t925NO5pMdUfol59vcDVDSPYckH9+OiDRhA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A clean up to make variable names more clear and to improve code readability. No functional change. Signed-off-by: Kinsey Ho --- mm/memcontrol.c | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 84de46ece9a9..87a0dc9d779a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1026,8 +1026,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css; - struct mem_cgroup *memcg = NULL; - struct mem_cgroup *pos = NULL; + struct mem_cgroup *pos; + struct mem_cgroup *next = NULL; if (mem_cgroup_disabled()) return NULL; @@ -1039,10 +1039,9 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, restart: if (reclaim) { int gen; - struct mem_cgroup_per_node *mz; + int nid = reclaim->pgdat->node_id; - mz = root->nodeinfo[reclaim->pgdat->node_id]; - iter = &mz->iter; + iter = &root->nodeinfo[nid]->iter; gen = atomic_read(&iter->generation); /* @@ -1055,29 +1054,22 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, goto out_unlock; pos = rcu_dereference(iter->position); - } else if (prev) { + } else pos = prev; - } css = pos ? &pos->css : NULL; - for (;;) { - css = css_next_descendant_pre(css, &root->css); - if (!css) { - break; - } - + while ((css = css_next_descendant_pre(css, &root->css))) { /* * Verify the css and acquire a reference. The root * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { + if (css == &root->css || css_tryget(css)) break; - } } - memcg = mem_cgroup_from_css(css); + next = mem_cgroup_from_css(css); if (reclaim) { /* @@ -1085,13 +1077,13 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (cmpxchg(&iter->position, pos, next) != pos) { if (css && css != &root->css) css_put(css); goto restart; } - if (!memcg) { + if (!next) { atomic_inc(&iter->generation); /* @@ -1110,7 +1102,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, if (prev && prev != root) css_put(&prev->css); - return memcg; + return next; } /**