From patchwork Tue Aug 27 23:07:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13780208 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EF0EC54742 for ; Tue, 27 Aug 2024 23:11:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A42066B0088; Tue, 27 Aug 2024 19:11:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CB356B008A; Tue, 27 Aug 2024 19:11:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81C066B008C; Tue, 27 Aug 2024 19:11:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 62E726B0088 for ; Tue, 27 Aug 2024 19:11:43 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 93735120573 for ; Tue, 27 Aug 2024 23:11:42 +0000 (UTC) X-FDA: 82499574444.28.35E6D56 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) by imf02.hostedemail.com (Postfix) with ESMTP id C781580012 for ; Tue, 27 Aug 2024 23:11:40 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=oNMMifdo; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3K13OZggKCFwECHM8SBIAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3K13OZggKCFwECHM8SBIAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724800203; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uLJnrvY/TNbJZgveXjXzKwi1waLm2OswD6XrKXtY0Dg=; b=BoXXS7paHl29T4eFbi1h42yT9A2cYNwUsKRZSmcLS5qZ4S4cQ+dtFD9caWcDHTcHf7XNCE RVBaTeWBHu6aeAlXWj+ROb7g39PsiLOIHFNCUeBhoKvBTC+oSmXOdSvEhX+kukdNOrEzgu 6gawmihZA6ZBwbNaRpBzDU7pKOnnaqo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724800203; a=rsa-sha256; cv=none; b=66bRhQeRDZ0OhgsYSs4cJLY0tX1Ra3rHzrldoAArx7rJVRRwag+AamgfJ8vLgcx/nLf2Hl 5qeDbbJUk9gcngalrDgvxrcJcvtk6lztmDjO0kpwjnme3G7MWn9hvlvJ5vglxr0J21Ly8/ JxdJxVsnO3o6IyvD2mJ3hggScHkqPnA= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=oNMMifdo; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf02.hostedemail.com: domain of 3K13OZggKCFwECHM8SBIAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--kinseyho.bounces.google.com designates 209.85.215.201 as permitted sender) smtp.mailfrom=3K13OZggKCFwECHM8SBIAIIAF8.6IGFCHOR-GGEP46E.ILA@flex--kinseyho.bounces.google.com Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-7cd7cd2f915so4785097a12.3 for ; Tue, 27 Aug 2024 16:11:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724800299; x=1725405099; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=uLJnrvY/TNbJZgveXjXzKwi1waLm2OswD6XrKXtY0Dg=; b=oNMMifdoyd4HGcN88WBLwJV/zLBNE4DujjHmVTxT0UQ+D9vPltRDCyPPHm6VsqekH6 tPkXNA3V9rrcdMygW6bVDbkJd1KFqzn5HP0uHbma2emx0+nlBQIBlXoUs89ciEuZeGs+ QHzoGiOPVSApNZwZjrx7+COLWb/VvhH6QIk+CBDTFZ4lxfn1Es+3rU7oumXfBZD926o8 4XtbBDcysWHNcByG0OcT48RelCvU6OSe5p8k+pCkAb7NK32pVOZlzsclRFHziGOpvEM4 /acvJOx3LTsid4pKbsuKgWvLueDSQGRPDinHpk6YYI57hBsNYktu/tiqAG7CJzNXtduO mfTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724800299; x=1725405099; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=uLJnrvY/TNbJZgveXjXzKwi1waLm2OswD6XrKXtY0Dg=; b=A9QDw3dIQIFIYr0j0HXCDVsEVFrdUQZKe4hvLGdzXBywGAa66+i1LryXxvPX7lOtYV 2h96qt/hO6nudwN+u00mHNoSbxnfgipI31yRTu7NqOWIEZgt8aNnJPz8IKzLXi+ucdB3 Sb8ArAjG/okNorqrxJd7i+QnHTs+PRnj3i/F3/KeQ7wZq/RyctNq99w1ZlLBFfznzhWy pMKediepT+fr+Ej6c2zgysIVul1NTLoJwC3pI8VC3c0oH8PlMVooAVX6ejzuUwHQzYBh iIzoytTH1+GwP/Sx1EhqmldZWJ8G1hGD5oixGh4OqSvzK+oI1+LjYm4oPN8Feu9PWjWX WXog== X-Gm-Message-State: AOJu0YxXdOS24hOTJ4Io6ruxi8/fV+q/ubDznMxc8iQfQb4QYO18Utbw J8qSz9F8dEkROfhgpS+/9dn6DMn29Y361JO9YHo4KCEIG8uGbNUdKQi72YKNviHnifXi9SExCWE S+nTChIPR6g== X-Google-Smtp-Source: AGHT+IEokv9e6LuULnuwid4WZdlPlarG8J+pXfvpDb7CgbDsiCb9RtBxcqB82FKfSIIge11nvt7XccQqszzV4Q== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:6a02:5aa:b0:6e7:95d3:b35c with SMTP id 41be03b00d2f7-7d2228eb22fmr302a12.5.1724800299140; Tue, 27 Aug 2024 16:11:39 -0700 (PDT) Date: Tue, 27 Aug 2024 23:07:38 +0000 In-Reply-To: <20240827230753.2073580-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240827230753.2073580-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.295.g3b9ea8a38a-goog Message-ID: <20240827230753.2073580-2-kinseyho@google.com> Subject: [PATCH mm-unstable v3 1/5] cgroup: clarify css sibling linkage is protected by cgroup_mutex or RCU From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C781580012 X-Stat-Signature: 7ak8i3k36p5kgj3mhqqj7tu48isdzd4k X-Rspam-User: X-HE-Tag: 1724800300-310666 X-HE-Meta: U2FsdGVkX18jjg/IbB9eS17AaPN++cXOv6qAAtgGNMYe0jrWU3hbRfhnp8Xsn033CuCdv7WgvkvXl1qAbyP9SAWGq+gy8d9Ecae9uc+iW3L42M/wPRsaUEEn7HnePRoWQrnd1SaGERSAxCDu/lHxSL3u+QjN4sW1Q5R5105yucIXuAB9yw5m2jU4p0udOJLWaIezhKGkJVLkwjCZ+jicERJP1cu4OAhzQZ3k9TM2PWGXRQzVExm5Jma3Ptx7Gb1VZZe8h7nChBVaBOlTx44WCfxz9SR8zbu8Hj0cMmt+uqv6aUPoE+z873JgTbRGBGncu+bsVKHx9LqM3Nvki/sTZFRXurUXQYyeZZBp4DHFVxvK9ypPMwTJGOeh6QSgJHLz3mjVZ2aXrOSjQVANSTxrg9MuqFv2fNwml8a8thGXX6ltxR1qIRtiD/yo2wdvcs8OULcwSJ5AND4aSdHB7Fg+5zg+RsYzEnTp/MFrNFSMewPpzyjcJiu6JbxQ2ANAx5UWndtKUpMm5Vu1kLKHu7MQmIUDcZMFwr2FI2DSOU7l2b2wGCkx8H196D3oE25nk/1RtpUN3cmKyDYkGEQ2Qbs6cSWrimEFg816RXZn83/6BTIe0XS4cAQ0rgqG5BPCQn/l0g7OoVT1kZGlHKqguYhi2HUZMGGrd8x2A68DB6Axs+0hU3iNJknUhhsYAOkpUoTwK5okJrdI3iReg7Qoy9VkIOIXVEzrR9F925AU6ujG30VvunjSuMOaCbeqCAmxj9gCI+jAdYzmNGCGm+V2J5zMsGhNqQP7HpfV1w/OF+E3jvSmSNPPTxS9HQR9hvTEaKDr2eyXlX+miqWlxMn/QULMJkuU3smzK/i44AcooqgvrZKDRv/bPcfeBcPROc3UoFR1sqdOVEIy3Mob0xAEcpopayTvrJeQjnJ0Fe5HAzQyf4600+AbnhgaL2Vkpw0uscXahdcg8aaQNRuvbh4p/p9 JIXjKgQb BRgRBd5ZF4M/KNayytFvuqK1k4EHcBfkSaFXZPApt56yBfYIHFXpllLQOpcj3bPWWGUIbD1lW5tlBkH+KBiocoz1Xng0rrpir/Fn9N3lHWvAP+aSncRv9YjeSalpIaMGAchzZD1DQnv2ksh8CwTc4eIYzrNdkcOTZf79o5GKsboJi5QN3oX7eGknmQXa76ELv6lqJ08FiRgni7DJzP+bF72T7Kc1HkuTx/XP0YwJNsa/zOf/dAQxQvVMwsWGsGQ9Jdfkm5pjcGWo3L/DylGB2t8i/r/oVA10Ai/CjROKrrOIaFO0shgbITCoq3Rbh5gdXUMNLhAyLRTsWS9nJV8JMWaV1YdjPpKe56DKxgsAsmrbdfjQa2SGO3tV/QbOB00gBbX+/H/LQw+TIxVmZSzTghu0sWmEHYN9DeduSXoeH+vm4jCsJF+d5lTNL8WuxJyehSReWlON5VDsi+Qh0mWQNDrkF84osMs6KGfDMhYlto77OzjM1+84znIm0VLRZwyWIHLu8F5w2PDmB6WRz5svrcNsOpxOe6AZya8gErt2rkSZf/gdpYHeHySp7AhRh39NrIojKvCnFe/3inqw1cJfcevbT+MIcbzUzxMJOUpe6tEbKSpSFCEX42gipT8QJHIBRr78BAR94Wl7t83gOBkmawjHpoj1wHewG/JjJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Explicitly document that css sibling/descendant linkage is protected by cgroup_mutex or RCU. Also, document in css_next_descendant_pre() and similar functions that it isn't necessary to hold a ref on @pos. The following changes in this patchset rely on this clarification for simplification in memcg iteration code. Suggested-by: Yosry Ahmed Reviewed-by: Michal Koutný Signed-off-by: Kinsey Ho --- include/linux/cgroup-defs.h | 6 +++++- kernel/cgroup/cgroup.c | 16 +++++++++------- 2 files changed, 14 insertions(+), 8 deletions(-) diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h index 7fc2d0195f56..ca7e912b8355 100644 --- a/include/linux/cgroup-defs.h +++ b/include/linux/cgroup-defs.h @@ -172,7 +172,11 @@ struct cgroup_subsys_state { /* reference count - access via css_[try]get() and css_put() */ struct percpu_ref refcnt; - /* siblings list anchored at the parent's ->children */ + /* + * siblings list anchored at the parent's ->children + * + * linkage is protected by cgroup_mutex or RCU + */ struct list_head sibling; struct list_head children; diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 0a97cb2ef124..ece2316e2bca 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -4602,8 +4602,9 @@ struct cgroup_subsys_state *css_next_child(struct cgroup_subsys_state *pos, * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @root are accessible and @pos is a descendant of @root. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct next descendant as long as both @pos + * and @root are accessible and @pos is a descendant of @root. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the @@ -4651,8 +4652,9 @@ EXPORT_SYMBOL_GPL(css_next_descendant_pre); * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct rightmost descendant as - * long as @pos is accessible. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct rightmost descendant as long as @pos + * is accessible. */ struct cgroup_subsys_state * css_rightmost_descendant(struct cgroup_subsys_state *pos) @@ -4696,9 +4698,9 @@ css_leftmost_descendant(struct cgroup_subsys_state *pos) * * While this function requires cgroup_mutex or RCU read locking, it * doesn't require the whole traversal to be contained in a single critical - * section. This function will return the correct next descendant as long - * as both @pos and @cgroup are accessible and @pos is a descendant of - * @cgroup. + * section. Additionally, it isn't necessary to hold onto a reference to @pos. + * This function will return the correct next descendant as long as both @pos + * and @cgroup are accessible and @pos is a descendant of @cgroup. * * If a subsystem synchronizes ->css_online() and the start of iteration, a * css which finished ->css_online() is guaranteed to be visible in the From patchwork Tue Aug 27 23:07:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13780209 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 775BDC5473F for ; Tue, 27 Aug 2024 23:11:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5FFF96B008A; Tue, 27 Aug 2024 19:11:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 475696B008C; Tue, 27 Aug 2024 19:11:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2C5FA6B0092; Tue, 27 Aug 2024 19:11:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0B0B26B008A for ; Tue, 27 Aug 2024 19:11:44 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A6BB2A19E0 for ; Tue, 27 Aug 2024 23:11:43 +0000 (UTC) X-FDA: 82499574486.28.6C6348C Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf30.hostedemail.com (Postfix) with ESMTP id EA0B18000D for ; Tue, 27 Aug 2024 23:11:41 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uhpmkNJd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3LF3OZggKCF0FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3LF3OZggKCF0FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724800282; a=rsa-sha256; cv=none; b=gZJTGDEhSNigXwRY3Mb+GnNAKXlKOR4YVCMSYRTImSmI9etF6MjyjQQNYm39I/iAAEQtw7 cd6agpIQcyQp2/Ym1jbBwpc24+9BlTKiBNmZ6+RP5vqQnxweQywWS+IL+AWHoC0r0ArxD3 aEy626a13+0sPEZtnj2+CN6wjBYtg0o= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uhpmkNJd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of 3LF3OZggKCF0FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3LF3OZggKCF0FDIN9TCJBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724800282; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QXZSgIWrSh7o4TMYH5HXuFFT6CvWnPSrTikMMv9Xu6c=; b=1mLOGo3cxKKFMN4HdJ7ZYWOkhcuyGPUwEhhjXNIaihZPlRMz/IksXSjGzLcR4G711qzbw8 lCq3IUZAaTtQ507aoRkXHVC4lg9K31vMWLjT1hGlZjnrx14TSAQrNNxGs2iFk3gfj9vLrM hRaX0EyDzLphYKSRonaMuDGwo/g6GOg= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e13c3dee397so10563661276.3 for ; Tue, 27 Aug 2024 16:11:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724800301; x=1725405101; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QXZSgIWrSh7o4TMYH5HXuFFT6CvWnPSrTikMMv9Xu6c=; b=uhpmkNJdxQxSLn464NZHDJg9lsVhucwRlZbwiDHGsIGpXiEQwJMs/O7A6mY8JZnMRV jZMU2tSRgW1WmYzrKSGaZ7e+kZpUAQCtuPiL87e/vnLsSgo9d7B2YrjnMajRDZdkh7EJ csFOn9f55ZgxX4VcCkto4PxTXIX3O8q5enXZr1/XfBi8ODfgi8ZOjJLpo0gAudCQrHVA Jm4ck4vXNoA107o1CMCcyETCWq6/1CZooIbkAAYS90KpQ6I83yqc3XBNp4H0pmhA8s48 9lOfBFtGxwG/MzYKEtyViE2CWlcKCp8CNwtZDXpyNo6uKB6oh7H5aVfNDRcZEo5yIXbd k4jQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724800301; x=1725405101; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QXZSgIWrSh7o4TMYH5HXuFFT6CvWnPSrTikMMv9Xu6c=; b=GocWSEZIu5R7pTRoaTzs+vJLuzWLabV2Kquet2eDj+r/aDzPf9AIwvoeY9OT7ZXvuP mvSm2q1mb0ilwrwG5uZRmeqL+ffIzxfWcCbfeVnd86b7nMIwnuTIM5PWkYYkINPqHunj qf/ZIu6aSncsXKtk9+y2eDIpJ/UD3X1LvZ1trQwwH5MUm7/eNWIyRcl7jrXdIYHejo/B fv9wLKycThxE/+1XYfuY779Y1CCSj7Eu8ScLQ87zDQnBjNezW+lv1cUnNzguJeZaa74V ubGJwIehlMSEB3fnH0iHYOSwRbCJkETukLkoCebER06KMgYr2W4y6IAN1tf57f0/YqAc w/Eg== X-Gm-Message-State: AOJu0YwAOYAoliBtgdVHtR0JOzP+JjPvL70iV2v0YQzUCQ4nGb4BY9+k sd+qoMjWt+3FiWTA1e0YmEpKfO1VDmYnDhENMQHIqJePJ6mVHo1qERmeT6feHLIyLwpHGeUg9K7 jv2aNx0tuoA== X-Google-Smtp-Source: AGHT+IH/OEMHPdWupnXyjW/u5NZJ+Iwna53tKUi6hE4QApyk+GLsJI4xuXktR0o4bdU85PWSweUUV8sgsDuT4w== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a25:abad:0:b0:e11:69f2:e39 with SMTP id 3f1490d57ef6-e1a4580c518mr380276.9.1724800300879; Tue, 27 Aug 2024 16:11:40 -0700 (PDT) Date: Tue, 27 Aug 2024 23:07:39 +0000 In-Reply-To: <20240827230753.2073580-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240827230753.2073580-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.295.g3b9ea8a38a-goog Message-ID: <20240827230753.2073580-3-kinseyho@google.com> Subject: [PATCH mm-unstable v3 2/5] mm: don't hold css->refcnt during traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspam-User: X-Rspamd-Queue-Id: EA0B18000D X-Rspamd-Server: rspam01 X-Stat-Signature: ebqi4fg58iuixpco8cs6njimmyd4bh85 X-HE-Tag: 1724800301-824657 X-HE-Meta: U2FsdGVkX18hQyONVPh3pYXtszZEHWkSH2ZkXOcgBhEm7xd9SMWdbcKxWtXcZ5+eyqYpkQzEhAQrwi5Xj4+zj26PXXU6izloNfKlM0tDS9jzSSEnJBofyiKgZ269z2Z8LkdX/Gwitw5m+MSfQaJgVLvo071+p2pfkd9MYmMO5cKo/ziROPo3+ITgyXk9Tqz0xtdAudiGGcgfZWKk1Zy0yMK8o4iZxxsf0F4tunBQ4klrTbHsdZCfcVB+FeDZLC/2lELquxKA5Ljf6aYe0W4j8vvHeA3MjWOviH9QjYQ/r4AbJtLARTWm6ewz+NcqHCnf0aRWH0MLwAiBE4B80RUoQWtKvYobwnY7VgVXDVSTi6+jHWh+Q/W8DaXrH1uJdQFsTHQsd35WuLYnC746crFlJ93BxkxqMS0iHVAaqi9l6DSm6aFrzbm1qg0OFeuNEW7ntPUxRFSzhyVdDXQNMLH0CVKskCukYj9/keETUPypVQZlykdKCBUGOdTLHlTCFZUnpkYJ4RadvmGITsqYGOyWb/R1+bTz3s4oyfaAkZwhEthXFJotXyXdBEwE2btPAJHBl2r10/A6slSWO06Zyki3UgskhjL7GrlRJ7BBCtDuqFV7+A1lp47DDLW+BNGXcG2rVUGDBYxitFDNhJklSHr33mlceWFVR2UenY3f3OTsFdKrAkoyDheCzduqDIkBW+DKFAHmrbjSRqSjLqAgtoSzTfodrD4cMYtM1Mqm6Ok0M8cvYh3cqKwksMC1GBqn9/iABDfM/5zzf2eUh0Elml8DGKlFRJZblQCZ4WUtRT0I2Koo6F/j1Xbnqs6reDa8W0A2OjmIVon872zYp8G5jW9CcIWYOOa4ek/i0thdAJYJh9+X94GPzDxDsU2lvZPuvaNaW+FWnGImIeG31TTdfcpz0UFWUWjBb9HtUU5oJAtzdLJx1zoR1O8tEeGd/dykrYmfVTGp5kBQ13F6yy8Eb9K 3NAc+P1B 2K/Z8f1cCOnSlFOcNO1fikFjpCP14lkbyyCk7KnS2qZIAqQ08eIzy8v7z59diXz8714340/ctLNw5vWr5v+3dPJ3KOY/goY94DD/B9DtKVXPstOfYb5fMxn31goVSaCPxNOnOGQF8dhitKl2SL4ar3GjUn0PCuSJhrkKWdlIdt/jK9+Enf87rpMLwYdVu5LujFbl5FH1VAQ5Vz9+0ARiz/VcM8F8eJAwIXtZCTG4/U3yOJuBK565qFQZo/d8W5NO1d4ugjkPVZaD8CQjxoOAX0Kne4PhPJHo1o8CpbKlRLAU8RratBQuUQcBoAR9Jx0AzJ8dgHIoYs6hKvpOSUx+/UR7CgkK0CPmIzHf+lmo12bXrZ05wxMzeGfOoyeb6TWloXwO12fyAa0tBCCux9XYqynkVpoomwXi087nRjrdGIDQLIuD4iCrWLYlrI1RRe6tujhUMpC+e+4fGQgDH2fPJLcpT/5sOtHSh2Q4hHPrmjt3RcR9zV6UOQdNIQVz6ktk+N5p8ncRY1y4qo/qD3FroXy5dC49leEos6aFkGsKIYyv0rAAaii0tvkHDeg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To obtain the pointer to the next memcg position, mem_cgroup_iter() currently holds css->refcnt during memcg traversal only to put css->refcnt at the end of the routine. This isn't necessary as an rcu_read_lock is already held throughout the function. The use of the RCU read lock with css_next_descendant_pre() guarantees that sibling linkage is safe without holding a ref on the passed-in @css. Remove css->refcnt usage during traversal by leveraging RCU. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier --- mm/memcontrol.c | 18 +----------------- 1 file changed, 1 insertion(+), 17 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 35431035e782..67b1994377b7 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1013,20 +1013,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, else if (reclaim->generation != iter->generation) goto out_unlock; - while (1) { - pos = READ_ONCE(iter->position); - if (!pos || css_tryget(&pos->css)) - break; - /* - * css reference reached zero, so iter->position will - * be cleared by ->css_released. However, we should not - * rely on this happening soon, because ->css_released - * is called from a work queue, and by busy-waiting we - * might block it. So we clear iter->position right - * away. - */ - (void)cmpxchg(&iter->position, pos, NULL); - } + pos = READ_ONCE(iter->position); } else if (prev) { pos = prev; } @@ -1067,9 +1054,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (pos) - css_put(&pos->css); - if (!memcg) iter->generation++; } From patchwork Tue Aug 27 23:07:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13780210 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE171C54743 for ; Tue, 27 Aug 2024 23:11:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B53B96B0092; Tue, 27 Aug 2024 19:11:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ADE656B0093; Tue, 27 Aug 2024 19:11:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9300C6B0095; Tue, 27 Aug 2024 19:11:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6D79C6B0092 for ; Tue, 27 Aug 2024 19:11:46 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 33221A70D3 for ; Tue, 27 Aug 2024 23:11:46 +0000 (UTC) X-FDA: 82499574612.11.F97B2D6 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf04.hostedemail.com (Postfix) with ESMTP id 6A70F40022 for ; Tue, 27 Aug 2024 23:11:44 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Rawjuc2t; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 3Ll3OZggKCF8HFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3Ll3OZggKCF8HFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724800233; a=rsa-sha256; cv=none; b=qyd/DLtzrPKgerK5kKls5uaSoXI+qxLttg2Vp01BuDDyK7FXnN+xcsR2YCr0I4dG1svtqo Cl97od7b7Zd9Iw77On/PTmEmQTByn7uLh0w+u5t9mGwWqPtKWkHu4iZhH4t9EknJuQTcmi /r7IdZ1s+6zjdqOe9G7JB6s4fjPKuaw= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Rawjuc2t; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 3Ll3OZggKCF8HFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3Ll3OZggKCF8HFKPBVELDLLDIB.9LJIFKRU-JJHS79H.LOD@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724800233; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pVbBxbMJe3JYZfDSh84GXFqWqtxHIvJybjs9yUa4Qwg=; b=4Q6g+dGvueEFM+ERJF7IYtRexRcuENQ4zTg6vjTTcWiia3h86wsxoWA5ll3MX92urmw+No 6Nm9eFqi2opiwZVqcLkdnA/lt9IZUJzJwjdxL6HgNFtcIaUEMcLNgQahwLZkkAUWnPiExM FKMbo06FfgQNfnLtotZfa56ZKpmXea4= Received: by mail-pg1-f202.google.com with SMTP id 41be03b00d2f7-7cd98f27becso5963654a12.3 for ; Tue, 27 Aug 2024 16:11:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724800303; x=1725405103; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=pVbBxbMJe3JYZfDSh84GXFqWqtxHIvJybjs9yUa4Qwg=; b=Rawjuc2tfwhCKjHDsXxXBGKUmpOO8csO5hcok6hwDJkJwDkpQmJpY368y7FipAeg5R BQqs3jCRnHQbc7mR7Wtlfaibhv8ZXkyjuQk7FVR3IKSprHaB3oB+UtAqgOpFgzHjJVZP JpNmCyJVxNhgj1vVvRt46Z292P4ETKePJXbae8/60H2uDd5HKGeXhNNempAgYAbEW424 SWHgrH1ATEK04CVGC/RxukKpdOXHwM/pckZ8z5xXBsGJ76gFdR1EGlfP+fNjYb0R5uE1 sTnsLgvh6ERna2Qd/eIrEpsGl9xIBzeY+Jon4bLuy98Dv15xIMJxkZ400WkhO2CMC1hX K5QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724800303; x=1725405103; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=pVbBxbMJe3JYZfDSh84GXFqWqtxHIvJybjs9yUa4Qwg=; b=ki/gMFpJnn3pN03P0jOZSQ+TUJtppjk4Z5uZ32zNw0dzNDNRZ2I0/Yw2NL6BWNJcEV 0teSzFwHeLI4L2CsYQG8S86cNpEF3uMH2Ze9crBUX0XZ3QN3WFyy6oW2+qW2dzyqnAt1 G56GH2Izl4PXa+RrrrrWyIB0k0Dbb21c/OohDOYLid0U94IrIDHA0rrLJz3+zfzkCwaK gxyFVvyjge30nvOPt8LjyMVMm/6XFf8PHN3syD40TxcCRPbmqQiaTuCm0cbejMP3i8mH 22mpCIxahyNRMlqNk7LRIx5wLCMsRaq+tGuNTmW0cH8kv6mBBtbnSy5qyZW+WoO7sCot 0lAA== X-Gm-Message-State: AOJu0YyRRDyn78wUrg2ZVBLE7rPCvfSYOgW2C4PBZYmFEGKWV1xSN/Aw gyR5YKSt//vLLeIimPxZMS6/wtoToLSw/cGgviVWaMgRzaRBUoOhg02yGtqqNAavNZUHKtbqap2 1q6LlCLJ5IQ== X-Google-Smtp-Source: AGHT+IFdVh/6ZDHdjnuvg4eof/eCc2TmxxMkSBRAT5EDf73D7CKtBh/Kuj5HdytE0rc9XtN+OFRBe12aA8e0gw== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a63:d70c:0:b0:7cb:c8c3:3811 with SMTP id 41be03b00d2f7-7d2228e5badmr309a12.5.1724800302880; Tue, 27 Aug 2024 16:11:42 -0700 (PDT) Date: Tue, 27 Aug 2024 23:07:40 +0000 In-Reply-To: <20240827230753.2073580-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240827230753.2073580-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.295.g3b9ea8a38a-goog Message-ID: <20240827230753.2073580-4-kinseyho@google.com> Subject: [PATCH mm-unstable v3 3/5] mm: increment gen # before restarting traversal From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 6A70F40022 X-Stat-Signature: m4t4wqxqpmasihytff7ddhec7fm7a5fz X-Rspam-User: X-HE-Tag: 1724800304-972282 X-HE-Meta: U2FsdGVkX1/qOq+Zv81dQCSSVwEcnij49dRzZuudchGfP0r5FKv1f5bncQW2pnXhdbXmwYC+/gO037ylYc/FpwjnDYHhDnU0FqSlmzMgqwrUgBq8wE7dy4ROQk7fd03R7caCvTirii6vKTnQU9AaRChMLJV58vyXDGKrK2RmoNeo5GJM2v0/SRTydqkw2aJwEg3CGQvuCwcv567fbhlCkfHGzU7/Xs5X/iRgrXHA9Cm3d4Xpj0RL4elDv2s5i8tjB+Q0rRa28tJTlLkl4qOGnJawjjMMxSN+2Ub0pPyZjUL2U9Sp/eReQDIJ68EmIvERCZl7iJxf1DZ/7YmnL2MBV7dFIXfvLDFuQQcm5ERI+dyAtzfJi/RPkksvtpXgvFFGNVSUA4r4/4rxFU62KPlrJ07RshMeUMcJL0zyYD87x8HcLUGItjwDXs9XAPsn8qbi8udMNT0i+uVFMx/ry3UTQx7JQVW900QlvlcuqQZjV9xq/rqSOmjcF3kdAdYsRVR5S+Q/Au9OGcCF20Ld4oEamx+nqNCu0dBjZX5mK39Bwnbg7wpeh9LcAR9umnTYTWRwm0lAD/Rx6Mnqtg/EMecuZtPR8V0GWo4a9nTh3z5yFJwZzh3S0q6tSoyXAsg25BgBGId3ZXA6VaaiAEYQajN+hTlEiGPcCHMM4WslGe/D2enwHfpOrAbrG9mJdERuh/v7g0AL50wEx45aizwCsdb9HTHKlLu1xAxYgzvqB2fru38eqoWi4kv5uaBFIrYAZmms/0V6EJc466/XnxR3wWOTOvnxgfoMbqoNwnnyIDAU7FnI06IgkvFUMcR5/ydKm+hrm5uosVIEL/Nu1wJ6BvxjACZ8yuJ333UbILBTlzme9m7CueMkY9l94flxTIWsNLw9wBWGYBUEnWzdIVnf6Dsn1cKQ+Vu1BI89vk4CuVrXAG06KJQQrHuglyySLNGMFooci/em/0ukG5hxUmG8moa MYiAcyiY IHZyIV/TMLOAaE0q+DRa/nObQ3OssZkIlWNzM1otJc/rBiUw+msrjJKieNutUivjxMg6hDYI3rNPop3T09EikN69J2hmGWGnWGfC6O01cU4s6RC892q4ughfXk5hmspe8XeOxZV5S/rsg+YzocIcLOqaBbOAUSn452RcYnZPVW/owyuHCtKo8yA6xnS5J15V1kERM7+TxJi3cC4/HEEXYj7vWCiu1k9ziACz9rDmWVJiP8MqgihWNa38N9HOzkcWk2N/beH6ZOou8Fq0AjxcubEoBetRAPiGuP8cT8SoF+iZdCOcTDa/aHXmRTaU4b6H34/bp3bCtMh4Jrsg/gVIHvg7pQvzqUhf6k6D8bpbbaotqL08FUUhxIUWuHQUYZOj33dUyf4nav46rBgNaxLOh0DisnDGAXtV/KbIwG9GAqW8SxWJDEVL8wfC2L7Q1Jg1r0s/xzvyJczYJDAxrFEph372rg7sSTkZP/b1uerTKbovtIMea/Se/04hKGlqbmccqGiRN0A+8kGhim/4FBO3t/kXwlNCi9I4lJ9oIPJoOlS68D3iZybqEQUjAow== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The generation number in struct mem_cgroup_reclaim_iter should be incremented on every round-trip. Currently, it is possible for a concurrent reclaimer to jump in at the end of the hierarchy, causing a traversal restart (resetting the iteration position) without incrementing the generation number. By resetting the position without incrementing the generation, it's possible for another ongoing mem_cgroup_iter() thread to walk the tree twice. Move the traversal restart such that the generation number is incremented before the restart. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier --- mm/memcontrol.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 67b1994377b7..51b194a4c375 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -997,7 +997,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, root = root_mem_cgroup; rcu_read_lock(); - +restart: if (reclaim) { struct mem_cgroup_per_node *mz; @@ -1024,14 +1024,6 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, for (;;) { css = css_next_descendant_pre(css, &root->css); if (!css) { - /* - * Reclaimers share the hierarchy walk, and a - * new one might jump in right at the end of - * the hierarchy - make sure they see at least - * one group and restart from the beginning. - */ - if (!prev) - continue; break; } @@ -1054,8 +1046,18 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, */ (void)cmpxchg(&iter->position, pos, memcg); - if (!memcg) + if (!memcg) { iter->generation++; + + /* + * Reclaimers share the hierarchy walk, and a + * new one might jump in right at the end of + * the hierarchy - make sure they see at least + * one group and restart from the beginning. + */ + if (!prev) + goto restart; + } } out_unlock: From patchwork Tue Aug 27 23:07:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13780211 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08F2CC54742 for ; Tue, 27 Aug 2024 23:11:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 49EDF6B0093; Tue, 27 Aug 2024 19:11:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 425C46B0095; Tue, 27 Aug 2024 19:11:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 205116B0096; Tue, 27 Aug 2024 19:11:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id F2CAB6B0093 for ; Tue, 27 Aug 2024 19:11:47 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6D77F140846 for ; Tue, 27 Aug 2024 23:11:47 +0000 (UTC) X-FDA: 82499574654.30.D6470BC Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf25.hostedemail.com (Postfix) with ESMTP id A9C38A000C for ; Tue, 27 Aug 2024 23:11:45 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uNEvlmjj; spf=pass (imf25.hostedemail.com: domain of 3MF3OZggKCGEJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3MF3OZggKCGEJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724800236; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GW/USJCjupc34zRwCQ9bHi7JmQHKfEiUl9nNlR9x2Hk=; b=gQ4bUYeP8QSpp3Khc8fjNwbBSuaQUhdAKOZsugruXUCkK2hh56EAj3lRizpmlrYeeKjy0I ph0mieHu6AbYl4yYcsU7EZ9UjDvd4f3zsa3/Rp05sssQLIsvx4SHgFgg5K5U/d5D6a2pzk usu4NHk3MgokNEj3RkLQoqB/Iqr7rXY= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=uNEvlmjj; spf=pass (imf25.hostedemail.com: domain of 3MF3OZggKCGEJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3MF3OZggKCGEJHMRDXGNFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--kinseyho.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724800236; a=rsa-sha256; cv=none; b=0EafIC2JRDBja5MrUKkH5Bp/IdVN7v54GabPoAfUbKIOiGdnjAOJN8Y1a8VC5zXZSL3H4O Cy/Nh3UPdTkDp8V7BEzzjh7y2LmBBXsH6AmOXVYsDAb2B2BCZREz8qogLYt4BF8ra75Njy zuwyYbTYRQQnAf5lyln4cx4SbC2/29U= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6b46a237b48so127400937b3.1 for ; Tue, 27 Aug 2024 16:11:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724800305; x=1725405105; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=GW/USJCjupc34zRwCQ9bHi7JmQHKfEiUl9nNlR9x2Hk=; b=uNEvlmjjkQw/6NXBHvv6lNDKwca/e/nYSfHuLxHBm0IHz7NgbwMES7KY5VAxWNOEq0 3wVQXdIx//0/vBwGBv9fWHPKe0Z+VMXaVssKBdBitfVm0NtbfsR0tZoEYFlR1i1jgWAG d132NQkCk+X6qu0IlwvttDP2DvT27Lk0DI0dhZy4TvnHDhz3rjl/CI1jeMEkty1zSx6u y5WDfo80JxyEDtoFyPyX/Cj8paTULrhvn5AXuP3XNFK0JTdUMx1m5IaX2cyTbsL6+mCm o8klcQ4HqtlFXRfMu5qZ/+TTq1wyWlbeRJIFIo13dCSrXix/4+B1SO1B9kIeidRfj4p8 bNBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724800305; x=1725405105; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GW/USJCjupc34zRwCQ9bHi7JmQHKfEiUl9nNlR9x2Hk=; b=hVQcYLAHdn4gHylXPdqKmIt0d1y3JhuOliDRGTkZuwL6DhEHJ9ybnRnPKAmTNoqDVe BJyshc0yT4xF69vDqI8tKqQiEGwQ3c2fCfvT9DAN4bw5Mot0qCa64nOYgBNCMAbEmVp1 1hABpIvzAS9EE/U+63uQZBhlnJFTWqpfRuMJAzhM8B5yQy8a5yz1uhBhrwgzpuy1LvHR UpWWvBjGps1cmlTm3FBBMQPZgcvUFO52fPS8pYv4xNv2r7v/I7wSRohxulaYQiK2d/av 6B+G5vsMgDqajrstZ6BMGvIltwfnemAPu9+XKmJchQ7g7OXdEfHlo8GBlj0h/fYbeVDd o7mw== X-Gm-Message-State: AOJu0YyzM6LskS0LkXW42bNEt0COpQe5H2d4a/T0q1p/BmGG2LsI/Tzo l5OKoqK8aWhieX0VV0+go6Bm3etKiAUB+SxVKH7xAvbMqqi5bAZhNdJsJaMV6plyvrG47ugAI6m DlR59wyjAng== X-Google-Smtp-Source: AGHT+IF1yp8ipyeMEvESYLh84If45Gfi4ptWiqS2P4CUd5/mGA60SGsDPudut6D5G0yZlYICuZUQrkeSqUZvzQ== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a05:690c:46c3:b0:69a:536:afd3 with SMTP id 00721157ae682-6d172052874mr2467b3.5.1724800304855; Tue, 27 Aug 2024 16:11:44 -0700 (PDT) Date: Tue, 27 Aug 2024 23:07:41 +0000 In-Reply-To: <20240827230753.2073580-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240827230753.2073580-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.295.g3b9ea8a38a-goog Message-ID: <20240827230753.2073580-5-kinseyho@google.com> Subject: [PATCH mm-unstable v3 4/5] mm: restart if multiple traversals raced From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Stat-Signature: 3bqkmgszwcwm716hftu1ua51ce5e5fcb X-Rspam-User: X-Rspamd-Queue-Id: A9C38A000C X-Rspamd-Server: rspam02 X-HE-Tag: 1724800305-227313 X-HE-Meta: U2FsdGVkX1/OmijcPPgpyY+P8iP1Rkb7QaKBYrj/AykEcpuWKLq9sXIiOX7+nxg1xYCYWvOAO1F49PC4Rikoz8P1z01rWbS+EDxsU/A20rD6WRIpcB+4WX+2hxRfeOPMb+DkKOIeC0IMAAsmkdW0AnV6MdZiTvWVXnmaLGQbUgJPNT4L/YdtExIjZLkv1Oju/6JJbQonYClxLupx36L6dzyvrwZuGsvx1cQma2UIPAmULzRKcNn9f8KEG+4pHjOw2xzyFtrTxt6hD2M02DJxYYpA9qypMLZKEystxBj5biHEMUXj1k04j+DLbydu/SEHTtwSEvhEUjF4W/CNZagHVGB73hEPp301WAJjqFLQfCwYMAlaRwIps5I1lr5Hftxi+arXieznYrxlCKuZZCJOQUBnQSRijsmNaxNMERb4UQiAtLFl+JcdHQX8wE5xFheEZQ1sPWSX6tcafluDupnYvcgSlQvUIX/zg7kldCtllhm8YdH0aYBgIac3PSRcynFFsVAZYZRMLDyKxElXb8PfdmKImO/OKz+Qqil31qqTMfro8nkc2RW20YnuPno08IrBAgTef/4DceuFX0KqxA72LDy5zDS9eDancwJ5tEzNi1qpliWQ7SAgPTUU8Z+zy3z7DietqhujOEzXkjQ9qyxDnTc4EWNfwiGrHbfngj0B82/8N+GhIEAqf3ftBvdsuYbPl/8PoH3eDEjLsVqB7YH/UIPb3QmdkTaGN6rF57oCi/lLG4Q/1z9IgmBo+2lvju0fpevk+/KCNJc1J1dFUk5nGiQ2d1URxWEYSL9qEiY0F+YHW064HweDH4ru5O5W6vGqVrgPoTLGgSP/mdvCmG/bpLaRX3SBjyDa0zoVCdbeEsFjPBi2l+vD6qxiEuzOJal17NK9v8V1PVhX/RGClEGcDPhKYPrUaboHj8yfbvcmIqjQHKvTIjbxrINCv4TUvw/KCBnLPCUM87ZCO8tjcY/ bw0OkWWO X6sC/qMBD9QXGkTjb2Eh1ak1yGaKAsCBPK6YIORoX+MNEHe/Osp0qm7mQnGfdnECYPJhkMgllfs2WDN1WowVsbL3f/qLQL4/lHn16D1k7qJKJO0z1uooIR+ReagSTwMjP1WphsHozs0o9mhGKmKuQ6yyDfnrdj90vSRZnHPvM/2u5XJ2thTBHWAbwwduzN0GVEnaM16wZfVlp1HNyq6/W9xbCauC2xQoo9J8aRULajyc0JowT89Lzqud2Q1MpRDS3VQD+fMg/EfPFYSPWdac0PwsGpBj/sEFuPADaprbk4uzVMRsBYASQ4cszsZrS8VDU75LrYGU99dldwQw/heAhBmAeB+uxnZVtQTdxhsbGrXbXqVuh/0SmDiPaY/nRXQTXQ33Y7pswLDScs6fYfCagvzOgeqisYp8C0v3AzlWLgjNyJLO772Jc125dlYVns2iwkZbe8AYersQyjiiQv6ENTN+52YKGwvYUmRReMJ4I6uX5vix7lPkWiWqNDXwScgSidfWYG62Dpx/JPAyaXrxkQB9vxio0IG0L//Z7s4ue9/kCcOk0FBQFN5OyUw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently, if multiple reclaimers raced on the same position, the reclaimers which detect the race will still reclaim from the same memcg. Instead, the reclaimers which detect the race should move on to the next memcg in the hierarchy. So, in the case where multiple traversals race, jump back to the start of the mem_cgroup_iter() function to find the next memcg in the hierarchy to reclaim from. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier Signed-off-by: Hugh Dickins Acked-by: Kinsey Ho Reported-by: syzbot+e099d407346c45275ce9@syzkaller.appspotmail.com --- include/linux/memcontrol.h | 4 ++-- mm/memcontrol.c | 22 ++++++++++++++-------- 2 files changed, 16 insertions(+), 10 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index fe05fdb92779..2ef94c74847d 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -57,7 +57,7 @@ enum memcg_memory_event { struct mem_cgroup_reclaim_cookie { pg_data_t *pgdat; - unsigned int generation; + int generation; }; #ifdef CONFIG_MEMCG @@ -78,7 +78,7 @@ struct lruvec_stats; struct mem_cgroup_reclaim_iter { struct mem_cgroup *position; /* scan generation, increased every round-trip */ - unsigned int generation; + atomic_t generation; }; /* diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 51b194a4c375..33bd379c738b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -986,7 +986,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, struct mem_cgroup_reclaim_cookie *reclaim) { struct mem_cgroup_reclaim_iter *iter; - struct cgroup_subsys_state *css = NULL; + struct cgroup_subsys_state *css; struct mem_cgroup *memcg = NULL; struct mem_cgroup *pos = NULL; @@ -999,18 +999,20 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, rcu_read_lock(); restart: if (reclaim) { + int gen; struct mem_cgroup_per_node *mz; mz = root->nodeinfo[reclaim->pgdat->node_id]; iter = &mz->iter; + gen = atomic_read(&iter->generation); /* * On start, join the current reclaim iteration cycle. * Exit when a concurrent walker completes it. */ if (!prev) - reclaim->generation = iter->generation; - else if (reclaim->generation != iter->generation) + reclaim->generation = gen; + else if (reclaim->generation != gen) goto out_unlock; pos = READ_ONCE(iter->position); @@ -1018,8 +1020,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, pos = prev; } - if (pos) - css = &pos->css; + css = pos ? &pos->css : NULL; for (;;) { css = css_next_descendant_pre(css, &root->css); @@ -1033,21 +1034,26 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * and kicking, and don't take an extra reference. */ if (css == &root->css || css_tryget(css)) { - memcg = mem_cgroup_from_css(css); break; } } + memcg = mem_cgroup_from_css(css); + if (reclaim) { /* * The position could have already been updated by a competing * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - (void)cmpxchg(&iter->position, pos, memcg); + if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (css && css != &root->css) + css_put(css); + goto restart; + } if (!memcg) { - iter->generation++; + atomic_inc(&iter->generation); /* * Reclaimers share the hierarchy walk, and a From patchwork Tue Aug 27 23:07:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kinsey Ho X-Patchwork-Id: 13780212 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D9FFC54742 for ; Tue, 27 Aug 2024 23:11:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1209E6B0096; Tue, 27 Aug 2024 19:11:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CD036B0098; Tue, 27 Aug 2024 19:11:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9AB76B0099; Tue, 27 Aug 2024 19:11:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B76E26B0096 for ; Tue, 27 Aug 2024 19:11:49 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7C1D41C6319 for ; Tue, 27 Aug 2024 23:11:49 +0000 (UTC) X-FDA: 82499574738.03.B7C931A Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf07.hostedemail.com (Postfix) with ESMTP id AC1BB40013 for ; Tue, 27 Aug 2024 23:11:47 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XKMcNwjd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3Ml3OZggKCGMLJOTFZIPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--kinseyho.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3Ml3OZggKCGMLJOTFZIPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--kinseyho.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724800236; a=rsa-sha256; cv=none; b=KcV8kVX9hpB6rU0LvTOjLS2CjVZmXC44cFhJSl+0z8aa9gY3LitLc4mxxt7ZY/+2eMC+9B Lqi6LqhLJ6UmVYJc5Aqy/T3duqkK7PoKTJDYrKzayVWh0RnIHwZ6bNg1615kxefu8GM2Ww t8qWPeIVucaNKeknALLAFj/jPCUls8Y= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=XKMcNwjd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of 3Ml3OZggKCGMLJOTFZIPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--kinseyho.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3Ml3OZggKCGMLJOTFZIPHPPHMF.DPNMJOVY-NNLWBDL.PSH@flex--kinseyho.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724800236; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=F9O1xAoZAOV8c4zW9/De55yG0OzoFjxVL+7Ggug8PCo=; b=eeydZ/jGUd+ZPMN7+GLWdcuyk5ubX+2U3BUVbUJgqjStEzyg5qXqQAFNusSCwE5wD66vIY yGWxKZPu7FJuF8aUWWKf7QTdDD62ai0484PmWUnCypxIPdFqRmP9bwm3G6mzNpRlKT/hPT ZkJXot+r4M+2UbvcCpI5/OtdTL6jV4w= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6b47ff8a5c4so107269827b3.2 for ; Tue, 27 Aug 2024 16:11:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724800307; x=1725405107; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=F9O1xAoZAOV8c4zW9/De55yG0OzoFjxVL+7Ggug8PCo=; b=XKMcNwjdZt3OKCG1RPfOjYYr4KvOKEmZ9q9fWhswL44nnHG2FNPAVlQiBvQp25Ao5P oqbOR3G2uMjynp2Qn5O53eMRQmPwYO62MZEBXuyK7cgekyxGlZew+RnUNDFb/DVRfEGy uobdGKbGdIVoF2F02VVOAIpZtsPuWvpOElq1DTn1pwPHEn3aUhamsVxM+WIO6A7qfT7w ICqZokWYkIg0UdG9Hk0kfiemvHeUmJFRb58NnTgtKbGBZGysv+taODCCG4c3evPerMRu bWB7quAcp2HH8VxMoDizybW0ONLfSa6JPnRNH7c1l8uKeF1Bo0nIPl/t7XomPfhWAhxF u81A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724800307; x=1725405107; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F9O1xAoZAOV8c4zW9/De55yG0OzoFjxVL+7Ggug8PCo=; b=Bfy9cGU1xs8QSX/+lkrhTykvytZb33vVySsVSDSVKED4pKEm1ESz9UbPIjtmj8O1Yc 3ZrWsw2/zDCBQT1G1NMZv6JztwEthS9Qtxe7KKbu2kvgj8d12rqfzbOB/0ImKozHEkng 7IGvvmk1dBBbOgUkasf5fHfSJxHRrIJiW0qgs0fSHOupLq7yFx8ZdnIKe4ApYe4V+FEy qFjkSYifn1AafYixOM5vt+AMqTywnuT2ZlKBhPsGRyygWl1mCbVmPQmhNIdnDDiVdCYR WPM8G2eKoYOS0zk4VxOJ94/G+VpeERuCCVch4huF/SBTdukv91lh6KMBuVTTGIJwXkMM fmEA== X-Gm-Message-State: AOJu0Yye5BbS5n1u8amGKzCd92I6qSHjuoKG831yF6/wUsUd9RYrDECv g8sRHITSALxn3EJcvBxr+njBugVgTn/NjWvE7nZ+1ZWvkteB4pre+aZZL6vIerosx439YWewHb2 +0IryW15Ncg== X-Google-Smtp-Source: AGHT+IGhyXg3/eIiJrPzV8LA1GsjfwoUYtS3Vb5uXpGkz/KSc2MBd8B85rwqll+iK9yx1Z8d0EFBFb/FbykjYg== X-Received: from kinseyct.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:46b]) (user=kinseyho job=sendgmr) by 2002:a25:2905:0:b0:e0b:f69b:da30 with SMTP id 3f1490d57ef6-e1a4580f12amr2713276.9.1724800306773; Tue, 27 Aug 2024 16:11:46 -0700 (PDT) Date: Tue, 27 Aug 2024 23:07:42 +0000 In-Reply-To: <20240827230753.2073580-1-kinseyho@google.com> Mime-Version: 1.0 References: <20240827230753.2073580-1-kinseyho@google.com> X-Mailer: git-send-email 2.46.0.295.g3b9ea8a38a-goog Message-ID: <20240827230753.2073580-6-kinseyho@google.com> Subject: [PATCH mm-unstable v3 5/5] mm: clean up mem_cgroup_iter() From: Kinsey Ho To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, Yosry Ahmed , Roman Gushchin , Johannes Weiner , Michal Hocko , Shakeel Butt , Muchun Song , Tejun Heo , Zefan Li , mkoutny@suse.com, Kinsey Ho X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: AC1BB40013 X-Stat-Signature: wf4dir7etdz7odkdxzarf66u6yj7rd8g X-Rspam-User: X-HE-Tag: 1724800307-634678 X-HE-Meta: U2FsdGVkX1/Eh/nOVehB8ifJ1/WxbKk9kf2QESjubHizY3gVgvD8En8QD1/7W/QoScOSJv4JPlHHOeNMN91T52whj+HSg4lDZzvNToaTH4JMaCtbaedCG2avWV0kkeuh9Cob6CF6hWvTOmJoTVcxNkTd3sPLcxW0zr7V7VKoq7s3NjyFuCqd2Q7o0OERUfQxSEET6aT4rPYL9mroVjwHmhRrxuyHnX476TgKUjKdZjRpEUSixu38wH6+KbN+EF+RduzS7nF22qiwRUIsQfgH31fx7Ed0msVWC1vy6PuaGlvJ0tmBWrfzofc/TmniW0+oise0MYNTWf++O0ccofaFgj9gx0cmd+daKAo03VvcAPGA7aWRmv/mJGGW1zJ/Zt2AmFB3kOPC5Jxrmj0bFqUjCZfUhYz9GAfZRpoH9mEr6gFJ7DByn0HQAK3+28Xf+d5KIiUh/D1hdkDFpdLIKUvAFyyYwFjgaRGIbUcjY06mhU1wQTK81jilv6ttlKhsHaF4nkjCZ4ggLhElVFbjnYT0bf0V8R8EJSimVNQQKK9A3Zr3FBTDkpzuNiJFC7BIEy+diceyDpj3YMjpIN/H07FVK6l4P2HR1r357/tI9p6Q06M8qc3P9NH48sl82LA9ITOTA61M4VmEqGeW06jPrYxD5kLu0d92DPcrY+yF65kzSyqaczP3zPahqnCWjqyFZemRU8WWmTi+XWvLEAT+DCsTxolLG1XK+//wViA2o5cWQBOz9oJPVh0fkre4QKZMXMfXQq8NanWSGwN6inLBTG7RGmPxUNDxs/eGMNOdZMVTCT7ILZGkFLPm7ts2Hw/m+VYMUFsnc7LZix9TEPUot55tmKdwkOYYIyuvcjOPtvnvcUkFVRwjamopywlPKcK+gCga5IlAZTEVTTDXkRPZbFviatt9HWTHh8eJjCPSW+gYaIrZyACskMk7Em3WQHdjK6zk23QrG869LUgWU5SVCWB Is6IPrhI 2Z/vRX1TqZAAeN6FXWcDg2sajMUW0tbjYhyFdxj0WN+xwBSKNEOLQVzJ82ybfj8KUm5OcJK3eQtvmnWRG9FWi8S2c4/ltNhvgOE4svK+rj4DHzZWkol719tdtJur9c0CdJMGexVWsZE7zvkQP8HM07SZsv0mkV0GRGrhkLA/VpxrQcOw2Svq0kDuLRfjlrfttvzwouemZ0+1ktrJNcXWX79AyYs/HFVG9Eptzu8X2LOrZvjdIa1XS4PIzCDIa6cFfbU6A0v84sjYtUtrfTv/oNpIn+rnSMg2RGq1o+qyTkQtYgjJmbKHV08Z2cwjpPwBOj95vUk3SEEsE+BekkbSN5yunbswQWnek0BvNzDmSnj095CVE5Zs2ywyEIZe/8AJx3TCb3wyGeaQ/ePAntzH2PZdtBCRKhZld6G5gtIlsd+xkMU04GwxdPjgwxAUgc3+Obqqkx4vW1s3TV7HXCobQZnIvm/ztbH2zX4yZ72LI4kEdUwVZHkVVI4MUAIslVedqf5RSSLZpGU+plgss6nwkxzac65mb8vN9RlgPFvPacJA0jX5vJuuubji2upFtw+9x2BrU X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A clean up to make variable names more clear and to improve code readability. No functional change. Signed-off-by: Kinsey Ho Reviewed-by: T.J. Mercier --- mm/memcontrol.c | 30 +++++++++++------------------- 1 file changed, 11 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 33bd379c738b..2bdad7c29ac0 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -987,8 +987,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, { struct mem_cgroup_reclaim_iter *iter; struct cgroup_subsys_state *css; - struct mem_cgroup *memcg = NULL; - struct mem_cgroup *pos = NULL; + struct mem_cgroup *pos; + struct mem_cgroup *next = NULL; if (mem_cgroup_disabled()) return NULL; @@ -1000,10 +1000,9 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, restart: if (reclaim) { int gen; - struct mem_cgroup_per_node *mz; + int nid = reclaim->pgdat->node_id; - mz = root->nodeinfo[reclaim->pgdat->node_id]; - iter = &mz->iter; + iter = &root->nodeinfo[nid]->iter; gen = atomic_read(&iter->generation); /* @@ -1016,29 +1015,22 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, goto out_unlock; pos = READ_ONCE(iter->position); - } else if (prev) { + } else pos = prev; - } css = pos ? &pos->css : NULL; - for (;;) { - css = css_next_descendant_pre(css, &root->css); - if (!css) { - break; - } - + while ((css = css_next_descendant_pre(css, &root->css))) { /* * Verify the css and acquire a reference. The root * is provided by the caller, so we know it's alive * and kicking, and don't take an extra reference. */ - if (css == &root->css || css_tryget(css)) { + if (css == &root->css || css_tryget(css)) break; - } } - memcg = mem_cgroup_from_css(css); + next = mem_cgroup_from_css(css); if (reclaim) { /* @@ -1046,13 +1038,13 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, * thread, so check that the value hasn't changed since we read * it to avoid reclaiming from the same cgroup twice. */ - if (cmpxchg(&iter->position, pos, memcg) != pos) { + if (cmpxchg(&iter->position, pos, next) != pos) { if (css && css != &root->css) css_put(css); goto restart; } - if (!memcg) { + if (!next) { atomic_inc(&iter->generation); /* @@ -1071,7 +1063,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, if (prev && prev != root) css_put(&prev->css); - return memcg; + return next; } /**