From patchwork Fri Apr 4 01:39:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 14037968 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EA3BC3600C for ; Fri, 4 Apr 2025 01:39:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01F036B0026; Thu, 3 Apr 2025 21:39:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F10156B0027; Thu, 3 Apr 2025 21:39:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DFF416B0028; Thu, 3 Apr 2025 21:39:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id C14A16B0026 for ; Thu, 3 Apr 2025 21:39:40 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 5BF50121BCF for ; Fri, 4 Apr 2025 01:39:41 +0000 (UTC) X-FDA: 83294654562.04.A510049 Received: from out-171.mta0.migadu.com (out-171.mta0.migadu.com [91.218.175.171]) by imf06.hostedemail.com (Postfix) with ESMTP id 9B188180005 for ; Fri, 4 Apr 2025 01:39:39 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=IAblp7+f; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf06.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.171 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1743730779; a=rsa-sha256; cv=none; b=H4Vwm2ZLB3NfF7BEJ1gVGyLS9lBj2vAI+/qPkdwNp4OkiQxpvQgd3I3wJNSoCaG681E+so 1eocF3Zkh/JKbQmtY/4PVZuE0uKznI1QlohXHSzSgQf+hcG7WJaxYqESJrH+31W5GMk3TG Y5ERCGz/F/o49jLolVk+ZPOmtlXyQeQ= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=IAblp7+f; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf06.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.171 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1743730779; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZwESqd7AZkcvjy2svN3vjPj4mb4ksMSsdpBE9pRlLtk=; b=Y3TwGRa4VKGlEYBZ4Apyh4iR255aM9R0Em8wQdblM+i1KkLrqrdsk4yvqffKse0WUnwawi xd56t+2J51/bvv8U/rt8bRlPidXQVK+/gc4rur1LZ6NHrh4X0RiQh26Tc6xEJ/tEkDygih 5KtuQaotVn/e8y/9vNYP+RTsTiP6JLU= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1743730777; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZwESqd7AZkcvjy2svN3vjPj4mb4ksMSsdpBE9pRlLtk=; b=IAblp7+fpjYEGUbuAy7stmx7sCATwj3MRNpK/SMhFEatyJbqyWryEzuHgIZ2GtmJPc1RUU fBBkXAT4axjSGU6L3payjnYBz+SQJ9SZtwFI5fgpfm4gq1gn3gcsZpDifoUpAsMKgMJPqQ LpBDY0nLoiac5uIBq6HqyzrixSBCTBI= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Sebastian Andrzej Siewior , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH v2 2/9] memcg: decouple drain_obj_stock from local stock Date: Thu, 3 Apr 2025 18:39:06 -0700 Message-ID: <20250404013913.1663035-3-shakeel.butt@linux.dev> In-Reply-To: <20250404013913.1663035-1-shakeel.butt@linux.dev> References: <20250404013913.1663035-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 9B188180005 X-Stat-Signature: 5ea6t986w7fww99wkzgpe16g4mfrb7oe X-HE-Tag: 1743730779-155042 X-HE-Meta: U2FsdGVkX1+243HEOjpBKX8jdw+aCEEsSQOghTcJctFp4I3VSsEjiNkyPjUjFitl9sTNlFl5+r8FIhN+2IrhkJ5C0ukWjLDuzSBDacLLXqsB/YamcGaILD+lyfZPBJUyWCxSsrFprext4S9LWDGV7CWgDR+bPcKXCLxcYqN/mkJ/YScIpIcrF9K75sZpSEYp1JT/Q5zuBi0TvFnHZig2eSMHdI2miwOceFNC9yGQ790Osu8iDVrPxsw4CLIF9aBFIvUpr3OF8tkudAmDBz8Y9BzpQiOlEAudxUbrWxE66Q/gvlPL9iMSAUisXd+zU4mI+xIfXmX1Luwy+D/Q6VJFiQYijuMXszwrCLmKfyH8XVfs0RpU6kce6Bbt2awA0+NN1caCgBzEAqb63JlYPPxeLnT8IIEWiADFwANMdTTg6ukNOlM90tj9lka2FBxr/7Pcz4lrKg5WPl/mnsvl7UxR1qCLJizGEYax0+mqLJ6AHGK4NKM0QgB3p5+Qs0H1J872KvGCdU0oPexMJCR87zo6ACzW5eavuKkVEr6laHIDTxm8RLFIPgchGMZ7q1fzCczBUQ9XRoT5+eKZOinsUs03jbov28kLB/UAKTfuZ4u8pUZaSt+Iq3KQneivd4ASH0mDf19gvp4im3hCLuQhlL+rKn35bMGOw1v2uj8GXJU6n4F9r7x0pRgo0RBVsgHjOaYVeH9MUtAbQ5tAHeYBinMzp1wARHgHbFJDgvXdvwDyV70KCrRr/TwoQ50xwChskL3h55q6sZARlqCKlwOpV+PbepQ54z2woebySIo05oi5/LknqoOdlugwzbQk9utch9JWfC2tIFTlYrF+58Si/kSSsdAWY11PmKogXFQQZPGrnkk1I5PIjaydrcq6M4IlKkoYZWhkv7sAPL0+PZRSupRud9E7osAdt51FzhC06wpiCiG85Rgi/f1aruSAQl80flX/9GxUBfdoBgPwqjalQNU ZBCbzOJg 1Rvlluw8zPrFtZAKbnwp5kSB3fgTXT8xcOJVCh08lIblNLqJMQzu8fwpG/j91IT0DdZjvgWQkEkz0jJ5FPlQlw337/20KvXjQl9f6DM99sbcty58mQKjpS5sk4CEBPBzD6m5sBl7bpsc/MvdaiUacRlgSs+rOi6p6q/x1jQC8LddeMUkf+CGhiD1qtS0EqxPSCVzv4+M77z6n7Mxb5TY4W/IccrcQmGJtm0I6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently drain_obj_stock() can potentially call __refill_stock which accesses local cpu stock and thus requires memcg stock's local_lock. However if we look at the code paths leading to drain_obj_stock(), there is never a good reason to refill the memcg stock at all from it. At the moment, drain_obj_stock can be called from reclaim, hotplug cpu teardown, mod_objcg_state() and refill_obj_stock(). For reclaim and hotplug there is no need to refill. For the other two paths, most probably the newly switched objcg would be used in near future and thus no need to refill stock with the older objcg. In addition, __refill_stock() from drain_obj_stock() happens on rare cases, so performance is not really an issue. Let's just uncharge directly instead of refill which will also decouple drain_obj_stock from local cpu stock and local_lock requirements. Acked-by: Vlastimil Babka Reviewed-by: Roman Gushchin Signed-off-by: Shakeel Butt --- mm/memcontrol.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ae1e953cead7..52be78515d70 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2876,7 +2876,12 @@ static struct obj_cgroup *drain_obj_stock(struct memcg_stock_pcp *stock) mod_memcg_state(memcg, MEMCG_KMEM, -nr_pages); memcg1_account_kmem(memcg, -nr_pages); - __refill_stock(memcg, nr_pages); + if (!mem_cgroup_is_root(memcg)) { + page_counter_uncharge(&memcg->memory, nr_pages); + if (do_memsw_account()) + page_counter_uncharge(&memcg->memsw, + nr_pages); + } css_put(&memcg->css); }