From patchwork Sat Mar 15 17:49:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 14018165 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE09AC28B30 for ; Sat, 15 Mar 2025 17:49:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CDCB280004; Sat, 15 Mar 2025 13:49:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 62CC3280001; Sat, 15 Mar 2025 13:49:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A97D280004; Sat, 15 Mar 2025 13:49:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 22FB7280001 for ; Sat, 15 Mar 2025 13:49:49 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 987E3AB063 for ; Sat, 15 Mar 2025 17:49:50 +0000 (UTC) X-FDA: 83224523340.10.3EC4B1A Received: from out-179.mta0.migadu.com (out-179.mta0.migadu.com [91.218.175.179]) by imf09.hostedemail.com (Postfix) with ESMTP id E1B8C140007 for ; Sat, 15 Mar 2025 17:49:48 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Ejy9vqLu; spf=pass (imf09.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.179 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742060989; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oUGjl5IUOeohrue8CflY3qReHtC2T034o0a04sP/4ac=; b=q+bqT6FsrNPtEFv+diI2HWE0F0Oa5BZVJzCYyHKSV0zvMOotfA7PROLfPUowvi0UJm4/EU 9y39GOxWvhwQMZTNb9ziVmGo4JS/iK/8GCP6rOsZKuskn0RrK3nRyefZu1fG/Eb8cKtolE RElrhrD6cdjc2jTVEt5fNeOk7vZlZ7s= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=Ejy9vqLu; spf=pass (imf09.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.179 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742060989; a=rsa-sha256; cv=none; b=cInw8npEBpVxjOSt+RmO2f8iSRQqMmFQbvKoyuNlfPydMirrQRT+yfTOi7AAB+HXslT/3u PeLOluYcTxQs36JRDNEPRCblJ1jiH2rdYli5RpEweABZwAC/aCz8C9+q2t7LRjtRQMHWs4 Jyfzay1i5FuUFjjTcxNg6cptWVghi/8= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1742060986; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oUGjl5IUOeohrue8CflY3qReHtC2T034o0a04sP/4ac=; b=Ejy9vqLuhMtkzwYa4BWZ2IiChp1s5sstyd4OAgBMSkax1T0pVT20KV+VkvWvY5R0mCrrCA CWNQcrKeNlCZoz9SnQEPbL3cjNDbBmqiM/2TtzFlraiYrRnTqDXLQBiBdgjMSa88A7UtW7 qGO2/7k9JcdwLvf0v1aCNfGdPeArjgs= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Sebastian Andrzej Siewior , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH 2/9] memcg: decouple drain_obj_stock from local stock Date: Sat, 15 Mar 2025 10:49:23 -0700 Message-ID: <20250315174930.1769599-3-shakeel.butt@linux.dev> In-Reply-To: <20250315174930.1769599-1-shakeel.butt@linux.dev> References: <20250315174930.1769599-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: wmr45uy49thwaij6n6pnw7insb9peacf X-Rspamd-Queue-Id: E1B8C140007 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1742060988-951903 X-HE-Meta: U2FsdGVkX18Ef+iOYSaGTNqZmrsO4pAw2WNS1rO8LhR54Nf4EJsm3yjU5zHEmL64IsZhDhLLEjwNV1I6xH+9+1JgxE0Oxl0D+TnI8tUSzaTegNIakfgeuNTJCR0V4lBPgEeZbqMUGqti3f++cSl7VE62QgblDI/LwIRscoIg05wCv/akExRjqloAJ9Ab8+bo5stfMvbm1X+vyK8JULFZkVyXEcjBRUPQMMdKXEmco/H+eS7j3q8YBC0b53d2DIH9AGW8vTf3v3dLxtIcCihMXosnUvwXH3iDEuzxQABeujo68H17G5ajdMWu5zja7RkQeU3HRNhoBwXW54gVoS1lp3HbjFEJg8KQUAaL3skAm4DCI5dUXYtLOHXEWfoplOnrBsVOXVTz+mr76JJQ9UjrscrGEvslU8fe3v5SBqDpGHSNRC71mbgL5XY/DvTuQDOuNKdq8XnuEDnYXZzERrhlt/Zk6nyUHqmqwt7I1eK44ABnrqn38UBQanqRG9PDf/fNpEPvKSIx6BhLbBvxEwpJLYrmBF/6D11GvRMru/65q6YctqlaLMAPus13OMLV4CRmOdjSJH4gHrgdO22jk9nwjtm+nkCKVjyVUWBNYe/toUhUNDvbOHXvhgudTWPEbzCmGtK3KCQc+QaOj9N7Q1MTxiQnOiu1Ylwm9BBsDojvxpxII6HAhSmq+9q+fNh+bM54DMv0hrW4VoidvjJ4b0tBkGI9wzSzrb/tOSUPKtkAbi6Eco9aZeqYHGu40dJS9AefUVFPY7WThMnoimWFPCOY1yDxJFh5JP844IK8ymzlXKhhLMFDBWBoc1xyNMVWK2jXVVjkFbRJ3u+Jc3t1oKQ5E0YaoChvUHtu+aIKRJagH5YyZN00ADlKxF8TvWRK9vMrAbPa+60sUvJlPnqEPWblLJNRddRCCfo2uK7dPSgPUF07q99z7H5MRLsvc/Fv9thESl2FG0nLNzpwHJuO1Th t9xsZ/nR w9D2tYzoPGKeBNpXboz0HimeIj/aX7qgxJGamqQ5CDbpEoZHMtTRHNc+CLjIgoS/Xw2LiqVcFcxtXuW4/Gu0POBgeOJu+ydS5Z9zGCuQW1bj3zy9CjxZL0hzHr0v4if4GFlDHxbgixkQnb9BFYP5oiaNAsQ+vfI2+kF3qRMaXS1jYqsm/0tzdjX8/Ur7uTFh599Ntfg1aqU2FnXuypuGrLol+1w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently drain_obj_stock() can potentially call __refill_stock which accesses local cpu stock and thus requires memcg stock's local_lock. However if we look at the code paths leading to drain_obj_stock(), there is never a good reason to refill the memcg stock at all from it. At the moment, drain_obj_stock can be called from reclaim, hotplug cpu teardown, mod_objcg_state() and refill_obj_stock(). For reclaim and hotplug there is no need to refill. For the other two paths, most probably the newly switched objcg would be used in near future and thus no need to refill stock with the older objcg. In addition, __refill_stock() from drain_obj_stock() happens on rare cases, so performance is not really an issue. Let's just uncharge directly instead of refill which will also decouple drain_obj_stock from local cpu stock and local_lock requirements. Signed-off-by: Shakeel Butt Acked-by: Vlastimil Babka --- mm/memcontrol.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index c09a32e93d39..28cb75b5bc66 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2855,7 +2855,12 @@ static struct obj_cgroup *drain_obj_stock(struct memcg_stock_pcp *stock) mod_memcg_state(memcg, MEMCG_KMEM, -nr_pages); memcg1_account_kmem(memcg, -nr_pages); - __refill_stock(memcg, nr_pages); + if (!mem_cgroup_is_root(memcg)) { + page_counter_uncharge(&memcg->memory, nr_pages); + if (do_memsw_account()) + page_counter_uncharge(&memcg->memsw, + nr_pages); + } css_put(&memcg->css); }