From patchwork Sat Mar 15 17:49:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 14018167 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 268AFC28B28 for ; Sat, 15 Mar 2025 17:49:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E60EA280006; Sat, 15 Mar 2025 13:49:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E1045280001; Sat, 15 Mar 2025 13:49:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD86C280006; Sat, 15 Mar 2025 13:49:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AF6E1280001 for ; Sat, 15 Mar 2025 13:49:56 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E989A121E91 for ; Sat, 15 Mar 2025 17:49:57 +0000 (UTC) X-FDA: 83224523634.15.F1E33AD Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) by imf09.hostedemail.com (Postfix) with ESMTP id 43575140009 for ; Sat, 15 Mar 2025 17:49:55 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=UCQl96Wa; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf09.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742060996; a=rsa-sha256; cv=none; b=lHlWcJJ8pYXpSrcFDWEXkwQU12DbHx1wA3g//AY5zDAhJwS8Xk7oZ2gaodofDHICxSCm3v 6JlJJPEvpRiuSEVwZIId/H/zeyTJ8ab7h5XM623+Xj7/Vk0mfhPqdFGH7eX95+Y8A+cRmK x7nEHK3Do/TZXMLMG6BfPc5lvvUM37s= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=UCQl96Wa; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf09.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742060996; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Bc6pQ9m45TBV5JfKg0PdI6TDpBclbZtCDqtLw9XzNuU=; b=4CogCOtDIIKqqTDTbBne8z5Oc1zDtp5fiD1iNeS64Hooj9xu+IM+p9oBhRO+mxXDmtqFHK 8vlY95E3U1F+io2Gcr/dxeIXET/nCL9A5nre48joN5xrvTTBXR6YntS+PMMVWiM9D6UCI+ dI55y/M0qyn0O4tqEp1PpP+JQ5ZXk8I= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1742060994; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Bc6pQ9m45TBV5JfKg0PdI6TDpBclbZtCDqtLw9XzNuU=; b=UCQl96Wa1/yEqZlafoPIygsX+2cWELzvlzoYrINa1G0wvMDEnFDPIjjN7q4Jw2b+9PzjIQ IdojMmemgziZOSXNmvSCv6u3VhSKT8RUTTHb5RfhZ0lRiieq8JAC75DNFjhgeKcfJA0DEC /WgjCitdjgMpbaZKjihNY1nTs+Dk+Tc= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Sebastian Andrzej Siewior , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH 4/9] memcg: manually inline __refill_stock Date: Sat, 15 Mar 2025 10:49:25 -0700 Message-ID: <20250315174930.1769599-5-shakeel.butt@linux.dev> In-Reply-To: <20250315174930.1769599-1-shakeel.butt@linux.dev> References: <20250315174930.1769599-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Queue-Id: 43575140009 X-Stat-Signature: 1rxk1wtbe1yq6c19ffs4u9s6wf56fhm8 X-Rspamd-Server: rspam06 X-HE-Tag: 1742060995-730710 X-HE-Meta: U2FsdGVkX180vmWl8hNdKFro4YKdUH8d3e6/MIzBQppriuGrvAj5Q7GLkLbS5jKjiIgXWLGHC7dgvGSzqPA0HItxrB3kHOxXP8tD2UJwBKW3Yd0hLPo/PoxhLxS5YMjtC2XhUcRe+GrxkRJdNLDhyk6tqscUoCfLRsN9qk6Gy0u69Yvl3kw1nlEFwjDJOT8cw3brewwd0JtpYjz2u2TmY5GUz5Owz3vkag9hJK+dPwISEN8o4racJrlPoIZlMMSTK315PesoKCWF+KD3WqQn/4S3/ZszIT09t1Geajjxh2HSVis2K+i2iA1g3X/riL/eU/zk/7TDUyAdX/gOVo/DP5NpYh5//gNqCGOXHqIJeWtM68N1WQUS6FwetnnJpU1PyWFIF2LxA+wuzMxiariWVO7vQnWbxQDHyanq0mYUFhvi2leLmZA4HJGSyIiLgQyH0lywRxX+A5SmoMDU3zIzX9y/dPMV6ajO5z/VQvDAy8X2kKQqTG6JWYEjvCniHsk10WJ/GocKa3SXh0gWLsG33RW/wxvDhOMIA1N07qq2tWa0qpU6n/43vul60xH83QV1RueFL0+FUdFKIaUKcxtm0fytKxsaYbqShwB/uhYzOfV6T+7FA+ph9vkqFWDaPYx14wso0/lA9Roton0ODI9wKHBFZnrFpZwz6qdHc357vbmNSOX+Shp+Lx0N7h1Bt478U/Se4aGBaN8Z3+1SMTmfujF4kEYJ32ps69QiF/qt3kvAMlKug8w5yn2gMOyHtaRVlAYaeJ7z2yC1IFAXEt4rd1TKWG9TTfKW392OPtOasHD2k/2euhGejk08OPnuf5aMQZDeT3ejlO2knYEPVEm3hbVaWd4oXigFqUwRe6O/yr4oIlB6Q+rs5F5/b9q5y1fQyG2YcIRuN8sDSzFlFyLYHHC6b8um/m5J6HCFBWmUtYbSdgazJNAimR0DLyR+0Rs4gxx85glNQs0DGVR/IgO vBvqwy3e sJGCMxSny5TsLRRe/sqJWXc8dW3pGZyJNgo0VelNWauFXNP+GN9rdv0MECCyJ9sn4hlX4z21etdVf/naJx9xWb/eiyKhCVqTZ0/xxpD80BeO2LplZCzPZMq04zwQ0Y8tcu/kkyKWI4B0v0Z57jHt+UodhTFd2rpn9GYCQesXtMrsoFca0F3Og+mXst9bLOSHIfZn6EeM3itimzuz3UnHWsD8bNlSDyjIpOG6N X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There are no more multiple callers of __refill_stock(), so simply inline it to refill_stock(). Signed-off-by: Shakeel Butt Acked-by: Vlastimil Babka --- mm/memcontrol.c | 32 ++++++++++++-------------------- 1 file changed, 12 insertions(+), 20 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b54e3a1d23bd..7054b0ebd207 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1865,14 +1865,21 @@ static void drain_local_stock(struct work_struct *dummy) obj_cgroup_put(old); } -/* - * Cache charges(val) to local per_cpu area. - * This will be consumed by consume_stock() function, later. - */ -static void __refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) +/* Should never be called with root_mem_cgroup. */ +static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) { struct memcg_stock_pcp *stock; unsigned int stock_pages; + unsigned long flags; + + if (!localtry_trylock_irqsave(&memcg_stock.stock_lock, flags)) { + /* + * In case of unlikely failure to lock percpu stock_lock + * uncharge memcg directly. + */ + memcg_uncharge(memcg, nr_pages); + return; + } stock = this_cpu_ptr(&memcg_stock); if (READ_ONCE(stock->cached) != memcg) { /* reset if necessary */ @@ -1885,22 +1892,7 @@ static void __refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) if (stock_pages > MEMCG_CHARGE_BATCH) drain_stock(stock); -} -/* Should never be called with root_mem_cgroup. */ -static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) -{ - unsigned long flags; - - if (!localtry_trylock_irqsave(&memcg_stock.stock_lock, flags)) { - /* - * In case of unlikely failure to lock percpu stock_lock - * uncharge memcg directly. - */ - memcg_uncharge(memcg, nr_pages); - return; - } - __refill_stock(memcg, nr_pages); localtry_unlock_irqrestore(&memcg_stock.stock_lock, flags); }