From patchwork Tue Dec 10 04:06:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 13900829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3614E7717F for ; Tue, 10 Dec 2024 04:08:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 22DD76B00EC; Mon, 9 Dec 2024 23:08:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DD976B00ED; Mon, 9 Dec 2024 23:08:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CD9C8D0001; Mon, 9 Dec 2024 23:08:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E3F306B00EC for ; Mon, 9 Dec 2024 23:08:16 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5C43D14020A for ; Tue, 10 Dec 2024 04:08:16 +0000 (UTC) X-FDA: 82877716362.22.B772079 Received: from out-177.mta0.migadu.com (out-177.mta0.migadu.com [91.218.175.177]) by imf27.hostedemail.com (Postfix) with ESMTP id E936840011 for ; Tue, 10 Dec 2024 04:07:47 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=bodIUL0I; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf27.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.177 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733803674; a=rsa-sha256; cv=none; b=memRLhZeuaNuPfd4HlJE82bJ2MkpBgCYuRKfrmIGlL1xTINUXnQ29Ju5y60KQ+/wPara12 Lj7ix2MYNcyGMGjLJsfy846mmwcxywS1B5LBTtXWWkT0xyIB3VUXnatV9gpXUnVBduqd2x ECAxK4u1ByM0CNuh526Q+DFfouEw8Ow= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=bodIUL0I; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf27.hostedemail.com: domain of shakeel.butt@linux.dev designates 91.218.175.177 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733803674; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=8RMk/oHGm+u3KEDKg9lygnIdO4nQMUWy6C7jj8RjEic=; b=z4Rru4/KqcK5Ao+KPaFtEY8ZPZbGq2EQqaKyA56nreUt2BjhaJgUdpNmKQQPU8kqC+bC1U hjvdh17/1GHsm2pXAYiptkKEzxgX7V8mRRavkoooR1Zbs357T5tgue1xET7PX9s4En1G96 baHBf6Q9Lgl3iZEWZIpDXNaIkE/l51c= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1733803690; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=8RMk/oHGm+u3KEDKg9lygnIdO4nQMUWy6C7jj8RjEic=; b=bodIUL0Is6mb8MAJO4BUgh+VmZQ1HWYkUTPb5XGpHvdTiL2C4DqIMW6OveUiG7JPULqauw uJNMAMGq1i70blX9TuyhgiHWniO8RMsDlboslPqyUBCTZqos8yilqPx1mVA8Qm83iHk/lp F6B8f37ALq2bO1FVzaJDtuVe0V3GOFw= From: Shakeel Butt To: Andrew Morton , Vlastimil Babka Cc: Yosry Ahmed , Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH] memcg: slub: fix SUnreclaim for post charged objects Date: Mon, 9 Dec 2024 20:06:57 -0800 Message-ID: <20241210040657.3441287-1-shakeel.butt@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Stat-Signature: 5gibttwt656i8156bc8ouyos7zuwi45t X-Rspam-User: X-Rspamd-Queue-Id: E936840011 X-Rspamd-Server: rspam08 X-HE-Tag: 1733803667-159192 X-HE-Meta: U2FsdGVkX19Lf5X2KaPENzrjeB1mKLlqS9Gq1yfU4OZZ19DHTU24pKQdMw8dvkYhiLoLbEkKOnQT7Ar9NaUdDb79qZjejRZpETlHJlPHviKHilI4GpiIKMVxS7R4rrQj6HdkQwl1WzQ5SY76NTSywNs9i5YQyZ+ZILJ6NmnRGPND8XVVIK7A6ddith40jam+GXQbhgyn5euEtA5P818p33yKuNsgCFFdD9gQVJ7n6SoIjRZICgOEX0Vh7bUjgTk+YLJ7EFPLBtISjwnz0R1bghTbW0OLJzpdZCAvFa9g2viaoArX2zd94RhTyDr7nYSWOF8C0GrSODfk+YkRDYpClbzeE2htQMqSPSMPYBkRgy2GRRBjnD3Yx9YlVX4SWwfp9S/PjN9FZfRk5uXfnfSpcH2hicDdwhFhVaDlA4RMgOsBL0WUzK262MvxGnrt8fzLb4hruQgf2NFMvYXwlM3HgVIteIj9vnqfBR4oHED/McF7tGb5MbZJWs64R7Kn3zRta2CyKXJt6Ms73/2ThbqBNwd1A5S2IuTDWyLB8xPh259SQh06/CTQjMQTWmS/vllClAG/AEdAUCYbYXAb3CCaG9lpHS/zFfo15j+mI/d3saz47hBX/R3+5YaZ69/Y4mih41bXeyax32Fcq5vvggdAl4quhWgLlE212Afcd75jKMJ5IXJZBKuZyGO967cG716wIpeuJvxZuhCaVW31JViMEWd+mjIDuzbbCZCwZChtNuv8MIeqSfbw7RM7Sa4etjjUrMjxGxJFbQNwQv3hdzBAHRKttdGlTxUngfTEfnMkca6MBlPsKG6oLEsMzQZ5CEXgMMP1aYcCo5fsYwoi9M7Hc3BQ4h+S20/ZvFRv0pOoockGxeOMyRZFCyLNM9kP1diMXOw68gKSfi+CHnV9T7SZ0CY19U9h9+CRqMFj9n+yoGjp2FEIVi+QOb2CU2w1KwNaBmJxX1SmkpggcN8Qv6I S5ap2SQh RPcsnCZdmN3zKNxrFOIm2X2D5Lhmqjhdp9oD//iJSY5jWnnmy3ntBIf1iJhBTkdxOqR3lUuw+sFB24G9j3GYluUrh29D3ID6KP6STHTA7ocEygsCj+tzJsRZ+3121ImCT7LNS3aIVGeemGyY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Large kmalloc directly allocates from the page allocator and then use lruvec_stat_mod_folio() to increment the unreclaimable slab stats for global and memcg. However when post memcg charging of slab objects was added in commit 9028cdeb38e1 ("memcg: add charging of already allocated slab objects"), it missed to correctly handle the unreclaimable slab stats for memcg. One user visisble effect of that bug is that the node level unreclaimable slab stat will work correctly but the memcg level stat can underflow as kernel correctly handles the free path but the charge path missed to increment the memcg level unreclaimable slab stat. Let's fix by correctly handle in the post charge code path. Fixes: 9028cdeb38e1 ("memcg: add charging of already allocated slab objects") Signed-off-by: Shakeel Butt --- mm/slub.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index f62c829b7b6b..88bf2bf51bd6 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2189,9 +2189,24 @@ bool memcg_slab_post_charge(void *p, gfp_t flags) folio = virt_to_folio(p); if (!folio_test_slab(folio)) { - return folio_memcg_kmem(folio) || - (__memcg_kmem_charge_page(folio_page(folio, 0), flags, - folio_order(folio)) == 0); + int size; + + if (folio_memcg_kmem(folio)) + return true; + + if (__memcg_kmem_charge_page(folio_page(folio, 0), flags, + folio_order(folio))) + return false; + + /* + * This folio has already been accounted in the global stats but + * not in the memcg stats. So, subtract from the global and use + * the interface which adds to both global and memcg stats. + */ + size = folio_size(folio); + node_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B, -size); + lruvec_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B, size); + return true; } slab = folio_slab(folio);