From patchwork Wed Jul 28 15:53:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shakeel Butt X-Patchwork-Id: 12406241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-26.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92B21C4338F for ; Wed, 28 Jul 2021 15:54:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2F4456101E for ; Wed, 28 Jul 2021 15:54:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2F4456101E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 905C46B005D; Wed, 28 Jul 2021 11:54:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B58C6B006C; Wed, 28 Jul 2021 11:54:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77CE08D0001; Wed, 28 Jul 2021 11:54:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0094.hostedemail.com [216.40.44.94]) by kanga.kvack.org (Postfix) with ESMTP id 5B2436B005D for ; Wed, 28 Jul 2021 11:54:07 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id E85B617847 for ; Wed, 28 Jul 2021 15:54:06 +0000 (UTC) X-FDA: 78412442892.12.244D04B Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf11.hostedemail.com (Postfix) with ESMTP id 94F2EF0081FA for ; Wed, 28 Jul 2021 15:54:06 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id o23-20020a17090a4217b02901774c248202so1840930pjg.9 for ; Wed, 28 Jul 2021 08:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=Nks6a/t+MySx7OFBPCEcNNVwXnFiVMAiOmt6uCggBWY=; b=kSAACNtPnggfD+XKxBN1GwQwowtS+CCiZGRw8Mp/udN5Pg5m1sMoJsMguCP1jV2RQJ e9hNinUoyttZFC3LfPFEDjt+MXRSiomKz+Ma2ZeyVDp/WPJFuxpBWIRWP1LDukzvm4MQ 808xP8Mpg3MxY3l7Vhx/YM3wgvk+j5JLC6plyumEdPC/+2+IUMmqWd3BTlxlQyl2shZC O1O19eZZ9mG7hwP1aJIteGpcc9WpHpAHTY+wbgd6lPEXoyEpIrCwDaqgCOvAl0eY44GH aimMV8khBJQ92dvidm/qtCpkWFpSst1z8268pZMYjpEtxxHVohvB0R9YqHpn5WrS/3Z/ nbOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=Nks6a/t+MySx7OFBPCEcNNVwXnFiVMAiOmt6uCggBWY=; b=YbyNutajOE7uKfgA3yLAl+LZJx2TXBgbPbBPi1oNi99HXTDrOmEEIuI5bLqHKCwqoN PLT20cNI/E0w+OY+98nZq9Ns3CYs4lL3F/1X7untqCWmJKb3+WtqDVONWTL0Tc4ahjrt WaUQIKKymtqibtFpY4P9IHwQd8iYiFyLLF4w9aB9o9s9NNxydoErNhLaJgAQUNljbZan ijeTjy0lAOSSgq5aC8wzd2SWeCUX0g2SuMh9hCilG++ARt9D2QQ868qTui47uOFxH8fV 2YcRb8TQhCFRfteSiPy3ReGYOkFBa3oFteDs1jl4fSN5+Lzn/IBbxnSmOEd/GzYEFTwg FCLw== X-Gm-Message-State: AOAM532pmza+A9+NPKl/Kg/HIPSysskAL3EpgYo5VZMKg/4TfWsW5Ev3 dPqQFhko2zn99zVjWhYTplUqjF/Az8+C8g== X-Google-Smtp-Source: ABdhPJy81kAiCqDg7f4ohI21A479NHMS9MXo6F2Sa6SsRdueNdiHFvczQs1Aoq8fzudRQEaEmqG51unyCHMjmA== X-Received: from shakeelb.svl.corp.google.com ([2620:15c:2cd:202:3056:c181:ea4:1833]) (user=shakeelb job=sendgmr) by 2002:a65:5083:: with SMTP id r3mr446248pgp.161.1627487645326; Wed, 28 Jul 2021 08:54:05 -0700 (PDT) Date: Wed, 28 Jul 2021 08:53:54 -0700 Message-Id: <20210728155354.3440560-1-shakeelb@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.32.0.432.gabb21c7263-goog Subject: [PATCH] slub: fix unreclaimable slab stat for bulk free From: Shakeel Butt To: Christoph Lameter , Pekka Enberg , David Rientjes , Vlastimil Babka Cc: Michal Hocko , Roman Gushchin , Wang Hai , Muchun Song , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Shakeel Butt X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 94F2EF0081FA Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20161025 header.b=kSAACNtP; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 3nX0BYQgKCKweTMWQQXNSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--shakeelb.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3nX0BYQgKCKweTMWQQXNSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--shakeelb.bounces.google.com X-Stat-Signature: 83peusi5icd4imano1o3e76onowxs13k X-HE-Tag: 1627487646-188456 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: SLUB uses page allocator for higher order allocations and update unreclaimable slab stat for such allocations. At the moment, the bulk free for SLUB does not share code with normal free code path for these type of allocations and have missed the stat update. So, fix the stat update by common code. The user visible impact of the bug is the potential of inconsistent unreclaimable slab stat visible through meminfo and vmstat. Fixes: 6a486c0ad4dc ("mm, sl[ou]b: improve memory accounting") Signed-off-by: Shakeel Butt Acked-by: Michal Hocko Acked-by: Roman Gushchin Reviewed-by: Muchun Song --- mm/slub.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 6dad2b6fda6f..03770291aa6b 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3238,6 +3238,16 @@ struct detached_freelist { struct kmem_cache *s; }; +static inline void free_nonslab_page(struct page *page) +{ + unsigned int order = compound_order(page); + + VM_BUG_ON_PAGE(!PageCompound(page), page); + kfree_hook(page_address(page)); + mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, -(PAGE_SIZE << order)); + __free_pages(page, order); +} + /* * This function progressively scans the array with free objects (with * a limited look ahead) and extract objects belonging to the same @@ -3274,9 +3284,7 @@ int build_detached_freelist(struct kmem_cache *s, size_t size, if (!s) { /* Handle kalloc'ed objects */ if (unlikely(!PageSlab(page))) { - BUG_ON(!PageCompound(page)); - kfree_hook(object); - __free_pages(page, compound_order(page)); + free_nonslab_page(page); p[size] = NULL; /* mark object processed */ return size; } @@ -4252,13 +4260,7 @@ void kfree(const void *x) page = virt_to_head_page(x); if (unlikely(!PageSlab(page))) { - unsigned int order = compound_order(page); - - BUG_ON(!PageCompound(page)); - kfree_hook(object); - mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, - -(PAGE_SIZE << order)); - __free_pages(page, order); + free_nonslab_page(page); return; } slab_free(page->slab_cache, page, object, NULL, 1, _RET_IP_);