From patchwork Mon May 4 16:07:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Konstantin Khlebnikov X-Patchwork-Id: 11526641 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8AD62912 for ; Mon, 4 May 2020 16:07:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 582DD20752 for ; Mon, 4 May 2020 16:07:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=yandex-team.ru header.i=@yandex-team.ru header.b="JmYLsNz6" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 582DD20752 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=yandex-team.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B7CEA8E0024; Mon, 4 May 2020 12:07:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B04D38E0058; Mon, 4 May 2020 12:07:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9575E8E0024; Mon, 4 May 2020 12:07:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id 7321C8E0058 for ; Mon, 4 May 2020 12:07:46 -0400 (EDT) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 35B03180AD806 for ; Mon, 4 May 2020 16:07:46 +0000 (UTC) X-FDA: 76779517332.11.mist03_43aff714c2755 X-Spam-Summary: 2,0,0,9357ed704a45f49a,d41d8cd98f00b204,khlebnikov@yandex-team.ru,,RULES_HIT:41:152:355:379:800:960:966:973:988:989:1260:1277:1311:1313:1314:1345:1437:1515:1516:1518:1534:1542:1593:1594:1711:1730:1747:1777:1792:2196:2198:2199:2200:2393:2553:2559:2562:3138:3139:3140:3141:3142:3353:3865:3866:3867:3868:3870:3871:4321:4385:5007:6119:6261:6653:7903:7904:9010:9207:10004:10400:11026:11473:11658:11914:12043:12297:12438:12517:12519:12555:12760:12986:13146:13161:13229:13230:14093:14097:14181:14394:14687:14721:14922:21067:21080:21451:21627:21795:21990:30051:30054:30090,0,RBL:5.45.199.163:@yandex-team.ru:.lbl8.mailshell.net-64.100.201.201 62.2.3.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:fp,MSBL:0,DNSBL:neutral,Custom_rules:0:0:0,LFtime:23,LUA_SUMMARY:none X-HE-Tag: mist03_43aff714c2755 X-Filterd-Recvd-Size: 4363 Received: from forwardcorp1j.mail.yandex.net (forwardcorp1j.mail.yandex.net [5.45.199.163]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Mon, 4 May 2020 16:07:45 +0000 (UTC) Received: from mxbackcorp1o.mail.yandex.net (mxbackcorp1o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::301]) by forwardcorp1j.mail.yandex.net (Yandex) with ESMTP id 74A802E1328; Mon, 4 May 2020 19:07:42 +0300 (MSK) Received: from vla5-58875c36c028.qloud-c.yandex.net (vla5-58875c36c028.qloud-c.yandex.net [2a02:6b8:c18:340b:0:640:5887:5c36]) by mxbackcorp1o.mail.yandex.net (mxbackcorp/Yandex) with ESMTP id XFRQHSghwv-7ebiFwpY; Mon, 04 May 2020 19:07:42 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1588608462; bh=KD7pqcqmqr2MDuVLcfG3e1Q0DT+u7iGuuqIXuZXITnw=; h=Message-ID:Date:To:From:Subject:Cc; b=JmYLsNz6oKNS/b/P0BSekGqLg7SlV9HxXkHUY0SCX0+PJ7BFzFQQBIsJAGGdw5qyU ywpUh0EPBM0n1SSFHQeEaI2nw6oEiCgiT9L4WWmg6JJbJTOb6cOd8WQjp98lk/ReMT dl2ymWrA7ba9e0ynoEkdA4GYlPBtqR2kpEgh6rZ4= Authentication-Results: mxbackcorp1o.mail.yandex.net; dkim=pass header.i=@yandex-team.ru Received: from dynamic-vpn.dhcp.yndx.net (dynamic-vpn.dhcp.yndx.net [2a02:6b8:b081:409::1:8]) by vla5-58875c36c028.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id hrZEXGjF63-7eY8meVx; Mon, 04 May 2020 19:07:40 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Subject: [PATCH] slub: limit count of partial slabs scanned to gather statistics From: Konstantin Khlebnikov To: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton Cc: Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim Date: Mon, 04 May 2020 19:07:39 +0300 Message-ID: <158860845968.33385.4165926113074799048.stgit@buzz> User-Agent: StGit/0.22-32-g6a05 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: To get exact count of free and used objects slub have to scan list of partial slabs. This may take at long time. Scanning holds spinlock and blocks allocations which move partial slabs to per-cpu lists and back. Example found in the wild: # cat /sys/kernel/slab/dentry/partial 14478538 N0=7329569 N1=7148969 # time cat /sys/kernel/slab/dentry/objects 286225471 N0=136967768 N1=149257703 real 0m1.722s user 0m0.001s sys 0m1.721s The same problem in slab was addressed in commit f728b0a5d72a ("mm, slab: faster active and free stats") by adding more kmem cache statistics. For slub same approach requires atomic op on fast path when object frees. Let's simply limit count of scanned slabs and print warning. Limit set in /sys/module/slub/parameters/max_partial_to_count. Default is 10000 which should be enough for most sane cases. Return linear approximation if list of partials is longer than limit. Nobody should notice difference. Signed-off-by: Konstantin Khlebnikov --- mm/slub.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/mm/slub.c b/mm/slub.c index 9bf44955c4f1..86a366f7acb6 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2407,16 +2407,29 @@ static inline unsigned long node_nr_objs(struct kmem_cache_node *n) #endif /* CONFIG_SLUB_DEBUG */ #if defined(CONFIG_SLUB_DEBUG) || defined(CONFIG_SYSFS) + +static unsigned long max_partial_to_count __read_mostly = 10000; +module_param(max_partial_to_count, ulong, 0644); + static unsigned long count_partial(struct kmem_cache_node *n, int (*get_count)(struct page *)) { + unsigned long counted = 0; unsigned long flags; unsigned long x = 0; struct page *page; spin_lock_irqsave(&n->list_lock, flags); - list_for_each_entry(page, &n->partial, slab_list) + list_for_each_entry(page, &n->partial, slab_list) { x += get_count(page); + + if (++counted > max_partial_to_count) { + pr_warn_once("SLUB: too much partial slabs to count all objects, increase max_partial_to_count.\n"); + /* Approximate total count of objects */ + x = mult_frac(x, n->nr_partial, counted); + break; + } + } spin_unlock_irqrestore(&n->list_lock, flags); return x; }