From patchwork Wed Mar 2 17:31:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12766419 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A07EDC433F5 for ; Wed, 2 Mar 2022 17:31:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4E3B8D0008; Wed, 2 Mar 2022 12:31:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CAB4D8D0007; Wed, 2 Mar 2022 12:31:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A66408D000A; Wed, 2 Mar 2022 12:31:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 7509E8D0007 for ; Wed, 2 Mar 2022 12:31:40 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 2D390620F6 for ; Wed, 2 Mar 2022 17:31:40 +0000 (UTC) X-FDA: 79200138360.05.31EFEAF Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf13.hostedemail.com (Postfix) with ESMTP id 3B3062001F for ; Wed, 2 Mar 2022 17:31:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 371DB1F39D; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=48xH8ZTiFHXoSQMDMbDhDVgxUz19c3sly3xHjZkeeao=; b=NV0bB1KwuS6luMyfYzLXSJiAzAf4yzCWKJ2kh0/udbFwtx4MR/G+ofKP1r49y/W4ldQrXH +CBFZqw1FD/mfjYdPpua1LUld141NK/7YhHt7j8dEJRvye20Z/ii8LwPUuaK0r2PMHbizk Ot3pl4psRMZtut0JIz/54+zwSXlJ2kg= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=48xH8ZTiFHXoSQMDMbDhDVgxUz19c3sly3xHjZkeeao=; b=c+F9XeRtLM7g3atREp577ucoAo9xHXR9USAm/S0qliucDd5QxflkAu4Txhkim6qfhY7hUq s9orFNcf9i0VZuDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E5C1F13D38; Wed, 2 Mar 2022 17:31:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id GOyCN/epH2LPTAAAMHmgww (envelope-from ); Wed, 02 Mar 2022 17:31:35 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin Cc: Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Marco Elver , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Imran Khan , Vlastimil Babka Subject: [PATCH v2 1/6] lib/stackdepot: allow requesting early initialization dynamically Date: Wed, 2 Mar 2022 18:31:17 +0100 Message-Id: <20220302173122.11939-2-vbabka@suse.cz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220302173122.11939-1-vbabka@suse.cz> References: <20220302173122.11939-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4062; h=from:subject; bh=fUAWYw/XGeQuLFiGDuRcbqQ62jmrmDVhwsfBAo33KGE=; b=owGbwMvMwMH4QPFU8cfOlQKMp9WSGJLkVz4tkfh50j/tfNQNLwPf7mus0968Kmb6qns1ictYKYRZ RFm3k9GYhYGRg0FWTJGl13sy40rTxxL7POLOwAxiZQKZwsDFKQATUZPjYFilajip33Vt8e2P/TO3fV upfuEJW7bWVAdGdv6Yh4KcaZc+GjtWWir7V/I4zjF1KNOxqil5ocWnejzrlYDC7vubPfueb/CN5qxX XJ7CwdFu9STbQnLGzKWf1k94mOe5dbWPXkQYw6pvSUWTK/NlfHqnW26wEiy7f6D32WJD/lVsT8Qu/8 +32trcUpjqUC18LHvPnw/K+9uCfxabzJm3/hP381iuQNt/7mV7GdrPBB29Fu6yx3hezL6WDy1MvXLv f7On/S9p0PZ8VbJZuk3F9PPRa+mmdq/XuhjXGtsvXFp5iiu4bP3r8p2K98WP/au6sdlBKEheXs2zhT 8pJCG7Kzzs5JnKBQIde/oidJoB X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Queue-Id: 3B3062001F X-Stat-Signature: 1e4gp4k9j3tfrigb43ciwouortuwxqum X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=NV0bB1Kw; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=c+F9XeRt; spf=pass (imf13.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Rspamd-Server: rspam07 X-HE-Tag: 1646242297-359556 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: In a later patch we want to add stackdepot support for object owner tracking in slub caches, which is enabled by slub_debug boot parameter. This creates a bootstrap problem as some caches are created early in boot when slab_is_available() is false and thus stack_depot_init() tries to use memblock. But, as reported by Hyeonggon Yoo [1] we are already beyond memblock_free_all(). Ideally memblock allocation should fail, yet it succeeds, but later the system crashes, which is a separately handled issue. To resolve this boostrap issue in a robust way, this patch adds another way to request stack_depot_early_init(), which happens at a well-defined point of time. In addition to build-time CONFIG_STACKDEPOT_ALWAYS_INIT, code that's e.g. processing boot parmeters (which happens early enough) can set a new variable stack_depot_want_early_init as true. In this patch we also convert page_owner to this approach. While it doesn't have the bootstrap issue as slub, it's also a functionality enabled by a boot param and can thus request stack_depot_early_init() with memblock allocation instead of later initialization with kvmalloc(). [1] https://lore.kernel.org/all/YhnUcqyeMgCrWZbd@ip-172-31-19-208.ap-northeast-1.compute.internal/ Reported-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Signed-off-by: Vlastimil Babka --- include/linux/stackdepot.h | 16 ++++++++++++++-- lib/stackdepot.c | 2 ++ mm/page_owner.c | 9 ++++++--- 3 files changed, 22 insertions(+), 5 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index 17f992fe6355..1217ba2b636e 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -15,6 +15,8 @@ typedef u32 depot_stack_handle_t; +extern bool stack_depot_want_early_init; + depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t gfp_flags, bool can_alloc); @@ -26,11 +28,21 @@ depot_stack_handle_t __stack_depot_save(unsigned long *entries, * The alternative is to select STACKDEPOT_ALWAYS_INIT to have stack depot * enabled as part of mm_init(), for subsystems where it's known at compile time * that stack depot will be used. + * + * Another alternative is to set stack_depot_want_early_init as true, when the + * decision to use stack depot is taken e.g. when evaluating kernel boot + * parameters, which precedes the call to stack_depot_want_early_init(). */ int stack_depot_init(void); -#ifdef CONFIG_STACKDEPOT_ALWAYS_INIT -static inline int stack_depot_early_init(void) { return stack_depot_init(); } +#ifdef CONFIG_STACKDEPOT +static inline int stack_depot_early_init(void) +{ + if (IS_ENABLED(CONFIG_STACKDEPOT_ALWAYS_INIT) + || stack_depot_want_early_init) + return stack_depot_init(); + return 0; +} #else static inline int stack_depot_early_init(void) { return 0; } #endif diff --git a/lib/stackdepot.c b/lib/stackdepot.c index bf5ba9af0500..02e2b5fcbf3b 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -66,6 +66,8 @@ struct stack_record { unsigned long entries[]; /* Variable-sized array of entries. */ }; +bool stack_depot_want_early_init = false; + static void *stack_slabs[STACK_ALLOC_MAX_SLABS]; static int depot_index; diff --git a/mm/page_owner.c b/mm/page_owner.c index 99e360df9465..40dce2b81d13 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -42,7 +42,12 @@ static void init_early_allocated_pages(void); static int __init early_page_owner_param(char *buf) { - return kstrtobool(buf, &page_owner_enabled); + int ret = kstrtobool(buf, &page_owner_enabled); + + if (page_owner_enabled) + stack_depot_want_early_init = true; + + return ret; } early_param("page_owner", early_page_owner_param); @@ -80,8 +85,6 @@ static __init void init_page_owner(void) if (!page_owner_enabled) return; - stack_depot_init(); - register_dummy_stack(); register_failure_stack(); register_early_stack(); From patchwork Wed Mar 2 17:31:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12766414 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD613C433F5 for ; Wed, 2 Mar 2022 17:31:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11DB58D0005; Wed, 2 Mar 2022 12:31:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 029B18D0001; Wed, 2 Mar 2022 12:31:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D1DCF8D0005; Wed, 2 Mar 2022 12:31:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id AB4298D0003 for ; Wed, 2 Mar 2022 12:31:38 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 683BB8248D52 for ; Wed, 2 Mar 2022 17:31:38 +0000 (UTC) X-FDA: 79200138276.21.5371FE4 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf26.hostedemail.com (Postfix) with ESMTP id B0CAE14000F for ; Wed, 2 Mar 2022 17:31:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 796F41F3A3; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r1N7lHefdQdRLrZdn0IGuF8+VmMNKPWirbLjcNDxaGY=; b=IDwyu0HRPpEDdX3qH4wxSuf3FbtkW1Sq3zxn73spk4TX9GhT7sboFX1MKRmUJf/VjurR3q h18ttIBZcoYyZaas0UDCc9F87KDQn17wA0zUPj53iZTIEAnyff6E+4l+LBCG6wFyi6naEP AX/eb9Xqo+YXAXPDlwjMDgLmEa6p5Eo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r1N7lHefdQdRLrZdn0IGuF8+VmMNKPWirbLjcNDxaGY=; b=aTf+RLXTMppbndGTrpCm9edJUqcEcfS5SqBs3R2NtOewR+I8RTE9HLYSpbcwrSVbOB56qo MFwg4DkR8mkP70Aw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3D91213A89; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id +Cz/DfipH2LPTAAAMHmgww (envelope-from ); Wed, 02 Mar 2022 17:31:36 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin Cc: Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Marco Elver , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Imran Khan , Vlastimil Babka Subject: [PATCH v2 2/6] mm/slub: move struct track init out of set_track() Date: Wed, 2 Mar 2022 18:31:18 +0100 Message-Id: <20220302173122.11939-3-vbabka@suse.cz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220302173122.11939-1-vbabka@suse.cz> References: <20220302173122.11939-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2040; h=from:subject; bh=5V02Ql5Wwt6ho1BAsi4n/qBKIm2FLxfXUIJYaK6h4Vs=; b=owEBbQGS/pANAwAIAeAhynPxiakQAcsmYgBiH6nmjmJgS1lZ546XQoaxj+A8/9t1bG5DvArQgDFu 1xtN6hKJATMEAAEIAB0WIQSNS5MBqTXjGL5IXszgIcpz8YmpEAUCYh+p5gAKCRDgIcpz8YmpECZ9CA Co15DjumUXFzKv14uunJSKOZY7cWSOuXlRF6M8bUCvUOz9SQPLNEVYQ6NA99DDIQltLe8NcWBlkIL0 nMFZpxbLP754+v0Tb5apmx2f/ixA9Uh8ja/gUAQ/h7AnVyD50tXpZ3uW2cYcoDRuCQIqtu+k7QBAml neMcM5aglvuFzaXdfZhR8kO92sZEnzpBJDPn2XiV/tyn+2F1H6FOYv2GE+BD7Z83KjeIb0i4mqxrLQ eVGTEmQxMsN1Fe81Z8zyaqSQevtlivkIUqwEaEi4QZQsjo7XTkrtvtR22ZmdUYegp0qhmjjwesmCdn p/08XmwOt9k25L46Isho9OYs3qL04V X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B0CAE14000F X-Stat-Signature: s3j6try3rijr5b374tr9g6ce3kw15bff Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=IDwyu0HR; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aTf+RLXT; spf=pass (imf26.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Rspam-User: X-HE-Tag: 1646242297-830479 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: set_track() either zeroes out the struct track or fills it, depending on the addr parameter. This is unnecessary as there's only one place that calls it for the initialization - init_tracking(). We can simply do the zeroing there, with a single memset() that covers both TRACK_ALLOC and TRACK_FREE as they are adjacent. Signed-off-by: Vlastimil Babka Reviewed-and-tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 261474092e43..1fc451f4fe62 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -729,34 +729,32 @@ static void set_track(struct kmem_cache *s, void *object, { struct track *p = get_track(s, object, alloc); - if (addr) { #ifdef CONFIG_STACKTRACE - unsigned int nr_entries; + unsigned int nr_entries; - metadata_access_enable(); - nr_entries = stack_trace_save(kasan_reset_tag(p->addrs), - TRACK_ADDRS_COUNT, 3); - metadata_access_disable(); + metadata_access_enable(); + nr_entries = stack_trace_save(kasan_reset_tag(p->addrs), + TRACK_ADDRS_COUNT, 3); + metadata_access_disable(); - if (nr_entries < TRACK_ADDRS_COUNT) - p->addrs[nr_entries] = 0; + if (nr_entries < TRACK_ADDRS_COUNT) + p->addrs[nr_entries] = 0; #endif - p->addr = addr; - p->cpu = smp_processor_id(); - p->pid = current->pid; - p->when = jiffies; - } else { - memset(p, 0, sizeof(struct track)); - } + p->addr = addr; + p->cpu = smp_processor_id(); + p->pid = current->pid; + p->when = jiffies; } static void init_tracking(struct kmem_cache *s, void *object) { + struct track *p; + if (!(s->flags & SLAB_STORE_USER)) return; - set_track(s, object, TRACK_FREE, 0UL); - set_track(s, object, TRACK_ALLOC, 0UL); + p = get_track(s, object, TRACK_ALLOC); + memset(p, 0, 2*sizeof(struct track)); } static void print_track(const char *s, struct track *t, unsigned long pr_time) From patchwork Wed Mar 2 17:31:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12766415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51838C4332F for ; Wed, 2 Mar 2022 17:31:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 544B28D0001; Wed, 2 Mar 2022 12:31:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 20D9C8D0006; Wed, 2 Mar 2022 12:31:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDE898D0003; Wed, 2 Mar 2022 12:31:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id B873D8D0001 for ; Wed, 2 Mar 2022 12:31:38 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8E1F325062 for ; Wed, 2 Mar 2022 17:31:38 +0000 (UTC) X-FDA: 79200138276.08.281706D Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf07.hostedemail.com (Postfix) with ESMTP id EFD9E40018 for ; Wed, 2 Mar 2022 17:31:37 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id B6E0C1F44D; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCaxkf/MeD15emqd3jto+7dldJPoNs7ZDgHl++2i7/Q=; b=TsnK/WQ/2RcSmnV4Av79OwvY8DJ5bhNRLF2IumvzNnz1v/ti25wBvAgDHwoBY0JCfHHhC4 RPN0HpUICRjRyo1vePf2u97nOTVwgoBFc6DQsql00g6trKNVeifQepNyhpx/rUz1SMM6J/ 7Q4ctTRCeEX7vGnSuzFhyg4LwIpl4wA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bCaxkf/MeD15emqd3jto+7dldJPoNs7ZDgHl++2i7/Q=; b=/TpSi0r9jG9stqerHnols26RX6IpDklcb9TLdzVX8PivEUYI/QWlcRgBDyALWg60ZhcPPa m4tKxTrQKoyL/CDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 7E6D513D38; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id GHjeHfipH2LPTAAAMHmgww (envelope-from ); Wed, 02 Mar 2022 17:31:36 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin Cc: Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Marco Elver , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Imran Khan , Vlastimil Babka Subject: [PATCH v2 3/6] mm/slub: use stackdepot to save stack trace in objects Date: Wed, 2 Mar 2022 18:31:19 +0100 Message-Id: <20220302173122.11939-4-vbabka@suse.cz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220302173122.11939-1-vbabka@suse.cz> References: <20220302173122.11939-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=7221; i=vbabka@suse.cz; h=from:subject; bh=eeOZ5fy5ud8Gx6oj27Gp5M6+txyzYk+05T/mhWZvDLw=; b=owEBbQGS/pANAwAIAeAhynPxiakQAcsmYgBiH6nn4maXb3uYS3GEk1NVGdCTCL8kvKjR22Fp51Pv IEfNg8SJATMEAAEIAB0WIQSNS5MBqTXjGL5IXszgIcpz8YmpEAUCYh+p5wAKCRDgIcpz8YmpEL2uB/ 4hv3bPDFSTeEcdTs+TugzyjKszd2ZLLlceH3dc634uVHMR0Un3bSdMfrIlivlWdpfjXJ3eEsC+guXs bVXf17LrGghCVn8iQw+pYOS6GqMg7KUGRp8YZ12XNJ6MB8eWQ6kJOsenhT+dG00yw/osT/IJSti6fi X4zR0SS1///t7QTwiEqZE9qCnLKes62VvbnJKs3jU+MCf1cG6v04VTY5NSSu5UMVHeBxfe4Xr53wFy dfxhO+6gg1wzvhJKeFRiEr2hb6nuqQGFeKfGxGYigx28X5D/Zo4yML/hrm9KMhDy2y+WYwOj0aqtnG VmmkoVdWD0hpDCHSdnvpgEd+j1bQcC X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Server: rspam10 X-Rspam-User: X-Stat-Signature: hfqr1nep5rwajrmda5o9j1piucpig5tb Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="TsnK/WQ/"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="/TpSi0r9"; spf=pass (imf07.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Rspamd-Queue-Id: EFD9E40018 X-HE-Tag: 1646242297-398401 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Oliver Glitta Many stack traces are similar so there are many similar arrays. Stackdepot saves each unique stack only once. Replace field addrs in struct track with depot_stack_handle_t handle. Use stackdepot to save stack trace. The benefits are smaller memory overhead and possibility to aggregate per-cache statistics in the following patch using the stackdepot handle instead of matching stacks manually. [ vbabka@suse.cz: rebase to 5.17-rc1 and adjust accordingly ] This was initially merged as commit 788691464c29 and reverted by commit ae14c63a9f20 due to several issues, that should now be fixed. The problem of unconditional memory overhead by stackdepot has been addressed by commit 2dba5eb1c73b ("lib/stackdepot: allow optional init and stack_table allocation by kvmalloc()"), so the dependency on stackdepot will result in extra memory usage only when a slab cache tracking is actually enabled, and not for all CONFIG_SLUB_DEBUG builds. The build failures on some architectures were also addressed, and the reported issue with xfs/433 test did not reproduce on 5.17-rc1 with this patch. Signed-off-by: Oliver Glitta Signed-off-by: Vlastimil Babka Cc: David Rientjes Cc: Christoph Lameter Cc: Pekka Enberg Cc: Joonsoo Kim Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- init/Kconfig | 1 + mm/slab_common.c | 5 ++++ mm/slub.c | 71 +++++++++++++++++++++++++++--------------------- 3 files changed, 46 insertions(+), 31 deletions(-) diff --git a/init/Kconfig b/init/Kconfig index e9119bf54b1f..b21dd3a4a106 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1871,6 +1871,7 @@ config SLUB_DEBUG default y bool "Enable SLUB debugging support" if EXPERT depends on SLUB && SYSFS + select STACKDEPOT if STACKTRACE_SUPPORT help SLUB has extensive debug support features. Disabling these can result in significant savings in code size. This also disables diff --git a/mm/slab_common.c b/mm/slab_common.c index 23f2ab0713b7..e51d50d03000 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -24,6 +24,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -314,9 +315,13 @@ kmem_cache_create_usercopy(const char *name, * If no slub_debug was enabled globally, the static key is not yet * enabled by setup_slub_debug(). Enable it if the cache is being * created with any of the debugging flags passed explicitly. + * It's also possible that this is the first cache created with + * SLAB_STORE_USER and we should init stack_depot for it. */ if (flags & SLAB_DEBUG_FLAGS) static_branch_enable(&slub_debug_enabled); + if (flags & SLAB_STORE_USER && IS_ENABLED(CONFIG_STACKDEPOT)) + stack_depot_init(); #endif mutex_lock(&slab_mutex); diff --git a/mm/slub.c b/mm/slub.c index 1fc451f4fe62..42cb79af70a0 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -26,6 +26,7 @@ #include #include #include +#include #include #include #include @@ -264,8 +265,8 @@ static inline bool kmem_cache_has_cpu_partial(struct kmem_cache *s) #define TRACK_ADDRS_COUNT 16 struct track { unsigned long addr; /* Called from address */ -#ifdef CONFIG_STACKTRACE - unsigned long addrs[TRACK_ADDRS_COUNT]; /* Called from address */ +#ifdef CONFIG_STACKDEPOT + depot_stack_handle_t handle; #endif int cpu; /* Was running on cpu */ int pid; /* Pid context */ @@ -724,22 +725,19 @@ static struct track *get_track(struct kmem_cache *s, void *object, return kasan_reset_tag(p + alloc); } -static void set_track(struct kmem_cache *s, void *object, +static void noinline set_track(struct kmem_cache *s, void *object, enum track_item alloc, unsigned long addr) { struct track *p = get_track(s, object, alloc); -#ifdef CONFIG_STACKTRACE +#ifdef CONFIG_STACKDEPOT + unsigned long entries[TRACK_ADDRS_COUNT]; unsigned int nr_entries; - metadata_access_enable(); - nr_entries = stack_trace_save(kasan_reset_tag(p->addrs), - TRACK_ADDRS_COUNT, 3); - metadata_access_disable(); - - if (nr_entries < TRACK_ADDRS_COUNT) - p->addrs[nr_entries] = 0; + nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 3); + p->handle = stack_depot_save(entries, nr_entries, GFP_NOWAIT); #endif + p->addr = addr; p->cpu = smp_processor_id(); p->pid = current->pid; @@ -759,20 +757,19 @@ static void init_tracking(struct kmem_cache *s, void *object) static void print_track(const char *s, struct track *t, unsigned long pr_time) { + depot_stack_handle_t handle __maybe_unused; + if (!t->addr) return; pr_err("%s in %pS age=%lu cpu=%u pid=%d\n", s, (void *)t->addr, pr_time - t->when, t->cpu, t->pid); -#ifdef CONFIG_STACKTRACE - { - int i; - for (i = 0; i < TRACK_ADDRS_COUNT; i++) - if (t->addrs[i]) - pr_err("\t%pS\n", (void *)t->addrs[i]); - else - break; - } +#ifdef CONFIG_STACKDEPOT + handle = READ_ONCE(t->handle); + if (handle) + stack_depot_print(handle); + else + pr_err("object allocation/free stack trace missing\n"); #endif } @@ -1532,6 +1529,8 @@ static int __init setup_slub_debug(char *str) global_slub_debug_changed = true; } else { slab_list_specified = true; + if (flags & SLAB_STORE_USER) + stack_depot_want_early_init = true; } } @@ -1549,6 +1548,8 @@ static int __init setup_slub_debug(char *str) } out: slub_debug = global_flags; + if (slub_debug & SLAB_STORE_USER) + stack_depot_want_early_init = true; if (slub_debug != 0 || slub_debug_string) static_branch_enable(&slub_debug_enabled); else @@ -4352,18 +4353,26 @@ void kmem_obj_info(struct kmem_obj_info *kpp, void *object, struct slab *slab) objp = fixup_red_left(s, objp); trackp = get_track(s, objp, TRACK_ALLOC); kpp->kp_ret = (void *)trackp->addr; -#ifdef CONFIG_STACKTRACE - for (i = 0; i < KS_ADDRS_COUNT && i < TRACK_ADDRS_COUNT; i++) { - kpp->kp_stack[i] = (void *)trackp->addrs[i]; - if (!kpp->kp_stack[i]) - break; - } +#ifdef CONFIG_STACKDEPOT + { + depot_stack_handle_t handle; + unsigned long *entries; + unsigned int nr_entries; + + handle = READ_ONCE(trackp->handle); + if (handle) { + nr_entries = stack_depot_fetch(handle, &entries); + for (i = 0; i < KS_ADDRS_COUNT && i < nr_entries; i++) + kpp->kp_stack[i] = (void *)entries[i]; + } - trackp = get_track(s, objp, TRACK_FREE); - for (i = 0; i < KS_ADDRS_COUNT && i < TRACK_ADDRS_COUNT; i++) { - kpp->kp_free_stack[i] = (void *)trackp->addrs[i]; - if (!kpp->kp_free_stack[i]) - break; + trackp = get_track(s, objp, TRACK_FREE); + handle = READ_ONCE(trackp->handle); + if (handle) { + nr_entries = stack_depot_fetch(handle, &entries); + for (i = 0; i < KS_ADDRS_COUNT && i < nr_entries; i++) + kpp->kp_free_stack[i] = (void *)entries[i]; + } } #endif #endif From patchwork Wed Mar 2 17:31:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12766416 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17576C433EF for ; Wed, 2 Mar 2022 17:31:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D471F8D0006; Wed, 2 Mar 2022 12:31:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CD2E38D0007; Wed, 2 Mar 2022 12:31:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A84AC8D0006; Wed, 2 Mar 2022 12:31:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 71EA98D0003 for ; Wed, 2 Mar 2022 12:31:39 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 41CC71D2B for ; Wed, 2 Mar 2022 17:31:39 +0000 (UTC) X-FDA: 79200138318.05.4A33AC8 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf12.hostedemail.com (Postfix) with ESMTP id 8B9504001A for ; Wed, 2 Mar 2022 17:31:38 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id EE40C1F8A8; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LQnHXhrO7on2owbSpKE5n1md35S3c5ELU5Z9yrYqevE=; b=pGtPuEgrk5zRWMLCylIQqL/l5kwpBUKLLwQw+/Nv36eFxoT56EUcD44bCY77kbRWNRxjO7 0MviCuveY2snGyyOS6++DZvGY3GDWxRBA8bSTYyJpj9twCLQrpbwZBdjSQiVwVDAY27ZBr doHMwNTSu1zvA9yQ79eVAO4O8N8V/o0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1646242296; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LQnHXhrO7on2owbSpKE5n1md35S3c5ELU5Z9yrYqevE=; b=EtbnYSUXQ8iCWjGO9SRXOrH2uy/pJIQHgf7+YT2dM+RIqmxlnlP+aUZ4wnGbXsPaIOw8WC 3JKikr+weZdjYNAQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id BA77413A89; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 8EnuLPipH2LPTAAAMHmgww (envelope-from ); Wed, 02 Mar 2022 17:31:36 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin Cc: Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Marco Elver , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Imran Khan , Vlastimil Babka Subject: [PATCH v2 4/6] mm/slub: distinguish and print stack traces in debugfs files Date: Wed, 2 Mar 2022 18:31:20 +0100 Message-Id: <20220302173122.11939-5-vbabka@suse.cz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220302173122.11939-1-vbabka@suse.cz> References: <20220302173122.11939-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2691; i=vbabka@suse.cz; h=from:subject; bh=i+vTj/bLGNkS06QEkNuKLLTXKRcpqAmsd8AhSKRmP/4=; b=owEBbQGS/pANAwAIAeAhynPxiakQAcsmYgBiH6noW6mVlFyLbj1aJzDiTKdnm9vD+CY6DbeO2YmL hzn1zXKJATMEAAEIAB0WIQSNS5MBqTXjGL5IXszgIcpz8YmpEAUCYh+p6AAKCRDgIcpz8YmpEPTTB/ 9Nm9Dxg3WSur7zB1Abt51hsqTZJ6TRd/u/Jx+tqEfmRCycJ89uyjl30SrhfjtLpAnUI2jha9AS6e+e +Zp5ZgM/67CLBHvLOQ9xD04qmnsl9+6aIM9LFj409LI5UQwJnttguLW1zdpea4kOQOF/1Zq8dK91eT rKBG8LGz+V+9fXO9x6vbN6TBzxvRLRzZqFWk4khMGMH7WY0PyktwnuIRbaU4s5n+rmvXroeLUNJ2ZY CgRpRTS3ykCDnUHj5fROFNHpJMWOSxjaln/3LAlX3svyq/tkZX1hflr8g/0Nxz6DimghkUEg0ILehb /cYy/J8v+gjNitirJ9GpHv/ObwEE/s X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Queue-Id: 8B9504001A X-Stat-Signature: 9rnsq1yhdub3ajgggun3gnufgb1st9uz Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=pGtPuEgr; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=EtbnYSUX; spf=pass (imf12.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1646242298-463777 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Oliver Glitta Aggregate objects in slub cache by unique stack trace in addition to caller address when producing contents of debugfs files alloc_traces and free_traces in debugfs. Also add the stack traces to the debugfs output. This makes it much more useful to e.g. debug memory leaks. Signed-off-by: Oliver Glitta Signed-off-by: Vlastimil Babka Reviewed-and-tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 42cb79af70a0..25138cd7a4ef 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5074,6 +5074,7 @@ EXPORT_SYMBOL(validate_slab_cache); */ struct location { + depot_stack_handle_t handle; unsigned long count; unsigned long addr; long long sum_time; @@ -5126,9 +5127,13 @@ static int add_location(struct loc_track *t, struct kmem_cache *s, { long start, end, pos; struct location *l; - unsigned long caddr; + unsigned long caddr, chandle; unsigned long age = jiffies - track->when; + depot_stack_handle_t handle = 0; +#ifdef CONFIG_STACKDEPOT + handle = READ_ONCE(track->handle); +#endif start = -1; end = t->count; @@ -5143,7 +5148,8 @@ static int add_location(struct loc_track *t, struct kmem_cache *s, break; caddr = t->loc[pos].addr; - if (track->addr == caddr) { + chandle = t->loc[pos].handle; + if ((track->addr == caddr) && (handle == chandle)) { l = &t->loc[pos]; l->count++; @@ -5168,6 +5174,8 @@ static int add_location(struct loc_track *t, struct kmem_cache *s, if (track->addr < caddr) end = pos; + else if (track->addr == caddr && handle < chandle) + end = pos; else start = pos; } @@ -5190,6 +5198,7 @@ static int add_location(struct loc_track *t, struct kmem_cache *s, l->max_time = age; l->min_pid = track->pid; l->max_pid = track->pid; + l->handle = handle; cpumask_clear(to_cpumask(l->cpus)); cpumask_set_cpu(track->cpu, to_cpumask(l->cpus)); nodes_clear(l->nodes); @@ -6101,6 +6110,21 @@ static int slab_debugfs_show(struct seq_file *seq, void *v) seq_printf(seq, " nodes=%*pbl", nodemask_pr_args(&l->nodes)); +#ifdef CONFIG_STACKDEPOT + { + depot_stack_handle_t handle; + unsigned long *entries; + unsigned int nr_entries, j; + + handle = READ_ONCE(l->handle); + if (handle) { + nr_entries = stack_depot_fetch(handle, &entries); + seq_puts(seq, "\n"); + for (j = 0; j < nr_entries; j++) + seq_printf(seq, " %pS\n", (void *)entries[j]); + } + } +#endif seq_puts(seq, "\n"); } From patchwork Wed Mar 2 17:31:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12766418 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF293C4332F for ; Wed, 2 Mar 2022 17:31:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F8928D0009; Wed, 2 Mar 2022 12:31:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 693308D0008; Wed, 2 Mar 2022 12:31:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35FA78D0007; Wed, 2 Mar 2022 12:31:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id EA9188D0008 for ; Wed, 2 Mar 2022 12:31:39 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id CA623251FC for ; Wed, 2 Mar 2022 17:31:39 +0000 (UTC) X-FDA: 79200138318.01.A143BD3 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf19.hostedemail.com (Postfix) with ESMTP id 31B571A0014 for ; Wed, 2 Mar 2022 17:31:39 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 3C1501F8AA; Wed, 2 Mar 2022 17:31:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1646242297; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LwcP1iJrYs9CiAU77l19QAKomEttfa24uCohxTMqv9A=; b=YGwfIf+naYMjndyRe1xaiqBlc0LEwBeqFk2Rgqi7wvpAEeuDNTfz8hZlqi0AKOY0xneEG6 mnhbvBk/rxNxxqG1sDJUbpBYWssViMCnuxgIR2aSSrq2Dval/FZf6Kt8+V0VwHiYGxXLxy WFJsq8E/AdY9bfOCMYeU/vfEMQcwzTk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1646242297; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LwcP1iJrYs9CiAU77l19QAKomEttfa24uCohxTMqv9A=; b=dmywO0lYe0j27g6NwDAGRsVMvyc04njnj0FW6VpDmK1nSudERrNP1/y5dyuyTArsS0/A6v MhRgzTJ99tJQDSDw== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id F1F8013D38; Wed, 2 Mar 2022 17:31:36 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id cBiDOvipH2LPTAAAMHmgww (envelope-from ); Wed, 02 Mar 2022 17:31:36 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin Cc: Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Marco Elver , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Imran Khan , Vlastimil Babka Subject: [PATCH v2 5/6] mm/slub: sort debugfs output by frequency of stack traces Date: Wed, 2 Mar 2022 18:31:21 +0100 Message-Id: <20220302173122.11939-6-vbabka@suse.cz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220302173122.11939-1-vbabka@suse.cz> References: <20220302173122.11939-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1585; i=vbabka@suse.cz; h=from:subject; bh=tlKVMC4Mx/hSa/FZSyCKskcpV5VFc4MSXPAqJSKQmvo=; b=owEBbQGS/pANAwAIAeAhynPxiakQAcsmYgBiH6npw4Jqu7vak6nvRHvxTCzeJjHQMlEYpCh9kQiZ Kc2CKi6JATMEAAEIAB0WIQSNS5MBqTXjGL5IXszgIcpz8YmpEAUCYh+p6QAKCRDgIcpz8YmpECnaB/ 43KXlOKcXMrSiW0sg7KEoTPuicTHQe25b5a2/l8Nr1v/N2Ay7agOO3pWPD/Mah/V27RAjM5T1NhTnl I++Nw27NO3RD7qNmvmA0VAMvpSG3RAT7nne7V83BJmSFFng/8QHp1hgDXQ+ctDY1K1zKqYh3xoXTdc vxcsAS8uOKvH5/TCVM6Ihw/ZC11qGthce8a2RFEGpxdwKjnP0BfuvcsJO2CU50HYXLzspWPHnZie2i xprk+SPoEB01FhHfqdAj5ZprQ+GiAxILxnMTFRVFs4DAnk6PIbSOAIH5vAVIOGdaUuRO8YhlDWj2ub O18MaKRRTTv8heY4DqKzc2FzIZmQqb X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 31B571A0014 X-Rspam-User: Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=YGwfIf+n; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=dmywO0lY; spf=pass (imf19.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Stat-Signature: x5n1daueb6rnu1xgoj1ncq6qrzd9kdf6 X-HE-Tag: 1646242299-29852 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Oliver Glitta Sort the output of debugfs alloc_traces and free_traces by the frequency of allocation/freeing stack traces. Most frequently used stack traces will be printed first, e.g. for easier memory leak debugging. Signed-off-by: Oliver Glitta Signed-off-by: Vlastimil Babka Reviewed-and-tested-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> --- mm/slub.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/mm/slub.c b/mm/slub.c index 25138cd7a4ef..39da762fbceb 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include @@ -6149,6 +6150,17 @@ static void *slab_debugfs_next(struct seq_file *seq, void *v, loff_t *ppos) return NULL; } +static int cmp_loc_by_count(const void *a, const void *b, const void *data) +{ + struct location *loc1 = (struct location *)a; + struct location *loc2 = (struct location *)b; + + if (loc1->count > loc2->count) + return -1; + else + return 1; +} + static void *slab_debugfs_start(struct seq_file *seq, loff_t *ppos) { struct loc_track *t = seq->private; @@ -6210,6 +6222,10 @@ static int slab_debug_trace_open(struct inode *inode, struct file *filep) spin_unlock_irqrestore(&n->list_lock, flags); } + /* Sort locations by count */ + sort_r(t->loc, t->count, sizeof(struct location), + cmp_loc_by_count, NULL, NULL); + bitmap_free(obj_map); return 0; } From patchwork Wed Mar 2 17:31:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 12766417 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD888C433F5 for ; Wed, 2 Mar 2022 17:31:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 126048D0003; Wed, 2 Mar 2022 12:31:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 048398D0009; Wed, 2 Mar 2022 12:31:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE7368D0003; Wed, 2 Mar 2022 12:31:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0106.hostedemail.com [216.40.44.106]) by kanga.kvack.org (Postfix) with ESMTP id 90E768D0007 for ; Wed, 2 Mar 2022 12:31:39 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4EF759F864 for ; Wed, 2 Mar 2022 17:31:39 +0000 (UTC) X-FDA: 79200138318.21.30376C7 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf30.hostedemail.com (Postfix) with ESMTP id 9217080020 for ; Wed, 2 Mar 2022 17:31:38 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 830C6218BB; Wed, 2 Mar 2022 17:31:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1646242297; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eVsefkriocSzUjBw8OkHFxHxhJgQ5EMDJypHn5jHs0k=; b=efE1c3vjkSOemRtKwzggI20vo6vnHKG2TEVFRCBo0662+au5WFaFYiw1+X1T8Gzw4Ess6B Pd3H5D2RFZrhhYNkiNCa1WRyCrvNZW/V6TWOpNQLicZ408BDRaUBJRwPsby7OymdWYZ3HK JEHDMGAqzV6AMpNiqXn/uAuSoth5CuA= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1646242297; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eVsefkriocSzUjBw8OkHFxHxhJgQ5EMDJypHn5jHs0k=; b=isl/wtgOnDtwene/o7J0JbzT7+nbsR71FRtkBz2833QpQlfGxHZVOcXlpOZH+9ilpxn3wW znHdK/GtW6UyEDBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3F9FD13A89; Wed, 2 Mar 2022 17:31:37 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CLroDvmpH2LPTAAAMHmgww (envelope-from ); Wed, 02 Mar 2022 17:31:37 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Joonsoo Kim , Pekka Enberg , Roman Gushchin Cc: Andrew Morton , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Oliver Glitta , Faiyaz Mohammed , Marco Elver , Mike Rapoport , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Imran Khan , Vlastimil Babka , Jonathan Corbet , Randy Dunlap , linux-doc@vger.kernel.org Subject: [PATCH v2 6/6] slab, documentation: add description of debugfs files for SLUB caches Date: Wed, 2 Mar 2022 18:31:22 +0100 Message-Id: <20220302173122.11939-7-vbabka@suse.cz> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220302173122.11939-1-vbabka@suse.cz> References: <20220302173122.11939-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3322; i=vbabka@suse.cz; h=from:subject; bh=gbw9JIIw0x4FMb8yzBSEH95gTlbrc/vprmL4/cGUu4k=; b=owEBbQGS/pANAwAIAeAhynPxiakQAcsmYgBiH6np64HPxdLs4MUHrwO0Y4xm1Ad8bnLMOPVjGHXe zCMzIdKJATMEAAEIAB0WIQSNS5MBqTXjGL5IXszgIcpz8YmpEAUCYh+p6QAKCRDgIcpz8YmpEAm4B/ 0b/rHHThVdfnOzXz8uyxBJehaveMN/UrVVj6FHBJ+wxO0SrxtdJlt/QcXKO7lQT+lTzFWS5LLW118v JRXOzmeUgqS62n8CSoZgv+sXiTR+E8rD5wryLR+yIV6CWtmGSEipTilQDz6ReKR3SdfWf7hoNMEko4 7tME/YbnVnsSlQumOei5l6TfiYWXVebOSYd26A/vufzXldRah9PeDnuSY3G7boMo3aCPaUTO9EHTYy KpVIiknnJe25ofizUuISTBn7krTyIP+1wBdowdXjjrp14nvqgcmGd2oIJL7TOtzJz7S7th+7T/iVIh Mz4nDDdd80wDHnpqO1SprbDJvE8raQ X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Rspamd-Queue-Id: 9217080020 X-Stat-Signature: 1te5tfjsaj1zs36kp6gqzmc4k8zfww1w X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=efE1c3vj; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="isl/wtgO"; spf=pass (imf30.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.28 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Rspamd-Server: rspam07 X-HE-Tag: 1646242298-628089 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Oliver Glitta Add description of debugfs files alloc_traces and free_traces to SLUB cache documentation. [ vbabka@suse.cz: some rewording ] Signed-off-by: Oliver Glitta Signed-off-by: Vlastimil Babka Cc: Jonathan Corbet Cc: Randy Dunlap Cc: linux-doc@vger.kernel.org Reviewed-by: Hyeonggon Yoo <42.hyeyoo@gmail.com> Acked-by: Mike Rapoport --- Documentation/vm/slub.rst | 64 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/Documentation/vm/slub.rst b/Documentation/vm/slub.rst index d3028554b1e9..43063ade737a 100644 --- a/Documentation/vm/slub.rst +++ b/Documentation/vm/slub.rst @@ -384,5 +384,69 @@ c) Execute ``slabinfo-gnuplot.sh`` in '-t' mode, passing all of the 40,60`` range will plot only samples collected between 40th and 60th seconds). + +DebugFS files for SLUB +====================== + +For more information about current state of SLUB caches with the user tracking +debug option enabled, debugfs files are available, typically under +/sys/kernel/debug/slab// (created only for caches with enabled user +tracking). There are 2 types of these files with the following debug +information: + +1. alloc_traces:: + + Prints information about unique allocation traces of the currently + allocated objects. The output is sorted by frequency of each trace. + + Information in the output: + Number of objects, allocating function, minimal/average/maximal jiffies since alloc, + pid range of the allocating processes, cpu mask of allocating cpus, and stack trace. + + Example::: + + 1085 populate_error_injection_list+0x97/0x110 age=166678/166680/166682 pid=1 cpus=1:: + __slab_alloc+0x6d/0x90 + kmem_cache_alloc_trace+0x2eb/0x300 + populate_error_injection_list+0x97/0x110 + init_error_injection+0x1b/0x71 + do_one_initcall+0x5f/0x2d0 + kernel_init_freeable+0x26f/0x2d7 + kernel_init+0xe/0x118 + ret_from_fork+0x22/0x30 + + +2. free_traces:: + + Prints information about unique freeing traces of the currently allocated + objects. The freeing traces thus come from the previous life-cycle of the + objects and are reported as not available for objects allocated for the first + time. The output is sorted by frequency of each trace. + + Information in the output: + Number of objects, freeing function, minimal/average/maximal jiffies since free, + pid range of the freeing processes, cpu mask of freeing cpus, and stack trace. + + Example::: + + 1980 age=4294912290 pid=0 cpus=0 + 51 acpi_ut_update_ref_count+0x6a6/0x782 age=236886/237027/237772 pid=1 cpus=1 + kfree+0x2db/0x420 + acpi_ut_update_ref_count+0x6a6/0x782 + acpi_ut_update_object_reference+0x1ad/0x234 + acpi_ut_remove_reference+0x7d/0x84 + acpi_rs_get_prt_method_data+0x97/0xd6 + acpi_get_irq_routing_table+0x82/0xc4 + acpi_pci_irq_find_prt_entry+0x8e/0x2e0 + acpi_pci_irq_lookup+0x3a/0x1e0 + acpi_pci_irq_enable+0x77/0x240 + pcibios_enable_device+0x39/0x40 + do_pci_enable_device.part.0+0x5d/0xe0 + pci_enable_device_flags+0xfc/0x120 + pci_enable_device+0x13/0x20 + virtio_pci_probe+0x9e/0x170 + local_pci_probe+0x48/0x80 + pci_device_probe+0x105/0x1c0 + Christoph Lameter, May 30, 2007 Sergey Senozhatsky, October 23, 2015