From patchwork Mon Nov 20 08:42:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oscar Salvador X-Patchwork-Id: 13460930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E7B0C54E76 for ; Mon, 20 Nov 2023 08:43:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DDB876B0405; Mon, 20 Nov 2023 03:43:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D8C526B0406; Mon, 20 Nov 2023 03:43:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C54D96B0407; Mon, 20 Nov 2023 03:43:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B61CB6B0405 for ; Mon, 20 Nov 2023 03:43:15 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 81EAB1407E2 for ; Mon, 20 Nov 2023 08:43:15 +0000 (UTC) X-FDA: 81477693150.03.7BBE59C Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf13.hostedemail.com (Postfix) with ESMTP id 6AD6A20023 for ; Mon, 20 Nov 2023 08:43:13 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=I2B+KXB0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=kFHm8z2I; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.220.28 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700469793; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MGpg1D+QkOzBj9HvH6Pm0yN+PlTUNsrk3hG0zRKlbo0=; b=3k/v23tZUkBIYeqp+iy9TiEWpWyUC0VhrAUrADvuj5k7RtOiHQPxbGqMpCauOMoeBkbsLM R++Z8X3JLBiHnp5Hs+z0S+ASEpoyGhwpY6FVnUIFcYtd2NjzbcCW1s0DN9/rmomM/8kPkE haeVOTCdpzhqIsOMSNgmpE7ZgeML2ck= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=I2B+KXB0; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=kFHm8z2I; dmarc=pass (policy=none) header.from=suse.de; spf=pass (imf13.hostedemail.com: domain of osalvador@suse.de designates 195.135.220.28 as permitted sender) smtp.mailfrom=osalvador@suse.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700469793; a=rsa-sha256; cv=none; b=WD3tCPBE6EKKTCi4UZlpZTylUjQGaxNlhL/4CP7XPRgAQ3XmwqhftlkAP190FMhV1/Q8Qo E/SX5gfPD9IevJ+aimdcEkyscZvDcefQiP1+aqot1Kn0pOFbHcbtmB81B5doKg8sy2hY9f LVU1IVrwNqqcxykpzGg/FLsWeYdTIac= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A23E2218EA; Mon, 20 Nov 2023 08:43:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1700469791; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MGpg1D+QkOzBj9HvH6Pm0yN+PlTUNsrk3hG0zRKlbo0=; b=I2B+KXB0e9Z8ba/sISmr7L1ZKTOJNxenGu9vqCIQj6M7fRTZftawKFLK4iEL8j5xND0rdf FE99H50tlrN3e4sKT2JKUpEOL9Hk8MzDeOwGBI9689VKXtifZUalbA1rd0lQdVAvZu+9oi lPd0IiBuPwkSK6eJv6UDXx53+W0mUlQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1700469791; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MGpg1D+QkOzBj9HvH6Pm0yN+PlTUNsrk3hG0zRKlbo0=; b=kFHm8z2IL7kLEdY1UWV1ldcG4z+LFZcCdrWBKgAyr8a324wFwRAr5+772aJ5JFlZVKf0It 09G4NT9vwC+18jDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id E420F13912; Mon, 20 Nov 2023 08:43:10 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id qFTTNB4cW2U3VgAAMHmgww (envelope-from ); Mon, 20 Nov 2023 08:43:10 +0000 From: Oscar Salvador To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Michal Hocko , Vlastimil Babka , Waiman Long , Suren Baghdasaryan , Marco Elver , Andrey Konovalov , Eric Dumazet , Alexander Potapenko , Oscar Salvador Subject: [PATCH v6 1/4] lib/stackdepot: Add a refcount field in stack_record Date: Mon, 20 Nov 2023 09:42:57 +0100 Message-Id: <20231120084300.4368-2-osalvador@suse.de> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20231120084300.4368-1-osalvador@suse.de> References: <20231120084300.4368-1-osalvador@suse.de> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6AD6A20023 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: w91tdyg7e7me6zygsu91qckybe94upqd X-HE-Tag: 1700469793-311261 X-HE-Meta: U2FsdGVkX18LAiIZYYTns9qBTmEdEOULHIhzLNsWl3GnJZgwmnIYvSpZAY7De3k/mJCqq9aEDmCOkpS209elAotLbumbnADhUwxwbUeYNYpVhmSVHuJMekqKKxW0/ROJZvIL6RV8Rpu2CLy8qbJmdKKEkSUxWjmr5fE5c0P0KGVvfROgWhpCJub2sDv2s6Zdp1UyHwquYJjZhKAKnJp8uiZr6zjqTqQ1ZOMogqs+s7TsLfznqmKwj7ZG97YXZbxl57CY9pNGp51q3iSkQDiLeADnsKmobfZQ9iHJpKbW372WVjeYjIQDZv+6K89lRmdXz4o+rloL/QpFkuLrcQ1mmIN9NrL8Y8j6szn1Kjmj8JoKHiMf2Iqx3/X+PWecgU/msB+xjEXmzRFatQKJpwcuy6wi3THOeUW9IqcIFvDeL98G7I1nic83O05HQQOZb6LG2S3gZGmM4yK1JUrKMHk9s0567c+Hc6p7d+etKesyhODeKphZtHKI7hMieuJfmqVJfPxhzphDPslC/or28IQdPlppJIsaG5SWr97Z2x+LhbLooKfBgDQ79a/pIJAnz974sV49/SUZsUCibkSKcF83SLTR+vH7dPHD2vsqNg7aORpwxdXH0ujnq0oN5akkom5JpSeZQmxyTfynDpeNoFPY1ULiEOUo0lihfopnqcb9wUisfVg/XmXhNZGqAlfGm/89Wm9SLChC/SqRUSyv59HZzLghFOAgB7/1g0lZ/OnnaIDuhtuAMzwLv88TFT7SZCGgUULgjzOz5EWgDiI6h4FpXO0GlYyDnH7QTkUJvZeXmHyxRAI7aRO5JM/Z9+9mkouUBpOs+xhFZqhrMG285geg0NCMULcxX9kx9yI3OnC1Vi8+N9LZ+w2/HUIwQkyFE6FJnwtiiwLF054QhUynxtyYPQzh0AKdq1kmalrFa3yZ4IAYTB72JOYN9bZsaLNYpG+PhJzb0MZa+BAzhdfGJcP bJVoe59a LT/JpiiAYDmY8YOw4EebIY8qEotX5HeeREWEQQZ58WBACnBPX4fSM1v3B/miWcvodUPRJI9A5HNsz9w+/Eo0wWiapHfppnMtEd9R4EqpAxDcCtnpmS3SgoOsGoC6m9YWEBWrm19AbalDg/oBqas2BTu+VZ6d2nRJYSWGsttB6OJuX8Ep609LExxGwcoZsQtRhFNaFGk+V06ceiZ9sklnOGzPCkhZbYK7PJhxQg63WIyNyHwCO4iQv6Ss5cEDP32w7fzpx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We want to filter out the page_owner output and print only those stacks for which their counter (number of outstanding allocations) goes beyond a certain threshold. This gives us the chance to get rid of a lot of noise. In order to do that, we need to know how many outstanding allocations with a particular stack (for allocation) do we have, so we add a new refcount_t field in the stack_record struct. Note that this might increase the size of the struct for some architectures. E.g: x86_64 is not affected due to alignment, but x86 32bits might. Besides adding the refcount, this patch also introduces stack_depot_{inc,dec}_count for the allocation/free handle. Signed-off-by: Oscar Salvador --- include/linux/stackdepot.h | 2 ++ lib/stackdepot.c | 53 +++++++++++++++++++++++++++++++------- mm/page_owner.c | 6 +++++ 3 files changed, 51 insertions(+), 10 deletions(-) diff --git a/include/linux/stackdepot.h b/include/linux/stackdepot.h index e58306783d8e..6ba4fcdb0c5f 100644 --- a/include/linux/stackdepot.h +++ b/include/linux/stackdepot.h @@ -94,6 +94,8 @@ static inline int stack_depot_early_init(void) { return 0; } depot_stack_handle_t __stack_depot_save(unsigned long *entries, unsigned int nr_entries, gfp_t gfp_flags, bool can_alloc); +void stack_depot_inc_count(depot_stack_handle_t handle); +void stack_depot_dec_count(depot_stack_handle_t handle); /** * stack_depot_save - Save a stack trace to stack depot diff --git a/lib/stackdepot.c b/lib/stackdepot.c index 2f5aa851834e..d35edac430c4 100644 --- a/lib/stackdepot.c +++ b/lib/stackdepot.c @@ -60,6 +60,7 @@ struct stack_record { u32 hash; /* Hash in the hash table */ u32 size; /* Number of stored frames */ union handle_parts handle; + refcount_t count; /* Number of the same repeated stacks */ unsigned long entries[]; /* Variable-sized array of frames */ }; @@ -305,6 +306,7 @@ depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **prealloc) stack->handle.offset = pool_offset >> DEPOT_STACK_ALIGN; stack->handle.valid = 1; stack->handle.extra = 0; + refcount_set(&stack->count, 1); memcpy(stack->entries, entries, flex_array_size(stack, entries, size)); pool_offset += required_size; /* @@ -457,8 +459,7 @@ depot_stack_handle_t stack_depot_save(unsigned long *entries, } EXPORT_SYMBOL_GPL(stack_depot_save); -unsigned int stack_depot_fetch(depot_stack_handle_t handle, - unsigned long **entries) +static struct stack_record *stack_depot_getstack(depot_stack_handle_t handle) { union handle_parts parts = { .handle = handle }; /* @@ -470,6 +471,26 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, size_t offset = parts.offset << DEPOT_STACK_ALIGN; struct stack_record *stack; + if (!handle) + return NULL; + + if (parts.pool_index > pool_index_cached) { + WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", + parts.pool_index, pool_index_cached, handle); + return NULL; + } + pool = stack_pools[parts.pool_index]; + if (!pool) + return NULL; + stack = pool + offset; + return stack; +} + +unsigned int stack_depot_fetch(depot_stack_handle_t handle, + unsigned long **entries) +{ + struct stack_record *stack; + *entries = NULL; /* * Let KMSAN know *entries is initialized. This shall prevent false @@ -480,21 +501,33 @@ unsigned int stack_depot_fetch(depot_stack_handle_t handle, if (!handle) return 0; - if (parts.pool_index > pool_index_cached) { - WARN(1, "pool index %d out of bounds (%d) for stack id %08x\n", - parts.pool_index, pool_index_cached, handle); - return 0; - } - pool = stack_pools[parts.pool_index]; - if (!pool) + stack = stack_depot_getstack(handle); + if (!stack) return 0; - stack = pool + offset; *entries = stack->entries; return stack->size; } EXPORT_SYMBOL_GPL(stack_depot_fetch); +void stack_depot_inc_count(depot_stack_handle_t handle) +{ + struct stack_record *stack = NULL; + + stack = stack_depot_getstack(handle); + if (stack) + refcount_inc(&stack->count); +} + +void stack_depot_dec_count(depot_stack_handle_t handle) +{ + struct stack_record *stack = NULL; + + stack = stack_depot_getstack(handle); + if (stack) + refcount_dec(&stack->count); +} + void stack_depot_print(depot_stack_handle_t stack) { unsigned long *entries; diff --git a/mm/page_owner.c b/mm/page_owner.c index 4f13ce7d2452..d53316d0d9be 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -139,6 +139,7 @@ void __reset_page_owner(struct page *page, unsigned short order) int i; struct page_ext *page_ext; depot_stack_handle_t handle; + depot_stack_handle_t alloc_handle; struct page_owner *page_owner; u64 free_ts_nsec = local_clock(); @@ -146,6 +147,9 @@ void __reset_page_owner(struct page *page, unsigned short order) if (unlikely(!page_ext)) return; + page_owner = get_page_owner(page_ext); + alloc_handle = page_owner->handle; + handle = save_stack(GFP_NOWAIT | __GFP_NOWARN); for (i = 0; i < (1 << order); i++) { __clear_bit(PAGE_EXT_OWNER_ALLOCATED, &page_ext->flags); @@ -155,6 +159,7 @@ void __reset_page_owner(struct page *page, unsigned short order) page_ext = page_ext_next(page_ext); } page_ext_put(page_ext); + stack_depot_dec_count(alloc_handle); } static inline void __set_page_owner_handle(struct page_ext *page_ext, @@ -196,6 +201,7 @@ noinline void __set_page_owner(struct page *page, unsigned short order, return; __set_page_owner_handle(page_ext, handle, order, gfp_mask); page_ext_put(page_ext); + stack_depot_inc_count(handle); } void __set_page_owner_migrate_reason(struct page *page, int reason)