From patchwork Thu Apr 1 22:30:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Trofimovich X-Patchwork-Id: 12179759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 230A8C433B4 for ; Thu, 1 Apr 2021 22:30:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AAFD96108F for ; Thu, 1 Apr 2021 22:30:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AAFD96108F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gentoo.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2CA626B0125; Thu, 1 Apr 2021 18:30:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 27AB46B0127; Thu, 1 Apr 2021 18:30:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11BE76B0128; Thu, 1 Apr 2021 18:30:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0183.hostedemail.com [216.40.44.183]) by kanga.kvack.org (Postfix) with ESMTP id E36E76B0125 for ; Thu, 1 Apr 2021 18:30:26 -0400 (EDT) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 95292282D for ; Thu, 1 Apr 2021 22:30:26 +0000 (UTC) X-FDA: 77985243252.28.35CE70F Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by imf12.hostedemail.com (Postfix) with ESMTP id E8468D5 for ; Thu, 1 Apr 2021 22:30:24 +0000 (UTC) Received: by sf.home (Postfix, from userid 1000) id 1CBBC5A22061; Thu, 1 Apr 2021 23:30:19 +0100 (BST) From: Sergei Trofimovich To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Sergei Trofimovich , Ingo Molnar , Peter Zijlstra , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Andrew Morton Subject: [PATCH] mm: page_owner: detect page_owner recursion via task_struct Date: Thu, 1 Apr 2021 23:30:10 +0100 Message-Id: <20210401223010.3580480-1-slyfox@gentoo.org> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E8468D5 X-Stat-Signature: ubq4t545nayyybuj3y8qf7oisrzdxj4g Received-SPF: none (gentoo.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=smtp.gentoo.org; client-ip=140.211.166.183 X-HE-DKIM-Result: none/none X-HE-Tag: 1617316224-162692 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Before the change page_owner recursion was detected via fetching backtrace and inspecting it for current instruction pointer. It has a few problems: - it is slightly slow as it requires extra backtrace and a linear stack scan of the result - it is too late to check if backtrace fetching required memory allocation itself (ia64's unwinder requires it). To simplify recursion tracking let's use page_owner recursion depth as a counter in 'struct task_struct'. The change make page_owner=on work on ia64 bu avoiding infinite recursion in: kmalloc() -> __set_page_owner() -> save_stack() -> unwind() [ia64-specific] -> build_script() -> kmalloc() -> __set_page_owner() [we short-circuit here] -> save_stack() -> unwind() [recursion] CC: Ingo Molnar CC: Peter Zijlstra CC: Juri Lelli CC: Vincent Guittot CC: Dietmar Eggemann CC: Steven Rostedt CC: Ben Segall CC: Mel Gorman CC: Daniel Bristot de Oliveira CC: Andrew Morton CC: linux-mm@kvack.org Signed-off-by: Sergei Trofimovich --- include/linux/sched.h | 9 +++++++++ init/init_task.c | 3 +++ mm/page_owner.c | 41 +++++++++++++++++------------------------ 3 files changed, 29 insertions(+), 24 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index ef00bb22164c..35771703fd89 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1371,6 +1371,15 @@ struct task_struct { struct llist_head kretprobe_instances; #endif +#ifdef CONFIG_PAGE_OWNER + /* + * Used by page_owner=on to detect recursion in page tracking. + * Is it fine to have non-atomic ops here if we ever access + * this variable via current->page_owner_depth? + */ + unsigned int page_owner_depth; +#endif + /* * New fields for task_struct should be added above here, so that * they are included in the randomized portion of task_struct. diff --git a/init/init_task.c b/init/init_task.c index 3711cdaafed2..f579f2b2eca8 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -213,6 +213,9 @@ struct task_struct init_task #ifdef CONFIG_SECCOMP .seccomp = { .filter_count = ATOMIC_INIT(0) }, #endif +#ifdef CONFIG_PAGE_OWNER + .page_owner_depth = 0, +#endif }; EXPORT_SYMBOL(init_task); diff --git a/mm/page_owner.c b/mm/page_owner.c index 7147fd34a948..422558605fcc 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -20,6 +20,16 @@ */ #define PAGE_OWNER_STACK_DEPTH (16) +/* + * How many reenters we allow to page_owner. + * + * Sometimes metadata allocation tracking requires more memory to be allocated: + * - when new stack trace is saved to stack depot + * - when backtrace itself is calculated (ia64) + * Instead of falling to infinite recursion give it a chance to recover. + */ +#define PAGE_OWNER_MAX_RECURSION_DEPTH (1) + struct page_owner { unsigned short order; short last_migrate_reason; @@ -97,42 +107,25 @@ static inline struct page_owner *get_page_owner(struct page_ext *page_ext) return (void *)page_ext + page_owner_ops.offset; } -static inline bool check_recursive_alloc(unsigned long *entries, - unsigned int nr_entries, - unsigned long ip) -{ - unsigned int i; - - for (i = 0; i < nr_entries; i++) { - if (entries[i] == ip) - return true; - } - return false; -} - static noinline depot_stack_handle_t save_stack(gfp_t flags) { unsigned long entries[PAGE_OWNER_STACK_DEPTH]; depot_stack_handle_t handle; unsigned int nr_entries; - nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 2); - - /* - * We need to check recursion here because our request to - * stackdepot could trigger memory allocation to save new - * entry. New memory allocation would reach here and call - * stack_depot_save_entries() again if we don't catch it. There is - * still not enough memory in stackdepot so it would try to - * allocate memory again and loop forever. - */ - if (check_recursive_alloc(entries, nr_entries, _RET_IP_)) + /* Avoid recursion. Used in stack trace generation code. */ + if (current->page_owner_depth >= PAGE_OWNER_MAX_RECURSION_DEPTH) return dummy_handle; + current->page_owner_depth++; + + nr_entries = stack_trace_save(entries, ARRAY_SIZE(entries), 2); + handle = stack_depot_save(entries, nr_entries, flags); if (!handle) handle = failure_handle; + current->page_owner_depth--; return handle; }