From patchwork Tue Oct 22 11:47:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lance Yang X-Patchwork-Id: 13845622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E499AD1713D for ; Tue, 22 Oct 2024 11:48:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6373F6B0088; Tue, 22 Oct 2024 07:48:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D7746B0089; Tue, 22 Oct 2024 07:48:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 451706B008A; Tue, 22 Oct 2024 07:48:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2498E6B0088 for ; Tue, 22 Oct 2024 07:48:16 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3DD40161159 for ; Tue, 22 Oct 2024 11:47:57 +0000 (UTC) X-FDA: 82701064446.21.FE1596D Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf15.hostedemail.com (Postfix) with ESMTP id C5FC8A001F for ; Tue, 22 Oct 2024 11:47:57 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XzqqOfvs; spf=pass (imf15.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729597544; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hKl6U/5kYqIul35IkUGRHOkC+Sxs+wAsHazeOfjnayo=; b=hhBytLJcmvDLMpN7MXa0e/E8cIfifupM8uSljl4gmM+REmb1N1WyuU9yb5elfKCnwffgvy +kD5Cv9FMRJTFpWk+TO3/tqbMLhiCDu2B7+qI/UXWvaRHoNaTDdlBYanvqyzuZ+GP5034Z 7981QAieNOoOc4R9J2gDfvfYtANtCbk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729597544; a=rsa-sha256; cv=none; b=wt6UoexbZkZgNBCtLdbRHdMMN1w9UsRWdysfU4aLNciy70gAJya7GYaASQ31F8rq6CfPd/ 4P0MzZJRLxYlkadIL8MLaDLjVo0SPBXUVYnuKQN+9sWlIBCyYB3tvrfQ0oOFQAJTmjiqkA yg6oe6XDd6b3i6XsEghS5ojIjE1hneU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XzqqOfvs; spf=pass (imf15.hostedemail.com: domain of ioworker0@gmail.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=ioworker0@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-20ce65c8e13so47350075ad.1 for ; Tue, 22 Oct 2024 04:48:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729597692; x=1730202492; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hKl6U/5kYqIul35IkUGRHOkC+Sxs+wAsHazeOfjnayo=; b=XzqqOfvslK53++0QwMqudZ1xes2/zGXiqkHxNv2bdlfcd5NikyllphTj1jdgFejk4m RJcUpSnlWn9exE1Q7/UlP0ezHcrqU+sQ7GahKVBXibeJ3bmTTu/2en1+loiMvr6xIR3c MOk1BwFYe/dJxRjEk9oO3nvjD2uG8OOX/DgSkn63wMU2UrLHY0iDS6A8MtoxwFF4kkBQ Jd6Ft89zChebCckUL0edyeRKHtdU6dEbtnKIeEkxTdecF31HWuYj8WhJKpH2PO46l89i iLE6VAmgbepSae0QNJqo7VwEfLM7jz1l3x2UVeLSWczCfsjlQAnd6IG7rr6xEzpWaMwC evfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729597692; x=1730202492; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hKl6U/5kYqIul35IkUGRHOkC+Sxs+wAsHazeOfjnayo=; b=LCDuDYx4T72JWXKpuf4yDcMSnu6k/+A8Ejq5T9QzQpyBaCCj7rfVd1i7quKIBiAhu+ NKYCb++MvzV4qBGxGMkvtfBapHPSTGDBIzLJfUPIU8xOxW71+WtVTzW9NEyRSxCR1GGq YY7VeWOZXH/lfcNvCFuOnlfRTkxP7AsOarP8TT7PKjonS+O/O0z6C8TflPkSUY5Norwf P7ONL3AUvBn3syfJW1xZwE3yfqZm1HJ9YtcaXVs2NIV1+xAj01mmvtAjccUgFHn2dQTu c0SIBV4E091yzk5VxmkE6oBzWGLZGEepnaPacAsqv16XpwDSYG4wgFpi1x7vBjdP5uff tRYw== X-Forwarded-Encrypted: i=1; AJvYcCX9yRRb/iHWSwkrWSAWOO3XqDtYzxzb4Zl0n7YhvQcAIpf+C+iIGGHWCFvNodz+WClXBfcD6vGPRg==@kvack.org X-Gm-Message-State: AOJu0YxmZrNq+DBRHM183BTjsmfsD7h+RGpDCFl6UvP4J5rW6YAQa4le +YKOLq4Rag/psowLc8nZyVQ/hO59hHANkd1opkrn0AMxn30DyDIz X-Google-Smtp-Source: AGHT+IHiGn0rGlYqLqA+F/OWg8LpaBp6Ymm27XpKX4Az/xFYXPaV3FzGzu4YKfbwVcSJ4WTJHHB2HQ== X-Received: by 2002:a17:902:ea0a:b0:20b:70b4:69d8 with SMTP id d9443c01a7336-20e984a5a57mr20839455ad.37.1729597692576; Tue, 22 Oct 2024 04:48:12 -0700 (PDT) Received: from EBJ9932692.tcent.cn ([43.129.202.66]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20e7ef0e13bsm41147145ad.65.2024.10.22.04.48.07 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 22 Oct 2024 04:48:12 -0700 (PDT) From: Lance Yang To: akpm@linux-foundation.org Cc: cunhuang@tencent.com, leonylgao@tencent.com, j.granados@samsung.com, jsiddle@redhat.com, kent.overstreet@linux.dev, 21cnbao@gmail.com, ryan.roberts@arm.com, david@redhat.com, ziy@nvidia.com, libang.li@antgroup.com, baolin.wang@linux.alibaba.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Lance Yang , Mingzhe Yang Subject: [PATCH 1/2] hung_task: add detect count for hung tasks Date: Tue, 22 Oct 2024 19:47:35 +0800 Message-ID: <20241022114736.83285-2-ioworker0@gmail.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20241022114736.83285-1-ioworker0@gmail.com> References: <20241022114736.83285-1-ioworker0@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: C5FC8A001F X-Stat-Signature: 8qbua48kykq41a6ify94b4e9ryb1b6xh X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1729597677-235378 X-HE-Meta: U2FsdGVkX1/sDKAPEMpZGYM4Mg9ocvr/TWLFN7blZiZMMMNDZ9TZbz6OQlyaj5dqJNGPTeYdpHnqY3TqUJjjyTSsDaHuNfyZTi9aG6jUcJB5o7FeIgRFh70GHUPKqggAw9Sx5+S14rRncyjt9MYHR05I3NmAuXG0IN383gVYo6eRDLzaqjTuH2fsfjIydf6fpFHSNWS003+7Eig/cFD7VJIJyzG9JaamXGiF6uIBwRw68AYcY42zFtYRDzoEz4aiN2Jg4zSc+nbnAbTRh/qf/kXrAwO5B+kZ+XZBffOBuq++53/dGzlNPJsaONhWQktFkJzFvFcWEZE0HCmHXWZNVmNR4x9GSZuKgupC9Wa9QhuOoIawIU5Gr3Kh9tLLb1mNlrm/IMr4iVb1ifGWltkBRdZW5FREWFrqQRUX7BVCjUggRWx2JraL+OUrycSdDMQsg/8scWs4vvXHfgQ5kuz/78S64vfTVjODpmRkpJYBDzRGeAB8l+P7onf9zpe0b55O1bFH7fstRrokRATwBAOpCMBRncszOXn3e5R5BO3EFYVaRDFiaBsefxte0WQSCevElzyTpMO2Hvd7ESr01YSG+HmUmU7wJkYF2rGYcoud9NvMJA3Rt0A6ZYWv8WywTvq/61Uu8/LjXdH1kcgx16nqgX8K4qbFP7STPOw6FanxHXtG+Nsoyh69qCIh5os9ulTP8XJFlblJioi8F/4KjxTndkppHWM79czE08mCgiqtAcEgzYRu57j/xWv/9B9vKen+r7NJ0MXj9ht1+9aZm+MjVuOkmUmzjBOnSRCZRJU093ZLcNWlE00YY+qRfm9hXRci8uEaztcQnKPyrF/LqAtIQrrujWaSAPFIghQfG71KSRRbXhgmopGGy0SZp3dJsWOvjfevo+wLkCQS3T2mlzR+O7ng5YvYi1FKYxzfI+U33GSnn7UBRSmnYjBT5dtbnoNCyU+YNHReSVSI/fUtRg7 awM4v/T9 xftwRDgNaShvWfdEGO2z6/37tT2/egaIV13hOVf8kHTCs5fwShUc33DfudL1H/4oWj9+ruydXttUDQzfLNVGIymFPRU9IlfE4LcQgMiI93jAcbNO0nAl/Nn+IOEfbG3q/aP0HVd0/FAcO2jTLLjHDU7+iOciCVa87U4jgp4MICY/5C3UMTCU0X/N65Wjd6H9dIOAe6DI4PO185ftVcx5Jk1zW9v4fQ3/3ySAzSC1F52zIfsIapXhvWweYLfjAILU2cL0OHR5r3KIdJOqubjroUMOrCSmEkKGRKlVkMBCaUdxGAftxGq4K069xiaTXZC+Wvofa0dnLjFDx+3bXU5Dc5/LBCo4yNAAv3lYoTNFYi3hJetUEAMVjL8AdJP2VfoRaiZG0kaax68HFurOXoAXrBgufuuRGsWSse7xYtJeoOjnI7LrSBQ0xIN7+FgRZTAd2o+mPkHDq93QUvsIp76x230EJGqvjBVW74bESYcA72egjuW8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000310, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This commit adds a counter, hung_task_detect_count, to track the number of times hung tasks are detected. This counter provides a straightforward way to monitor hung task events without manually checking dmesg logs. With this counter in place, system issues can be spotted quickly, allowing admins to step in promptly before system load spikes occur, even if the hung_task_warnings value has been decreased to 0 well before. Signed-off-by: Mingzhe Yang Signed-off-by: Lance Yang --- kernel/hung_task.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/kernel/hung_task.c b/kernel/hung_task.c index 959d99583d1c..229ff3d4e501 100644 --- a/kernel/hung_task.c +++ b/kernel/hung_task.c @@ -30,6 +30,11 @@ */ static int __read_mostly sysctl_hung_task_check_count = PID_MAX_LIMIT; +/* + * Total number of tasks detected as hung since boot: + */ +static unsigned long __read_mostly sysctl_hung_task_detect_count; + /* * Limit number of tasks checked in a batch. * @@ -115,6 +120,12 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout) if (time_is_after_jiffies(t->last_switch_time + timeout * HZ)) return; + /* + * This counter tracks the total number of tasks detected as hung + * since boot. + */ + sysctl_hung_task_detect_count++; + trace_sched_process_hang(t); if (sysctl_hung_task_panic) { @@ -314,6 +325,13 @@ static struct ctl_table hung_task_sysctls[] = { .proc_handler = proc_dointvec_minmax, .extra1 = SYSCTL_NEG_ONE, }, + { + .procname = "hung_task_detect_count", + .data = &sysctl_hung_task_detect_count, + .maxlen = sizeof(unsigned long), + .mode = 0444, + .proc_handler = proc_dointvec, + }, }; static void __init hung_task_sysctl_init(void)