From patchwork Tue Jan 31 08:39:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122428 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46E7EC636CC for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BE97C6B0074; Tue, 31 Jan 2023 03:40:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B40FB6B0075; Tue, 31 Jan 2023 03:40:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 992C26B0078; Tue, 31 Jan 2023 03:40:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 84EA96B0074 for ; Tue, 31 Jan 2023 03:40:00 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 577331C600F for ; Tue, 31 Jan 2023 08:40:00 +0000 (UTC) X-FDA: 80414446560.07.0FEFA4E Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf08.hostedemail.com (Postfix) with ESMTP id EDA86160007 for ; Tue, 31 Jan 2023 08:39:57 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf08.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154398; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=AOXI2niMw5pMQofmXYWujFqGse/4IkLn2XEkaU2SwvA=; b=laMTgo8KNjpyG78e7lP7l5kTUnaU9toAqEfOmSO5I4PimDp1lkYvqLhDIKeo9sELTVTbTZ YpCV7wbuRItkRZqn02x3CrRNor9D4Audjil9y8KX3Oq5RGJXvH1Zq9hM+SlNqCT5d8SMNt /CjNNHLMMgmURdKw2cMnLqz11y9NhhQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf08.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154398; a=rsa-sha256; cv=none; b=sv9jJemTbChq5VVjm/UKEFylh07LgAjM/pO/uq8XOBOtnx+nBAcl/6IWHgfRY4HSg3jdDJ 85WV9h5zzsNZMZ6cMvZia/b/ht9ThLEy9fyC1waM7rvV7bAaL5iMIzKw9/LBDLeklcW8nL hKM5XgY2UJ8Ci0O/3gwdecEv03Koofo= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:39:55 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:55 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 01/25] llist: Move llist_{head,node} definition to types.h Date: Tue, 31 Jan 2023 17:39:30 +0900 Message-Id: <1675154394-25598-2-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: EDA86160007 X-Stat-Signature: rz16ceu4r1y166cmwo51aqos4jhukga1 X-HE-Tag: 1675154397-887408 X-HE-Meta: U2FsdGVkX1/GF2nLn9fEazu+C2MxAkEzn0+YLmN6xUBXT2bbCquOr4jimlsloQMTYoN3iMs8TvnjhKpQP6yt+ITkSEGTvSpzusfyAOfM82sf0OCmnZNAjmKRY5ZEsFKaYIrebTzJiEZPIscVy2M6az1XrmVLvDW3b/Ombjcvw4inuHVkDNgK0YORX0TixLCmw7rVgUeOYU7h4LjEfc3uypnUHgaoglNXgMfTC3088lWf5kDbtilN9keVh5FGUKbctqmbkN3SjznhwCIpU5RGuNXXo7nGvARq9JDX19y+5KMQ+FssdxQTPPNfO8KXffqcZp3Ef+8wBEZPfIeJK+3laV/N53uFTcGVW0vUFVndrn7wXMYKFRZZNK+2mPl/CurS7wlvBv45Jk7rbjn3GupUYh7c/u7+BWwfDe5VRsYzJFMWC1lpZ3BmmLvBO103/7G2DStrpkWLhr+DsTgv1JUESfTrL1OMMoooki4KBHgPSQIOuPjVKYmyViwXoiqUfHasHFhMPhmyMsLGXbG2sYZGcq8RRCF01MQT2YKS4Xj7WsB4Sp6+rUUZkXtc9bqL7+gz1ONgz4mFB0qS+rbor7expIqaUkUZ5ZqD/XGlDWGOeiicCjbv4wqfkMqQ903qxBjrccuPL/MuslQ0d9Bvfp0WeT/CmPTHLzs9uwK/OTnD6Pda5TUifWXN9Tg6O1AxACCLPrIt4OnMlxeYUzJunhn3w396Nw55l7qyWF/PR/m/lETm/XHvjVz8fYU+/yLJXRk97a1Unj+m46iUM0s9hC20fVR6RVEOPCJduMpjLvmquRqTKUBLyt1mOjlo2IP0PWVjqCEAezXV7xkSrhQMngR4J4cOUOycVICzEWgfWnZZNawntaMblliOkGmaHLmskhWXTP88G6fb5f8xIUuupQ7QuUsEBwvm39yVAkq0ByZKaCCu40pnk14xG3ne8Al5m6VuW0SHlbzldVVaafrbyd2 Cjxsr46N 2oybsfcAxx0MiAcBjcyePFE2lKDybKHjB8RvuGaZAVHdFcsYtMwmkHsJ87aR4F7m/VQ4GWeReIOkfe+mzGqmewcAWfhSoXj30LIBEObHUkETytx8kkOkznBrkWG5QoaHPj8VFcFDhFQ6OVqtUyLePWAAbgb0gDm40SGm+rPspfzfGYmNTQi7EvRwQIWNMpu1xW127dHsEbbhZWrwBeDkOBKxCLkFe73WRzcc1tmIPJPTVxHBczJcN6vWINNxYVNAUB5It8AondIpTmv1QzDjcefMmmF7uWeop2zToky/46ZGOu8nSHc03ZcYSeayjFqPFG1F8SAzPGhLN/ilqFIl+96vLCoW2h67S/iZwZAKyt0MPajs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: llist_head and llist_node can be used by very primitives. For example, Dept for tracking dependency uses llist things in its header. To avoid header dependency, move those to types.h. Signed-off-by: Byungchul Park --- include/linux/llist.h | 8 -------- include/linux/types.h | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/llist.h b/include/linux/llist.h index 85bda2d..99cc3c3 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -53,14 +53,6 @@ #include #include -struct llist_head { - struct llist_node *first; -}; - -struct llist_node { - struct llist_node *next; -}; - #define LLIST_HEAD_INIT(name) { NULL } #define LLIST_HEAD(name) struct llist_head name = LLIST_HEAD_INIT(name) diff --git a/include/linux/types.h b/include/linux/types.h index ea8cf60a..b12a444 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -187,6 +187,14 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +struct llist_head { + struct llist_node *first; +}; + +struct llist_node { + struct llist_node *next; +}; + struct ustat { __kernel_daddr_t f_tfree; #ifdef CONFIG_ARCH_32BIT_USTAT_F_TINODE From patchwork Tue Jan 31 08:39:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F51DC636D3 for ; Tue, 31 Jan 2023 08:40:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCC926B008C; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B1A466B0089; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 366F86B008A; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CE8196B0088 for ; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8EA0CAB074 for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.30.9C9D5BC Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf12.hostedemail.com (Postfix) with ESMTP id EE58F40011 for ; Tue, 31 Jan 2023 08:39:57 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf12.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=SouZHCYG96wJoH/otxGyp7sax/u5paaa9QAJIOZJWZ4=; b=MaRQYPxSHW7bj3kJ04KuxG7TlrTAkKl0CjSRUozTKkdWoEQU1ZOaiHVq7/FPyPvnC7k6g3 lcYhMweyR4uen1D28MKqWsQJpFL/7VPyaimnDJ2YZ1atsrAlB0zqFEcL50Qyt6ntFgAg05 ns+7i2jedY/EET4f+AZ820c+/XA6rw8= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf12.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154399; a=rsa-sha256; cv=none; b=LOYCNCiL2pPH1mhIbJDN7xWFKM2uxbXrOKsOzLPJbyodPAuyTaS21l1Ob+ycqhzck154tb bAzX7VQCMqCRafWgplr/Xn+fdG8QX1raWy8xiQ6h2k+30IdX85ojlQ3cAyxMzn7k3bY37F 3T0cvW65gw5Dv68ZJe50WhlUULhXjKQ= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:39:55 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:55 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 02/25] dept: Implement Dept(Dependency Tracker) Date: Tue, 31 Jan 2023 17:39:31 +0900 Message-Id: <1675154394-25598-3-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EE58F40011 X-Stat-Signature: 8yjsfdy3bab6opjtkjenhwj811imjng3 X-HE-Tag: 1675154397-63235 X-HE-Meta: U2FsdGVkX1/6+2nom/YiElrq4GbXtynAUQaBqvI8/gG9TiGg6ERHuRvtHZWL9/Z5CYnMCWWcAxqW2ienV9MbtEozPowH8EDXMoO/scCAcGAFbWnwOsLy+nr1292DXYjliNgbN+1VukR6BYqdvW8UxfcA5HCBxSnX/q5ilkNVzsI190LjkAs7BWdXboLA79Wxad4p0hln41RMWaSmFFNYWLl09CnhBrqxLH3RXlvNqEunwZSiV00MM76KL0Z/uRo8aiAyfx/c/LK8YpRovUBy+V2iDnXhlFrf2yTPlQnMJr08EDHg9FvQhmzWIlDDZU4lhjobcmlePF7zBS6snChUVmanJ8QKGeWp1CvpVE3p23WdSS4aN3Lc+h7F12+Md/WgNK/lL9Fn2GxC7gYuZqUOZBi332750BQ8TlsbBctqWGo7XBBC/R8NCtAzXrsdhYB8Om5cgiMORT/HT81LXuG+FI9kSd/2nWmAZMgvQf1rsyyZ5pNNryGXPiHY0OLguTdobRqfxj/dG4tzauoDoweO/xxBaHSo0r3rnIZvoWdbBze/LgLMtg3mBLuPdsqma0jPDUqzerqlHcIhdEIke9dCX2X8PBwJfjU7CrYPUCZDC/i8l11HobvT6nGF89/N5aHOoECt8XlOWf5+bMy8isdTwVUsLQ2C52CSHTJHWIpGrj093yZMejl/scrdma0mBw1Gi0x/+JeDawT06bMBbd4YcdmGoqnCgP4k0RO1Vss4I8cs5Au9RrrJVCSRYJmpTIhZ6gSZFLWcsjw1kiyfpaH0h7vhnbQq0+MVeIDFaA5zJfOj2Pe5rveKxTVtm16bIClLxlKAGxph7PmOA+BhHS412P5AFgjeXCytyaxjWKmmQ4SAt6w0HnB6zOC18aF1OSjpC4GCHzzbDK1Wmn90l+L6i3tD3nUdgbv4/FrArm6pqVQz3/kVnQfvZp/+720fEFlnlSZEfYLnmp8jJVcoNBR 3N37zCsb wjvubWjpLXlT2togD0uDMSMlqUaqajRTuC4UUMbNchyB/kGMkpJKN3ffSwb3ji06yHymom2nNYm3PCy3SQYQumKAm1sAimA64lQpmnRTugzkXa1f4ER8QDECkm3M3I+1OFbv7bCznslQT65blNDinh2L0KY2JZmZ9igCvdle0zkkyVWlN2nmUQvBB0h7+lbDjUTO7kuzqZQhgUJrAbzfb8n8ROZnoaNArizq3tWSXTrxI9zITOzSfHSXXDJ9HldKU1Ee9NUXFUdyb5Isp+7zdWwCM9dlcZLiLy7We88mHMe6+/al0kIUCkClBSLq3otNODog0hMQWULSDGVkrLKHtEOgMhtkP0+48C5Gk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: CURRENT STATUS -------------- Lockdep tracks acquisition order of locks in order to detect deadlock, and IRQ and IRQ enable/disable state as well to take accident acquisitions into account. Lockdep should be turned off once it detects and reports a deadlock since the data structure and algorithm are not reusable after detection because of the complex design. PROBLEM ------- *Waits* and their *events* that never reach eventually cause deadlock. However, Lockdep is only interested in lock acquisition order, forcing to emulate lock acqusition even for just waits and events that have nothing to do with real lock. Even worse, no one likes Lockdep's false positive detection because that prevents further one that might be more valuable. That's why all the kernel developers are sensitive to Lockdep's false positive. Besides those, by tracking acquisition order, it cannot correctly deal with read lock and cross-event e.g. wait_for_completion()/complete() for deadlock detection. Lockdep is no longer a good tool for that purpose. SOLUTION -------- Again, *waits* and their *events* that never reach eventually cause deadlock. The new solution, Dept(DEPendency Tracker), focuses on waits and events themselves. Dept tracks waits and events and report it if any event would be never reachable. Dept does: . Works with read lock in the right way. . Works with any wait and event e.i. cross-event. . Continue to work even after reporting multiple times. . Provides simple and intuitive APIs. . Does exactly what dependency checker should do. Q & A ----- Q. Is this the first try ever to address the problem? A. No. Cross-release feature (b09be676e0ff2 locking/lockdep: Implement the 'crossrelease' feature) addressed it 2 years ago that was a Lockdep extension and merged but reverted shortly because: Cross-release started to report valuable hidden problems but started to give report false positive reports as well. For sure, no one likes Lockdep's false positive reports since it makes Lockdep stop, preventing reporting further real problems. Q. Why not Dept was developed as an extension of Lockdep? A. Lockdep definitely includes all the efforts great developers have made for a long time so as to be quite stable enough. But I had to design and implement newly because of the following: 1) Lockdep was designed to track lock acquisition order. The APIs and implementation do not fit on wait-event model. 2) Lockdep is turned off on detection including false positive. Which is terrible and prevents developing any extension for stronger detection. Q. Do you intend to totally replace Lockdep? A. No. Lockdep also checks if lock usage is correct. Of course, the dependency check routine should be replaced but the other functions should be still there. Q. Do you mean the dependency check routine should be replaced right away? A. No. I admit Lockdep is stable enough thanks to great efforts kernel developers have made. Lockdep and Dept, both should be in the kernel until Dept gets considered stable. Q. Stronger detection capability would give more false positive report. Which was a big problem when cross-release was introduced. Is it ok with Dept? A. It's ok. Dept allows multiple reporting thanks to simple and quite generalized design. Of course, false positive reports should be fixed anyway but it's no longer as a critical problem as it was. Signed-off-by: Byungchul Park --- include/linux/dept.h | 577 ++++++++ include/linux/hardirq.h | 3 + include/linux/sched.h | 3 + init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/dependency/Makefile | 3 + kernel/dependency/dept.c | 3009 +++++++++++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_object.h | 13 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/module/main.c | 2 + kernel/sched/core.c | 9 + lib/Kconfig.debug | 27 + lib/locking-selftest.c | 2 + 16 files changed, 3666 insertions(+) create mode 100644 include/linux/dept.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_object.h diff --git a/include/linux/dept.h b/include/linux/dept.h new file mode 100644 index 00000000..b6d45b4 --- /dev/null +++ b/include/linux/dept.h @@ -0,0 +1,577 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DEPT(DEPendency Tracker) - runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_H +#define __LINUX_DEPT_H + +#ifdef CONFIG_DEPT + +#include + +struct task_struct; + +#define DEPT_MAX_STACK_ENTRY 16 +#define DEPT_MAX_WAIT_HIST 64 +#define DEPT_MAX_ECXT_HELD 48 + +#define DEPT_MAX_SUBCLASSES 16 +#define DEPT_MAX_SUBCLASSES_EVT 2 +#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) +#define DEPT_MAX_SUBCLASSES_CACHE 2 + +#define DEPT_SIRQ 0 +#define DEPT_HIRQ 1 +#define DEPT_IRQS_NR 2 +#define DEPT_SIRQF (1UL << DEPT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_HIRQ) + +struct dept_ecxt; +struct dept_iecxt { + struct dept_ecxt *ecxt; + int enirq; + /* + * for preventing to add a new ecxt + */ + bool staled; +}; + +struct dept_wait; +struct dept_iwait { + struct dept_wait *wait; + int irq; + /* + * for preventing to add a new wait + */ + bool staled; + bool touched; +}; + +struct dept_class { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * unique information about the class + */ + const char *name; + unsigned long key; + int sub_id; + + /* + * for BFS + */ + unsigned int bfs_gen; + int bfs_dist; + struct dept_class *bfs_parent; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking all classes + */ + struct list_head all_node; + + /* + * for associating its dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for tracking IRQ dependencies + */ + struct dept_iecxt iecxt[DEPT_IRQS_NR]; + struct dept_iwait iwait[DEPT_IRQS_NR]; + + /* + * classified by a map embedded in task_struct, + * not an explicit map + */ + bool sched_map; + }; + }; +}; + +struct dept_key { + union { + /* + * Each byte-wise address will be used as its key. + */ + char base[DEPT_MAX_SUBCLASSES]; + + /* + * for caching the main class pointer + */ + struct dept_class *classes[DEPT_MAX_SUBCLASSES_CACHE]; + }; +}; + +struct dept_map { + const char *name; + struct dept_key *keys; + + /* + * subclass that can be set from user + */ + int sub_u; + + /* + * It's local copy for fast access to the associated classes. + * Also used for dept_key for static maps. + */ + struct dept_key map_key; + + /* + * wait timestamp associated to this map + */ + unsigned int wgen; + + /* + * whether this map should be going to be checked or not + */ + bool nocheck; +}; + +#define DEPT_MAP_INITIALIZER(n, k) \ +{ \ + .name = #n, \ + .keys = (struct dept_key *)(k), \ + .sub_u = 0, \ + .map_key = { .classes = { NULL, } }, \ + .wgen = 0U, \ + .nocheck = false, \ +} + +struct dept_stack { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * backtrace entries + */ + unsigned long raw[DEPT_MAX_STACK_ENTRY]; + int nr; + }; + }; +}; + +struct dept_ecxt { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function that entered to this ecxt + */ + const char *ecxt_fn; + + /* + * event function + */ + const char *event_fn; + + /* + * associated class + */ + struct dept_class *class; + + /* + * flag indicating which IRQ has been + * enabled within the event context + */ + unsigned long enirqf; + + /* + * where the IRQ-enabled happened + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + + /* + * where the event context started + */ + unsigned long ecxt_ip; + struct dept_stack *ecxt_stack; + + /* + * where the event triggered + */ + unsigned long event_ip; + struct dept_stack *event_stack; + }; + }; +}; + +struct dept_wait { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function causing this wait + */ + const char *wait_fn; + + /* + * the associated class + */ + struct dept_class *class; + + /* + * which IRQ the wait was placed in + */ + unsigned long irqf; + + /* + * where the IRQ wait happened + */ + unsigned long irq_ip[DEPT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_IRQS_NR]; + + /* + * where the wait happened + */ + unsigned long wait_ip; + struct dept_stack *wait_stack; + + /* + * whether this wait is for commit in scheduler + */ + bool sched_sleep; + }; + }; +}; + +struct dept_dep { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * key data of dependency + */ + struct dept_ecxt *ecxt; + struct dept_wait *wait; + + /* + * This object can be referred without dept_lock + * held but with IRQ disabled, e.g. for hash + * lookup. So deferred deletion is needed. + */ + struct rcu_head rh; + + /* + * for BFS + */ + struct list_head bfs_node; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking to a class object + */ + struct list_head dep_node; + struct list_head dep_rev_node; + }; + }; +}; + +struct dept_hash { + /* + * hash table + */ + struct hlist_head *table; + + /* + * size of the table e.i. 2^bits + */ + int bits; +}; + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +struct dept_ecxt_held { + /* + * associated event context + */ + struct dept_ecxt *ecxt; + + /* + * unique key for this dept_ecxt_held + */ + struct dept_map *map; + + /* + * class of the ecxt of this dept_ecxt_held + */ + struct dept_class *class; + + /* + * the wgen when the event context started + */ + unsigned int wgen; + + /* + * subclass that only works in the local context + */ + int sub_l; +}; + +struct dept_wait_hist { + /* + * associated wait + */ + struct dept_wait *wait; + + /* + * unique id of all waits system-wise until wrapped + */ + unsigned int wgen; + + /* + * local context id to identify IRQ context + */ + unsigned int ctxt_id; +}; + +struct dept_task { + /* + * all event contexts that have entered and before exiting + */ + struct dept_ecxt_held ecxt_held[DEPT_MAX_ECXT_HELD]; + int ecxt_held_pos; + + /* + * ring buffer holding all waits that have happened + */ + struct dept_wait_hist wait_hist[DEPT_MAX_WAIT_HIST]; + int wait_hist_pos; + + /* + * sequential id to identify each IRQ context + */ + unsigned int irq_id[DEPT_IRQS_NR]; + + /* + * for tracking IRQ-enabled points with cross-event + */ + unsigned int wgen_enirq[DEPT_IRQS_NR]; + + /* + * for keeping up-to-date IRQ-enabled points + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + + /* + * current effective IRQ-enabled flag + */ + unsigned long eff_enirqf; + + /* + * for reserving a current stack instance at each operation + */ + struct dept_stack *stack; + + /* + * for preventing recursive call into DEPT engine + */ + int recursive; + + /* + * for staging data to commit a wait + */ + struct dept_map stage_m; + bool stage_sched_map; + const char *stage_w_fn; + unsigned long stage_ip; + + /* + * the number of missing ecxts + */ + int missing_ecxt; + + /* + * for tracking IRQ-enable state + */ + bool hardirqs_enabled; + bool softirqs_enabled; + + /* + * whether the current is on do_exit() + */ + bool task_exit; + + /* + * whether the current is running __schedule() + */ + bool in_sched; +}; + +#define DEPT_TASK_INITIALIZER(t) \ +{ \ + .wait_hist = { { .wait = NULL, } }, \ + .ecxt_held_pos = 0, \ + .wait_hist_pos = 0, \ + .irq_id = { 0U }, \ + .wgen_enirq = { 0U }, \ + .enirq_ip = { 0UL }, \ + .eff_enirqf = 0UL, \ + .stack = NULL, \ + .recursive = 0, \ + .stage_m = DEPT_MAP_INITIALIZER((t)->stage_m, NULL), \ + .stage_sched_map = false, \ + .stage_w_fn = NULL, \ + .stage_ip = 0UL, \ + .missing_ecxt = 0, \ + .hardirqs_enabled = false, \ + .softirqs_enabled = false, \ + .task_exit = false, \ + .in_sched = false, \ +} + +extern void dept_on(void); +extern void dept_off(void); +extern void dept_init(void); +extern void dept_task_init(struct task_struct *t); +extern void dept_task_exit(struct task_struct *t); +extern void dept_free_range(void *start, unsigned int sz); +extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_copy(struct dept_map *to, struct dept_map *from); + +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_request_event_wait_commit(void); +extern void dept_clean_stage(void); +extern void dept_stage_event(struct task_struct *t, unsigned long ip); +extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); +extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); +extern void dept_request_event(struct dept_map *m); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); +extern void dept_sched_enter(void); +extern void dept_sched_exit(void); + +static inline void dept_ecxt_enter_nokeep(struct dept_map *m) +{ + dept_ecxt_enter(m, 0UL, 0UL, NULL, NULL, 0); +} + +/* + * for users who want to manage external keys + */ +extern void dept_key_init(struct dept_key *k); +extern void dept_key_destroy(struct dept_key *k); +extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, struct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const char *new_c_fn, const char *new_e_fn, int new_sub_l); + +extern void dept_softirq_enter(void); +extern void dept_hardirq_enter(void); +extern void dept_softirqs_on_ip(unsigned long ip); +extern void dept_hardirqs_on(void); +extern void dept_hardirqs_on_ip(unsigned long ip); +extern void dept_softirqs_off_ip(unsigned long ip); +extern void dept_hardirqs_off(void); +extern void dept_hardirqs_off_ip(unsigned long ip); +#else /* !CONFIG_DEPT */ +struct dept_key { }; +struct dept_map { }; +struct dept_task { }; + +#define DEPT_MAP_INITIALIZER(n, k) { } +#define DEPT_TASK_INITIALIZER(t) { } + +#define dept_on() do { } while (0) +#define dept_off() do { } while (0) +#define dept_init() do { } while (0) +#define dept_task_init(t) do { } while (0) +#define dept_task_exit(t) do { } while (0) +#define dept_free_range(s, sz) do { } while (0) +#define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_copy(t, f) do { } while (0) + +#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_request_event_wait_commit() do { } while (0) +#define dept_clean_stage() do { } while (0) +#define dept_stage_event(t, ip) do { } while (0) +#define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) +#define dept_ecxt_holding(m, e_f) false +#define dept_request_event(m) do { } while (0) +#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_ecxt_exit(m, e_f, ip) do { } while (0) +#define dept_sched_enter() do { } while (0) +#define dept_sched_exit() do { } while (0) +#define dept_ecxt_enter_nokeep(m) do { } while (0) +#define dept_key_init(k) do { (void)(k); } while (0) +#define dept_key_destroy(k) do { (void)(k); } while (0) +#define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_sl) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) + +#define dept_softirq_enter() do { } while (0) +#define dept_hardirq_enter() do { } while (0) +#define dept_softirqs_on_ip(ip) do { } while (0) +#define dept_hardirqs_on() do { } while (0) +#define dept_hardirqs_on_ip(ip) do { } while (0) +#define dept_softirqs_off_ip(ip) do { } while (0) +#define dept_hardirqs_off() do { } while (0) +#define dept_hardirqs_off_ip(ip) do { } while (0) +#endif +#endif /* __LINUX_DEPT_H */ diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index d57cab4..bb279db 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ static __always_inline void rcu_irq_enter_check_tick(void) */ #define __nmi_enter() \ do { \ + dept_off(); \ lockdep_off(); \ arch_nmi_enter(); \ BUG_ON(in_nmi() == NMI_MASK); \ @@ -128,6 +130,7 @@ static __always_inline void rcu_irq_enter_check_tick(void) __preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \ arch_nmi_exit(); \ lockdep_on(); \ + dept_on(); \ } while (0) #define nmi_exit() \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 853d08f..fcb0099 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -37,6 +37,7 @@ #include #include #include +#include /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -1168,6 +1169,8 @@ struct task_struct { struct held_lock held_locks[MAX_LOCK_DEPTH]; #endif + struct dept_task dept_task; + #if defined(CONFIG_UBSAN) && !defined(CONFIG_UBSAN_TRAP) unsigned int in_ubsan; #endif diff --git a/init/init_task.c b/init/init_task.c index ff6c4b9..eb36ad6 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -12,6 +12,7 @@ #include #include #include +#include #include @@ -194,6 +195,7 @@ struct task_struct init_task .curr_chain_key = INITIAL_CHAIN_KEY, .lockdep_recursion = 0, #endif + .dept_task = DEPT_TASK_INITIALIZER(init_task), #ifdef CONFIG_FUNCTION_GRAPH_TRACER .ret_stack = NULL, .tracing_graph_pause = ATOMIC_INIT(0), diff --git a/init/main.c b/init/main.c index e1c3911..6e5b492 100644 --- a/init/main.c +++ b/init/main.c @@ -66,6 +66,7 @@ #include #include #include +#include #include #include #include @@ -1080,6 +1081,7 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void) panic_param); lockdep_init(); + dept_init(); /* * Need to run this when irqs are enabled, because it wants diff --git a/kernel/Makefile b/kernel/Makefile index 10ef068..d1eb49e 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -51,6 +51,7 @@ obj-y += livepatch/ obj-y += dma/ obj-y += entry/ obj-$(CONFIG_MODULES) += module/ +obj-y += dependency/ obj-$(CONFIG_KCMP) += kcmp.o obj-$(CONFIG_FREEZER) += freezer.o diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile new file mode 100644 index 00000000..b5cfb8a --- /dev/null +++ b/kernel/dependency/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DEPT) += dept.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c new file mode 100644 index 00000000..03d82e9 --- /dev/null +++ b/kernel/dependency/dept.c @@ -0,0 +1,3009 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DEPT(DEPendency Tracker) - Runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * + * DEPT provides a general way to detect deadlock possibility in runtime + * and the interest is not limited to typical lock but to every + * syncronization primitives. + * + * The following ideas were borrowed from LOCKDEP: + * + * 1) Use a graph to track relationship between classes. + * 2) Prevent performance regression using hash. + * + * The following items were enhanced from LOCKDEP: + * + * 1) Cover more deadlock cases. + * 2) Allow muliple reports. + * + * TODO: Both LOCKDEP and DEPT should co-exist until DEPT is considered + * stable. Then the dependency check routine should be replaced with + * DEPT after. It should finally look like: + * + * + * + * As is: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | +-------------------------------------+ | + * | | Dependency check | | + * | | (by tracking lock acquisition order)| | + * | +-------------------------------------+ | + * | | + * +-----------------------------------------+ + * + * DEPT + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + * + * + * + * To be: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | (Request dependency check) | + * | T | + * +--------------------|--------------------+ + * | + * DEPT V + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static int dept_stop; +static int dept_per_cpu_ready; + +#define DEPT_READY_WARN (!oops_in_progress) + +/* + * Make all operations using DEPT_WARN_ON() fail on oops_in_progress and + * prevent warning message. + */ +#define DEPT_WARN_ON_ONCE(c) \ + ({ \ + int __ret = 0; \ + \ + if (likely(DEPT_READY_WARN)) \ + __ret = WARN_ONCE(c, "DEPT_WARN_ON_ONCE: " #c); \ + __ret; \ + }) + +#define DEPT_WARN_ONCE(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) \ + WARN_ONCE(1, "DEPT_WARN_ONCE: " s); \ + }) + +#define DEPT_WARN_ON(c) \ + ({ \ + int __ret = 0; \ + \ + if (likely(DEPT_READY_WARN)) \ + __ret = WARN(c, "DEPT_WARN_ON: " #c); \ + __ret; \ + }) + +#define DEPT_WARN(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) \ + WARN(1, "DEPT_WARN: " s); \ + }) + +#define DEPT_STOP(s...) \ + ({ \ + WRITE_ONCE(dept_stop, 1); \ + if (likely(DEPT_READY_WARN)) \ + WARN(1, "DEPT_STOP: " s); \ + }) + +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) + +static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t stage_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; + +/* + * DEPT internal engine should be careful in using outside functions + * e.g. printk at reporting since that kind of usage might cause + * untrackable deadlock. + */ +static atomic_t dept_outworld = ATOMIC_INIT(0); + +static inline void dept_outworld_enter(void) +{ + atomic_inc(&dept_outworld); +} + +static inline void dept_outworld_exit(void) +{ + atomic_dec(&dept_outworld); +} + +static inline bool dept_outworld_entered(void) +{ + return atomic_read(&dept_outworld); +} + +static inline bool dept_lock(void) +{ + while (!arch_spin_trylock(&dept_spin)) + if (unlikely(dept_outworld_entered())) + return false; + return true; +} + +static inline void dept_unlock(void) +{ + arch_spin_unlock(&dept_spin); +} + +/* + * whether to stack-trace on every wait or every ecxt + */ +static bool rich_stack = true; + +enum bfs_ret { + BFS_CONTINUE, + BFS_CONTINUE_REV, + BFS_DONE, + BFS_SKIP, +}; + +static inline bool after(unsigned int a, unsigned int b) +{ + return (int)(b - a) < 0; +} + +static inline bool before(unsigned int a, unsigned int b) +{ + return (int)(a - b) < 0; +} + +static inline bool valid_stack(struct dept_stack *s) +{ + return s && s->nr > 0; +} + +static inline bool valid_class(struct dept_class *c) +{ + return c->key; +} + +static inline void invalidate_class(struct dept_class *c) +{ + c->key = 0UL; +} + +static inline struct dept_ecxt *dep_e(struct dept_dep *d) +{ + return d->ecxt; +} + +static inline struct dept_wait *dep_w(struct dept_dep *d) +{ + return d->wait; +} + +static inline struct dept_class *dep_fc(struct dept_dep *d) +{ + return dep_e(d)->class; +} + +static inline struct dept_class *dep_tc(struct dept_dep *d) +{ + return dep_w(d)->class; +} + +static inline const char *irq_str(int irq) +{ + if (irq == DEPT_SIRQ) + return "softirq"; + if (irq == DEPT_HIRQ) + return "hardirq"; + return "(unknown)"; +} + +static inline struct dept_task *dept_task(void) +{ + return ¤t->dept_task; +} + +/* + * Dept doesn't work either when it's stopped by DEPT_STOP() or in a nmi + * context. + */ +static inline bool dept_working(void) +{ + return !READ_ONCE(dept_stop) && !in_nmi(); +} + +/* + * Even k == NULL is considered as a valid key because it would use + * &->map_key as the key in that case. + */ +struct dept_key __dept_no_validate__; +static inline bool valid_key(struct dept_key *k) +{ + return &__dept_no_validate__ != k; +} + +/* + * Pool + * ===================================================================== + * DEPT maintains pools to provide objects in a safe way. + * + * 1) Static pool is used at the beginning of booting time. + * 2) Local pool is tried first before the static pool. Objects that + * have been freed will be placed. + */ + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +#define OBJECT(id, nr) \ +static struct dept_##id spool_##id[nr]; \ +static DEFINE_PER_CPU(struct llist_head, lpool_##id); + #include "dept_object.h" +#undef OBJECT + +static struct dept_pool pool[OBJECT_NR] = { +#define OBJECT(id, nr) { \ + .name = #id, \ + .obj_sz = sizeof(struct dept_##id), \ + .obj_nr = ATOMIC_INIT(nr), \ + .node_off = offsetof(struct dept_##id, pool_node), \ + .spool = spool_##id, \ + .lpool = &lpool_##id, }, + #include "dept_object.h" +#undef OBJECT +}; + +/* + * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is + * enabled or not because NMI and other contexts in the same CPU never + * run inside of DEPT concurrently by preventing reentrance. + */ +static void *from_pool(enum object_t t) +{ + struct dept_pool *p; + struct llist_head *h; + struct llist_node *n; + + /* + * llist_del_first() doesn't allow concurrent access e.g. + * between process and IRQ context. + */ + if (DEPT_WARN_ON(!irqs_disabled())) + return NULL; + + p = &pool[t]; + + /* + * Try local pool first. + */ + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + n = llist_del_first(h); + if (n) + return (void *)n - p->node_off; + + /* + * Try static pool. + */ + if (atomic_read(&p->obj_nr) > 0) { + int idx = atomic_dec_return(&p->obj_nr); + + if (idx >= 0) + return p->spool + (idx * p->obj_sz); + } + + DEPT_INFO_ONCE("---------------------------------------------\n" + " Some of Dept internal resources are run out.\n" + " Dept might still work if the resources get freed.\n" + " However, the chances are Dept will suffer from\n" + " the lack from now. Needs to extend the internal\n" + " resource pools. Ask max.byungchul.park@gmail.com\n"); + return NULL; +} + +static void to_pool(void *o, enum object_t t) +{ + struct dept_pool *p = &pool[t]; + struct llist_head *h; + + preempt_disable(); + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + llist_add(o + p->node_off, h); + preempt_enable(); +} + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); \ +static void (*dtor_##id)(struct dept_##id *a); \ +static inline struct dept_##id *new_##id(void) \ +{ \ + struct dept_##id *a; \ + \ + a = (struct dept_##id *)from_pool(OBJECT_##id); \ + if (unlikely(!a)) \ + return NULL; \ + \ + atomic_set(&a->ref, 1); \ + \ + if (ctor_##id) \ + ctor_##id(a); \ + \ + return a; \ +} \ + \ +static inline struct dept_##id *get_##id(struct dept_##id *a) \ +{ \ + atomic_inc(&a->ref); \ + return a; \ +} \ + \ +static inline void put_##id(struct dept_##id *a) \ +{ \ + if (!atomic_dec_return(&a->ref)) { \ + if (dtor_##id) \ + dtor_##id(a); \ + to_pool(a, OBJECT_##id); \ + } \ +} \ + \ +static inline void del_##id(struct dept_##id *a) \ +{ \ + put_##id(a); \ +} \ + \ +static inline bool id##_consumed(struct dept_##id *a) \ +{ \ + return a && atomic_read(&a->ref) > 1; \ +} +#include "dept_object.h" +#undef OBJECT + +#define SET_CONSTRUCTOR(id, f) \ +static void (*ctor_##id)(struct dept_##id *a) = f + +static void initialize_dep(struct dept_dep *d) +{ + INIT_LIST_HEAD(&d->bfs_node); + INIT_LIST_HEAD(&d->dep_node); + INIT_LIST_HEAD(&d->dep_rev_node); +} +SET_CONSTRUCTOR(dep, initialize_dep); + +static void initialize_class(struct dept_class *c) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iecxt *ie = &c->iecxt[i]; + struct dept_iwait *iw = &c->iwait[i]; + + ie->ecxt = NULL; + ie->enirq = i; + ie->staled = false; + + iw->wait = NULL; + iw->irq = i; + iw->staled = false; + iw->touched = false; + } + c->bfs_gen = 0U; + + INIT_LIST_HEAD(&c->all_node); + INIT_LIST_HEAD(&c->dep_head); + INIT_LIST_HEAD(&c->dep_rev_head); +} +SET_CONSTRUCTOR(class, initialize_class); + +static void initialize_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + e->enirq_stack[i] = NULL; + e->enirq_ip[i] = 0UL; + } + e->ecxt_ip = 0UL; + e->ecxt_stack = NULL; + e->enirqf = 0UL; + e->event_ip = 0UL; + e->event_stack = NULL; +} +SET_CONSTRUCTOR(ecxt, initialize_ecxt); + +static void initialize_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + w->irq_stack[i] = NULL; + w->irq_ip[i] = 0UL; + } + w->wait_ip = 0UL; + w->wait_stack = NULL; + w->irqf = 0UL; +} +SET_CONSTRUCTOR(wait, initialize_wait); + +static void initialize_stack(struct dept_stack *s) +{ + s->nr = 0; +} +SET_CONSTRUCTOR(stack, initialize_stack); + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_CONSTRUCTOR + +#define SET_DESTRUCTOR(id, f) \ +static void (*dtor_##id)(struct dept_##id *a) = f + +static void destroy_dep(struct dept_dep *d) +{ + if (dep_e(d)) + put_ecxt(dep_e(d)); + if (dep_w(d)) + put_wait(dep_w(d)); +} +SET_DESTRUCTOR(dep, destroy_dep); + +static void destroy_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (e->enirq_stack[i]) + put_stack(e->enirq_stack[i]); + if (e->class) + put_class(e->class); + if (e->ecxt_stack) + put_stack(e->ecxt_stack); + if (e->event_stack) + put_stack(e->event_stack); +} +SET_DESTRUCTOR(ecxt, destroy_ecxt); + +static void destroy_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (w->irq_stack[i]) + put_stack(w->irq_stack[i]); + if (w->class) + put_class(w->class); + if (w->wait_stack) + put_stack(w->wait_stack); +} +SET_DESTRUCTOR(wait, destroy_wait); + +#define OBJECT(id, nr) \ +static void (*dtor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_DESTRUCTOR + +/* + * Caching and hashing + * ===================================================================== + * DEPT makes use of caching and hashing to improve performance. Each + * object can be obtained in O(1) with its key. + * + * NOTE: Currently we assume all the objects in the hashs will never be + * removed. Implement it when needed. + */ + +/* + * Some information might be lost but it's only for hashing key. + */ +static inline unsigned long mix(unsigned long a, unsigned long b) +{ + int halfbits = sizeof(unsigned long) * 8 / 2; + unsigned long halfmask = (1UL << halfbits) - 1UL; + + return (a << halfbits) | (b & halfmask); +} + +static bool cmp_dep(struct dept_dep *d1, struct dept_dep *d2) +{ + return dep_fc(d1)->key == dep_fc(d2)->key && + dep_tc(d1)->key == dep_tc(d2)->key; +} + +static unsigned long key_dep(struct dept_dep *d) +{ + return mix(dep_fc(d)->key, dep_tc(d)->key); +} + +static bool cmp_class(struct dept_class *c1, struct dept_class *c2) +{ + return c1->key == c2->key; +} + +static unsigned long key_class(struct dept_class *c) +{ + return c->key; +} + +#define HASH(id, bits) \ +static struct hlist_head table_##id[1 << (bits)]; \ + \ +static inline struct hlist_head *head_##id(struct dept_##id *a) \ +{ \ + return table_##id + hash_long(key_##id(a), bits); \ +} \ + \ +static inline struct dept_##id *hash_lookup_##id(struct dept_##id *a) \ +{ \ + struct dept_##id *b; \ + \ + hlist_for_each_entry_rcu(b, head_##id(a), hash_node) \ + if (cmp_##id(a, b)) \ + return b; \ + return NULL; \ +} \ + \ +static inline void hash_add_##id(struct dept_##id *a) \ +{ \ + get_##id(a); \ + hlist_add_head_rcu(&a->hash_node, head_##id(a)); \ +} \ + \ +static inline void hash_del_##id(struct dept_##id *a) \ +{ \ + hlist_del_rcu(&a->hash_node); \ + put_##id(a); \ +} +#include "dept_hash.h" +#undef HASH + +static inline struct dept_dep *lookup_dep(struct dept_class *fc, + struct dept_class *tc) +{ + struct dept_ecxt onetime_e = { .class = fc }; + struct dept_wait onetime_w = { .class = tc }; + struct dept_dep onetime_d = { .ecxt = &onetime_e, + .wait = &onetime_w }; + return hash_lookup_dep(&onetime_d); +} + +static inline struct dept_class *lookup_class(unsigned long key) +{ + struct dept_class onetime_c = { .key = key }; + + return hash_lookup_class(&onetime_c); +} + +/* + * Report + * ===================================================================== + * DEPT prints useful information to help debuging on detection of + * problematic dependency. + */ + +static inline void print_ip_stack(unsigned long ip, struct dept_stack *s) +{ + if (ip) + print_ip_sym(KERN_WARNING, ip); + + if (valid_stack(s)) { + pr_warn("stacktrace:\n"); + stack_trace_print(s->raw, s->nr, 5); + } + + if (!ip && !valid_stack(s)) + pr_warn("(N/A)\n"); +} + +#define print_spc(spc, fmt, ...) \ + pr_warn("%*c" fmt, (spc) * 4, ' ', ##__VA_ARGS__) + +static void print_diagram(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + bool firstline = true; + int spc = 1; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + if (!firstline) + pr_warn("\nor\n\n"); + firstline = false; + + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, " <%s interrupt>\n", irq_str(irq)); + print_spc(spc + 1, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } + + if (!irqf) { + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } +} + +static void print_dep(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + pr_warn("%s has been enabled:\n", irq_str(irq)); + print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); + pr_warn("\n"); + + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d) in %s context:\n", + w_fn, tc_n, tc->sub_id, irq_str(irq)); + print_ip_stack(w->irq_ip[irq], w->irq_stack[irq]); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } + + if (!irqf) { + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d):\n", w_fn, tc_n, tc->sub_id); + print_ip_stack(w->wait_ip, w->wait_stack); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } +} + +static void save_current_stack(int skip); + +/* + * Print all classes in a circle. + */ +static void print_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + int i; + + dept_outworld_enter(); + save_current_stack(6); + + pr_warn("===================================================\n"); + pr_warn("DEPT: Circular dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + pr_warn("summary\n"); + pr_warn("---------------------------------------------------\n"); + + if (fc == tc) + pr_warn("*** AA DEADLOCK ***\n\n"); + else + pr_warn("*** DEADLOCK ***\n\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + if (fc != c) + pr_warn("\n"); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("\n"); + pr_warn("[S]: start of the event context\n"); + pr_warn("[W]: the wait blocked\n"); + pr_warn("[E]: the event not reachable\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c's detail\n", 'A' + i); + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + pr_warn("\n"); + print_dep(d); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +/* + * BFS(Breadth First Search) + * ===================================================================== + * Whenever a new dependency is added into the graph, search the graph + * for a new circular dependency. + */ + +static inline void enqueue(struct list_head *h, struct dept_dep *d) +{ + list_add_tail(&d->bfs_node, h); +} + +static inline struct dept_dep *dequeue(struct list_head *h) +{ + struct dept_dep *d; + + d = list_first_entry(h, struct dept_dep, bfs_node); + list_del(&d->bfs_node); + return d; +} + +static inline bool empty(struct list_head *h) +{ + return list_empty(h); +} + +static void extend_queue(struct list_head *h, struct dept_class *cur) +{ + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_head, dep_node) { + struct dept_class *next = dep_tc(d); + + if (cur->bfs_gen == next->bfs_gen) + continue; + next->bfs_gen = cur->bfs_gen; + next->bfs_dist = cur->bfs_dist + 1; + next->bfs_parent = cur; + enqueue(h, d); + } +} + +static void extend_queue_rev(struct list_head *h, struct dept_class *cur) +{ + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_rev_head, dep_rev_node) { + struct dept_class *next = dep_fc(d); + + if (cur->bfs_gen == next->bfs_gen) + continue; + next->bfs_gen = cur->bfs_gen; + next->bfs_dist = cur->bfs_dist + 1; + next->bfs_parent = cur; + enqueue(h, d); + } +} + +typedef enum bfs_ret bfs_f(struct dept_dep *d, void *in, void **out); +static unsigned int bfs_gen; + +/* + * NOTE: Must be called with dept_lock held. + */ +static void bfs(struct dept_class *c, bfs_f *cb, void *in, void **out) +{ + LIST_HEAD(q); + enum bfs_ret ret; + + if (DEPT_WARN_ON(!cb)) + return; + + /* + * Avoid zero bfs_gen. + */ + bfs_gen = bfs_gen + 1 ?: 1; + + c->bfs_gen = bfs_gen; + c->bfs_dist = 0; + c->bfs_parent = c; + + ret = cb(NULL, in, out); + if (ret == BFS_DONE) + return; + if (ret == BFS_SKIP) + return; + if (ret == BFS_CONTINUE) + extend_queue(&q, c); + if (ret == BFS_CONTINUE_REV) + extend_queue_rev(&q, c); + + while (!empty(&q)) { + struct dept_dep *d = dequeue(&q); + + ret = cb(d, in, out); + if (ret == BFS_DONE) + break; + if (ret == BFS_SKIP) + continue; + if (ret == BFS_CONTINUE) + extend_queue(&q, dep_tc(d)); + if (ret == BFS_CONTINUE_REV) + extend_queue_rev(&q, dep_fc(d)); + } + + while (!empty(&q)) + dequeue(&q); +} + +/* + * Main operations + * ===================================================================== + * Add dependencies - Each new dependency is added into the graph and + * checked if it forms a circular dependency. + * + * Track waits - Waits are queued into the ring buffer for later use to + * generate appropriate dependencies with cross-event. + * + * Track event contexts(ecxt) - Event contexts are pushed into local + * stack for later use to generate appropriate dependencies with waits. + */ + +static inline unsigned long cur_enirqf(void); +static inline int cur_irq(void); +static inline unsigned int cur_ctxt_id(void); + +static inline struct dept_iecxt *iecxt(struct dept_class *c, int irq) +{ + return &c->iecxt[irq]; +} + +static inline struct dept_iwait *iwait(struct dept_class *c, int irq) +{ + return &c->iwait[irq]; +} + +static inline void stale_iecxt(struct dept_iecxt *ie) +{ + if (ie->ecxt) + put_ecxt(ie->ecxt); + + WRITE_ONCE(ie->ecxt, NULL); + WRITE_ONCE(ie->staled, true); +} + +static inline void set_iecxt(struct dept_iecxt *ie, struct dept_ecxt *e) +{ + /* + * ->ecxt will never be updated once getting set until the class + * gets removed. + */ + if (ie->ecxt) + DEPT_WARN_ON(1); + else + WRITE_ONCE(ie->ecxt, get_ecxt(e)); +} + +static inline void stale_iwait(struct dept_iwait *iw) +{ + if (iw->wait) + put_wait(iw->wait); + + WRITE_ONCE(iw->wait, NULL); + WRITE_ONCE(iw->staled, true); +} + +static inline void set_iwait(struct dept_iwait *iw, struct dept_wait *w) +{ + /* + * ->wait will never be updated once getting set until the class + * gets removed. + */ + if (iw->wait) + DEPT_WARN_ON(1); + else + WRITE_ONCE(iw->wait, get_wait(w)); + + iw->touched = true; +} + +static inline void touch_iwait(struct dept_iwait *iw) +{ + iw->touched = true; +} + +static inline void untouch_iwait(struct dept_iwait *iw) +{ + iw->touched = false; +} + +static inline struct dept_stack *get_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + return s ? get_stack(s) : NULL; +} + +static inline void prepare_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + /* + * The dept_stack is already ready. + */ + if (s && !stack_consumed(s)) { + s->nr = 0; + return; + } + + if (s) + put_stack(s); + + s = dept_task()->stack = new_stack(); + if (!s) + return; + + get_stack(s); + del_stack(s); +} + +static void save_current_stack(int skip) +{ + struct dept_stack *s = dept_task()->stack; + + if (!s) + return; + if (valid_stack(s)) + return; + + s->nr = stack_trace_save(s->raw, DEPT_MAX_STACK_ENTRY, skip); +} + +static void finish_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + if (stack_consumed(s)) + save_current_stack(2); +} + +/* + * FIXME: For now, disable LOCKDEP while DEPT is working. + * + * Both LOCKDEP and DEPT report it on a deadlock detection using + * printk taking the risk of another deadlock that might be caused by + * locks of console or printk between inside and outside of them. + * + * For DEPT, it's no problem since multiple reports are allowed. But it + * would be a bad idea for LOCKDEP since it will stop even on a singe + * report. So we need to prevent LOCKDEP from its reporting the risk + * DEPT would take when reporting something. + */ +#include + +void dept_off(void) +{ + dept_task()->recursive++; + lockdep_off(); +} + +void dept_on(void) +{ + dept_task()->recursive--; + lockdep_on(); +} + +static inline unsigned long dept_enter(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + dept_off(); + prepare_current_stack(); + return flags; +} + +static inline void dept_exit(unsigned long flags) +{ + finish_current_stack(); + dept_on(); + arch_local_irq_restore(flags); +} + +static inline unsigned long dept_enter_recursive(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + return flags; +} + +static inline void dept_exit_recursive(unsigned long flags) +{ + arch_local_irq_restore(flags); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static struct dept_dep *__add_dep(struct dept_ecxt *e, + struct dept_wait *w) +{ + struct dept_dep *d; + + if (DEPT_WARN_ON(!valid_class(e->class))) + return NULL; + + if (DEPT_WARN_ON(!valid_class(w->class))) + return NULL; + + if (lookup_dep(e->class, w->class)) + return NULL; + + d = new_dep(); + if (unlikely(!d)) + return NULL; + + d->ecxt = get_ecxt(e); + d->wait = get_wait(w); + + /* + * Add the dependency into hash and graph. + */ + hash_add_dep(d); + list_add(&d->dep_node, &dep_fc(d)->dep_head); + list_add(&d->dep_rev_node, &dep_tc(d)->dep_rev_head); + return d; +} + +static enum bfs_ret cb_check_dl(struct dept_dep *d, + void *in, void **out) +{ + struct dept_dep *new = (struct dept_dep *)in; + + /* + * initial condition for this BFS search + */ + if (!d) { + dep_tc(new)->bfs_parent = dep_fc(new); + + if (dep_tc(new) != dep_fc(new)) + return BFS_CONTINUE; + + /* + * AA circle does not make additional deadlock. We don't + * have to continue this BFS search. + */ + print_circle(dep_tc(new)); + return BFS_DONE; + } + + /* + * Allow multiple reports. + */ + if (dep_tc(d) == dep_fc(new)) + print_circle(dep_tc(new)); + + return BFS_CONTINUE; +} + +/* + * This function is actually in charge of reporting. + */ +static inline void check_dl_bfs(struct dept_dep *d) +{ + bfs(dep_tc(d), cb_check_dl, (void *)d, NULL); +} + +static enum bfs_ret cb_find_iw(struct dept_dep *d, void *in, void **out) +{ + int irq = *(int *)in; + struct dept_class *fc; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + /* + * initial condition for this BFS search + */ + if (!d) + return BFS_CONTINUE_REV; + + fc = dep_fc(d); + iw = iwait(fc, irq); + + /* + * If any parent's ->wait was set, then the children would've + * been touched. + */ + if (!iw->touched) + return BFS_SKIP; + + if (!iw->wait) + return BFS_CONTINUE_REV; + + *out = iw; + return BFS_DONE; +} + +static struct dept_iwait *find_iw_bfs(struct dept_class *c, int irq) +{ + struct dept_iwait *iw = iwait(c, irq); + struct dept_iwait *found = NULL; + + if (iw->wait) + return iw; + + /* + * '->touched == false' guarantees there's no parent that has + * been set ->wait. + */ + if (!iw->touched) + return NULL; + + bfs(c, cb_find_iw, (void *)&irq, (void **)&found); + + if (found) + return found; + + untouch_iwait(iw); + return NULL; +} + +static enum bfs_ret cb_touch_iw_find_ie(struct dept_dep *d, void *in, + void **out) +{ + int irq = *(int *)in; + struct dept_class *tc; + struct dept_iecxt *ie; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + /* + * initial condition for this BFS search + */ + if (!d) + return BFS_CONTINUE; + + tc = dep_tc(d); + ie = iecxt(tc, irq); + iw = iwait(tc, irq); + + touch_iwait(iw); + + if (!ie->ecxt) + return BFS_CONTINUE; + + if (!*out) + *out = ie; + + return BFS_CONTINUE; +} + +static struct dept_iecxt *touch_iw_find_ie_bfs(struct dept_class *c, + int irq) +{ + struct dept_iecxt *ie = iecxt(c, irq); + struct dept_iwait *iw = iwait(c, irq); + struct dept_iecxt *found = ie->ecxt ? ie : NULL; + + touch_iwait(iw); + bfs(c, cb_touch_iw_find_ie, (void *)&irq, (void **)&found); + return found; +} + +/* + * Should be called with dept_lock held. + */ +static void __add_idep(struct dept_iecxt *ie, struct dept_iwait *iw) +{ + struct dept_dep *new; + + /* + * There's nothing to do. + */ + if (!ie || !iw || !ie->ecxt || !iw->wait) + return; + + new = __add_dep(ie->ecxt, iw->wait); + + /* + * Deadlock detected. Let check_dl_bfs() report it. + */ + if (new) { + check_dl_bfs(new); + stale_iecxt(ie); + stale_iwait(iw); + } + + /* + * If !new, it would be the case of lack of object resource. + * Just let it go and get checked by other chances. Retrying is + * meaningless in that case. + */ +} + +static void set_check_iecxt(struct dept_class *c, int irq, + struct dept_ecxt *e) +{ + struct dept_iecxt *ie = iecxt(c, irq); + + set_iecxt(ie, e); + __add_idep(ie, find_iw_bfs(c, irq)); +} + +static void set_check_iwait(struct dept_class *c, int irq, + struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + set_iwait(iw, w); + __add_idep(touch_iw_find_ie_bfs(c, irq), iw); +} + +static void add_iecxt(struct dept_class *c, int irq, struct dept_ecxt *e, + bool stack) +{ + /* + * This access is safe since we ensure e->class has set locally. + */ + struct dept_task *dt = dept_task(); + struct dept_iecxt *ie = iecxt(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(ie->staled))) + return; + + /* + * Skip add_iecxt() if ie->ecxt has ever been set at least once. + * Which means it has a valid ->ecxt or been staled. + */ + if (READ_ONCE(ie->ecxt)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(ie->staled)) + goto unlock; + if (ie->ecxt) + goto unlock; + + e->enirqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * enirq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(e->enirq_ip[irq]); + DEPT_WARN_ON(e->enirq_stack[irq]); + + e->enirq_ip[irq] = dt->enirq_ip[irq]; + e->enirq_stack[irq] = stack ? get_current_stack() : NULL; + + set_check_iecxt(c, irq, e); +unlock: + dept_unlock(); +} + +static void add_iwait(struct dept_class *c, int irq, struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(iw->staled))) + return; + + /* + * Skip add_iwait() if iw->wait has ever been set at least once. + * Which means it has a valid ->wait or been staled. + */ + if (READ_ONCE(iw->wait)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(iw->staled)) + goto unlock; + if (iw->wait) + goto unlock; + + w->irqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * irq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(w->irq_ip[irq]); + DEPT_WARN_ON(w->irq_stack[irq]); + + w->irq_ip[irq] = w->wait_ip; + w->irq_stack[irq] = get_current_stack(); + + set_check_iwait(c, irq, w); +unlock: + dept_unlock(); +} + +static inline struct dept_wait_hist *hist(int pos) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist + (pos % DEPT_MAX_WAIT_HIST); +} + +static inline int hist_pos_next(void) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist_pos % DEPT_MAX_WAIT_HIST; +} + +static inline void hist_advance(void) +{ + struct dept_task *dt = dept_task(); + + dt->wait_hist_pos++; + dt->wait_hist_pos %= DEPT_MAX_WAIT_HIST; +} + +static inline struct dept_wait_hist *new_hist(void) +{ + struct dept_wait_hist *wh = hist(hist_pos_next()); + + hist_advance(); + return wh; +} + +static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) +{ + struct dept_wait_hist *wh = new_hist(); + + if (likely(wh->wait)) + put_wait(wh->wait); + + wh->wait = get_wait(w); + wh->wgen = wg; + wh->ctxt_id = ctxt_id; +} + +/* + * Should be called after setting up e's iecxt and w's iwait. + */ +static void add_dep(struct dept_ecxt *e, struct dept_wait *w) +{ + struct dept_class *fc = e->class; + struct dept_class *tc = w->class; + struct dept_dep *d; + int i; + + if (lookup_dep(fc, tc)) + return; + + if (unlikely(!dept_lock())) + return; + + /* + * __add_dep() will lookup_dep() again with lock held. + */ + d = __add_dep(e, w); + if (d) { + check_dl_bfs(d); + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iwait *fiw = iwait(fc, i); + struct dept_iecxt *found_ie; + struct dept_iwait *found_iw; + + /* + * '->touched == false' guarantees there's no + * parent that has been set ->wait. + */ + if (!fiw->touched) + continue; + + /* + * find_iw_bfs() will untouch the iwait if + * not found. + */ + found_iw = find_iw_bfs(fc, i); + + if (!found_iw) + continue; + + found_ie = touch_iw_find_ie_bfs(tc, i); + __add_idep(found_ie, found_iw); + } + } + dept_unlock(); +} + +static atomic_t wgen = ATOMIC_INIT(1); + +static void add_wait(struct dept_class *c, unsigned long ip, + const char *w_fn, int sub_l, bool sched_sleep) +{ + struct dept_task *dt = dept_task(); + struct dept_wait *w; + unsigned int wg = 0U; + int irq; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + w = new_wait(); + if (unlikely(!w)) + return; + + WRITE_ONCE(w->class, get_class(c)); + w->wait_ip = ip; + w->wait_fn = w_fn; + w->wait_stack = get_current_stack(); + w->sched_sleep = sched_sleep; + + irq = cur_irq(); + if (irq < DEPT_IRQS_NR) + add_iwait(c, irq, w); + + /* + * Avoid adding dependency between user aware nested ecxt and + * wait. + */ + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + + /* + * the case of invalid key'ed one + */ + if (!eh->ecxt) + continue; + + if (eh->ecxt->class != c || eh->sub_l == sub_l) + add_dep(eh->ecxt, w); + } + + if (!wait_consumed(w) && !rich_stack) { + if (w->wait_stack) + put_stack(w->wait_stack); + w->wait_stack = NULL; + } + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + add_hist(w, wg, cur_ctxt_id()); + + del_wait(w); +} + +static bool add_ecxt(struct dept_map *m, struct dept_class *c, + unsigned long ip, const char *c_fn, + const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + unsigned long irqf; + int irq; + + if (DEPT_WARN_ON(!valid_class(c))) + return false; + + if (DEPT_WARN_ON_ONCE(dt->ecxt_held_pos >= DEPT_MAX_ECXT_HELD)) + return false; + + if (m->nocheck) { + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = NULL; + eh->map = m; + eh->class = get_class(c); + eh->wgen = atomic_read(&wgen); + eh->sub_l = sub_l; + + return true; + } + + e = new_ecxt(); + if (unlikely(!e)) + return false; + + e->class = get_class(c); + e->ecxt_ip = ip; + e->ecxt_stack = ip && rich_stack ? get_current_stack() : NULL; + e->event_fn = e_fn; + e->ecxt_fn = c_fn; + + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = get_ecxt(e); + eh->map = m; + eh->class = get_class(c); + eh->wgen = atomic_read(&wgen); + eh->sub_l = sub_l; + + irqf = cur_enirqf(); + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + add_iecxt(c, irq, e, false); + + del_ecxt(e); + return true; +} + +static int find_ecxt_pos(struct dept_map *m, struct dept_class *c, + bool newfirst) +{ + struct dept_task *dt = dept_task(); + int i; + + if (newfirst) { + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } else { + for (i = 0; i < dt->ecxt_held_pos; i++) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } + return -1; +} + +static bool pop_ecxt(struct dept_map *m, struct dept_class *c) +{ + struct dept_task *dt = dept_task(); + int pos; + int i; + + pos = find_ecxt_pos(m, c, true); + if (pos == -1) + return false; + + if (dt->ecxt_held[pos].class) + put_class(dt->ecxt_held[pos].class); + + if (dt->ecxt_held[pos].ecxt) + put_ecxt(dt->ecxt_held[pos].ecxt); + + dt->ecxt_held_pos--; + + for (i = pos; i < dt->ecxt_held_pos; i++) + dt->ecxt_held[i] = dt->ecxt_held[i + 1]; + return true; +} + +static inline bool good_hist(struct dept_wait_hist *wh, unsigned int wg) +{ + return wh->wait != NULL && before(wg, wh->wgen); +} + +/* + * Binary-search the ring buffer for the earliest valid wait. + */ +static int find_hist_pos(unsigned int wg) +{ + int oldest; + int l; + int r; + int pos; + + oldest = hist_pos_next(); + if (unlikely(good_hist(hist(oldest), wg))) { + DEPT_INFO_ONCE("Need to expand the ring buffer.\n"); + return oldest; + } + + l = oldest + 1; + r = oldest + DEPT_MAX_WAIT_HIST - 1; + for (pos = (l + r) / 2; l <= r; pos = (l + r) / 2) { + struct dept_wait_hist *p = hist(pos - 1); + struct dept_wait_hist *wh = hist(pos); + + if (!good_hist(p, wg) && good_hist(wh, wg)) + return pos % DEPT_MAX_WAIT_HIST; + if (good_hist(wh, wg)) + r = pos - 1; + else + l = pos + 1; + } + return -1; +} + +static void do_event(struct dept_map *m, struct dept_class *c, + unsigned int wg, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + struct dept_wait_hist *wh; + struct dept_ecxt_held *eh; + unsigned int ctxt_id; + int end; + int pos; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (m->nocheck) + return; + + /* + * The event was triggered before wait. + */ + if (!wg) + return; + + pos = find_ecxt_pos(m, c, false); + if (pos == -1) + return; + + eh = dt->ecxt_held + pos; + + if (DEPT_WARN_ON(!eh->ecxt)) + return; + + eh->ecxt->event_ip = ip; + eh->ecxt->event_stack = get_current_stack(); + + /* + * The ecxt already has done what it needs. + */ + if (!before(wg, eh->wgen)) + return; + + pos = find_hist_pos(wg); + if (pos == -1) + return; + + ctxt_id = cur_ctxt_id(); + end = hist_pos_next(); + end = end > pos ? end : end + DEPT_MAX_WAIT_HIST; + for (wh = hist(pos); pos < end; wh = hist(++pos)) { + if (after(wh->wgen, eh->wgen)) + break; + + if (dt->in_sched && wh->wait->sched_sleep) + continue; + + if (wh->ctxt_id == ctxt_id) + add_dep(eh->ecxt, wh->wait); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_ecxt *e; + + if (before(dt->wgen_enirq[i], wg)) + continue; + + e = eh->ecxt; + add_iecxt(e->class, i, e, false); + } +} + +static void del_dep_rcu(struct rcu_head *rh) +{ + struct dept_dep *d = container_of(rh, struct dept_dep, rh); + + preempt_disable(); + del_dep(d); + preempt_enable(); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_class(struct dept_class *c) +{ + struct dept_dep *d, *n; + int i; + + list_for_each_entry_safe(d, n, &c->dep_head, dep_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + list_for_each_entry_safe(d, n, &c->dep_rev_head, dep_rev_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + stale_iecxt(iecxt(c, i)); + stale_iwait(iwait(c, i)); + } +} + +/* + * Context control + * ===================================================================== + * Whether a wait is in {hard,soft}-IRQ context or whether + * {hard,soft}-IRQ has been enabled on the way to an event is very + * important to check dependency. All those things should be tracked. + */ + +static inline unsigned long cur_enirqf(void) +{ + struct dept_task *dt = dept_task(); + int he = dt->hardirqs_enabled; + int se = dt->softirqs_enabled; + + if (he) + return DEPT_HIRQF | (se ? DEPT_SIRQF : 0UL); + return 0UL; +} + +static inline int cur_irq(void) +{ + if (lockdep_softirq_context(current)) + return DEPT_SIRQ; + if (lockdep_hardirq_context()) + return DEPT_HIRQ; + return DEPT_IRQS_NR; +} + +static inline unsigned int cur_ctxt_id(void) +{ + struct dept_task *dt = dept_task(); + int irq = cur_irq(); + + /* + * Normal process context + */ + if (irq == DEPT_IRQS_NR) + return 0U; + + return dt->irq_id[irq] | (1UL << irq); +} + +static void enirq_transition(int irq) +{ + struct dept_task *dt = dept_task(); + int i; + + /* + * READ wgen >= wgen of an event with IRQ enabled has been + * observed on the way to the event means, the IRQ can cut in + * within the ecxt. Used for cross-event detection. + * + * wait context event context(ecxt) + * ------------ ------------------- + * wait event + * WRITE wgen + * observe IRQ enabled + * READ wgen + * keep the wgen locally + * + * on the event + * check the local wgen + */ + dt->wgen_enirq[irq] = atomic_read(&wgen); + + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + + eh = dt->ecxt_held + i; + e = eh->ecxt; + if (e) + add_iecxt(e->class, irq, e, true); + } +} + +static void enirq_update(unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long irqf; + unsigned long prev; + int irq; + + prev = dt->eff_enirqf; + irqf = cur_enirqf(); + dt->eff_enirqf = irqf; + + /* + * Do enirq_transition() only on an OFF -> ON transition. + */ + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + if (prev & (1UL << irq)) + continue; + + dt->enirq_ip[irq] = ip; + enirq_transition(irq); + } +} + +/* + * Ensure it has been called on ON/OFF transition. + */ +static void dept_enirq_transition(unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * IRQ ON/OFF transition might happen while Dept is working. + * We cannot handle recursive entrance. Just ingnore it. + * Only transitions outside of Dept will be considered. + */ + if (dt->recursive) + return; + + flags = dept_enter(); + + enirq_update(ip); + + dept_exit(flags); +} + +void dept_softirqs_on_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = true; + dept_enirq_transition(ip); +} + +void dept_hardirqs_on(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = true; + dept_enirq_transition(_RET_IP_); +} +EXPORT_SYMBOL_GPL(dept_hardirqs_on); + +void dept_hardirqs_on_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = true; + dept_enirq_transition(ip); +} +EXPORT_SYMBOL_GPL(dept_hardirqs_on_ip); + +void dept_softirqs_off_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = false; + dept_enirq_transition(ip); +} + +void dept_hardirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = false; + dept_enirq_transition(_RET_IP_); +} +EXPORT_SYMBOL_GPL(dept_hardirqs_off); + +void dept_hardirqs_off_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = false; + dept_enirq_transition(ip); +} +EXPORT_SYMBOL_GPL(dept_hardirqs_off_ip); + +/* + * Ensure it's the outmost softirq context. + */ +void dept_softirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; +} + +/* + * Ensure it's the outmost hardirq context. + */ +void dept_hardirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; +} + +void dept_sched_enter(void) +{ + dept_task()->in_sched = true; +} + +void dept_sched_exit(void) +{ + dept_task()->in_sched = false; +} + +/* + * Exposed APIs + * ===================================================================== + */ + +static inline void clean_classes_cache(struct dept_key *k) +{ + int i; + + for (i = 0; i < DEPT_MAX_SUBCLASSES_CACHE; i++) { + if (!READ_ONCE(k->classes[i])) + continue; + + WRITE_ONCE(k->classes[i], NULL); + } +} + +void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u < 0)) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u >= DEPT_MAX_SUBCLASSES_USR)) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + clean_classes_cache(&m->map_key); + + m->keys = k; + m->sub_u = sub_u; + m->name = n; + m->wgen = 0U; + m->nocheck = !valid_key(k); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_init); + +void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + if (k) { + clean_classes_cache(&m->map_key); + m->keys = k; + m->nocheck = !valid_key(k); + } + + if (sub_u >= 0 && sub_u < DEPT_MAX_SUBCLASSES_USR) + m->sub_u = sub_u; + + if (n) + m->name = n; + + m->wgen = 0U; + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_reinit); + +void dept_map_copy(struct dept_map *to, struct dept_map *from) +{ + if (unlikely(!dept_working())) { + to->nocheck = true; + return; + } + + *to = *from; + + /* + * XXX: 'to' might be in a stack or something. Using the address + * in a stack segment as a key is meaningless. Just ignore the + * case for now. + */ + if (!to->keys) { + to->nocheck = true; + return; + } + + /* + * Since the class cache can be modified concurrently we could + * observe half pointers (64bit arch using 32bit copy insns). + * Therefore clear the caches and take the performance hit. + * + * XXX: Doesn't work well with lockdep_set_class_and_subclass() + * since that relies on cache abuse. + */ + clean_classes_cache(&to->map_key); +} + +static LIST_HEAD(classes); + +static inline bool within(const void *addr, void *start, unsigned long size) +{ + return addr >= start && addr < start + size; +} + +void dept_free_range(void *start, unsigned int sz) +{ + struct dept_task *dt = dept_task(); + struct dept_class *c, *n; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Failed to successfully free Dept objects.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_free_range() should not fail. + * + * FIXME: Should be fixed if dept_free_range() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + list_for_each_entry_safe(c, n, &classes, all_node) { + if (!within((void *)c->key, start, sz) && + !within(c->name, start, sz)) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} + +static inline int sub_id(struct dept_map *m, int e) +{ + return (m ? m->sub_u : 0) + e * DEPT_MAX_SUBCLASSES_USR; +} + +static struct dept_class *check_new_class(struct dept_key *local, + struct dept_key *k, int sub_id, + const char *n, bool sched_map) +{ + struct dept_class *c = NULL; + + if (DEPT_WARN_ON(sub_id >= DEPT_MAX_SUBCLASSES)) + return NULL; + + if (DEPT_WARN_ON(!k)) + return NULL; + + /* + * XXX: Assume that users prevent the map from using if any of + * the cached keys has been invalidated. If not, the cache, + * local->classes should not be used because it would be racy + * with class deletion. + */ + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + c = READ_ONCE(local->classes[sub_id]); + + if (c) + return c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (c) + goto caching; + + if (unlikely(!dept_lock())) + return NULL; + + c = lookup_class((unsigned long)k->base + sub_id); + if (unlikely(c)) + goto unlock; + + c = new_class(); + if (unlikely(!c)) + goto unlock; + + c->name = n; + c->sched_map = sched_map; + c->sub_id = sub_id; + c->key = (unsigned long)(k->base + sub_id); + hash_add_class(c); + list_add(&c->all_node, &classes); +unlock: + dept_unlock(); +caching: + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + WRITE_ONCE(local->classes[sub_id], c); + + return c; +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool sched_map) +{ + int e; + + /* + * Be as conservative as possible. In case of mulitple waits for + * a single dept_map, we are going to keep only the last wait's + * wgen for simplicity - keeping all wgens seems overengineering. + * + * Of course, it might cause missing some dependencies that + * would rarely, probabily never, happen but it helps avoid + * false positive report. + */ + for_each_set_bit(e, &w_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, sched_map); + if (!c) + continue; + + add_wait(c, ip, w_fn, sub_l, sched_sleep); + } +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn, + bool sched_map) +{ + struct dept_class *c; + struct dept_key *k; + int e; + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (DEPT_WARN_ON(e >= DEPT_MAX_SUBCLASSES_EVT)) + goto exit; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); + + if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { + do_event(m, c, READ_ONCE(m->wgen), ip); + pop_ecxt(m, c); + } +exit: + /* + * Keep the map diabled until the next sleep. + */ + WRITE_ONCE(m->wgen, 0U); +} + +void dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (m->nocheck) + return; + + flags = dept_enter(); + + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_wait); + +void dept_stage_wait(struct dept_map *m, struct dept_key *k, + unsigned long ip, const char *w_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m && m->nocheck) + return; + + /* + * Either m or k should be passed. Which means Dept relies on + * either its own map or the caller's position in the code when + * determining its class. + */ + if (DEPT_WARN_ON(!m && !k)) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + arch_spin_lock(&stage_spin); + + /* + * Ensure the outmost dept_stage_wait() works. + */ + if (dt->stage_m.keys) + goto unlock; + + if (m) { + dt->stage_m = *m; + + /* + * Ensure dt->stage_m.keys != NULL and it works with the + * map's map_key, not stage_m's one when ->keys == NULL. + */ + if (!m->keys) + dt->stage_m.keys = &m->map_key; + } else { + dt->stage_m.name = w_fn; + dt->stage_sched_map = true; + } + + /* + * dept_map_reinit() includes WRITE_ONCE(->wgen, 0U) that + * effectively disables the map just in case real sleep won't + * happen. dept_request_event_wait_commit() will enable it. + */ + dept_map_reinit(&dt->stage_m, k, -1, NULL); + + dt->stage_w_fn = w_fn; + dt->stage_ip = ip; +unlock: + arch_spin_unlock(&stage_spin); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_stage_wait); + +void dept_clean_stage(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + arch_spin_lock(&stage_spin); + memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); + dt->stage_sched_map = false; + dt->stage_w_fn = NULL; + dt->stage_ip = 0UL; + arch_spin_unlock(&stage_spin); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_clean_stage); + +/* + * Always called from __schedule(). + */ +void dept_request_event_wait_commit(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + unsigned int wg; + unsigned long ip; + const char *w_fn; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + /* + * It's impossible that __schedule() is called while Dept is + * working that already disabled IRQ at the entrance. + */ + if (DEPT_WARN_ON(dt->recursive)) + return; + + flags = dept_enter(); + + /* + * Checks if current has staged a wait. + */ + if (!dt->stage_m.keys) + goto exit; + + w_fn = dt->stage_w_fn; + ip = dt->stage_ip; + sched_map = dt->stage_sched_map; + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + WRITE_ONCE(dt->stage_m.wgen, wg); + + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); +exit: + dept_exit(flags); +} + +/* + * Always called from try_to_wake_up(). + */ +void dept_stage_event(struct task_struct *t, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + struct dept_map m; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + flags = dept_enter(); + + arch_spin_lock(&stage_spin); + m = dt->stage_m; + sched_map = dt->stage_sched_map; + arch_spin_unlock(&stage_spin); + + /* + * ->stage_m.keys should not be NULL if it's in use. Should + * make sure that it's not NULL when staging a valid map. + */ + if (!m.keys) + goto exit; + + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); +exit: + dept_exit(flags); +} + +/* + * Modifies the latest ecxt corresponding to m and e_f. + */ +void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, + struct dept_key *new_k, unsigned long new_e_f, + unsigned long new_ip, const char *new_c_fn, + const char *new_e_fn, int new_sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_class *c; + struct dept_key *k; + unsigned long flags; + int pos = -1; + int new_e; + int e; + + if (unlikely(!dept_working())) + return; + + /* + * XXX: Couldn't handle re-enterance cases. Ingore it for now. + */ + if (dt->recursive) + return; + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + pos = find_ecxt_pos(m, c, true); + if (pos != -1) + break; + } + + if (unlikely(pos == -1)) + goto exit; + + eh = dt->ecxt_held + pos; + new_sub_l = new_sub_l >= 0 ? new_sub_l : eh->sub_l; + + new_e = find_first_bit(&new_e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (new_e < DEPT_MAX_SUBCLASSES_EVT) + /* + * Let it work with the first bit anyway. + */ + DEPT_WARN_ON(1UL << new_e != new_e_f); + else + new_e = e; + + pop_ecxt(m, c); + + /* + * Apply the key to the map. + */ + if (new_k) + dept_map_reinit(m, new_k, -1, NULL); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); + + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + goto exit; + + /* + * Successfully pop_ecxt()ed but failed to add_ecxt(). + */ + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_map_ecxt_modify); + +void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, + const char *c_fn, const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + struct dept_class *c; + struct dept_key *k; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt++; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (e >= DEPT_MAX_SUBCLASSES_EVT) + goto missing_ecxt; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); + + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + goto exit; +missing_ecxt: + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_enter); + +bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + bool ret = false; + int e; + + if (unlikely(!dept_working())) + return false; + + if (dt->recursive) + return false; + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + if (find_ecxt_pos(m, c, true) != -1) { + ret = true; + break; + } + } + + dept_exit(flags); + + return ret; +} +EXPORT_SYMBOL_GPL(dept_ecxt_holding); + +void dept_request_event(struct dept_map *m) +{ + unsigned long flags; + unsigned int wg; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + WRITE_ONCE(m->wgen, wg); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_request_event); + +void dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + /* + * Dept won't work with this even though an event + * context has been asked. Don't make it confused at + * handling the event. Disable it until the next. + */ + WRITE_ONCE(m->wgen, 0U); + return; + } + + if (m->nocheck) + return; + + flags = dept_enter(); + + __dept_event(m, e_f, ip, e_fn, false); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_event); + +void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, + unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt--; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + if (pop_ecxt(m, c)) + goto exit; + } + + dt->missing_ecxt--; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_exit); + +void dept_task_exit(struct task_struct *t) +{ + struct dept_task *dt = &t->dept_task; + int i; + + if (unlikely(!dept_working())) + return; + + raw_local_irq_disable(); + + if (dt->stack) + put_stack(dt->stack); + + for (i = 0; i < dt->ecxt_held_pos; i++) { + if (dt->ecxt_held[i].class) + put_class(dt->ecxt_held[i].class); + if (dt->ecxt_held[i].ecxt) + put_ecxt(dt->ecxt_held[i].ecxt); + } + + for (i = 0; i < DEPT_MAX_WAIT_HIST; i++) + if (dt->wait_hist[i].wait) + put_wait(dt->wait_hist[i].wait); + + dt->task_exit = true; + dept_off(); + + raw_local_irq_enable(); +} + +void dept_task_init(struct task_struct *t) +{ + memset(&t->dept_task, 0x0, sizeof(struct dept_task)); +} + +void dept_key_init(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Key initialization fails.\n"); + return; + } + + flags = dept_enter(); + + clean_classes_cache(k); + + /* + * dept_key_init() should not fail. + * + * FIXME: Should be fixed if dept_key_init() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + DEPT_STOP("The class(%s/%d) has not been removed.\n", + c->name, sub_id); + break; + } + + dept_unlock(); + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_key_init); + +void dept_key_destroy(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive == 1 && dt->task_exit) { + /* + * Need to allow to go ahead in this case where + * ->recursive has been set to 1 by dept_off() in + * dept_task_exit() and ->task_exit has been set to + * true in dept_task_exit(). + */ + } else if (dt->recursive) { + DEPT_STOP("Key destroying fails.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_key_destroy() should not fail. + * + * FIXME: Should be fixed if dept_key_destroy() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(dept_key_destroy); + +static void move_llist(struct llist_head *to, struct llist_head *from) +{ + struct llist_node *first = llist_del_all(from); + struct llist_node *last; + + if (!first) + return; + + for (last = first; last->next; last = last->next); + llist_add_batch(first, last, to); +} + +static void migrate_per_cpu_pool(void) +{ + const int boot_cpu = 0; + int i; + + /* + * The boot CPU has been using the temperal local pool so far. + * From now on that per_cpu areas have been ready, use the + * per_cpu local pool instead. + */ + DEPT_WARN_ON(smp_processor_id() != boot_cpu); + for (i = 0; i < OBJECT_NR; i++) { + struct llist_head *from; + struct llist_head *to; + + from = &pool[i].boot_pool; + to = per_cpu_ptr(pool[i].lpool, boot_cpu); + move_llist(to, from); + } +} + +#define B2KB(B) ((B) / 1024) + +/* + * Should be called after setup_per_cpu_areas() and before no non-boot + * CPUs have been on. + */ +void __init dept_init(void) +{ + size_t mem_total = 0; + + local_irq_disable(); + dept_per_cpu_ready = 1; + migrate_per_cpu_pool(); + local_irq_enable(); + +#define HASH(id, bits) BUILD_BUG_ON(1 << (bits) <= 0); + #include "dept_hash.h" +#undef HASH +#define OBJECT(id, nr) mem_total += sizeof(struct dept_##id) * nr; + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) mem_total += sizeof(struct hlist_head) * (1 << (bits)); + #include "dept_hash.h" +#undef HASH + + pr_info("DEPendency Tracker: Copyright (c) 2020 LG Electronics, Inc., Byungchul Park\n"); + pr_info("... DEPT_MAX_STACK_ENTRY: %d\n", DEPT_MAX_STACK_ENTRY); + pr_info("... DEPT_MAX_WAIT_HIST : %d\n", DEPT_MAX_WAIT_HIST); + pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); + pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); +#define OBJECT(id, nr) \ + pr_info("... memory used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct dept_##id) * nr)); + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) \ + pr_info("... hash list head used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); + #include "dept_hash.h" +#undef HASH + pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); +} diff --git a/kernel/dependency/dept_hash.h b/kernel/dependency/dept_hash.h new file mode 100644 index 00000000..fd85aab --- /dev/null +++ b/kernel/dependency/dept_hash.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * HASH(id, bits) + * + * id : Id for the object of struct dept_##id. + * bits: 1UL << bits is the hash table size. + */ + +HASH(dep, 12) +HASH(class, 12) diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h new file mode 100644 index 00000000..0b7eb16 --- /dev/null +++ b/kernel/dependency/dept_object.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * OBJECT(id, nr) + * + * id: Id for the object of struct dept_##id. + * nr: # of the object that should be kept in the pool. + */ + +OBJECT(dep, 1024 * 8) +OBJECT(class, 1024 * 8) +OBJECT(stack, 1024 * 32) +OBJECT(ecxt, 1024 * 16) +OBJECT(wait, 1024 * 32) diff --git a/kernel/exit.c b/kernel/exit.c index 15dc2ec..0f48752 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -916,6 +916,7 @@ void __noreturn do_exit(long code) exit_tasks_rcu_finish(); lockdep_free_task(tsk); + dept_task_exit(tsk); do_task_dead(); } diff --git a/kernel/fork.c b/kernel/fork.c index 9f7fe35..1d33fc3 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -97,6 +97,7 @@ #include #include #include +#include #include #include @@ -2219,6 +2220,7 @@ static __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_LOCKDEP lockdep_init_task(p); #endif + dept_task_init(p); #ifdef CONFIG_DEBUG_MUTEXES p->blocked_on = NULL; /* not blocked yet */ diff --git a/kernel/module/main.c b/kernel/module/main.c index 48568a0..2882ea2 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1194,6 +1194,7 @@ static void free_module(struct module *mod) /* Free lock-classes; relies on the preceding sync_rcu(). */ lockdep_free_key_range(mod->data_layout.base, mod->data_layout.size); + dept_free_range(mod->data_layout.base, mod->data_layout.size); /* Finally, free the core (containing the module structure) */ module_memfree(mod->core_layout.base); @@ -2893,6 +2894,7 @@ static int load_module(struct load_info *info, const char __user *uargs, free_module: /* Free lock-classes; relies on the preceding sync_rcu() */ lockdep_free_key_range(mod->data_layout.base, mod->data_layout.size); + dept_free_range(mod->data_layout.base, mod->data_layout.size); module_deallocate(mod, info); free_copy: diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 25b582b..0dc066c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -64,6 +64,7 @@ #include #include #include +#include #ifdef CONFIG_PREEMPT_DYNAMIC # ifdef CONFIG_GENERIC_ENTRY @@ -4070,6 +4071,7 @@ bool ttwu_state_match(struct task_struct *p, unsigned int state, int *success) int cpu, success = 0; preempt_disable(); + dept_stage_event(p, _RET_IP_); if (p == current) { /* * We're waking current, this means 'p->on_rq' and 'task_cpu(p) @@ -6446,6 +6448,12 @@ static void __sched notrace __schedule(unsigned int sched_mode) rq = cpu_rq(cpu); prev = rq->curr; + prev_state = READ_ONCE(prev->__state); + if (sched_mode != SM_PREEMPT && prev_state & TASK_NORMAL) + dept_request_event_wait_commit(); + + dept_sched_enter(); + schedule_debug(prev, !!sched_mode); if (sched_feat(HRTICK) || sched_feat(HRTICK_DL)) @@ -6560,6 +6568,7 @@ static void __sched notrace __schedule(unsigned int sched_mode) __balance_callbacks(rq); raw_spin_rq_unlock_irq(rq); } + dept_sched_exit(); } void __noreturn do_task_dead(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 881c3f8..611fd01 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1255,6 +1255,33 @@ config DEBUG_PREEMPT menu "Lock Debugging (spinlocks, mutexes, etc...)" +config DEPT + bool "Dependency tracking (EXPERIMENTAL)" + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + select DEBUG_SPINLOCK + select DEBUG_MUTEXES + select DEBUG_RT_MUTEXES if RT_MUTEXES + select DEBUG_RWSEMS + select DEBUG_WW_MUTEX_SLOWPATH + select DEBUG_LOCK_ALLOC + select TRACE_IRQFLAGS + select STACKTRACE + select FRAME_POINTER if !MIPS && !PPC && !ARM && !S390 && !MICROBLAZE && !ARC && !X86 + select KALLSYMS + select KALLSYMS_ALL + select PROVE_LOCKING + default n + help + Check dependencies between wait and event and report it if + deadlock possibility has been detected. Multiple reports are + allowed if there are more than a single problem. + + This feature is considered EXPERIMENTAL that might produce + false positive reports because new dependencies start to be + tracked, that have never been tracked before. It's worth + noting, to mitigate the impact by the false positives, multi + reporting has been supported. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c index 8d24279..cd89138 100644 --- a/lib/locking-selftest.c +++ b/lib/locking-selftest.c @@ -1398,6 +1398,8 @@ static void reset_locks(void) local_irq_disable(); lockdep_free_key_range(&ww_lockdep.acquire_key, 1); lockdep_free_key_range(&ww_lockdep.mutex_key, 1); + dept_free_range(&ww_lockdep.acquire_key, 1); + dept_free_range(&ww_lockdep.mutex_key, 1); I1(A); I1(B); I1(C); I1(D); I1(X1); I1(X2); I1(Y1); I1(Y2); I1(Z1); I1(Z2); From patchwork Tue Jan 31 08:39:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B6A8C636D3 for ; Tue, 31 Jan 2023 08:40:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9E586B0075; Tue, 31 Jan 2023 03:40:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E74EC6B007B; Tue, 31 Jan 2023 03:40:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACBBD6B007B; Tue, 31 Jan 2023 03:40:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9CBC16B0074 for ; Tue, 31 Jan 2023 03:40:00 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 68AF4A0250 for ; Tue, 31 Jan 2023 08:40:00 +0000 (UTC) X-FDA: 80414446560.06.B58F06F Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf02.hostedemail.com (Postfix) with ESMTP id 0D0FC80006 for ; Tue, 31 Jan 2023 08:39:57 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; spf=softfail (imf02.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154398; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=uND5ZC1VBzEX1PsDvQaWhOtJ4Giv67cqRAht/cJxS5E=; b=n/fVErUyyNmUOK9TTeZlHm+08Dat5yNBcfL7t/ifej4IaXgllLSB54+rjjhzN4KyqklnD4 MyymdvYPP/GpcoWCMKQNzvz+b8/5XSqzoGIVZ7sk28ojFA7WIUecJa1XqqCn/eYsA0Kmab 33ZFrzOr2h5wkUQ/KbtONcjJqQfEDFM= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; spf=softfail (imf02.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154398; a=rsa-sha256; cv=none; b=NWX2m7EeWgQOGB+Wx5Bndd8XAjxlxiG3zeX0b6nv/w7OXI1dQIYf4oQTqnxgYpphioycwj J1VLoezi9eHBfMbIa+wGMdABLNc11rvCqUJ79cG0ZnO5PMysL6EfFba88wf2cBbm77P54M eVrT8F6FjZF94pRY9SfalT60jCXvwW4= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:39:56 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:56 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 03/25] dept: Add single event dependency tracker APIs Date: Tue, 31 Jan 2023 17:39:32 +0900 Message-Id: <1675154394-25598-4-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: qho64edqo84bmixczsbg15xunhwqpwyx X-Rspamd-Queue-Id: 0D0FC80006 X-HE-Tag: 1675154397-567941 X-HE-Meta: U2FsdGVkX18Quy2/5p3Wr3WKIPW57481IhAXhnYaysNp+0oTDXVaBFhgJy2Hq4+pe+iWjWAA+SFB0uWNZ4YkgsqUxNeWbuwue1fkPEzl3+Qm1LKMqTYUWr1fxQ3qfMpH78lv7dOoPhB3dfquIDNix5nAqnm0aE16HCZdJJVJSG1XDqqOftbnL4zLvpdiNWJ+GMokTKCeBgR5wu7p3NXguoUiM1mKlFvJpqLTdV8jHpUfD+xurDru4J23d63Afa0OIpuSKqvc1Z6EplIsP4WcshhGK1r3Kj43DsKLpKrCJksz5GBgMUjOBvwXC0UlQbUUtM3BR9KalkhV9hpq/IUi0x1IUJjXOyudomMm50RZpd6Cj9w1vmGir0q/XLmQoZXpOUAX2Y1RIOu8Bolr8aePJ0xpIKxOf28NIVQ9c+qGeT0+H+87QJNeAKFDh+NoqdV2B7YuOG3CVuQdxg1+4UG9kUKZSlib8JOHH7J4soHZ9w78Kq3+xAwopBnbv5UQPacX3fwQCnc2ftE7bXqb29I+wQv7St3Mp2Tjz1HZZhq2+PUshh97fR0A9D1h2sN3AjYV0dtQtB5K4aNZsyCYOqOALuGfD5WmnNvIJjr/TcUJcPiT27/gE4bUI8Ldw0aaztl5bwMHLDYC/sbp3FTw7AtPlsuy4ePbW+WNe9oFeKbdT0VLyKFva6YtE2yli4NVhA3rcJiwgupnFAgMKR9a6aVeDeQBgB9S0xa4LAhmZx6QoyPkraguvGZ3Zs5nk3kNQbUZ5Z+/mReJIVdoCuu2o++aLvcgt0mdt2g6HGO47MXCvY3qeXbJXcw8DEcH9sAby0YJDOSBmbv0MlKCLXdZ7xRjh/dCVAICLpMoo0d2o3laemEDVICxKhM/+UFWqPWwW7sPB7f3uWIaUoSHv5CO1LJde1BJpB7ROK6EX/N6SMDwYhyTMlCk72Lps32HEJu+8Z8+WGyt5Yl3PyZENFNaY7B R/B8J3Gm BFr3elxya5q9Ods1bLLB4FCV9UZfKIgmSJf5HBQGGIT6my5cNEUvAlsQs04WZcThxnkW7iVhHgMaeJAPounBBvya+B5JX4tOHrMreZLijG3cKndv6QEaCkcR/FlktBuqZ9txibaEF5ASFWbmdiOOLftEvPcoljaATMVUBW3PPfCzyggOEPQW3DJwxUec+kf5QT8vhhS0Gr03B4U1NWFB73PUgLEb+HAoVH8QyKs0DvsyjVRw0vgg+KaV8eiDcQMvGi9wkc4TCdqmb9jUZU/kSycTWTjFl+Y85TsyaRnGsS7vpcfY9K+YN+jBM6QM9/Sxau1Z0N1qb/GX+o6YX3qWb8S131/1gnIUvwE3+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Wrapped the base APIs for easier annotation on wait and event. Start with supporting waiters on each single event. More general support for multiple events is a future work. Do more when the need arises. How to annotate (the simplest way): 1. Initaialize a map for the interesting wait. /* * Recommand to place along with the wait instance. */ struct dept_map my_wait; /* * Recommand to place in the initialization code. */ sdt_map_init(&my_wait); 2. Place the following at the wait code. sdt_wait(&my_wait); 3. Place the following at the event code. sdt_event(&my_wait); That's it! Signed-off-by: Byungchul Park --- include/linux/dept_sdt.h | 62 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 include/linux/dept_sdt.h diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h new file mode 100644 index 00000000..12a793b --- /dev/null +++ b/include/linux/dept_sdt.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Single-event Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_SDT_H +#define __LINUX_DEPT_SDT_H + +#include +#include + +#ifdef CONFIG_DEPT +#define sdt_map_init(m) \ + do { \ + static struct dept_key __key; \ + dept_map_init(m, &__key, 0, #m); \ + } while (0) + +#define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) + +#define sdt_wait(m) \ + do { \ + dept_request_event(m); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + } while (0) + +/* + * sdt_might_sleep() and its family will be committed in __schedule() + * when it actually gets to __schedule(). Both dept_request_event() and + * dept_wait() will be performed on the commit. + */ + +/* + * Use the code location as the class key if an explicit map is not used. + */ +#define sdt_might_sleep_start(m) \ + do { \ + struct dept_map *__m = m; \ + static struct dept_key __key; \ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + } while (0) + +#define sdt_might_sleep_end() dept_clean_stage() + +#define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) +#else /* !CONFIG_DEPT */ +#define sdt_map_init(m) do { } while (0) +#define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start(m) do { } while (0) +#define sdt_might_sleep_end() do { } while (0) +#define sdt_ecxt_enter(m) do { } while (0) +#define sdt_event(m) do { } while (0) +#define sdt_ecxt_exit(m) do { } while (0) +#endif +#endif /* __LINUX_DEPT_SDT_H */ From patchwork Tue Jan 31 08:39:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EB21C636CC for ; Tue, 31 Jan 2023 08:40:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5224D6B0078; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B0AF6B007E; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 269366B0078; Tue, 31 Jan 2023 03:40:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B9E166B0078 for ; Tue, 31 Jan 2023 03:40:00 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 99AB480463 for ; Tue, 31 Jan 2023 08:40:00 +0000 (UTC) X-FDA: 80414446560.05.1E6FA1A Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf17.hostedemail.com (Postfix) with ESMTP id 6763340009 for ; Tue, 31 Jan 2023 08:39:58 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=none; spf=softfail (imf17.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=QUPYVXukvkhJQyvCaOVez+3E4bKK4emd7/QpFX7Le/Y=; b=jkUfSlgre4qqOSpQV1MQYKMxIEliQ1pjAWThkC54ndZETz6cGRb9xj0/E+gcn1PBf5wUyY IaVs+udnHUjBGeos/bILI71iydflFFNh/iaJfav+7o50ZP6zZ0mHYHE1Bu+lR6UP21xXLP XpAztq1p85DXqyQDvEeIWhJR+pVnZM8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=none; spf=softfail (imf17.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154399; a=rsa-sha256; cv=none; b=JQM9raGlNAee1S3bhLTBkFuuaO3ca10Ab8nwDB7acUk+aIRIX0VtFBNEhfkjaKqdQuGyCF TFuZKSXo8i6Cq1xgbSoo1fjvDf1K4JYky537zeiq/Q8QM96Lwyb9rCB1eNw4Lc8mrs4BZI rCY6DEz4K6K1QgByPxFE31oTmAC1VUg= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:39:56 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:56 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 04/25] dept: Add lock dependency tracker APIs Date: Tue, 31 Jan 2023 17:39:33 +0900 Message-Id: <1675154394-25598-5-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Stat-Signature: nc93ch86fdyzf6hoqa6w8cwqmth8j7xe X-Rspam-User: X-Rspamd-Queue-Id: 6763340009 X-Rspamd-Server: rspam06 X-HE-Tag: 1675154398-526904 X-HE-Meta: U2FsdGVkX1/gi43VIuE+z+wBfpKAfe4Ug+rDzcbZN6JZ+V/zo3Wstg/2hEKdaTJr7HjXy0EC5wuuMqN/yEhpCXUedNSb4+7AzrPJwF0Z5fnxHjI7QdLFlAu2lK+vbCWT8SDrtNhNoVCnUpNeD0mUYTYDuTOZ1pGF+CdaPRTIaFrnWkbFTa9e6itGRtoX6HtBNnXro2xw7LvoVGr+VhUuAiu68/4XA4Uh3GHdCPwNPz8ur749KHojBGtX1FS6GJ6hof7OGNKH0kq1ZiL17YZLD9YMCnS8lZjpm3ospD6y4T3zgbOe5xlpyeF9g7922ewdRc2gXUW9mz3JlSNk9HFMKKvQsgJXHMVKggB3vkOtSe/oYocFnbsKc7Ow9WKY2ychFUMxJ13vw3/qdtJcIRFErQZFaLL6bW0YDwFGFEKxm2FVf6ILajszjMOuvvNkQAqUlRjOQW+CXWD8LkSWE029YKxVJV51C3J7OKy/zqwYa5yDX3bk6pQTc7IkUI/MfvNepQieawl5F6K5iGnLvMK3oYxxUoQeYxigHJHs3jUn6mmHHVBi6uD0faYiE9uZz722hlq0iqQmBOmuxhEgt/MXbyBqCfrxf7vzJAMY0RIauvYvD6PQKOTuzCNENy8KwbPHmS+jEYymfjLKbP4Z/qnNrHmRdIjVr9T8ygUhg83JnPCWWctFO/80Z6b8SN4xW5M/XZrsw0j7c03eEnMeA3tU0rdAj+rySGHPBtg7XU1ZU1m/aFqQlyOdeV0g0C2js3nOPevtF5Ykvl01vMUD21kwj5dYAEZjffKQlizL4VVzBk81R2NGnpC6GJGho92+VbMhZtC7g7zVHsHdwrAj7F6sBWkKKWvRi7VXSRy4MtkEA+pu/ROyQvsuIBIe3ej88DIa7ac/OZb/TZTooyXEJLCyOzBhoXYbI6i7DZNv7kIUpyQxXN9MxWxsfTg0qPNT0m0YFKPC/bvgpItVFFwFyc/ lxzaN1cH Y7TaOiW4zROc8vTN38HXZL3WLSueQTelFQI/XEqyNekuz9LUJE7rYD+gp3ytetQQ65YEFGm/Wg3WlCa1gjmt7Xb5Hio6j9FYs+rRskXA/MxzM6Mo8dp9M4xQ1oD+FPwKl/HQpxNJcHw/6yJGzcVsPTDRPIIjFJm3QNCFQKXTDBVFiUgR5wcxDANRpZ8ZSFC6vO4asJjcuuIdQl0yVs8M+f6JTi+X4Wym7bDbUDU++ZVIH59vFn+R7SiebtJyxowMw+UhEVZAf7gb5CU9XWa6z3kdq3nI4ta/5i2xhUHwyoqV1RMW4UVw1ERYQqMStzMCJb6TG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Wrapped the base APIs for easier annotation on typical lock. Signed-off-by: Byungchul Park --- include/linux/dept_ldt.h | 77 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 include/linux/dept_ldt.h diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h new file mode 100644 index 00000000..062613e --- /dev/null +++ b/include/linux/dept_ldt.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Lock Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_LDT_H +#define __LINUX_DEPT_LDT_H + +#include + +#ifdef CONFIG_DEPT +#define LDT_EVT_L 1UL +#define LDT_EVT_R 2UL +#define LDT_EVT_W 1UL +#define LDT_EVT_RW (LDT_EVT_R | LDT_EVT_W) +#define LDT_EVT_ALL (LDT_EVT_L | LDT_EVT_RW) + +#define ldt_init(m, k, su, n) dept_map_init(m, k, su, n) +#define ldt_lock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ + } \ + } while (0) + +#define ldt_rlock(m, sl, t, n, i, q) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ + else { \ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ + } \ + } while (0) + +#define ldt_wlock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ + } \ + } while (0) + +#define ldt_unlock(m, i) dept_ecxt_exit(m, LDT_EVT_ALL, i) + +#define ldt_downgrade(m, i) \ + do { \ + if (dept_ecxt_holding(m, LDT_EVT_W)) \ + dept_map_ecxt_modify(m, LDT_EVT_W, NULL, LDT_EVT_R, i, "downgrade", "read_unlock", -1);\ + } while (0) + +#define ldt_set_class(m, n, k, sl, i) dept_map_ecxt_modify(m, LDT_EVT_ALL, k, 0UL, i, "lock_set_class", "(any)unlock", sl) +#else /* !CONFIG_DEPT */ +#define ldt_init(m, k, su, n) do { (void)(k); } while (0) +#define ldt_lock(m, sl, t, n, i) do { } while (0) +#define ldt_rlock(m, sl, t, n, i, q) do { } while (0) +#define ldt_wlock(m, sl, t, n, i) do { } while (0) +#define ldt_unlock(m, i) do { } while (0) +#define ldt_downgrade(m, i) do { } while (0) +#define ldt_set_class(m, n, k, sl, i) do { } while (0) +#endif +#endif /* __LINUX_DEPT_LDT_H */ From patchwork Tue Jan 31 08:39:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A84DDC63797 for ; Tue, 31 Jan 2023 08:40:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A80436B0085; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A0AAF6B0083; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BC616B0082; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 57C4B6B007D for ; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3973D140337 for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.28.40917DE Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf01.hostedemail.com (Postfix) with ESMTP id 7C62F40015 for ; Tue, 31 Jan 2023 08:39:58 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf01.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=9UWwahqiAs7jUJvxNwUHLk+Vin2j16pczOaZQP7p9wU=; b=xaBMSdsi46cnXspe7PbPr8V1eACvxNZCUeEDvr0NFGNDlKZYTr0wee2cBYm93GOQEtOA44 bjzETqe5A5jLZ8m9tJNpwEw6L4qQOh+m624DLH4jV98pHYJufIsUJ8DIqRD1WxGqcfj9Zt waNcUx7EdMuFyH9ycwroOAi4lI/m1pk= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf01.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154399; a=rsa-sha256; cv=none; b=Qd8uzWprVJ7+g2iaPv8IK8XcuXGpQcFpq6kx9As6wMZxVx5xSF29CR4UVVTwDXWUGpJwGv gu5e8kEceSKUyGPk4yexsUx2wumPN23amnNpGJQS9bCRPyALmUurXOrqLqVF9Y/iG1vnD0 /h+a3ytwhDlLsoKrDjqbVihwEKKtkUA= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:39:56 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:56 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 05/25] dept: Tie to Lockdep and IRQ tracing Date: Tue, 31 Jan 2023 17:39:34 +0900 Message-Id: <1675154394-25598-6-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7C62F40015 X-Stat-Signature: xcbeyixywk8kk1f6h8ix55ms4oim9fes X-Rspam-User: X-HE-Tag: 1675154398-510364 X-HE-Meta: U2FsdGVkX18SbtXVPnNaaxFJNYBtISDG+gc4IX67Tc4pvlcB5jt4AoUf6Evsxgtrin+crEoXzJN3we3X5Q+prA81ROBWnqUwUUoE3eRifF+RywElEp8Q4zgXddJNn+Q2wL4tbcpQjiGTw5D9srVZvGBsLHKrDo2SJHfc4qqPJ5PFavnNV43Tma+AiPCClJmHz65pyZMZC7XS8kgOJlr2aX8ZvIdMvPXtoVYaIwu/DRGwcqYt+Ii7//LXTSLF3wwUVu01sgXOEIknJ8sN08ddc6fViJERfcjVInW4+QpvpWyICTGjRL6ztgU5rRKiXqKga+AY2lbSy4PkvO2yRBkhMjfrL71Q1IoxdO2OWSA21GIjdrEqkOHcuXKOLaH5h3Ly/Fd31OhPZl3AElFkI/BSUcBCen6+Fb7/N1mp6wp8rClCZfhc4s+5d8iJYBG7CTlSMJDB8tbUAhFDRST7gqWpLs8Td7D5YIggs5X3okbfJLWeujIU1w6raNMyb1V6ES7CmxdGNNpWOSpyBGNOtcZP0f60B/X+uNJe3dxoJkYGiZ4S73k6h6KKUX6V+9iBF/k2aRY1jevi859sVyldZ5LNBDONKe6wG4y6AHE2VizyGic0ol0pWDMIwXAu6Rii2jT9V34Ev26/VP58USjzGWNPKmyQCSfJ32kt3qzn1QvCltXneT+cFofIe083bFaKfxhW3m4cKyTMeJa9IWr3dioeeTDwM6Jpu12T5vpES8xcznPuFH5Y4t11y0JpP/mlRihKrV+HndE3dh3XcGHZqqeUEaOpzonO1UEZGz3LFajMIigcNJJ4RrrCcDJAWV72PBsNuFiwj/pZnL/N2YJKu/fhZKodrK7y3OiYlaqbpgtF2u9eAtp3iPz7XfprufH3seJ/uBsmDnIf0DTSD7/xOy9RvZcgETFOC+6cYxtrmXwpCugZpcbK/2+KcXID1eEBHqeFyUXzXjt1hrvBZBudH1S OUmLvgIJ 9Oy5bKPo+XOBw1dZxqRNMGcntPRXbaf5tbf3iI/1kTuH0jIE2nByc2o7BkDKAKNvl8rabnOOqMDBW/M043EQNgn9Z+AMdYmgKv3KbaB9+o2ivqI0Yo4G9ccCG0zkkQA5k28u062L5RbGMDSb86zA4a2//Chl7vPKPyCsTWlzcmjG6bX4ILhb2g9jRn7OWj3ZyEXdo1TG3uRJ5hgrdfy+N2K2x6nNwLFV9anV++FdBFdp3IPWmFR5qoVWQklOHlF0/B5S2lRJJv0vjEG+QluF1ogZIMhIyA57DDreqV1FxjdJGG/jP7d8rUmFqtC5phflPVBEi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Yes. How to place Dept in here looks so ugly. But it's inevitable as long as relying on Lockdep. The way should be enhanced gradually. 1. Basically relies on Lockdep to track typical locks and IRQ things. 2. Dept fails to recognize IRQ situation so it generates false alarms when raw_local_irq_*() APIs are used. So made it track those too. 3. Lockdep doesn't track the outmost {hard,soft}irq entracnes but Dept makes use of it. So made it track those too. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 22 ++++++-- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++++++++-------- include/linux/lockdep_types.h | 3 ++ include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 ++ include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 4 +- kernel/locking/lockdep.c | 23 ++++++++ 14 files changed, 139 insertions(+), 29 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 5ec0fa7..0ebc5ec 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -13,6 +13,7 @@ #define _LINUX_TRACE_IRQFLAGS_H #include +#include #include #include @@ -60,8 +61,10 @@ struct irqtrace_events { # define lockdep_softirqs_enabled(p) ((p)->softirqs_enabled) # define lockdep_hardirq_enter() \ do { \ - if (__this_cpu_inc_return(hardirq_context) == 1)\ + if (__this_cpu_inc_return(hardirq_context) == 1) { \ current->hardirq_threaded = 0; \ + dept_hardirq_enter(); \ + } \ } while (0) # define lockdep_hardirq_threaded() \ do { \ @@ -136,6 +139,8 @@ struct irqtrace_events { # define lockdep_softirq_enter() \ do { \ current->softirq_context++; \ + if (current->softirq_context == 1) \ + dept_softirq_enter(); \ } while (0) # define lockdep_softirq_exit() \ do { \ @@ -170,17 +175,28 @@ struct irqtrace_events { /* * Wrap the arch provided IRQ routines to provide appropriate checks. */ -#define raw_local_irq_disable() arch_local_irq_disable() -#define raw_local_irq_enable() arch_local_irq_enable() +#define raw_local_irq_disable() \ + do { \ + arch_local_irq_disable(); \ + dept_hardirqs_off(); \ + } while (0) +#define raw_local_irq_enable() \ + do { \ + dept_hardirqs_on(); \ + arch_local_irq_enable(); \ + } while (0) #define raw_local_irq_save(flags) \ do { \ typecheck(unsigned long, flags); \ flags = arch_local_irq_save(); \ + dept_hardirqs_off(); \ } while (0) #define raw_local_irq_restore(flags) \ do { \ typecheck(unsigned long, flags); \ raw_check_bogus_irq_restore(); \ + if (!arch_irqs_disabled_flags(flags)) \ + dept_hardirqs_on(); \ arch_local_irq_restore(flags); \ } while (0) #define raw_local_save_flags(flags) \ diff --git a/include/linux/local_lock_internal.h b/include/linux/local_lock_internal.h index 975e33b..39f6778 100644 --- a/include/linux/local_lock_internal.h +++ b/include/linux/local_lock_internal.h @@ -21,6 +21,7 @@ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner = NULL, diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index 1f1099d..9996102 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -12,6 +12,7 @@ #include #include +#include #include struct task_struct; @@ -39,6 +40,8 @@ static inline void lockdep_copy_map(struct lockdep_map *to, */ for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) to->class_cache[i] = NULL; + + dept_map_copy(&to->dmap, &from->dmap); } /* @@ -441,7 +444,8 @@ enum xhlock_context_t { * Note that _name must not be NULL. */ #define STATIC_LOCKDEP_MAP_INIT(_name, _key) \ - { .name = (_name), .key = (void *)(_key), } + { .name = (_name), .key = (void *)(_key), \ + .dmap = DEPT_MAP_INITIALIZER(_name, _key) } static inline void lockdep_invariant_state(bool force) {} static inline void lockdep_free_task(struct task_struct *task) {} @@ -523,33 +527,89 @@ static inline void print_irqtrace_events(struct task_struct *curr) #define lock_acquire_shared(l, s, t, n, i) lock_acquire(l, s, t, 1, 1, n, i) #define lock_acquire_shared_recursive(l, s, t, n, i) lock_acquire(l, s, t, 2, 1, n, i) -#define spin_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define spin_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define spin_release(l, i) lock_release(l, i) - -#define rwlock_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) +#define spin_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define spin_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define spin_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwlock_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) #define rwlock_acquire_read(l, s, t, i) \ do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, !read_lock_is_recursive());\ if (read_lock_is_recursive()) \ lock_acquire_shared_recursive(l, s, t, NULL, i); \ else \ lock_acquire_shared(l, s, t, NULL, i); \ } while (0) - -#define rwlock_release(l, i) lock_release(l, i) - -#define seqcount_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define seqcount_acquire_read(l, s, t, i) lock_acquire_shared_recursive(l, s, t, NULL, i) -#define seqcount_release(l, i) lock_release(l, i) - -#define mutex_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define mutex_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define mutex_release(l, i) lock_release(l, i) - -#define rwsem_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define rwsem_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define rwsem_acquire_read(l, s, t, i) lock_acquire_shared(l, s, t, NULL, i) -#define rwsem_release(l, i) lock_release(l, i) +#define rwlock_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define seqcount_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_acquire_read(l, s, t, i) \ +do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, false); \ + lock_acquire_shared_recursive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define mutex_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define mutex_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define mutex_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwsem_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define rwsem_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define rwsem_acquire_read(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_shared(l, s, t, NULL, i); \ +} while (0) +#define rwsem_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) #define lock_map_acquire(l) lock_acquire_exclusive(l, 0, 0, NULL, _THIS_IP_) #define lock_map_acquire_read(l) lock_acquire_shared_recursive(l, 0, 0, NULL, _THIS_IP_) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index d224308..50c8879 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -11,6 +11,7 @@ #define __LINUX_LOCKDEP_TYPES_H #include +#include #define MAX_LOCKDEP_SUBCLASSES 8UL @@ -76,6 +77,7 @@ struct lock_class_key { struct hlist_node hash_entry; struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES]; }; + struct dept_key dkey; }; extern struct lock_class_key __lockdep_no_validate__; @@ -185,6 +187,7 @@ struct lockdep_map { int cpu; unsigned long ip; #endif + struct dept_map dmap; }; struct pin_cookie { unsigned int val; }; diff --git a/include/linux/mutex.h b/include/linux/mutex.h index 8f226d4..58bf314 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -25,6 +25,7 @@ , .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define __DEP_MAP_MUTEX_INITIALIZER(lockname) diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index 36b942b..e871aca 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -21,7 +21,7 @@ struct percpu_rw_semaphore { }; #ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) }, #else #define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) #endif diff --git a/include/linux/rtmutex.h b/include/linux/rtmutex.h index 7d04988..35889ac 100644 --- a/include/linux/rtmutex.h +++ b/include/linux/rtmutex.h @@ -81,6 +81,7 @@ static inline void rt_mutex_debug_task_free(struct task_struct *tsk) { } .dep_map = { \ .name = #mutexname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(mutexname, NULL),\ } #else #define __DEP_MAP_RT_MUTEX_INITIALIZER(mutexname) diff --git a/include/linux/rwlock_types.h b/include/linux/rwlock_types.h index 1948442..6e58dfc 100644 --- a/include/linux/rwlock_types.h +++ b/include/linux/rwlock_types.h @@ -10,6 +10,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL), \ } #else # define RW_DEP_MAP_INIT(lockname) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index efa5c32..4f856e7 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -21,6 +21,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, #else # define __RWSEM_DEP_MAP_INIT(lockname) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index 3926e90..6ba00bc 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -81,7 +81,7 @@ static inline void __seqcount_init(seqcount_t *s, const char *name, #ifdef CONFIG_DEBUG_LOCK_ALLOC # define SEQCOUNT_DEP_MAP_INIT(lockname) \ - .dep_map = { .name = #lockname } + .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) } /** * seqcount_init() - runtime initializer for seqcount_t diff --git a/include/linux/spinlock_types_raw.h b/include/linux/spinlock_types_raw.h index 91cb36b..3dcc551 100644 --- a/include/linux/spinlock_types_raw.h +++ b/include/linux/spinlock_types_raw.h @@ -31,11 +31,13 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SPIN, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define SPIN_DEP_MAP_INIT(lockname) \ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define LOCAL_SPIN_DEP_MAP_INIT(lockname) \ @@ -43,6 +45,7 @@ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define RAW_SPIN_DEP_MAP_INIT(lockname) diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 9b9d0bb..c934158 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const char *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) -#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name }, +#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name, .dmap = DEPT_MAP_INITIALIZER(srcu_name, NULL) }, #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ int init_srcu_struct(struct srcu_struct *ssp); diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 03d82e9..c92fe94 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -245,10 +245,10 @@ static inline bool dept_working(void) * Even k == NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__; static inline bool valid_key(struct dept_key *k) { - return &__dept_no_validate__ != k; + return &__lockdep_no_validate__.dkey != k; } /* diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index e3375bc..7ff3fb4 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -1220,6 +1220,8 @@ void lockdep_register_key(struct lock_class_key *key) struct lock_class_key *k; unsigned long flags; + dept_key_init(&key->dkey); + if (WARN_ON_ONCE(static_obj(key))) return; hash_head = keyhashentry(key); @@ -4327,6 +4329,8 @@ void noinstr lockdep_hardirqs_on(unsigned long ip) { struct irqtrace_events *trace = ¤t->irqtrace; + dept_hardirqs_on_ip(ip); + if (unlikely(!debug_locks)) return; @@ -4392,6 +4396,8 @@ void noinstr lockdep_hardirqs_on(unsigned long ip) */ void noinstr lockdep_hardirqs_off(unsigned long ip) { + dept_hardirqs_off_ip(ip); + if (unlikely(!debug_locks)) return; @@ -4436,6 +4442,8 @@ void lockdep_softirqs_on(unsigned long ip) { struct irqtrace_events *trace = ¤t->irqtrace; + dept_softirqs_on_ip(ip); + if (unlikely(!lockdep_enabled())) return; @@ -4474,6 +4482,9 @@ void lockdep_softirqs_on(unsigned long ip) */ void lockdep_softirqs_off(unsigned long ip) { + + dept_softirqs_off_ip(ip); + if (unlikely(!lockdep_enabled())) return; @@ -4806,6 +4817,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, { int i; + ldt_init(&lock->dmap, &key->dkey, subclass, name); + for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) lock->class_cache[i] = NULL; @@ -5544,6 +5557,12 @@ void lock_set_class(struct lockdep_map *lock, const char *name, { unsigned long flags; + /* + * dept_map_(re)init() might be called twice redundantly. But + * there's no choice as long as Dept relies on Lockdep. + */ + ldt_set_class(&lock->dmap, name, &key->dkey, subclass, ip); + if (unlikely(!lockdep_enabled())) return; @@ -5561,6 +5580,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigned long ip) { unsigned long flags; + ldt_downgrade(&lock->dmap, ip); + if (unlikely(!lockdep_enabled())) return; @@ -6333,6 +6354,8 @@ void lockdep_unregister_key(struct lock_class_key *key) unsigned long flags; bool found = false; + dept_key_destroy(&key->dkey); + might_sleep(); if (WARN_ON_ONCE(static_obj(key))) From patchwork Tue Jan 31 08:39:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00AAEC636D3 for ; Tue, 31 Jan 2023 08:40:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 463FB6B007D; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A1F06B0089; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D40CA6B0083; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 946646B0080 for ; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5CF97A015F for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.04.290A5CB Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf15.hostedemail.com (Postfix) with ESMTP id 121A9A0006 for ; Tue, 31 Jan 2023 08:39:58 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf15.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=1UlKdUXSGiDBuujO+OOrn1D/CZOdr4G7JLVSQA/4bUw=; b=0Uaqqxb6KDMetl349oO6y2nC3yxP/pXOOInRPO9So1c0pBAr+XkowspMnPzgBiinaNmslr SLEecMrbeDyiqSiaO14qxEQOG/VWfiXNUKj+Kuy0wV9D7JL0juAlKn/f+Q8A5u1pVtTG8N GalaL7VPZr8U+pxAw2fC5dLe71hABP8= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf15.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154399; a=rsa-sha256; cv=none; b=uD8pKX+0rVpyrvlr7IpbAxYJr+VMGxr7K3aHgRE9NBGQHlY8dJNEbazMJ9JpWqexPWGWcc 5xnNaVGlxBxvxRy3pT1vlF3GDzu6fiRwO/VRJHHwvxZw5GA5aYw3GcNxVlRP/6bMRBaqC+ K7HGSd+VShKy0FBJNdGmYa+U66kHXWo= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:39:57 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:57 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 06/25] dept: Add proc knobs to show stats and dependency graph Date: Tue, 31 Jan 2023 17:39:35 +0900 Message-Id: <1675154394-25598-7-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 121A9A0006 X-Stat-Signature: 7fq5nsmjcbiey88s6p3i6pdqktex6s8r X-HE-Tag: 1675154398-377949 X-HE-Meta: U2FsdGVkX18XAyThrc8qPR+bPkate1YRwVshW/ERRTcNfgr5vtYRxPfI6YRQWFAWF2Q7Dla9TItfP50SsCKXg1ahOvmMVfX5qfwL5C1cPX+DD1TCiq3rxcG0XiXYmBuF05aIAHil+fkqbwpwfd8CwVQbtLBcwW5mWj8NyhAotJKyXKhfn39EptNqH7hJKt8eo5Zhe9puvaW+dH1LV8YMnuI7zTPq90xzvQAWrtFmC3aDMFc6J/p9lP48QSaZ9oPtKixQI1+BaDTbZs1zOknTQpVXjBuDuU5Oxx/7yyYt9nW3nfAU0RKp3D59gkb7pL0FBCjXx1iexVj0+xzVNj1ygvBtlKcvGSwghJBSmjCniVj54ptN+lOoKHue8V+5fwvQCgjkw05acDeTCJ/IOCe+I6ORE4fKu2nhwkEZ6EEOtSfEjxm6M9N6+INVF4LvNgmzBHW7jkJu4BI2lsKgBRvKn89/nh1G2HFbPQIDooGoYq65wcl+FVbwyXTXZqR3Iz0J4i1dBlwsdMR0+7zHZDDcqNuTWI8yWnlPsasC4sLl9s6yL0Ayhawq4Beo+zwbZdpyDiWG42hnjRfSxfGIlZlEgV9jgbo+5Fx0mWByTrFejWEiIMU6MMBX6O5K5WRcnVgAS7viZ5HCiUjdFAILkhQCNo0ucJpoRwTwKp5hzuRMNFGaJR5AzmDAKLHVXeHypnqu6H++vLhC6WsMZ7Kt6abJqneNmsUmT2OSLgucO6Bv1VqeRSVNN1PqIPBXnrhYD1vyTtcLB3/reYl9Iz/lR8SNFqrdJKYAr4JLTincUFsja+qsf4/3u2uKF2Ji5b6H822CHrcCljKxmruXWuXXzEDxwDA7MU+Tf59PfUahYs25i3LxQKmZk2FGIjZXSaZwkpYD4BwPveNwdr+r4NI0MXVxQJoynvKbrYy127quJEySsPdgcokxAXtTMEzHnB+tbVKpK8ZwGIWhSUOA3rPw4Ie SJAHjnlL 6B/mPppLjw8zRgVsG1dNH4ad/7Fr0QPX0NLy/gRT6rRmiKro4SfD1NJ+4m4+VUsNrmgTtniuZsD0rUazwp/Q8RGlt6lc/Y7kCQgSmnt1u5/dfgymbwAFcxNSV3MHM27Z1Iooz7neFmj3SWb3w8F3TNyGpN0iBqhLVfSnu3WMXSsVVbIgyqvGyUW87e+xWjF/sXVcOqINSkur3+ltD5VJ8b/WsWZGXIBdNmrrJqjaMMSLY8HX5fjcjHXOfr4J9t/z8zOFbm757MDtjSwMVJtNQNevXllkvwv2zh+p9bTfSwSAZ8Ex97OeI58ARE05yYReEgVgE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It'd be useful to show Dept internal stats and dependency graph on runtime via proc for better information. Introduced the knobs. Signed-off-by: Byungchul Park --- kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 24 ++++------ kernel/dependency/dept_internal.h | 26 +++++++++++ kernel/dependency/dept_proc.c | 95 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 15 deletions(-) create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_proc.c diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index b5cfb8a..92f1654 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DEPT) += dept.o +obj-$(CONFIG_DEPT) += dept_proc.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index c92fe94..ede82cc 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,7 @@ #include #include #include +#include "dept_internal.h" static int dept_stop; static int dept_per_cpu_ready; @@ -261,20 +262,13 @@ static inline bool valid_key(struct dept_key *k) * have been freed will be placed. */ -enum object_t { -#define OBJECT(id, nr) OBJECT_##id, - #include "dept_object.h" -#undef OBJECT - OBJECT_NR, -}; - #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT -static struct dept_pool pool[OBJECT_NR] = { +struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ @@ -304,7 +298,7 @@ static void *from_pool(enum object_t t) if (DEPT_WARN_ON(!irqs_disabled())) return NULL; - p = &pool[t]; + p = &dept_pool[t]; /* * Try local pool first. @@ -339,7 +333,7 @@ static void *from_pool(enum object_t t) static void to_pool(void *o, enum object_t t) { - struct dept_pool *p = &pool[t]; + struct dept_pool *p = &dept_pool[t]; struct llist_head *h; preempt_disable(); @@ -2136,7 +2130,7 @@ void dept_map_copy(struct dept_map *to, struct dept_map *from) clean_classes_cache(&to->map_key); } -static LIST_HEAD(classes); +LIST_HEAD(dept_classes); static inline bool within(const void *addr, void *start, unsigned long size) { @@ -2168,7 +2162,7 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); - list_for_each_entry_safe(c, n, &classes, all_node) { + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) continue; @@ -2244,7 +2238,7 @@ static struct dept_class *check_new_class(struct dept_key *local, c->sub_id = sub_id; c->key = (unsigned long)(k->base + sub_id); hash_add_class(c); - list_add(&c->all_node, &classes); + list_add(&c->all_node, &dept_classes); unlock: dept_unlock(); caching: @@ -2958,8 +2952,8 @@ static void migrate_per_cpu_pool(void) struct llist_head *from; struct llist_head *to; - from = &pool[i].boot_pool; - to = per_cpu_ptr(pool[i].lpool, boot_cpu); + from = &dept_pool[i].boot_pool; + to = per_cpu_ptr(dept_pool[i].lpool, boot_cpu); move_llist(to, from); } } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_internal.h new file mode 100644 index 00000000..007c1ee --- /dev/null +++ b/kernel/dependency/dept_internal.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dept(DEPendency Tracker) - runtime dependency tracker internal header + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __DEPT_INTERNAL_H +#define __DEPT_INTERNAL_H + +#ifdef CONFIG_DEPT + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +extern struct list_head dept_classes; +extern struct dept_pool dept_pool[]; + +#endif +#endif /* __DEPT_INTERNAL_H */ diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c new file mode 100644 index 00000000..7d61dfb --- /dev/null +++ b/kernel/dependency/dept_proc.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Procfs knobs for Dept(DEPendency Tracker) + * + * Started by Byungchul Park : + * + * Copyright (C) 2021 LG Electronics, Inc. , Byungchul Park + */ +#include +#include +#include +#include "dept_internal.h" + +static void *l_next(struct seq_file *m, void *v, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_next(v, &dept_classes, pos); +} + +static void *l_start(struct seq_file *m, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_start_head(&dept_classes, *pos); +} + +static void l_stop(struct seq_file *m, void *v) +{ +} + +static int l_show(struct seq_file *m, void *v) +{ + struct dept_class *fc = list_entry(v, struct dept_class, all_node); + struct dept_dep *d; + const char *prefix; + + if (v == &dept_classes) { + seq_puts(m, "All classes:\n\n"); + return 0; + } + + prefix = fc->sched_map ? " " : ""; + seq_printf(m, "[%p] %s%s\n", (void *)fc->key, prefix, fc->name); + + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + list_for_each_entry(d, &fc->dep_head, dep_node) { + struct dept_class *tc = d->wait->class; + + prefix = tc->sched_map ? " " : ""; + seq_printf(m, " -> [%p] %s%s\n", (void *)tc->key, prefix, tc->name); + } + seq_puts(m, "\n"); + + return 0; +} + +static const struct seq_operations dept_deps_ops = { + .start = l_start, + .next = l_next, + .stop = l_stop, + .show = l_show, +}; + +static int dept_stats_show(struct seq_file *m, void *v) +{ + int r; + + seq_puts(m, "Availability in the static pools:\n\n"); +#define OBJECT(id, nr) \ + r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ + if (r < 0) \ + r = 0; \ + seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + #include "dept_object.h" +#undef OBJECT + + return 0; +} + +static int __init dept_proc_init(void) +{ + proc_create_seq("dept_deps", S_IRUSR, NULL, &dept_deps_ops); + proc_create_single("dept_stats", S_IRUSR, NULL, dept_stats_show); + return 0; +} + +__initcall(dept_proc_init); From patchwork Tue Jan 31 08:39:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DAFE0C636CC for ; Tue, 31 Jan 2023 08:40:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 135A06B0081; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 028996B007D; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7C7A6B007D; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 956AD6B0081 for ; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 60243160406 for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.29.54E9CC6 Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf30.hostedemail.com (Postfix) with ESMTP id 2CB448000A for ; Tue, 31 Jan 2023 08:39:58 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; spf=softfail (imf30.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=YWxyCP6k2sR51QC5g5Xns6OAkyCtz4mg3vnZr2yU7HE=; b=B2kpG2YpwIwk0yzUXnLr/a1pumwvfqHEEXK7RVsBj3byiWQyeOqq9aDUA6cSz6V74GEE9v uEOeV1hkcAKf040dZgFzFMi2MOPaxjTFirj+cnQ8Df47TBz7hSNyfD6HQGlfzUa8Xp6u+q 6z7vdcvS1yXe6IcRjr9d+nsc0P7U5c0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; spf=softfail (imf30.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154399; a=rsa-sha256; cv=none; b=S+fGopE6XKniJonknWLjPglcHwSZJY73wIdoBE8JvSVkeXwQHL7wUpmqyqIECHT2sDxAxD UA1WnKqSH31jN4Owv5CxBwPP5BnkIHgUcb22HDtsOeuyUCwdJZbJ0F0ii/yk+Kc74sjbwG 7piKuKwPNV+qzUpUJvZCjj1xJWkyyOQ= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:39:57 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:57 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 07/25] dept: Apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() Date: Tue, 31 Jan 2023 17:39:36 +0900 Message-Id: <1675154394-25598-8-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Stat-Signature: n1j1q1axgsrezmhib7der9fehc5chfhb X-Rspam-User: X-Rspamd-Queue-Id: 2CB448000A X-Rspamd-Server: rspam06 X-HE-Tag: 1675154398-659447 X-HE-Meta: U2FsdGVkX19dOm5CAzoQ4z3toGIRG4QqS8u64VTkJ1FA6VV+huFtnZeh8cu08L7GF/F2n4UEPu3k1O5Q8Y+rU4ZsnsjmQ7/b+0Bm4qunGqu8ZtZWYJgXwFdpkopZ0+DdVcom+uuITFL8E34h/lQzLojHDd2TmWI/ciAeCdTaVzz1WJBi1ZI39rR885gZaqd9Nz7xbYg4IitNCD5pnFTEHFPsRJjcvF2jefAm4DS9F5wI6PsSQEJUvYNkm1DILB9Iy+NEoj7xtLExfUwNPwSah7dQMjtrP7YiZk/cyLJljp1U8O2nngvgrbCimmQvEiAhUsX/a9moWI/+9Iu0AhuPcDQQm/DwlM30+H142i501S2Rsq92/DmUX1aZQtlpuzRV1IiF26zk430VNjsisx1u+NcZ+3M1Bt3t/1nQWgokIm1PyEpz275hZEsWWuRDAULf5lhasiTWtF0IImjWfFRhyl9yRi8b1zZLT/3vBsQmLxQPXUuuOhOsCs8QVZWeFbziKJHXBtNnsv8+1hfaOASEqivRO2JFuY18/bcnoxLWzFXXbUlr5JEQJkI0U7D4en6CVwSEumPrmlmPUAVUoePqKrZGUizpJkFylz0FIL7Ef+c6VkKbxZ73On4KttmLxXdB+bzFQpIXasLcD0ubM+8mvmMLapp4ub+z5+BueQS62l2yiqXI+7fQRdlhEFC05EzeSQyol2pcWZgtwTQUVlSaVFBBTYZKuKPrrdVRAeF0WA9xa2MlLDCI8hRRghB19uxrNhIUJG9K5FLP45AZprCKcNEVj2EG2Ubmc+FAx3FqEgeDcnzeGzrabIAwknj1Kp2lRBUtqfMChKk7Y3xfPxGh8i2z3TJmIAVToVqdQR234IZ1SKdAIOQMgV6JkK9IauAu+sxLoaCBzhqG8n5XhUEIH7N5+FgBgaTGKRMsq30WqXBSZOV2stEchT1eO4GgTsGO6h+HupWWwO2HE7mZQQp NuoMtiWj 7vfcZj4coisUrF5wyxJUn8cBR+ninVAzQXcU8LXfo+NyR1dC5EK5MHNMrkmuk1OQympVoqlOYcZBuxlJ2GV4ceTjSnHlIhApCUnTn+9ZRlLEH2l2CS3OyyCdsRdLH7kfqXBlc0lpq3Vyc/Ld8Ihjeluy0DXfYVX90LynuMGFcrCJSoA46D+5LDClhlYX1ykLmyVhd46CARRe5jWQYn6fNCPwRJd/5q+aep/c5cbJpDXIV4MzF3pPIV255lcNO2Utzwb76fzUejphwlhw4z7V1VfdritnzqZcafayTkr7YQJfgNoMx+I3E7Pxrk+WJ/wperclGtqfBOTa4SbrGKoE/oiQKziqUnQzTtbL31q9WAf7oawk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Makes Dept able to track dependencies by wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index 62b32b1..32d535a 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -10,6 +10,7 @@ */ #include +#include /* * struct completion - structure used to maintain state for a "completion" @@ -26,14 +27,33 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map dmap; }; +#define init_completion(x) \ +do { \ + sdt_map_init(&(x)->dmap); \ + __init_completion(x); \ +} while (0) + +/* + * XXX: No use cases for now. Fill the body when needed. + */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) {} -static inline void complete_release(struct completion *x) {} + +static inline void complete_acquire(struct completion *x) +{ + sdt_might_sleep_start(&x->dmap); +} + +static inline void complete_release(struct completion *x) +{ + sdt_might_sleep_end(); +} #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ + .dmap = DEPT_MAP_INITIALIZER(work, NULL), } #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -75,13 +95,13 @@ static inline void complete_release(struct completion *x) {} #endif /** - * init_completion - Initialize a dynamically allocated completion + * __init_completion - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized * * This inline function will initialize a dynamically created completion * structure. */ -static inline void init_completion(struct completion *x) +static inline void __init_completion(struct completion *x) { x->done = 0; init_swait_queue_head(&x->wait); From patchwork Tue Jan 31 08:39:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122432 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9329EC636CD for ; Tue, 31 Jan 2023 08:40:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D838C6B0080; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C542F6B0081; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87C2A6B007E; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6D6DE6B0081 for ; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4D58D1602B0 for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.04.340909E Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf21.hostedemail.com (Postfix) with ESMTP id 1D0FD1C0005 for ; Tue, 31 Jan 2023 08:39:58 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf21.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=J/LqlnG5zc04uMC4V9U7W+u/BW18Wd8gDOP8crsz6R4=; b=bB5grkK++QbOj4hV2tU7w2IVbNPaTLwaaAxuVc4r3YvtPoo2r1P17ZivRU2m9YWzNX4amL 5SHHpBKLnL2FlQFUBtIHkD1l24b6s9S61LYi8Syi6HZfYA+mY63BGxrSNcxLCEdgWkTcbF yqzWzH17rP+i2490pYV7lgSzoAxFgTo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf21.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154399; a=rsa-sha256; cv=none; b=ViXDO35Phcg/dVPDes23DOlgphygXNfeDe7c1S7AuPVvvkNo1pOOQzLKoMOybmY0i9zTsq PWb8r5aIxYoOykd+pSMXmKcmRZ7u6dUgIHeIqcJat4ATI7l/gHjIKtMO9gu/wvIuf1xG+k z9miz7Tc2VYaGMscDJ9xBtzoO6sjX+8= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:39:57 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:57 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 08/25] dept: Apply sdt_might_sleep_{start,end}() to PG_{locked,writeback} wait Date: Tue, 31 Jan 2023 17:39:37 +0900 Message-Id: <1675154394-25598-9-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Queue-Id: 1D0FD1C0005 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 75km8kh6ijsddhpnrgqeiedzd6zykoo3 X-HE-Tag: 1675154398-750623 X-HE-Meta: U2FsdGVkX19IgCRglulUQ63C5l2J2OxIebZk6Ppol5BlWGc9fwcJun+xB3ElBx5HgzWx8QW8JDLjl1Zy8NhFVbL+th5ZEZhzRVO3C5yXBdTYTbSKPEJPTOnvrPzNF7isFJJjDyHC/t+XCdQVGU0RcT+PZlyEC4Uk+6QPphZC0cbjpUELLm9uaM6LFCf8Y5k6acT8zPgIm4IIs6uYNN7NoReow+Fhy+xCuL9qqfUCNIQMGe793c9L7y9LVXnvO1BFdpCtv7EhJZg0tM6WCi0V2ogmfMN8v9RZiDo3vvI+OIJoetoyclYM5ZZ8XHWJVwqSy19n8nlmE5ua7CNUg1RanDpscs6JX1ItUyzfU8B2DoGVfXi/57RuAxUDo4laWkG62Gpb0EV/6YZ3BRtzTsqEq9qiDL2I7AiRCxKz5LIR/urUK2OkNF/poKdw6pR4Z8zaK+5630PUfP6PLBHV7trf86KZHiMnpB28Y2vcJ+Tk76DaaJmwv+L4eVRJDSu+2sM5rjG8oa0zF37hopIZpLsY6mO4wuTQMaAXAn1zPqvv3l6rdliuh7GrBus3hE9nMaDmJotN1kBiTmp3zab1SicOI5lcLTwD/cD6p+jt/xRXj1LP+pRUT5EeMRbsV6GDTqdhjLdztqeBmpmCcSlAu4kMiBBAamJdp15M4xn3m8aEQaCPcXxeIn7O/16q1t5NGHkcFb7vKxhiq86m9RDNg+h68Gf0bk7+2AB9ZSfcS02UgfQyXlyQUyFcoc4O+Bnn4PEMB3v0nn252sRc33k9h7dW162MdRYycoPxpDHUl8lx9TxCaSGEttPqyyjIZcpp/S35cVB+lRj5ZjL1NV0M3czDYuMEU1MEcLDGZqQ4lmUWuUR8LE6tC4KkovAycPR2/Flhy9UdkGln3KXJwKR1wC4LIEeBmpyCh4zT/4ViXEGTPSyXfrT5dXszhga7SGzbCPHjLR4OrIv1/AkOWQGJS3o 9Z7NjFQn MWxWF1aq2CTo9yV1xTS+c6fEyHSP57itKV06jKhJEg43Z9aftjPCuld0n4rWMxhWH/rFKucPZSHJGGjcJ60wujltGcJ7WANutZrQ0xKiHym6LKlBnqzy1nwHwweQ/BEcGxYWc/AS499kur4X2SO8tzxqUwHRpcfKKXGgEe1cVYhR/lgXzWIomiMjrLweTNzHSWNaQBczm4a/4xHyd4hqskfKYDiQgtWIVDbIS8e0xTqxJzbeeh77jTdTJxAGeNrkYoy5efj6tKTmooo5/PRoN2pJt5x/Om0u81st6nJnI+hR/BYrPGqJTfP8rlsbxL7V61PyYfUeq563otWHhyh+YIQBnd+QaiJMr/LHVNpDPQ3JEYrtPWwfIzzPYIB0QsdixUlWpA7XNyvkzImebjbBGX3AJBmII+duvGLz5vcYtKEKcgJEqEw0OiL+oyMRfeOKsHmexXJ6gJGP5HZaXcie0w0WgQsNvIbTZZQ3ljzWPLrGvPmw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Makes Dept able to track dependencies by PG_{locked,writeback} waits. Signed-off-by: Byungchul Park --- mm/filemap.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/filemap.c b/mm/filemap.c index c4d4ace..adc49cb 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -42,6 +42,7 @@ #include #include #include +#include #include #include #include "internal.h" @@ -1215,6 +1216,9 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, /* How many times do we accept lock stealing from under a waiter? */ int sysctl_page_lock_unfairness = 5; +static struct dept_map __maybe_unused PG_locked_map = DEPT_MAP_INITIALIZER(PG_locked_map, NULL); +static struct dept_map __maybe_unused PG_writeback_map = DEPT_MAP_INITIALIZER(PG_writeback_map, NULL); + static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, int state, enum behavior behavior) { @@ -1226,6 +1230,11 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, unsigned long pflags; bool in_thrashing; + if (bit_nr == PG_locked) + sdt_might_sleep_start(&PG_locked_map); + else if (bit_nr == PG_writeback) + sdt_might_sleep_start(&PG_writeback_map); + if (bit_nr == PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { delayacct_thrashing_start(&in_thrashing); @@ -1327,6 +1336,8 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, */ finish_wait(q, wait); + sdt_might_sleep_end(); + if (thrashing) { delayacct_thrashing_end(&in_thrashing); psi_memstall_leave(&pflags); From patchwork Tue Jan 31 08:39:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36D5FC636CD for ; Tue, 31 Jan 2023 08:40:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F0536B007E; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 68E5F6B008C; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 092E96B007E; Tue, 31 Jan 2023 03:40:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C51A86B007E for ; Tue, 31 Jan 2023 03:40:01 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9C0AEC030A for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.01.B347AA2 Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf28.hostedemail.com (Postfix) with ESMTP id 63715C0015 for ; Tue, 31 Jan 2023 08:39:59 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf28.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154400; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=f2yoJfgpaAqeAcS022E5rJAY5Fzp6QDkzYTsXafUzh0=; b=kXLwmt0BqGBEfKLZTTsSWo+uUioY4BSMxQIJEtSpsmN0Fm3/8JqGV1tPX58fbDR3At55Ce rNuOaJfgM7oGwpRruhEKsr/Py7hgq8wT5qE+D6VEcHJ8Vq9cJmbVMHU3jtiFSQ0Fy32G/o awQqjmEwG43ZvwoLFvBU6cGIOwk8i0k= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf28.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154400; a=rsa-sha256; cv=none; b=d86nKmzTYe4uye1v2BqUSRULy6CU5UoLUabwzkJokOgUYyRmaS9QztBGnswoqeVntQyCuj W/tAM7zB/pFZZVvEvXZBDt9RQ0JHgz0tUBlAIHYyG308Q81f0A8qwx9foWiq00Mpcupxgn uLk4U+tjsQE07pIdkdw+WtOhe65w5ss= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:39:58 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:58 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 09/25] dept: Apply sdt_might_sleep_{start,end}() to swait Date: Tue, 31 Jan 2023 17:39:38 +0900 Message-Id: <1675154394-25598-10-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 63715C0015 X-Stat-Signature: 3qzdkw1hgr9imbtp75u1knpcdkcfxj1u X-HE-Tag: 1675154399-685122 X-HE-Meta: U2FsdGVkX1/RaTnjjI9o5IiJSLmYmJuFmLEocJQIo0LtGbaglbM09ZIKYx0KX9WChXiOgNwdv0QjmzJbjuHiJI0YJQem/7ENtp10j8qxotsrPHf0NXsfSbBZLLtyrwS3oiBKb/fOh3Gia5AuXIKhwPSs0LdWyoI+fdyj34D4+k3vExw6eL/ohi3FdCo/Y873VJf7QAER4LFQ8Cwhgyur5bfo0CgubTSpYIZf4Al6NccDmk9uHIXB94LLSeRKeCTqXhCfU9FjFMza0SAATkTlAjQxJWldGDf1HiJzuHLX5xMERVlFtpgwG+ot4DoqXB8Z818EwU1KLLtFn6h3ri62SXkYMutuycSVkqhtGGjIummrmkavaDyR0XNOsEC/Lty9OAs2yPCMcubdIGT+dcw0qlLa8qlXBjkJrIoDN+5igJ1+6bpnSPxbpTZiEeHBIQxHWfpMtWaRjFIWAY1FXk96P9N1h45hYJaSfmZJ3ugV1vsqd9681mnBFEpePWkCadxeTX43swarzwgL6U6JND6B6vVgcHUxpOJwSuxcpBec1PSBXPQcTSFhJcfFGZqY3vDP8gBnRPtPGqDVkUVcYB2jh8swTHyg7t1u3HNjHguCN/UWytf9dpwe3lUHA5q3QFNsWdxFQhyFhNmwyQ92v7TYJx0ssHzAOlQ9nIOMCip+TOCywhtEHHozsdtKAqjKL7BzMZonUKyJAFnv5fPEYEsCNGhsJXTzWMSpKmBxsw1z4/ORpyo94ITI/jVBfDlW2YfEZsiD1mP2Mem+fXudP7EKEcUjzCvs95Zq2S0biSW3PjlkOTM7ERr6ZfBpP6HVtN3zoJjPxX5ShD2FXTpagJ8pSRi8VALVBI7UwQUvteXgA3dZXt8lpM53/hwQNOJqXuX7jNgzR/+zDLeoesl206jhlyWbHlLZoY5sb4JyAOtIbcxo6p48mauQr58ZCsE/l60J1NrZ/Ca8JXI/tT00aQ3 zAoGc+wO MHy69evX+/6PSL49+Pz8f8DkSoGgEjq1oA5dxDP2WSfNdhrHST7r62lOQ81WrOcNKxcXTllsMPFRf8k8JqLKXtfk7SzN29FirxZRTnnkh2vKe4eOlbyMOn3Ksjcnhc4nK/kXzYSLIINMz4YZWwXZBfMfzctpPMCMikejGsev8koKv3wtqd+lVjs/zeoSVua8jZ9xxNCeXd+DwcyR9neuvgvUw8Ky+bMlNvPfdBUg0y0/fO8oUt3VcGed5wje0S1VUr5QqB8YdFuO2u7QrcCXOz3G/yCEqtUa8BhmTR7vdnF7xZD/fyqNMrGjflanOmnCeax6e/w9PwPSxGvH4g/N1t+0Q2NZJCn6AJZsQt/QHXt9JpNeShsVccpOjANQ5FSuK05HV8w2Tae8bwWyggIqDbgGYaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Makes Dept able to track dependencies by swaits. Signed-off-by: Byungchul Park --- include/linux/swait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/swait.h b/include/linux/swait.h index 6a8c22b..0284821 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -6,6 +6,7 @@ #include #include #include +#include #include /* @@ -161,6 +162,7 @@ static inline bool swq_has_sleeper(struct swait_queue_head *wq) struct swait_queue __wait; \ long __ret = ret; \ \ + sdt_might_sleep_start(NULL); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ @@ -176,6 +178,7 @@ static inline bool swq_has_sleeper(struct swait_queue_head *wq) cmd; \ } \ finish_swait(&wq, &__wait); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Tue Jan 31 08:39:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 721CEC38142 for ; Tue, 31 Jan 2023 08:40:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F261A6B0089; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DFF5A6B0087; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6FD646B0088; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2B7E86B0087 for ; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id F3D641405EB for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) X-FDA: 80414446602.28.6A72447 Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf16.hostedemail.com (Postfix) with ESMTP id B40FC180005 for ; Tue, 31 Jan 2023 08:39:59 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; spf=softfail (imf16.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154400; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=ooQM6sVKv/AbGeYgDV1oAafMBNKP6iHIYrOBZ5wL7lw=; b=KMrbEt7DAI8AaDBINEv6OE4Yg52cVBItaVQGdD5wfZy1Mankei9QoPOdn4F1QDdLRKGt9e fQz9i5kZAz7DQvgrcDxg+/I+G9ndQKBW63Yi2aCwz0cg03NAwEWUffPG2riiGH9omL8WIA 72x00ZXIWsKsjIa6mqiI9OSg6H386cc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; spf=softfail (imf16.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154400; a=rsa-sha256; cv=none; b=JxU9CAAD1u/qUHVWMY/XkGmAt9bxocCh1/vTrOAku5AwKeGa2fAY29rcSu4so9+h/GWGAi 3V9R+rY18Et4DbN9ZlLQKc5+PkDqlrh3QUkZNjYaWM1PCs0U4UX2lUQztZFgz+xZEYn7Hy TWqUgGLzv4p7fVVv4SHdPrBvYpXOk+8= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:39:58 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:58 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 10/25] dept: Apply sdt_might_sleep_{start,end}() to waitqueue wait Date: Tue, 31 Jan 2023 17:39:39 +0900 Message-Id: <1675154394-25598-11-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 85krapoj3ow45dy9uin6cypfxm3qkd7r X-Rspamd-Queue-Id: B40FC180005 X-HE-Tag: 1675154399-719539 X-HE-Meta: U2FsdGVkX1+/uHgf4JzH3oim4T2zOBQKSzr8RisJ2+4lc+FStBOzKYHYxoVnPG19YErTGPlnUnRFHoJCwrj+eaRUXHiJV35JBm8N9Upr5QZZ6NXW4N8wuMRjlYfFlaoENxpmFeDE+5HuKIT7jqkU7Kdyo6mW3kbAtm02OOHJohhfjLSleVeO4+HJkpmrk/92rld4dYd1yolYbljjI2w9qd5vqn4RzzpUycbYfh4I1/1/lydjRVvm3fmqnH/WPgQLQIwOuFeaE5lOSJVySVJIe392Sgy3ZI0IKDeAuzX+lr/lB/mJ2Rx0LldoO0OuwK21TlIu+4nMYfLaf5Uza03JZDi1DQkWU/Z2abF4b87vjzKEGFQ73j52fn1Mr92C7oFmFh5eDVYhqDpWM0TweEYL9TMpWt2CScjB3hgdPn3pzfl427rPtgYrIbFFua66WNbBLtZNJDUO/vdyfuzj93xmLpoVR/r+FPkwZx/wpAAJS5c2uoRkleLuuKvZ4XU+WB4BkdhMTA695h9aiod/I2S7D2diOFGbPLBmjGAXofrLfxLZOuBdWeit3E17Hyg4VEQzReyEyIxGrNcvSj4GDnS6nQ4OAHTW+hIQ402xffshplHRSb5PZ7eYYBINx/PqMXz4FqaX5Jaul4bah5GAnI3YUFS6qbnTHJCvd0LhkXqP7ZgtAYTTwnX043z5hRmyEg/ecNeCZYYZn8iQHuyn87EnRN+fA4loQue+j2btMXIVt/LHhFwy1ZZqop0Aqv8lh/ONWDbFkZerVZuBqCiMuqw7HgNvKlFH56zeaUi1DSJDNfepJzRwmVnY1oKdz66T6N0VpNqzB3G1yRyFIrQ6djTvNR0Q0+auu0AcwQzKdV96Lj44OhNB5xWQtMNwIm/YCzolKvzbp1BB88Bly6LJw2mdRoIM5f1oTdBH53bDliw2g74iKgjsWdL9TLBmCWQCDrV9ix3IYFiLrRNQR0YRhzR uQNCg3I1 m/ut+39t5VhFq3SH3Chy58wrlR+eus4alXq669/AIElm5bzfttY6JfoqggrwBaCET7LzAQNi1ecj0m1BWAPZZIm+5CkVZcDmjv/pUtmSh8rOReAu1OkN6aIUoLPTH22BsOWzvGyv4MJkJyPbE71dIdiiteNKr85v5M0xR3QRu518/9guODF1Hu+senXZ+6TmfvbjxXgl+fnKNrft4tNKV6Y0eRZv+0j4kS2KKAckTaCUU+frOT/TFadddAedLpacWx1AaAJhXnUjte+/CnW72gfsoka3+HRa6FFwIy28BkUwEMkionNhGgweCOqmYz70xyVQqouKKx7fOS7AMW/yd0ZRTKVPEWAljwBzJr+L5msZV07zlD+XOaRYMTUj6tNbHZ2TuQiY1BErn+jmBROZWfT4iFw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Makes Dept able to track dependencies by waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index a0307b5..ff349e6 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -303,6 +304,7 @@ static inline void wake_up_pollfree(struct wait_queue_head *wq_head) struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ @@ -318,6 +320,7 @@ static inline void wake_up_pollfree(struct wait_queue_head *wq_head) cmd; \ } \ finish_wait(&wq_head, &__wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Tue Jan 31 08:39:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122438 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD0C5C64ED8 for ; Tue, 31 Jan 2023 08:40:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26B296B0083; Tue, 31 Jan 2023 03:40:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A1ED6B008A; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CCC7E6B0083; Tue, 31 Jan 2023 03:40:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 53E7F6B0082 for ; Tue, 31 Jan 2023 03:40:02 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 36883AADB7 for ; Tue, 31 Jan 2023 08:40:02 +0000 (UTC) X-FDA: 80414446644.26.157FE57 Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf30.hostedemail.com (Postfix) with ESMTP id 11C2980007 for ; Tue, 31 Jan 2023 08:39:59 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=none; spf=softfail (imf30.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154400; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=YFB+mTTRMO0UpYua4CshxSpPgO7v1xqcrwX+V2Zz91Y=; b=6bpyfiGJRUMrKJTV6f9qFKZEoCf7rcEgpNWtVqckHd+hP5w6SPaby3/4MPDAVSx7aU2i2t P062f0kUNI8b52VrLyr8apqAZmOdL5tW/o1S8XUJxNAZjr6NvRU3zBP1pLN/8NZ2f18OZn INnoARNo+4iGz/9ue78jZzqJujSZiok= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=none; spf=softfail (imf30.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154400; a=rsa-sha256; cv=none; b=UDRejCRIhSyRUgYrEzs6bNIojWbt4ROI2UToehlDDQ9bRi/ljPQx2+3PGMf6yInhY9n+02 qfC5lOFu+dPdRXwTT+ojrVwGFH+n7o+SGt1BecvWImqBaxUKx0ALcgdu8mT9Od6/PXm+t5 y+lzCos0fmhk0467HggKHOv6pws+F6U= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:39:58 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:58 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 11/25] dept: Apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait Date: Tue, 31 Jan 2023 17:39:40 +0900 Message-Id: <1675154394-25598-12-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Stat-Signature: krezkwypghiw38qoytbnpnh14mdyecb7 X-Rspam-User: X-Rspamd-Queue-Id: 11C2980007 X-Rspamd-Server: rspam06 X-HE-Tag: 1675154399-691389 X-HE-Meta: U2FsdGVkX18ydb/nusrtbK3Hr9OPRBe671IE46mH5UsAMXUKXaL207n2tVvknfnc+R3R44ASsPiygtS67AgyEnRVjzQDF3JyEIMNkBUEKo3o3jly5AcJ8LI0dR/y2q1hPE+Lek0of0lv8UNHF4lhhCcA8JMy2r6FMUHDmYyRDvIHCtvgP1fRYkhSQ5rloX/8pY+ihd6pz5TYqt0yJ3Jtaioy0k8Ndzyujbw3h3e/DQ+5gAwOTapk2RGbKMAyIlgoD/cdf0OGUYP7kQjH+rhjHR+Grp26JY9AHgM78i3Ohu5VS9Ro6SoKQtpKLP8P+m6MV3LcS8bJYul6RBoglo/58dlcHpTle+7xua0ft5dL72TdITyzju6QyJiJ5NHmt/WvTkPMUSz9MCXoLQlJWAXp7qORGE9X7JGnOZEXNmhjTK1lOwPBNyVRiaMt5eT1njV4OkjF+JpMQIf54iJ1zOgFRPfGU4veMeUgH6J6pithmG1Cv5DPDC3p+Q+YJT8YMNTIyhvGGTz+eXqosLiXsCoL3ZEUIU7PpgqyjdkZuhTrLjorehQgz0Zmv0iZzUJVdmKNP7mF3bj5csNPJsV9QZCXo2kR40gEd+LI1jKB7ylxsmwwEkDgdZhfde99E/riT9oqzCS8jwbuiI42INuX6kqrqXWsemqb+nKHu+6v8PCtrAu4fN8OmhFvtEhbLjPcAcwFlRJ8TPxXDTMwgkeg57W2QsjTCE5LJCEjq4TwBrycYVMPQOO4XqTT5NEVwm5zV+tiXEYHBugItT5654gFr4832c5iYe/3FS4Qqkst9ZTJcpIeidkSk3bYDokE52RseoUoEiWXp1I3n6cRqdfmKNhmHa46XOIYLFI8X6OOyFGYxDdAI36C6Vhyj6nBcxDYVPRAftGes1ffABtyW93RsnXA5r/tS/TjR3171aWVLtXn0Mu8gH8sTAHKbzUvgtEAga2y5yGMfGSty3uBE+LCx13 8UPxaX1o XLluTMtPGGYj9BZ5ZK4ayDA4i2WEqRQYyfMhSWeNh+qr4w8TEximC+66Ci8dT2Y+Q6ph+QC4Sux6zirl0GJKV6bw1gx/2d4/XBaqBdMCa/hmLNuiT7LaQwkCnL6exN1p0SqSq54BwAab0Z/KoT1QOecHfW9PPlfTTpDwZGxIEOjoThSyeM893cHY+bB8Xsi8VfVzY2Z8PWXFpdWbJtSB1cNcduVjpQ7YEM23NWAXB78TWKcp8OrrO8U2ADnExvbpTgso3as1tv16xNz4tFeRgYXGTFtAh/pLw0xKES5f3AAVmLjwHMS0EFH9nHdmLeYVO1Wg1RIORQxOoEqpuWv8NuApgLZpIRZWgm3d5dZDpWb3Wcog= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Makes Dept able to track dependencies by hashed-waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 7725b75..fe89282 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -6,6 +6,7 @@ * Linux wait-bit related types and methods: */ #include +#include struct wait_bit_key { void *flags; @@ -246,6 +247,7 @@ struct wait_bit_queue_entry { struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ @@ -263,6 +265,7 @@ struct wait_bit_queue_entry { cmd; \ } \ finish_wait(__wq_head, &__wbq_entry.wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Tue Jan 31 08:39:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C383C38142 for ; Tue, 31 Jan 2023 08:40:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B43FF6B0088; Tue, 31 Jan 2023 03:40:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AF3DD6B008A; Tue, 31 Jan 2023 03:40:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 996406B0092; Tue, 31 Jan 2023 03:40:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 825FA6B008A for ; Tue, 31 Jan 2023 03:40:03 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 476F3140337 for ; Tue, 31 Jan 2023 08:40:03 +0000 (UTC) X-FDA: 80414446686.14.B0A4EAC Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf14.hostedemail.com (Postfix) with ESMTP id 6A126100011 for ; Tue, 31 Jan 2023 08:40:00 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=softfail (imf14.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=XOMk1bfmKx3HAbPUprCHIfabBqGgr5qu2VjXUgG/rnE=; b=MTk6Yiv6Uy2q48MVE/XZxoCJ5km7+mH9vEhT6iFbrrYpiQTR8n229KT26VCQfwxCdsXhQD 0wkhFb5AQKiKchDH7/V1i2Nq7cVE7UfVLVU5jolmQlO+9ueisRMIqlOj8BGPJ/Kku6BgNb YgAJoMfqIi1ktgcFbdaZHlhL1K+NGjo= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=softfail (imf14.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154401; a=rsa-sha256; cv=none; b=yHC/ogiT3/9ItJ+ifeMkPQF89OFmggUnr8bmn2F1KpDGriwbi45vbw3YJSqzz3pyVx2sYf XiNn/yh+51K2rmIzOGENK4euV6nQImGxyKa9NhS5KZooyPNuTJSrFZ2Ctw1qAPeo1qwWo8 EaLgU9HKLHP+RGvqen9vj2zhEbVGYPw= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:39:59 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:59 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 12/25] dept: Distinguish each syscall context from another Date: Tue, 31 Jan 2023 17:39:41 +0900 Message-Id: <1675154394-25598-13-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: ogw4fbn5k7phs1wzenb7shzrnofttb1j X-Rspamd-Queue-Id: 6A126100011 X-HE-Tag: 1675154400-844253 X-HE-Meta: U2FsdGVkX1/5Zn1l79QLBXB6GtLFzbURtD8hcDIfjSqL1OWcOGAQRkviDk97iWvJxb0qhje5+DrPo7NyybQnovqz0FnxTzZpr5u0INpmY0H49Uvc8jvNpa1jUuyZ52GnXWgWnobORd2ozoRF6nuNkKiTplJtetF2JikRGknMUVEYu9hJiJ3SD2/mN4neLqwwBDZprgESkj6hcz8iDYsEs6891GTqtD/fWiXROFN++4/GYFAltcLU8snzueeJJKEJjD/7p+SJmhcLNz4JkVuVfzvpss/V3ToIhDOKSMrD+NIIebpLUSyuo6lLTs62crnHycNIzpmWdmV3uCM+cfuZerGCScwfdB3b7DbVVH/ZhRAfbDvzZ9axhr5Jlz7KQVfxhy7kIKV+I/q439F8MAey6hRyoeafz0Epp9En+KW5gYA1dLhWTLwIq/Cv1WYPsslu23SbLdqHyfuIOnhP+BTEEWKwkLP08X03IitNxJZxdec7wgSrO4sBciEUPITNJemRUwGTbOYpS1K2Zd1QYGxrSzC9tt56xoWv7lgnVV6a78ewiHEdIhbZsGMWifCrXnsfQ1IM9gQ7joXF/cMx771ajg+IEJvx2Y8x4V11otXcqMSciBWsqXX6hzZKQoA67sm/i796ibd2puaPPDzcLQFIsfZ4iy3ELuBCDob2HA7qb7BYb4GMFibjT2BC3OC4rXg0urWRq5CJM/YD/3kpqcsTyGbkchJhvLxW2MNP7AKKBrA876xWVE0kk7ZARqsq32FmLrkbfrQJkeBzJRiTGpJALDMDpTM8IfUgLamyvZMvOOROoz+ZZEeLgvuQatVb/uY9q6fuhTViclLbht/EQbW011nNQFUJeQ7DEePCHOIM2PlX0aJvv359pJxgbIJeA4Uk170V1Rltg1Kzqnw3sPnz0OIQixalUpvWH2ZfmZSSLUCljl01j+PjtMtrqJfobAe37MAYgcrOOVJLSBh4Ucz 2NGS+k3O eZmhofCOr2Uc/gf6FjGWMi855Stg67z2N9k/gwULZcT9hEjhTu3r1e3eHO+xkcbZ8vLJq9Ck8gOlJjM6oXJxmKERcE6hXSt6/wRUZHCnDz7lM+r0s7Bit0iDUGwLEdAPcC0UMoNVY7IUCeVgfb+POOom93fSLLrstH/uJyphXY8N1ySed9nfxw5KJZFy3YFWXCyPGf3ie7eKudOx3ZC7ChednOYHiLJw1O9x7HyfEwq/XmRc6m0f6tak5jgfrS7/q3yqwKIidPGfdNme274lMnIV+fT8+2M0skvrOwFl1X9jxqvKd87tmH5DqpyVB+8ZzP42GFAql4r7x7l3w226I6mKmEa1cFqjcktlohywptotUOirmEQ1tGjh74XYkbfeVWT+zntNT/22EUMLwysqgyAcb8i0yXEtYznbgn7AtRExvI2g= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It enters kernel mode on each syscall and each syscall handling should be considered independently from the point of view of Dept. Otherwise, Dept may wrongly track dependencies across different syscalls. That might be a real dependency from user mode. However, now that Dept just started to work, conservatively let Dept not track dependencies across different syscalls. Signed-off-by: Byungchul Park --- arch/arm64/kernel/syscall.c | 2 ++ arch/x86/entry/common.c | 4 +++ include/linux/dept.h | 39 +++++++++++++++----------- kernel/dependency/dept.c | 67 +++++++++++++++++++++++---------------------- 4 files changed, 63 insertions(+), 49 deletions(-) diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index a5de47e..e26d0ca 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -105,6 +106,7 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, */ local_daif_restore(DAIF_PROCCTX); + dept_kernel_enter(); if (flags & _TIF_MTE_ASYNC_FAULT) { /* diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 6c28264..7cdd27a 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -19,6 +19,7 @@ #include #include #include +#include #ifdef CONFIG_XEN_PV #include @@ -72,6 +73,7 @@ static __always_inline bool do_syscall_x32(struct pt_regs *regs, int nr) __visible noinstr void do_syscall_64(struct pt_regs *regs, int nr) { + dept_kernel_enter(); add_random_kstack_offset(); nr = syscall_enter_from_user_mode(regs, nr); @@ -120,6 +122,7 @@ __visible noinstr void do_int80_syscall_32(struct pt_regs *regs) { int nr = syscall_32_enter(regs); + dept_kernel_enter(); add_random_kstack_offset(); /* * Subtlety here: if ptrace pokes something larger than 2^31-1 into @@ -140,6 +143,7 @@ static noinstr bool __do_fast_syscall_32(struct pt_regs *regs) int nr = syscall_32_enter(regs); int res; + dept_kernel_enter(); add_random_kstack_offset(); /* * This cannot use syscall_enter_from_user_mode() as it has to diff --git a/include/linux/dept.h b/include/linux/dept.h index b6d45b4..f62c7b6 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -25,11 +25,16 @@ #define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) #define DEPT_MAX_SUBCLASSES_CACHE 2 -#define DEPT_SIRQ 0 -#define DEPT_HIRQ 1 -#define DEPT_IRQS_NR 2 -#define DEPT_SIRQF (1UL << DEPT_SIRQ) -#define DEPT_HIRQF (1UL << DEPT_HIRQ) +enum { + DEPT_CXT_SIRQ = 0, + DEPT_CXT_HIRQ, + DEPT_CXT_IRQS_NR, + DEPT_CXT_PROCESS = DEPT_CXT_IRQS_NR, + DEPT_CXTS_NR +}; + +#define DEPT_SIRQF (1UL << DEPT_CXT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_CXT_HIRQ) struct dept_ecxt; struct dept_iecxt { @@ -94,8 +99,8 @@ struct dept_class { /* * for tracking IRQ dependencies */ - struct dept_iecxt iecxt[DEPT_IRQS_NR]; - struct dept_iwait iwait[DEPT_IRQS_NR]; + struct dept_iecxt iecxt[DEPT_CXT_IRQS_NR]; + struct dept_iwait iwait[DEPT_CXT_IRQS_NR]; /* * classified by a map embedded in task_struct, @@ -207,8 +212,8 @@ struct dept_ecxt { /* * where the IRQ-enabled happened */ - unsigned long enirq_ip[DEPT_IRQS_NR]; - struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_CXT_IRQS_NR]; /* * where the event context started @@ -252,8 +257,8 @@ struct dept_wait { /* * where the IRQ wait happened */ - unsigned long irq_ip[DEPT_IRQS_NR]; - struct dept_stack *irq_stack[DEPT_IRQS_NR]; + unsigned long irq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_CXT_IRQS_NR]; /* * where the wait happened @@ -406,19 +411,19 @@ struct dept_task { int wait_hist_pos; /* - * sequential id to identify each IRQ context + * sequential id to identify each context */ - unsigned int irq_id[DEPT_IRQS_NR]; + unsigned int cxt_id[DEPT_CXTS_NR]; /* * for tracking IRQ-enabled points with cross-event */ - unsigned int wgen_enirq[DEPT_IRQS_NR]; + unsigned int wgen_enirq[DEPT_CXT_IRQS_NR]; /* * for keeping up-to-date IRQ-enabled points */ - unsigned long enirq_ip[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; /* * current effective IRQ-enabled flag @@ -470,7 +475,7 @@ struct dept_task { .wait_hist = { { .wait = NULL, } }, \ .ecxt_held_pos = 0, \ .wait_hist_pos = 0, \ - .irq_id = { 0U }, \ + .cxt_id = { 0U }, \ .wgen_enirq = { 0U }, \ .enirq_ip = { 0UL }, \ .eff_enirqf = 0UL, \ @@ -509,6 +514,7 @@ struct dept_task { extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); +extern void dept_kernel_enter(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -560,6 +566,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) +#define dept_kernel_enter() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index ede82cc..dd90746 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -221,9 +221,9 @@ static inline struct dept_class *dep_tc(struct dept_dep *d) static inline const char *irq_str(int irq) { - if (irq == DEPT_SIRQ) + if (irq == DEPT_CXT_SIRQ) return "softirq"; - if (irq == DEPT_HIRQ) + if (irq == DEPT_CXT_HIRQ) return "hardirq"; return "(unknown)"; } @@ -407,7 +407,7 @@ static void initialize_class(struct dept_class *c) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iecxt *ie = &c->iecxt[i]; struct dept_iwait *iw = &c->iwait[i]; @@ -432,7 +432,7 @@ static void initialize_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { e->enirq_stack[i] = NULL; e->enirq_ip[i] = 0UL; } @@ -448,7 +448,7 @@ static void initialize_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { w->irq_stack[i] = NULL; w->irq_ip[i] = 0UL; } @@ -487,7 +487,7 @@ static void destroy_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (e->enirq_stack[i]) put_stack(e->enirq_stack[i]); if (e->class) @@ -503,7 +503,7 @@ static void destroy_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (w->irq_stack[i]) put_stack(w->irq_stack[i]); if (w->class) @@ -652,7 +652,7 @@ static void print_diagram(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { if (!firstline) pr_warn("\nor\n\n"); firstline = false; @@ -685,7 +685,7 @@ static void print_dep(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { pr_warn("%s has been enabled:\n", irq_str(irq)); print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); pr_warn("\n"); @@ -911,7 +911,7 @@ static void bfs(struct dept_class *c, bfs_f *cb, void *in, void **out) */ static inline unsigned long cur_enirqf(void); -static inline int cur_irq(void); +static inline int cur_cxt(void); static inline unsigned int cur_ctxt_id(void); static inline struct dept_iecxt *iecxt(struct dept_class *c, int irq) @@ -1459,7 +1459,7 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) if (d) { check_dl_bfs(d); - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iwait *fiw = iwait(fc, i); struct dept_iecxt *found_ie; struct dept_iwait *found_iw; @@ -1495,7 +1495,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, struct dept_task *dt = dept_task(); struct dept_wait *w; unsigned int wg = 0U; - int irq; + int cxt; int i; if (DEPT_WARN_ON(!valid_class(c))) @@ -1511,9 +1511,9 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; - irq = cur_irq(); - if (irq < DEPT_IRQS_NR) - add_iwait(c, irq, w); + cxt = cur_cxt(); + if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) + add_iwait(c, cxt, w); /* * Avoid adding dependency between user aware nested ecxt and @@ -1594,7 +1594,7 @@ static bool add_ecxt(struct dept_map *m, struct dept_class *c, eh->sub_l = sub_l; irqf = cur_enirqf(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) add_iecxt(c, irq, e, false); del_ecxt(e); @@ -1746,7 +1746,7 @@ static void do_event(struct dept_map *m, struct dept_class *c, add_dep(eh->ecxt, wh->wait); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_ecxt *e; if (before(dt->wgen_enirq[i], wg)) @@ -1788,7 +1788,7 @@ static void disconnect_class(struct dept_class *c) call_rcu(&d->rh, del_dep_rcu); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { stale_iecxt(iecxt(c, i)); stale_iwait(iwait(c, i)); } @@ -1813,27 +1813,21 @@ static inline unsigned long cur_enirqf(void) return 0UL; } -static inline int cur_irq(void) +static inline int cur_cxt(void) { if (lockdep_softirq_context(current)) - return DEPT_SIRQ; + return DEPT_CXT_SIRQ; if (lockdep_hardirq_context()) - return DEPT_HIRQ; - return DEPT_IRQS_NR; + return DEPT_CXT_HIRQ; + return DEPT_CXT_PROCESS; } static inline unsigned int cur_ctxt_id(void) { struct dept_task *dt = dept_task(); - int irq = cur_irq(); + int cxt = cur_cxt(); - /* - * Normal process context - */ - if (irq == DEPT_IRQS_NR) - return 0U; - - return dt->irq_id[irq] | (1UL << irq); + return dt->cxt_id[cxt] | (1UL << cxt); } static void enirq_transition(int irq) @@ -1884,7 +1878,7 @@ static void enirq_update(unsigned long ip) /* * Do enirq_transition() only on an OFF -> ON transition. */ - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { if (prev & (1UL << irq)) continue; @@ -1983,6 +1977,13 @@ void dept_hardirqs_off_ip(unsigned long ip) } EXPORT_SYMBOL_GPL(dept_hardirqs_off_ip); +void dept_kernel_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + /* * Ensure it's the outmost softirq context. */ @@ -1990,7 +1991,7 @@ void dept_softirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_SIRQ] += 1UL << DEPT_CXTS_NR; } /* @@ -2000,7 +2001,7 @@ void dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_HIRQ] += 1UL << DEPT_CXTS_NR; } void dept_sched_enter(void) From patchwork Tue Jan 31 08:39:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98537C636CD for ; Tue, 31 Jan 2023 08:40:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D6586B0087; Tue, 31 Jan 2023 03:40:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 687096B0088; Tue, 31 Jan 2023 03:40:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 54EA96B008A; Tue, 31 Jan 2023 03:40:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 30B466B0087 for ; Tue, 31 Jan 2023 03:40:03 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E3FFBA03B8 for ; Tue, 31 Jan 2023 08:40:02 +0000 (UTC) X-FDA: 80414446644.17.D0C6BFB Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf10.hostedemail.com (Postfix) with ESMTP id B84C8C0021 for ; Tue, 31 Jan 2023 08:40:00 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf10.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154401; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=U3VZbsvkCtR5jiGjYPucAe8KHSg+BSaRd3Oz0QAcyXI=; b=0INhHMgbdPZKrkfm9gR07bzQw4WaTdDGPicPdoH3t0PHgOnub5T554+61Ne9QhqO6hvV0p vHIquPAAUL98qGDxF3xSYjv9SY3pk68PVBP6U0WqwBEuGDv4lDMFPWrrWgjeuPZ36lSWTs Ogo4LentudOjfi9szKXkbgoc97dNAuY= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf10.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154401; a=rsa-sha256; cv=none; b=jWWybYKJcADghxuneakhY59ZZxK2lD3dbjHsNmUI+gOM+KKK/dt13TtzULCRYZkuhrxorz c5CVENKgZHqybbj3tzIKPFOoKiWzUBuRvlESqIfyDg/cnFXtMBr3CCMhf+4itW8uW/HAPg 2s556SgOQk1MKG7XhKXUOFeo2VBRgSY= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:39:59 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:59 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 13/25] dept: Distinguish each work from another Date: Tue, 31 Jan 2023 17:39:42 +0900 Message-Id: <1675154394-25598-14-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B84C8C0021 X-Rspam-User: X-Stat-Signature: 5d7rkgor3ried9noacg3i4n86m69d3mt X-HE-Tag: 1675154400-294530 X-HE-Meta: U2FsdGVkX188Aemy7tinnqQ02uLDAp+5tkV64d+7v60BxQ0wzc9PKhehPZNNDVVHM3+vZzlI3wCnuTdCcxvlOC4ueRRKWxHCkPm0JX76q/ISVzDRU8rmLDrOLY5nd/dzSWZiQl2iMINTWtddvV/R2YCs0SjTniks1DryVI+kJVAJUCyseSHp+kEHg4qPMvx/ASkK1EeiYp7xpgA9mbgY5ted0vtCx+n3zAEz+n+Hry3sNfmicFIfrUnVUgbEKlgpfXDtkhqjBHFfWrDcO5FFzokWrL4A4apNbF1pCmiRDZXBBcQjhFaZgBKVz0l6Gg6cwRE7kEgK0Uq1iearQsvJP/2fvsRI0xFqFhOElEp+K1GA7MbYQDYTs76cY276C3COQqsOhfCos3PuUCSRd4lShwRx8F1nVVW+8sTgYM09pcGCcPttSX34BG773kqU4KIoys458l7LDz/NP79DcrMTJboq2KKqSWrxbqDXJYz83Rh1wXQJrkXivv+MNXpHmyNR4drxcW+59nJjFLc7BvPYpEGdeYOiK32wWyiXU8qeyMeHFauiJWPxJ2yHOIP3JbE/TfCcEFbFyVvNODOt0a4Rb34quK48aqSczcEAbrd7LsY2VFIXs7OOzjZww9kmipyb5s+4mQQzg6DiEzA3Ow0XVLfL2PHb9uIq/95bVR3gGp3HhO7jVuJW6HvDzTykBpU5bsLTXyAbLxzOWFT93eAFP43Qqb0vf0LzwOtdUrxEKaVdboFzU/LTMs5eyIeBM6dMGdSr46AAmcXGqBxa/5GUWn46IaJ/KvDjEV+oR9HCfRax5ITSgK5+8+JLsMqT40VMx/r7ph469JrF5Z9JGBiUK+9EFRgQ3PYLz51X8gxp6w2jgcXQlryrCMBwMwuERac5eAOYHpkSFohE0OPS3bCPCEQnrOF8lgoTRQ2e52aCNQQK7bFWuVQxkkMZr1ezyIku+UGk4AcxJHMUTwgmZsQ GClSoLLV WVW50wuB8VBKkR583n2IpsjXQfcSyQlkIYJQYJgk5DgxreurxBI5TFQF5ymkjVHAWxgncCvSg5htIx1C5Tk4ZZBfUpRT7nkSgiI/fyX7QPVB6EuD2r4y5SIx1/+lYlGv5tmNRLNxdMi73ACbSsyH7j3qty8Vp//WdZH28YsYVMDE4L8/OKSAZyNE45lpA5/pHRIkmyoX0JncZT2kbaaXEfv7esld6H49bB3xw8GnQgRMI8DtSZdPEU2vCtZlJGhYgDccauggRfPb7D9+XEAbZfvXPqopNhu50KeSYdqw0T/hdAIWLxBSzUuGXTJVcqS5S+z1j X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Workqueue already provides concurrency control. By that, any wait in a work doesn't prevents events in other works with the control enabled. Thus, each work would better be considered a different context. So let Dept assign a different context id to each work. Signed-off-by: Byungchul Park --- include/linux/dept.h | 2 ++ kernel/dependency/dept.c | 10 ++++++++++ kernel/workqueue.c | 3 +++ 3 files changed, 15 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index f62c7b6..d9ca9dd 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -515,6 +515,7 @@ struct dept_task { extern void dept_sched_enter(void); extern void dept_sched_exit(void); extern void dept_kernel_enter(void); +extern void dept_work_enter(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -567,6 +568,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) #define dept_kernel_enter() do { } while (0) +#define dept_work_enter() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index dd90746..1f1aca4 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1977,6 +1977,16 @@ void dept_hardirqs_off_ip(unsigned long ip) } EXPORT_SYMBOL_GPL(dept_hardirqs_off_ip); +/* + * Assign a different context id to each work. + */ +void dept_work_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + void dept_kernel_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 07895de..69c4f46 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -51,6 +51,7 @@ #include #include #include +#include #include "workqueue_internal.h" @@ -2199,6 +2200,8 @@ static void process_one_work(struct worker *worker, struct work_struct *work) lockdep_copy_map(&lockdep_map, &work->lockdep_map); #endif + dept_work_enter(); + /* ensure we're on the correct CPU */ WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); From patchwork Tue Jan 31 08:39:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122444 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F08C5C636CD for ; Tue, 31 Jan 2023 08:40:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26B1A6B0099; Tue, 31 Jan 2023 03:40:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A4FB6B0096; Tue, 31 Jan 2023 03:40:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7BCB6B0098; Tue, 31 Jan 2023 03:40:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C109F6B0093 for ; Tue, 31 Jan 2023 03:40:04 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 95B74AAFB8 for ; Tue, 31 Jan 2023 08:40:04 +0000 (UTC) X-FDA: 80414446728.06.8AAC0E2 Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf14.hostedemail.com (Postfix) with ESMTP id 287F010001F for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf14.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=YrtxP3iNfNB6GMz6Yene4KD2LoXzXVa4AcHpJ/MRGX4=; b=QX1V4tgOV64OzwDop/tLiWn6spyM6ui1uSCkbGgvK9oUoV3Shpx7JUhon6FmZ4rav3b3s/ 1uFT0jxJIH2+r19juYwvCDAmSlzOCTI5S+EhvH39UJDL/2lubTmbE5ApQSxAyzECLsxi1Z 0OdoP3O7vMSwrBkakFtARTpaf480kys= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf14.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154402; a=rsa-sha256; cv=none; b=H1Jf4VGZirf5noQ3Mvat9RF2AvwalI/OY9Ztp70GpzIJYgQhAuP7aOit59CPA1eVYUs+AA tV+NPR5HYYvg9so3zikxGTNszF1jS7lg8JDHkS1pvAs5EbHH9ZJyxwxIY3ccyJ+KE0LkdS IagipdV/RI15AOypy8lF+/ldIRhuOtQ= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:39:59 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:39:59 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 14/25] dept: Add a mechanism to refill the internal memory pools on running out Date: Tue, 31 Jan 2023 17:39:43 +0900 Message-Id: <1675154394-25598-15-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Queue-Id: 287F010001F X-Stat-Signature: rcaawy9auiwgjna4azjnywfnaet9ws4i X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1675154401-231745 X-HE-Meta: U2FsdGVkX1//yTzs6i3IfSgQIp5PMOi7rJUeopCrcGWgekKQuwa5jI/qljyRqExbEtLS4ZkUs5tz1D4A9mNZR5eOE3kpH1CEFbR0J3FEDIZpErf6Jkxw6KRsE6dnRSURBDa71KGTKIgr9S89Od/n5Lk9H21ltvqTKUDl0Q0+k1UiQLaQJTRduv1hO+GL5KBwf3YA+HKQ8IPd+DG7q2fwr+cIRR1VOPblk5CGWG8SQ/tF/5X7evgqw3CAOoNFsn8/I142tLMmMm/7T4hp0eJogQFToU36tb8+doDI5LLLcr5Ti2KGeLGIbn2mmaC1XLB1G6KiyZXMYZyMsj+9aCyzf6cNHoWk/J48mBEd4/NY1Q7TZkGXybr4OZNYrhtZrTbxStY/3YBRYhpvUnh76m6/v7lgRXpyZegCfFVNZvLF8h4KP5j0uh1xghc1iz7qRrMPPpkOosiXL+TpVcKs45LGyQjqxT7kTL5V2qQV2NA/JEMxz4OoeCvTvDtXB6vrpl2VdQZkSwyPo3GZaqHJVPW+JVrXY4cy5It7r00SWPnRPdIXJs9bxsbHXTCioum7CdgZAVrFCWe9nrxeqMqxlnhYHtk9cx39RVB1Twb5bBRr6+IiJinCfFU3udV0wFziqaH2sT2dVDr6BBGYbsTXESSNjX6x/ur/cFp3kwswGhOoo5zoup+0GzwToWAUpUVjr5WJfW9McoeOlx52TpEosjhVkM795rQkJ9fGO7HcPT3TMdq3F2sC+DCVYFOKpiRWtjqklLd+KwuztAudsLrvpjMI6qSA6ktrUcqPrmBN8U1wBtoofKz+wp5Sdu5UPOPWoQ3b8Ouf89c4ExeQcLnBiy2cwPLIDEIUAMOj+5ENa8wUFno2WkzfFpMP4SoaaI53k9+8Bm4QIsizFluwDK/a2I1JxjwuO82FT4xXFHGRMlCRVcZA1JyMZa7DiSjaQgiIzNGyJi2oebLIvHq/kP7POPu mgyhCcgG HQEYactbdJHgSP2SnHzmHMmjncWsI1diSuJJvmG5F1exqzzPgiPo4STGtVCnnlxZYFlLfW0YYMR0KdlTPX2huD05DBSk0MshVnp3NSyDrJ74u9UMl0HBWt7PeBBL3Wwn50xoEwUV9+XDwcfYcDFGGSrqvmYQ8Vy8SQ6497E8BdiIYDv84dsc7ru0OnzqSqhFl040AiQDO6o2MPURuQ5Mey5NI1Pa5+mOIfEa9jCiqx0ESge0pyzAP7F2YcFiKw5/N1urtpC0DSO0g/21APr7UG7Rlwr1pJg6yNMZTRRXPN7azzA+LK0CUF9cpqaAKpkzlWTio X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Dept engine works in a constrained environment. For example, Dept cannot make use of dynamic allocation e.g. kmalloc(). So Dept has been using static pools to keep memory chunks Dept uses. However, Dept would barely work once any of the pools gets run out. So implemented a mechanism for the refill on the lack by any chance, using irq work and workqueue that fits on the contrained environment. Signed-off-by: Byungchul Park --- include/linux/dept.h | 19 ++++++-- kernel/dependency/dept.c | 104 ++++++++++++++++++++++++++++++++++------ kernel/dependency/dept_object.h | 10 ++-- kernel/dependency/dept_proc.c | 8 ++-- 4 files changed, 112 insertions(+), 29 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index d9ca9dd..583e8fe 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -336,9 +336,19 @@ struct dept_pool { size_t obj_sz; /* - * the number of the static array + * the remaining number of the object in spool */ - atomic_t obj_nr; + int obj_nr; + + /* + * the number of the object in spool + */ + int tot_nr; + + /* + * accumulated amount of memory used by the object in byte + */ + atomic_t acc_sz; /* * offset of ->pool_node @@ -348,9 +358,10 @@ struct dept_pool { /* * pointer to the pool */ - void *spool; + void *spool; /* static pool */ + void *rpool; /* reserved pool */ struct llist_head boot_pool; - struct llist_head __percpu *lpool; + struct llist_head __percpu *lpool; /* local pool */ }; struct dept_ecxt_held { diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 1f1aca4..0ac6442 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,9 @@ #include #include #include +#include +#include +#include #include "dept_internal.h" static int dept_stop; @@ -122,10 +125,12 @@ WARN(1, "DEPT_STOP: " s); \ }) -#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; static arch_spinlock_t stage_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t dept_pool_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; /* * DEPT internal engine should be careful in using outside functions @@ -264,6 +269,7 @@ static inline bool valid_key(struct dept_key *k) #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ +static struct dept_##id rpool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT @@ -272,14 +278,70 @@ struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ - .obj_nr = ATOMIC_INIT(nr), \ + .obj_nr = nr, \ + .tot_nr = nr, \ + .acc_sz = ATOMIC_INIT(sizeof(spool_##id) + sizeof(rpool_##id)), \ .node_off = offsetof(struct dept_##id, pool_node), \ .spool = spool_##id, \ + .rpool = rpool_##id, \ .lpool = &lpool_##id, }, #include "dept_object.h" #undef OBJECT }; +static void dept_wq_work_fn(struct work_struct *work) +{ + int i; + + for (i = 0; i < OBJECT_NR; i++) { + struct dept_pool *p = dept_pool + i; + int sz = p->tot_nr * p->obj_sz; + void *rpool; + bool need; + + arch_spin_lock(&dept_pool_spin); + need = !p->rpool; + arch_spin_unlock(&dept_pool_spin); + + if (!need) + continue; + + rpool = vmalloc(sz); + + if (!rpool) { + DEPT_STOP("Failed to extend internal resources.\n"); + break; + } + + arch_spin_lock(&dept_pool_spin); + if (!p->rpool) { + p->rpool = rpool; + rpool = NULL; + atomic_add(sz, &p->acc_sz); + } + arch_spin_unlock(&dept_pool_spin); + + if (rpool) + vfree(rpool); + else + DEPT_INFO("Dept object(%s) just got refilled successfully.\n", p->name); + } +} + +static DECLARE_WORK(dept_wq_work, dept_wq_work_fn); + +static void dept_irq_work_fn(struct irq_work *w) +{ + schedule_work(&dept_wq_work); +} + +static DEFINE_IRQ_WORK(dept_irq_work, dept_irq_work_fn); + +static void request_rpool_refill(void) +{ + irq_work_queue(&dept_irq_work); +} + /* * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is * enabled or not because NMI and other contexts in the same CPU never @@ -315,19 +377,31 @@ static void *from_pool(enum object_t t) /* * Try static pool. */ - if (atomic_read(&p->obj_nr) > 0) { - int idx = atomic_dec_return(&p->obj_nr); + arch_spin_lock(&dept_pool_spin); + + if (!p->obj_nr) { + p->spool = p->rpool; + p->obj_nr = p->rpool ? p->tot_nr : 0; + p->rpool = NULL; + request_rpool_refill(); + } + + if (p->obj_nr) { + void *ret; + + p->obj_nr--; + ret = p->spool + (p->obj_nr * p->obj_sz); + arch_spin_unlock(&dept_pool_spin); - if (idx >= 0) - return p->spool + (idx * p->obj_sz); + return ret; } + arch_spin_unlock(&dept_pool_spin); - DEPT_INFO_ONCE("---------------------------------------------\n" - " Some of Dept internal resources are run out.\n" - " Dept might still work if the resources get freed.\n" - " However, the chances are Dept will suffer from\n" - " the lack from now. Needs to extend the internal\n" - " resource pools. Ask max.byungchul.park@gmail.com\n"); + DEPT_INFO("------------------------------------------\n" + " Dept object(%s) is run out.\n" + " Dept is trying to refill the object.\n" + " Nevertheless, if it fails, Dept will stop.\n", + p->name); return NULL; } @@ -3000,8 +3074,8 @@ void __init dept_init(void) pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); #define OBJECT(id, nr) \ - pr_info("... memory used by %s: %zu KB\n", \ - #id, B2KB(sizeof(struct dept_##id) * nr)); + pr_info("... memory initially used by %s: %zu KB\n", \ + #id, B2KB(sizeof(spool_##id) + sizeof(rpool_##id))); #include "dept_object.h" #undef OBJECT #define HASH(id, bits) \ @@ -3009,6 +3083,6 @@ void __init dept_init(void) #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); #include "dept_hash.h" #undef HASH - pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... total memory initially used by objects and hashs: %zu KB\n", B2KB(mem_total)); pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); } diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h index 0b7eb16..4f936ad 100644 --- a/kernel/dependency/dept_object.h +++ b/kernel/dependency/dept_object.h @@ -6,8 +6,8 @@ * nr: # of the object that should be kept in the pool. */ -OBJECT(dep, 1024 * 8) -OBJECT(class, 1024 * 8) -OBJECT(stack, 1024 * 32) -OBJECT(ecxt, 1024 * 16) -OBJECT(wait, 1024 * 32) +OBJECT(dep, 1024 * 4 * 2) +OBJECT(class, 1024 * 4) +OBJECT(stack, 1024 * 4 * 8) +OBJECT(ecxt, 1024 * 4 * 2) +OBJECT(wait, 1024 * 4 * 4) diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c index 7d61dfb..f07a512 100644 --- a/kernel/dependency/dept_proc.c +++ b/kernel/dependency/dept_proc.c @@ -73,12 +73,10 @@ static int dept_stats_show(struct seq_file *m, void *v) { int r; - seq_puts(m, "Availability in the static pools:\n\n"); + seq_puts(m, "Accumulated amount of memory used by pools:\n\n"); #define OBJECT(id, nr) \ - r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ - if (r < 0) \ - r = 0; \ - seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + r = atomic_read(&dept_pool[OBJECT_##id].acc_sz); \ + seq_printf(m, "%s\t%d KB\n", #id, r / 1024); #include "dept_object.h" #undef OBJECT From patchwork Tue Jan 31 08:39:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD516C636D3 for ; Tue, 31 Jan 2023 08:40:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D20A06B0092; Tue, 31 Jan 2023 03:40:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C537B6B0096; Tue, 31 Jan 2023 03:40:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACC6D6B0095; Tue, 31 Jan 2023 03:40:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 994D16B0092 for ; Tue, 31 Jan 2023 03:40:04 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 950A1C0A6D for ; Tue, 31 Jan 2023 08:40:03 +0000 (UTC) X-FDA: 80414446686.14.0369FE7 Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf24.hostedemail.com (Postfix) with ESMTP id 6D7ED180013 for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=softfail (imf24.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=705GyNEIJXEWyOVKsCQN1qIBphC8YCGkT9Dvo0ANs9k=; b=sDyLpNkGn8lnLlUTX2NcxoAjgBQ6gmhRnRxwfRfgPyDfynT+l3kcP2Z2yg55NvHi276mnp OSG2wxGBtYA8LM2d4xy9Q1Pf6PuPoHfPsM1ikfN854Wd8Je5hKursrahggcSDpOAptMJLy tCiS+nRaNa3dtYwBo7BngQ8Zw3ROoxM= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=softfail (imf24.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154402; a=rsa-sha256; cv=none; b=EN6fDlbqUFEDAPJyao7d5QK76/rAe9QAjd2ib3d7MgRFI0a3KdNeST+8rQkLd7LOgXZbDS GkKBYplxQblyTXCIlaE1ZYZOvEWlHhvH3c03JR0EeGlXUyzZN26e6YjyQp194jlKcW6RWk VdpIJGZJxQzOn2J7xIsbryP1TDj1gWM= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:40:00 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:00 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 15/25] locking/lockdep, cpu/hotplus: Use a weaker annotation in AP thread Date: Tue, 31 Jan 2023 17:39:44 +0900 Message-Id: <1675154394-25598-16-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Stat-Signature: di81to5mgqytyw568cxdc14oc6xb6e5p X-Rspam-User: X-Rspamd-Queue-Id: 6D7ED180013 X-Rspamd-Server: rspam06 X-HE-Tag: 1675154401-597121 X-HE-Meta: U2FsdGVkX19EaXNoSAziywrCkOXlVZR3nADQu09HVuqGdEOAKqB9ULJuMxOnVob5GATKkEOHz4D69zrGPDj1DGbFixQUaXceTXd+j/KAcXXNF7Qo5WrLkiiLHmRHn/NS8d2gpPPNFaPs9G6ZRCSeH3Jphk0JXOnpyTEkUwKvVnuww1cuUieEq4raM/6DA3enF/QPUcns085ItDkPynE3/UndTPbtxEjAh4GkAMpPQ2iRFt4AQ0s9LBV9H5b6QOfCZr0IWvUqtbB5KqnV9jNIVhS8puC3ReUUfClRHyTVXkSJyoqDjJ34AktldgkxlqYsrF8WhhyIcRLH4IkpoNBZhbtCOuc+wL0f1Y1Y7lbM3IIZnll2hoYMF0wRzUwWYAkcMevjgKZCvSvYRvVO38eqX6ms2GGZmOWQhuJ4KM6vJGYJzHILMM37rqRF6WNU8F6xnjySTSxegvoJWcgB1CfbR6+KKZep5v02cAEN25pLL1OGJW3cs9xTWDYTk9jWFP7T4iT/aC5WUVsjJsRiHOnltkbBoadZ6zbtd7N1I9FDn4Y6c2U8Nc5Yu94FxM2WfenB7XFA+dOlF+eIPTe2Rs5T3b607bDBCa2GU5gXkFwc6g0nzBEIX/mWtrEO1R6WH00qalJLgKT3ks55zJ/vrTXnnBC3W91FVX6yp+TuoyWYEPesRls+AJWx6Nz57LxkU8QBOttQ8lv0CoyU2UBf5JtQtyw/iqBkf936wCs5Fd9Yx4+xO51CLjrJX+a+kecR1/CStT0ABe5GG+AYiHEX0DPdUJoj1YJC7YPpsHqnrmmKnh4RPqqV2/tMEwIeIO8TGMm1f+ShMsq01+aSFBG7w+uBPLdeO3d8KnByP74Y2UkmSGw+rqOc448coOXU2cW1Lw4clvWT7OSNTyTlmZDsjErbL4cGsUWmw0WLwg0h/nJGUoTXKKf2fWVKBOvrj6+bAfPviKjzDokxingv4j8AkTw bT/g7fwS O8QM7r6rK39xk05hZrue0tRwv1xFtLz99QenlUOQYzsI0Rozc55mOCJdud0bxAbwmrO+GySpX1PQIzfGG72SGz9y9rjGbavCJwuO1CJZCAHGKxfaHeRyYWp+om6NlRykv/Ogl3INz/MLRxzb9ji5KgichA5DStUSSrBUcRjsrRZCFoNF0B1XaMW2wnrV2UjwLK0z0QTygYzZGyi68i3399L0HDffwdQVTEXByNVr0o4wXbkojGJW3t3KyX/PXlxyfpZEgmloI1ZbKiGmn94VOAMcyLE+lzeyP2s90XcAQQESv2zpf6LfIWd0snkaKIkCrqLwAmqA64NBNv4TFIVRce7zLbvF62dRnqUGhNvue3fe4y9lCOcrGSIwQounMHwOKr57jwqKkU4wYgk35dEN2Kyk6MJPCVRv9qMLQBOLOITfKt8I= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: cb92173d1f0 ("locking/lockdep, cpu/hotplug: Annotate AP thread") was introduced to make lockdep_assert_cpus_held() work in AP thread. However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. Furthermore, now that Dept was introduced, false positive alarms was reported by that. Replaced it with try lock annotation. Signed-off-by: Byungchul Park --- kernel/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index 6c0a92c..6a9b9c3 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -356,7 +356,7 @@ int lockdep_is_cpus_held(void) static void lockdep_acquire_cpus_lock(void) { - rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 0, _THIS_IP_); + rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 1, _THIS_IP_); } static void lockdep_release_cpus_lock(void) From patchwork Tue Jan 31 08:39:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4CAAC63797 for ; Tue, 31 Jan 2023 08:40:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 528536B008A; Tue, 31 Jan 2023 03:40:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B2506B0092; Tue, 31 Jan 2023 03:40:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DDB76B0093; Tue, 31 Jan 2023 03:40:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 0F6F76B008A for ; Tue, 31 Jan 2023 03:40:04 -0500 (EST) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D72B580463 for ; Tue, 31 Jan 2023 08:40:03 +0000 (UTC) X-FDA: 80414446686.11.8963932 Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf08.hostedemail.com (Postfix) with ESMTP id B05A9160004 for ; Tue, 31 Jan 2023 08:40:01 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf08.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=WDEL3F5nXEauQMgzscvBdoNAhznqE4SMECYcSkGVIJo=; b=2Ofw7vD051TQ2tkE6LlsxDcQKjTeXgWH+a3kmz11xiFTr2urv4vtXt5uxsrtI+VvEdZnZd Uspw3r9g7QUlDFqvJJSfvUq60xqUz0zfssbn+X2am0Qig88Bb3umpvOlpdWOn0poEIiESh XVq6i5oq0kO5GKfq8ADM+OTpO8GYAv0= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf08.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154402; a=rsa-sha256; cv=none; b=EpNsofCOBRVc+VykdYddaMf5r3NWcn79B0y2uC6gqzEVdAr+5dZ6VScta/8ra3BZGPcNMf ARfoJ04GJp6t4uJQJl30mTwwvlYAE6OwSenm+pscBfAtqGH06tVE3PM/rCAbOMXOYHFXlk 2qI+I1ekS3w7+UH6x02IFjaDrvscCZU= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:40:00 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:00 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 16/25] dept: Apply sdt_might_sleep_{start,end}() to dma fence wait Date: Tue, 31 Jan 2023 17:39:45 +0900 Message-Id: <1675154394-25598-17-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Queue-Id: B05A9160004 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: zb6thqw9aosz5utjm3eb5jbu9kesagkk X-HE-Tag: 1675154401-628607 X-HE-Meta: U2FsdGVkX19/5sEhginD9wxJhnXSjeV6eCF7CRvTMnCYloXqUgX4s8/iNDPlsu8X6QNcL3bB/wSJh0RZgfJfMGJfcqWKaQJhZFgq/2OT9Kl4TuNvEvLKrzNXeYcJBC9eg6x8ug9TnffdG6JvyBA5PRAkpPcz+uArkgjFg0LJ7RYrYl0Kge6TDzzZohDTFK94kI22qOs91M4DgwdYC+1JlYsxNDFn65BFn3Uueh8llBeTEudIt4XUs97lHglFYHuHsvhz86hpJ3g7GouKnVDLQTNBW+rj1Lr0kPloyQFi6QQlE8rPwd0mhRMSyT3ff0jzdd82i5E5J+RYt5LrREgTRyMNuPjth3jRk4DqaO5kpmLKKAsLH/hDd0XoiBshyluhCgDfmuRibl07/JHJiFFE4FYCop+I7EfsXNtsXRMOW9mdstQlK0Kpr2NvBHMfhaOb1o/JiOlJGeLKkkGe1qQzP1ppneCqCwtZqO1k353VsqzwsHrTqZcolbdG1gyV1DMtOtFfGKNZYtiIPJW9bpq1cQRBJk2HV1JWUNN5tGqoYXstLep+EyHURdqk4608t+ZwgwoF2caRI9X3dseOyLa922Nv+8e7IzJ1rVxKWKp8A9aTQyX2SXZ6oHXlehX2Q0bgsTm/ODGJB5X+cKNYqmcwwJCv5rHMhzuz1pDEmNAWm0l40Fa/h1ZCOT3HIAZb5lZR6/CdD3VBVPy12MNfjYoUSOqej5nhkh0LBn3myHSZC4bZ5RrpsyESu6HaptCQOXUcaJzPxAsVb6ncADa5sHpJkTJH5XvzxbO/834OMipzlyWl3DfRvgfAF+FkKZqqSKz/FbJu/0rWaN2gqKet6pADb3roM0mUQS0I1MstvN89augbKT4LVwKw5+kJHU2ps2TDdKPRK81GBZUB2oGv94sN/V8c1Snh9RbLEv9RlvaeZv9raSfegzg33J9p2+EG4bVvs9nSpCXUMKDkf5wMrB3 JOwIW1IH zKJ3RtcX7lX+EOzL7Cslt2WMB50LTeacBsH1pZJrNMmmr1X4K/HwUxb1OnfEGCXpvQpI/sr9cxPCzXR0WpeZ5Q28EwK2yWs5NEJ6gZuMpTa9zacKToQzVgu9kfolHJA91MjDfrDAS1RzXuAo/NdbisSHv1aZ/NhxUuXBh5LYahs4VPFqK9Z/x/uDiU55xU1LEkAAR9NJ7jNkQ1i36HDojMVt4Z9H2wt6f8GsWXsM7KLkhROQV7xzwsifp91TNr8Mckhv4fK/ZccVmNDfwxnrQQyPULe2EVuDsVHmG60ikYg907C0pHoqyxLMx1CKC2Qtj4J30+VTIDY9aUaBECpdC+jyJlydUU75MICHjbv3jt8KTA9p1FIOlSvTIjzOsy4FeK7DtYWWPvMovjj5OQKv/3lohS+mFMRxDwT+SAPhWHRyUsRTMt4TwFRIVyz7I2kphhWMeFjHOD9hE7hvKiVNGFnrBrwKsjVufoQkebGiQxnn/t90UaTEZHzvPbjxr/hEA3LwX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Makes Dept able to track dma fence waits. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 406b4e2..1db4bc0 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -16,6 +16,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -782,6 +783,7 @@ struct default_wait_cb { cb.task = current; list_add(&cb.base.node, &fence->cb_list); + sdt_might_sleep_start(NULL); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -795,6 +797,7 @@ struct default_wait_cb { if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); if (!list_empty(&cb.base.node)) list_del(&cb.base.node); @@ -884,6 +887,7 @@ struct default_wait_cb { } } + sdt_might_sleep_start(NULL); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); @@ -898,6 +902,7 @@ struct default_wait_cb { if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); __set_current_state(TASK_RUNNING); From patchwork Tue Jan 31 08:39:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67F8EC38142 for ; Tue, 31 Jan 2023 08:40:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7258A6B0093; Tue, 31 Jan 2023 03:40:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 701776B0096; Tue, 31 Jan 2023 03:40:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 267366B0093; Tue, 31 Jan 2023 03:40:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0D9A36B0095 for ; Tue, 31 Jan 2023 03:40:05 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E24F7404D0 for ; Tue, 31 Jan 2023 08:40:04 +0000 (UTC) X-FDA: 80414446728.01.72BD230 Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf19.hostedemail.com (Postfix) with ESMTP id 7BEE21A0008 for ; Tue, 31 Jan 2023 08:40:02 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf19.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154403; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=yccejmWFjoDlzEkWmxJYk0ze/Kuf7ybQyaMiH9GD9YQ=; b=GlTWpP5E8XipIf1/8dChSpDyJYNLQPo5Y0E4iBb6bItDnNOCCbvYSs6MjkNlbrJhFzeBEm 5S/LL5NWMJnJVm+2GsQ7ImukRUeYzPXIo4xLfN6N/1nKjc281NL7RCkHa53qIUk6xCV6tA NbDJEYSGQNNre3qZYxTvkPjTJKWq6kA= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf19.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154403; a=rsa-sha256; cv=none; b=bapz9jAGgpw/R94ab+LGwiAs7e1iUsVGqk67wKZyMenioG3hioTfPYccKJUeVpFIyX69mg 5o+zBwotQivRHl+aK1QA6YRTkC15Rfmqj8/P28ucbE34mX/hQ5wi7J/mqdn2X1J+dLKJCG lzHgpoe8IP4eqD33QbnbhYCokBxC4hU= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:40:00 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:00 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 17/25] dept: Track timeout waits separately with a new Kconfig Date: Tue, 31 Jan 2023 17:39:46 +0900 Message-Id: <1675154394-25598-18-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7BEE21A0008 X-Stat-Signature: n81w6dpy5aa4q8asc3cybb336gas7f75 X-HE-Tag: 1675154402-321183 X-HE-Meta: U2FsdGVkX18gQ4UZkFZrvteUkd7PLAOhVh4e1VxCXvIVK7Ho5AqG5RPvaAXJhEMI2Wd3LQCi5udN3P62oV2WaUza+WneF66WbDrqZBgjY1Ycvf8K9mHV28CjMFbEuCO7HtwjILwc7C4orIT1hBcMcqityn9U48uPPHL5jsH4NEMzAd8Lmz07bfHLwHDVEpcjRCQ73ijit7Yq8CiB94kE8/n7gY5Xjjzho1/+sU9uTmXpwXQKRhdSwJNbtn4lf1+JftYss++cOpi7MHKrCbDC1sfY0rQw77qcfTpqsexKU5IbuXJMII38VTTvT4aj9P/h1NF2RBV071Mn5FLnS+tT7loQ6PilGxkiucMtdK+t2Ha0fP5l1/E0s4flx0LfPm2+I9I62cng4tt2Cw94nItI+/03trqnk2sx1Vi8T9n66jI77O5CDrjoY3fDXo5AsZ6zDjdXIOTz2ctwIiJdoBaWcZYam12rpXrUg/xGpDH889Bq+yjfxVh91eEmSM7PknM+d+nx4GViNiwe2eiicmgB/PJSVz92zJlhoDPEgp6h+DeVm+aCkhb6DY4/CAHpsjL6veu8/9W6YIu7I6K3ak/vPW4Bpj2/sBcnjhIyc68Wu1CV6w/UGCu4/VMCiR3ayJEn0kOx5z8FWpcBx700PDZelrEIde00rGgX0fmtSxfTypBdOiot4hflPp98u144HjliwWYRRxlo9fJRYy0S31fsQ3Cikxtq4w5GqJ+0CyaTEiQs5KhmT5xpiU8yudlO0ZhyQ70l3Zt7ZfNKZiZIQ+s/nwNWAL7M4XfK5KeKluok2nbO+GXiDPbNQCQWC9YO9syEHIfIVqIZEVeUsXpDVsgtkILmxK2qvzZLlsNYVuZ0IYMTdP+0ZrwAx5hGg6zAX+iTXUu08QVLdUCqu0ggVb8J0qMBeabLQZgZpj1A4xhajkIKU0UPBBWJhMzf0sZwTFdN2gLUOHh1JquMPI8Kt5k TKo7LXGC SWqaSYyvz3SOFE5hGLSMaWUrORHbmXUGQynXJmDZ4rCYtfiOosFRxrNMQpdiqHNzb7NjCQRwugcLTkyDO9QawIF08l0S4EBU9Yi3TLEnzb88vQoEaCmC71pyeb8oh7DdtmhQNaQ/gRL4vk3vlMtEpFQF6HcZ/FF3HyW6nF05BAgzZSh+9A3VUIisp2PFbJ5ZMei4gJFyf4tTp/9Nu1aqjEw2shprwzda3F9C343IxoDaWvVsrW3lx69DBeP0mVvWAsrTYVX01hohS+hicc/bW3iMImE0dtY66RdmNq/h63v87Wreccpt7XeFSUcapxawSUo5j X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Waits with valid timeouts don't actually cause deadlocks. However, Dept has been reporting the cases as well because it's worth informing the circular dependency for some cases where, for example, timeout is used to avoid a deadlock but not meant to be expired. However, yes, there are also a lot of, even more, cases where timeout is used for its clear purpose and meant to be expired. Let Dept report these as an information rather than shouting DEADLOCK. Plus, introduced CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT Kconfig to make it optional so that any reports involving waits with timeouts can be turned on/off depending on the purpose. Signed-off-by: Byungchul Park --- include/linux/dept.h | 15 ++++++++--- include/linux/dept_ldt.h | 6 ++--- include/linux/dept_sdt.h | 12 ++++++--- kernel/dependency/dept.c | 66 +++++++++++++++++++++++++++++++++++++++++------- lib/Kconfig.debug | 10 ++++++++ 5 files changed, 89 insertions(+), 20 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 583e8fe..0aa8d90 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -270,6 +270,11 @@ struct dept_wait { * whether this wait is for commit in scheduler */ bool sched_sleep; + + /* + * whether a timeout is set + */ + bool timeout; }; }; }; @@ -458,6 +463,7 @@ struct dept_task { bool stage_sched_map; const char *stage_w_fn; unsigned long stage_ip; + bool stage_timeout; /* * the number of missing ecxts @@ -496,6 +502,7 @@ struct dept_task { .stage_sched_map = false, \ .stage_w_fn = NULL, \ .stage_ip = 0UL, \ + .stage_timeout = false, \ .missing_ecxt = 0, \ .hardirqs_enabled = false, \ .softirqs_enabled = false, \ @@ -513,8 +520,8 @@ struct dept_task { extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); -extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); -extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn, long timeout); extern void dept_request_event_wait_commit(void); extern void dept_clean_stage(void); extern void dept_stage_event(struct task_struct *t, unsigned long ip); @@ -566,8 +573,8 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_copy(t, f) do { } while (0) -#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) -#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); } while (0) #define dept_request_event_wait_commit() do { } while (0) #define dept_clean_stage() do { } while (0) #define dept_stage_event(t, ip) do { } while (0) diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h index 062613e..8adf298 100644 --- a/include/linux/dept_ldt.h +++ b/include/linux/dept_ldt.h @@ -27,7 +27,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_wait(m, LDT_EVT_L, i, "lock", sl, false); \ dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ } \ } while (0) @@ -39,7 +39,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ else { \ - dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ } \ } while (0) @@ -51,7 +51,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ } \ } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 12a793b..21fce52 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -22,11 +22,12 @@ #define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) -#define sdt_wait(m) \ +#define sdt_wait_timeout(m, t) \ do { \ dept_request_event(m); \ - dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) +#define sdt_wait(m) sdt_wait_timeout(m, -1L) /* * sdt_might_sleep() and its family will be committed in __schedule() @@ -37,12 +38,13 @@ /* * Use the code location as the class key if an explicit map is not used. */ -#define sdt_might_sleep_start(m) \ +#define sdt_might_sleep_start_timeout(m, t) \ do { \ struct dept_map *__m = m; \ static struct dept_key __key; \ - dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__, t);\ } while (0) +#define sdt_might_sleep_start(m) sdt_might_sleep_start_timeout(m, -1L) #define sdt_might_sleep_end() dept_clean_stage() @@ -52,7 +54,9 @@ #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait_timeout(m, t) do { } while (0) #define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start_timeout(m, t) do { } while (0) #define sdt_might_sleep_start(m) do { } while (0) #define sdt_might_sleep_end() do { } while (0) #define sdt_ecxt_enter(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 0ac6442..3ab999f 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -740,6 +740,8 @@ static void print_diagram(struct dept_dep *d) if (!irqf) { print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + if (w->timeout) + print_spc(spc, "--------------- >8 timeout ---------------\n"); print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); } } @@ -793,6 +795,24 @@ static void print_dep(struct dept_dep *d) static void save_current_stack(int skip); +static bool is_timeout_wait_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + + do { + struct dept_dep *d = lookup_dep(fc, tc); + + if (d->wait->timeout) + return true; + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + return false; +} + /* * Print all classes in a circle. */ @@ -815,10 +835,14 @@ static void print_circle(struct dept_class *c) pr_warn("summary\n"); pr_warn("---------------------------------------------------\n"); - if (fc == tc) + if (is_timeout_wait_circle(c)) { + pr_warn("NOT A DEADLOCK BUT A CIRCULAR DEPENDENCY\n"); + pr_warn("CHECK IF THE TIMEOUT IS INTENDED\n\n"); + } else if (fc == tc) { pr_warn("*** AA DEADLOCK ***\n\n"); - else + } else { pr_warn("*** DEADLOCK ***\n\n"); + } i = 0; do { @@ -1564,7 +1588,8 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) static atomic_t wgen = ATOMIC_INIT(1); static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep) + const char *w_fn, int sub_l, bool sched_sleep, + bool timeout) { struct dept_task *dt = dept_task(); struct dept_wait *w; @@ -1584,6 +1609,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_fn = w_fn; w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; + w->timeout = timeout; cxt = cur_cxt(); if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) @@ -2338,7 +2364,7 @@ static struct dept_class *check_new_class(struct dept_key *local, */ static void __dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, - bool sched_sleep, bool sched_map) + bool sched_sleep, bool sched_map, bool timeout) { int e; @@ -2361,7 +2387,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, if (!c) continue; - add_wait(c, ip, w_fn, sub_l, sched_sleep); + add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); } } @@ -2403,14 +2429,23 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, } void dept_wait(struct dept_map *m, unsigned long w_f, - unsigned long ip, const char *w_fn, int sub_l) + unsigned long ip, const char *w_fn, int sub_l, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (dt->recursive) return; @@ -2419,21 +2454,30 @@ void dept_wait(struct dept_map *m, unsigned long w_f, flags = dept_enter(); - __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false, timeout); dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_wait); void dept_stage_wait(struct dept_map *m, struct dept_key *k, - unsigned long ip, const char *w_fn) + unsigned long ip, const char *w_fn, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (m && m->nocheck) return; @@ -2481,6 +2525,7 @@ void dept_stage_wait(struct dept_map *m, struct dept_key *k, dt->stage_w_fn = w_fn; dt->stage_ip = ip; + dt->stage_timeout = timeout; unlock: arch_spin_unlock(&stage_spin); @@ -2506,6 +2551,7 @@ void dept_clean_stage(void) dt->stage_sched_map = false; dt->stage_w_fn = NULL; dt->stage_ip = 0UL; + dt->stage_timeout = false; arch_spin_unlock(&stage_spin); dept_exit_recursive(flags); @@ -2523,6 +2569,7 @@ void dept_request_event_wait_commit(void) unsigned long ip; const char *w_fn; bool sched_map; + bool timeout; if (unlikely(!dept_working())) return; @@ -2545,6 +2592,7 @@ void dept_request_event_wait_commit(void) w_fn = dt->stage_w_fn; ip = dt->stage_ip; sched_map = dt->stage_sched_map; + timeout = dt->stage_timeout; /* * Avoid zero wgen. @@ -2552,7 +2600,7 @@ void dept_request_event_wait_commit(void) wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(dt->stage_m.wgen, wg); - __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: dept_exit(flags); } diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 611fd01..912309b 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1282,6 +1282,16 @@ config DEPT noting, to mitigate the impact by the false positives, multi reporting has been supported. +config DEPT_AGGRESSIVE_TIMEOUT_WAIT + bool "Aggressively track even timeout waits" + depends on DEPT + default n + help + Timeout wait doesn't contribute to a deadlock. However, + informing a circular dependency might be helpful for cases + that timeout is used to avoid a deadlock. Say N if you'd like + to avoid verbose reports. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT From patchwork Tue Jan 31 08:39:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 616D0C38142 for ; Tue, 31 Jan 2023 09:10:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EFD8A6B0078; Tue, 31 Jan 2023 04:10:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E88126B007B; Tue, 31 Jan 2023 04:10:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D001F6B007D; Tue, 31 Jan 2023 04:10:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C1F7C6B0078 for ; Tue, 31 Jan 2023 04:10:05 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 7F46FA5BDD for ; Tue, 31 Jan 2023 09:10:05 +0000 (UTC) X-FDA: 80414522370.25.DCE3F0F Received: from lgeamrelo11.lge.com (lgeamrelo13.lge.com [156.147.23.53]) by imf25.hostedemail.com (Postfix) with ESMTP id 0FEE7A0016 for ; Tue, 31 Jan 2023 09:10:02 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf25.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675156203; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=thOsmF+EH8LVfGu7KxOVDTRvhmgkfNTM1DUTW5gELOY=; b=ZvxvFwYlp0hSkMex/kMAF8kL5dsvWLCfFWcmhACnJeDoiPjCtUQK2WbhwM3B0fFsQ5mXxN Z8SrUD22oGHD90OwlDTlFMBXMwMKMRgy4z8oBID1p7nmdMtUqKyaprA2KgwEkgK3L57UUr 1wU6x43q2fMdvrKMGU4iiZD1ewnrbwI= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf25.hostedemail.com: 156.147.23.53 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675156203; a=rsa-sha256; cv=none; b=yvEiS8F9pWLzc2+7xUzmWUp1YPYfrctb2GQNuo0p4uxGOV7xAjglYk9UITWqYDU6nBJFJd o8HBwTvP8575g+k77mm73Ge9i0RVWHbR17W6KFE/grgljdEcQg/g5BmtCLn76uRWcPqp7A o8NbVv/S9B/h3zU4yag1f3osz/4to/M= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.53 with ESMTP; 31 Jan 2023 17:40:01 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:01 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 18/25] dept: Apply timeout consideration to wait_for_completion()/complete() Date: Tue, 31 Jan 2023 17:39:47 +0900 Message-Id: <1675154394-25598-19-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 0FEE7A0016 X-Rspam-User: X-Stat-Signature: mrd744kp8khx3w7fp7bsqk7z1njrregk X-HE-Tag: 1675156202-406734 X-HE-Meta: U2FsdGVkX189xnD03M5gWPEDqbSJd8QIaRaLSWUpyVMZpg7QP4vLXjly34jvSy0ItwayH2xXfvrafd28iF8tF+1WU0ZqPIiKNmCYt2AbpwZBbe+yLq9McG3raUyd3waNpW+/86nkI8AHJdQGD78JEXRjguCloNZfbl+w4nQMEMdPHyQW1Q2NHgFS8F5IyuDRWFwMbvwp6vn7WDWxShSlZ65wa4McVPXx96JV7kheErs7OcmfSO/t9Rat3xDMZETo8mABVN8I/hdAP+s1JK330D6tr7jauNWKt9uRXcBJ51cyMgGjUTdOgU5FzeAFtXcgb8k7+s3nCD8f+Mtz6VbtbmidCt2oghRgmDg8CBa59GSttj9jyVFvXfjZ4VyU4I5Mz0am1jJzLGfL+rOYqXlbFyLwjZwEL6puZo9vXVLNlstLptt83oJ4XgPNHaz7uoMiLCGVSSla+5zXfYnPAdLXpeI+oIC5+4W3UcrDvX/G+DXJSsJF4jqeQugfAzgnZt7NVpR6tRf1FqgEabFkyrTOuYSk0q8DJc0wcPNeNtCVOfwz6VSmHDOvEc5bNHeLJVJ+W8rAD8BEQCvDOBWfRPYiXUNdNeau2HCZB8Q90NynoP7oU6lKB4yRYNyK3yQChvPZFOW3nTgV8InM1shMDWtSIY9uFjW4Iodfvs3WEPN9bhuoCA0+xBax/Bqb6fVfFipXSCptbYkSDe9KlwgjwVYNxHqz42ZY+thdUN/o6/SiHaQLEq8q3Jw+mljpwvpfrFZDIe7/xBXCu8blHTAv8hZm0hseU9dorgJuSXO8CVTTEWTz4j0qLqAKKOy0soQ8OJrmEHaoKRKTLPQjSD74gQn5u189mhEoVLhulg/lTuIW8HWqReNjKchXgoRPMs1c8sMB5LLG5IlW2KgEZXObK/3bwg2NzgepKSBXJbjbXj8i1sgDqg+WOgPtKu+Ov/n0CokH6QuunSQw+jin15JoJJX 5y79DWkv zSW/W2yREJtYR4kx53zi1L1H+VMjccuRaoCqEdphIa42TtHdeGeQveq86oG6MWHUYm0A0zuQFQviYMFBmXunFfaz4ai5gmcrXobvWpCGqZDM7JeZ/BR1lelcfjLWRlrE8A4B973CgoOlkyfx8d6+F9CoDGVugYGG/zabRSHLLXLw0d9n7yMM5KX39vP1wxeDQvbEwtgBcSXMMql52yVWLxulTDxBdudF6mCanzPuSAls3u506g1PzeETN5tbfdLYSum6wnfyYBKCMcVlLcpXDshxE1TnEmHI4nhEdbFyW2LCy6GYXszhoEA2OstgR6J4+C9XFBxrHftjsDHCOKw2IodM9cGN5YnovrDT0rB3fGSgi8pord1cMVLU8Ymzis/uwRLSI/tqH54zwjh/CTKMo7nRWj0xaZtIQnOs4SCXOKJEpQ48e/ueaUY2/tZuq6BbAC0ovixZ3FiHtF5z0oKs11/l1Qks2fjxrk3pw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 4 ++-- kernel/sched/completion.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index 32d535a..15eede0 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -41,9 +41,9 @@ struct completion { */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) +static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start(&x->dmap); + sdt_might_sleep_start_timeout(&x->dmap, timeout); } static inline void complete_release(struct completion *x) diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index d57a5c1..261807f 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -100,7 +100,7 @@ void complete_all(struct completion *x) { might_sleep(); - complete_acquire(x); + complete_acquire(x, timeout); raw_spin_lock_irq(&x->wait.lock); timeout = do_wait_for_common(x, action, timeout, state); From patchwork Tue Jan 31 08:39:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E545C636CD for ; Tue, 31 Jan 2023 08:40:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 358436B009A; Tue, 31 Jan 2023 03:40:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 26A056B0098; Tue, 31 Jan 2023 03:40:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB1EF6B009B; Tue, 31 Jan 2023 03:40:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C04506B0095 for ; Tue, 31 Jan 2023 03:40:05 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A5F38A0242 for ; Tue, 31 Jan 2023 08:40:05 +0000 (UTC) X-FDA: 80414446770.27.5583553 Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf27.hostedemail.com (Postfix) with ESMTP id 61EA140002 for ; Tue, 31 Jan 2023 08:40:02 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; spf=softfail (imf27.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154404; a=rsa-sha256; cv=none; b=TO5n129iaAPP8eeSz+DfJqEcasDyYrapZKN2NZ0CKiCOtss6vw+adWBlDrSAeVwPbOZ442 JUcN+YL21Qp7JMBPTr00iyuc6tTR3HjnUD2Ieog3x8LMyTnMHZZ9SBe7pUdgTvkrjT0wJN Rq9oQTiwm8XA04AvpQ6kXryrcGORsos= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; spf=softfail (imf27.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154404; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=pUBfq7kkLW//AjfV6VFMenp8BeTZ8uL79HZDpouyOfw=; b=FUdwCZj1IjzFtxe+2iSduZ0drIoxGQcmon22MoEDLJJ4XKEBViqDyo9mOoOqmJQGJvCLqZ LIP6GQ+zL8Tz1oRvLTnitE86ufs5KFrJ69LjsBd8jPrApPwiDV/A1Td52fMGLcNiAZzFFU ZxCtwNsHm5h+uExbYjlTEDrm3uU5cXY= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:40:01 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:01 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 19/25] dept: Apply timeout consideration to swait Date: Tue, 31 Jan 2023 17:39:48 +0900 Message-Id: <1675154394-25598-20-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Queue-Id: 61EA140002 X-Rspamd-Server: rspam01 X-Stat-Signature: 5uddaom41dgj4p5wbznao6a9shcguict X-HE-Tag: 1675154402-282137 X-HE-Meta: U2FsdGVkX1/WEdZRe71pWZGAvJPk+wky5xDeiM+RtQ6qbVUe34RKqjH9UECi0c5npezA330bNJ0jGUWlBnDEXr69DGgpHiECn+rAsopuzt9Ri19EXCCIasBPUyY42tWYizk/IIDj4KDchsQHJuQU8VCJp8zZc156Y0vXPr+54J365I0F+y/Jg5zdM2H7jltikPa9qrpdytZcqoAwN/97nMxj3a1kluyCGXowKd9xTM6zmEyU2pgD92dETvcBhsSr6e8BaxoOBwirKcuqjHRUZLd3JGQ65NCuP8vOMGNwKbIyCtTOEVk78Li7+0KmvJXFdez0d5WvdEn8vZQY7cLGkOZNtDCtO0Wu3sVtovbCDaqKXRgVBR7jpzNPwYbrqohf/K+3TxBU9PB5qIDyDDEK/HfPNFZ2zLHorfYce09NH+opqbgLcvZWZexsMacoz2kgVcHzfePlut1LaHh66wPA/a6dhzOYB+6xlUyVTR9kTuaA3Hv83Iz9w0ZSmpYO8bu76vJf5+qFhVnK3Ca5Bg+ZkKiS3EDQXMhJSeJlxHoCu6mfb23RUF2CeGeDAPunZ7cYZ2+3HIymU4H/BR3OuQmmLeY0HsR6CHmX6vE8+2lGTEIA3BznqQjtSMZIYRaphV7aHDvWY7oP3/EMYIapv221haFey2aK1g/4OwEuPO9IPqgJAeyJk0HL8Z1BQiab+gzimXjIfm73rAJs5YQUZHGK8NyID1AL0AQvNh1vEkRmQ/L9LzIafbiYYphAekm4OH+o7Hf2QL5QHrLNjkT+jRUJR8vR1n+1e5QUirMbyAuf3zWdKcqJr0N9GbT4crUtY1n1G8RIHpJkrBxMv/D02rkZsocOMAmVYvXom27P0OVfUZvyGjO8V5dcEfbN3pL5w7gvmEY5ZkV1ZdWmdSfkQEByq3OWMev4PdV0cUxUJpf3W6fPmDeoaXhvR+/P4bx9Oraw97VO1KYTe6yrdqaxNSK iqYybnpR GSX6oigylVNE/ZQ6XZdrZnaXVyDp3uSwQdZC5Q62Pc5rzUnSitzMtbXkYqeyR5dbe4DgdecTqklFbHr1IMia51OcKfZcNhCY8ncq7C2IxMc7a389UeaFpV8cmCglpKQmKQ9eEImX9PviNrXSEaClSwy3U4WeUMb9hVfZ5XxFBbrLLkVylUMMuKD8FRNi76F6IiOasgNZzUKUoM+FVlN6OhujvxuOgCNzNjG1AB0JiGALvxnh8KUWKYeH6xRgl9vbw9F03MM2OCv2uLU+Yl0Wx8v1+Oi7YLyf8lqY9D/oe1lCxq00z9868Vm/XKA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to swait, assuming an input 'ret' in ___swait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/swait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/swait.h b/include/linux/swait.h index 0284821..def1e47 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -162,7 +162,7 @@ static inline bool swq_has_sleeper(struct swait_queue_head *wq) struct swait_queue __wait; \ long __ret = ret; \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ From patchwork Tue Jan 31 08:39:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6DB3AC636CC for ; Tue, 31 Jan 2023 08:40:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0E9F86B009D; Tue, 31 Jan 2023 03:40:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 01F196B009A; Tue, 31 Jan 2023 03:40:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B1A3D6B0098; Tue, 31 Jan 2023 03:40:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9DB206B0095 for ; Tue, 31 Jan 2023 03:40:05 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 79347C058A for ; Tue, 31 Jan 2023 08:40:05 +0000 (UTC) X-FDA: 80414446770.27.80A8DDE Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf02.hostedemail.com (Postfix) with ESMTP id 52D1B80014 for ; Tue, 31 Jan 2023 08:40:02 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf02.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154403; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=O3brDvLbf1/7jmenG01cC+UF1fIt+v37BUH8hTnWxRw=; b=09siZoLYpq2FKiSwYug+M0RHQtg16ZpSCIWvyDW6II5GtS7eDwmCvBaWfqlj58h5AvT57k kmXaiAWu7eozDIdU5QCzwsmMY26c1+C3KYwBRLJQaraiiAXZ7hUQC3O7Zl1+WrX+9BZS5Y 9QHWKodYZ3QPKye8ARXWaWWxxlxyPeo= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf02.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154403; a=rsa-sha256; cv=none; b=hcGCCB+NfDW1KFdR5ezE6TVbYLjXnufNcFVR5zNhTAJyQM+nYekqqYcAJaIJqLT8PX72xe p+7V5YDZa8iS343b0PpSxd9E9+vhFtX9tOtegNbphppaCTwzvGoLWNIZdouT1a9SrnMcLt oSsDIFhWq11A26/wiTQEtatY5b4zHi8= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:40:01 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:01 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 20/25] dept: Apply timeout consideration to waitqueue wait Date: Tue, 31 Jan 2023 17:39:49 +0900 Message-Id: <1675154394-25598-21-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 52D1B80014 X-Stat-Signature: wdkec3q4bwghbsp49gge8kz3whrdgw7j X-HE-Tag: 1675154402-507186 X-HE-Meta: U2FsdGVkX18Ba84KfDzlBssimZmKGHDBehMztNflDHG0zEgTDsxjsw6kfkiAMs5DJDV/s5Dhvg5XJahsWrCvXoX6nsZMWP0i53+owF5lgfVdR1IWul6tfGOnGmo2rfKM6bCG2m32V/PItTQ4/pxhSuyjC+y9w9YyRepuWHtBaZznQ7UvGOFHq3TJjaVz9dq4x7i0hu8tYNtNkxFrIkdbdeV/rCrfG4wkRXHiJmfn3Qy6xcJPTj8ZYNXrXdrnBiHj29p5w+nDWIf9CQ3G+53TIRE5b9bwAPmFuJWnuOHZAY9nJbb1/w1z3lGd7TXM3vUok9W9STs/m2oadyKM4IUVcQwRlNMft9fQmZlzUZI2Y0aDxk55MPYa/lLD5KNxy6xhQ2CEkaYoyH5hBcy7jjuHsdvJBHuWhmFQmBEH0d5tp8fvrE95jrpWN1vVubhcN6akYXVY6igsQoO8x9JNNk1rVGlkHYbtI7KljtvgKQtPfwJ9y4KnT0VpV+wzCwJMMX6snY396/l7jYVZGRg77FQO/d+Du5r+RTo93BgPZbuVHU1JI4PP4gOiVctSpy4285dzdm6JcndKy9xgWP1mQ7gTkqPdpCZ/yy9BwbMCy8g82B0wAyZYG3qmnf5NBJq9UYabztmH9P8b/IUSrYDUAn7PtFLwJ9SPCbcGQbuS+OVRnYAs1Hs4S5t7l7ZEKifVVMGnrOHvdNNpVaJfGIEnsFdf7h7epdHD771o6lzRGQeLesLB6SMMjUAHo7KRL/SPLNU8BDrWVKr7My5nufzelW6YIuiz6gvnyszmYNxHotgtEkjP2Pq4N6mCLddpvccA+TZb/q5M2CpBzW+QFIfInzry9/KdkV4NgKqQ2u5M3+13IpR2o4XEUnMqpVSviv32KPhcdX6Lqc9L13QovfHKXnNfUv5LJYxEfYEuzKzMsh38f/dUNFNfW55PdEX6Vs6xEP0C3zJjblR67OQxgozphdm UNyVaCqo soFd+dci2Na92eSe3OJ85uIBzO8e9h+2hM7BNJqKCKXP3+7QhUAPA9varS0ujBdqtEvI0JXJBJ/Lyd0M0OOfikOOzrWrcRIO8X0XZc4seB/qSBYw6cWCUrkUHzzGfUKprPetudPeE0MPgJkpcb5tfvouIGgx7e27gs3dIFkS/oOCE1Z9LxfBvVZFu+ss+UOhBW5i68wC+Ev3SMVBwaNs+nk7EuEbqsfBmdWrDBiCf+trQa/3+7lms9dZEsJWn1DDIFcWQ0qBv603ykZHD9nzF3J5gkPDye/SzWXKBy9BffKtVUS+U6y00iRFAxPs3UmPi2j/V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to waitqueue wait, assuming an input 'ret' in ___wait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index ff349e6..aa1bd96 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -304,7 +304,7 @@ static inline void wake_up_pollfree(struct wait_queue_head *wq_head) struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ From patchwork Tue Jan 31 08:39:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFDF7C38142 for ; Tue, 31 Jan 2023 08:40:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4FDD76B0098; Tue, 31 Jan 2023 03:40:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 48EFB6B00A0; Tue, 31 Jan 2023 03:40:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A5B56B009F; Tue, 31 Jan 2023 03:40:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E4F7A6B009C for ; Tue, 31 Jan 2023 03:40:06 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C36F0806E8 for ; Tue, 31 Jan 2023 08:40:06 +0000 (UTC) X-FDA: 80414446812.19.3787CC3 Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf04.hostedemail.com (Postfix) with ESMTP id 8C71940004 for ; Tue, 31 Jan 2023 08:40:04 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=none; spf=softfail (imf04.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=/KVJe06Myih7eO0UjMzcYRwZE4PO0RMlkTLcvu4jDuU=; b=JTjKQ8qaB7EMC6o98tuZ15Dhq7bXgDEFMDimcaTQKccyu0C3YqkqCCJ9jI/QLJbnaQcsZn nwKuZtyA3blwZsh+7krgbNHOAvdkgffcFSF9bEqeDRTM9jsps9Gh5LpqSYGRLdxcs+MSLB LlDeQ8Gis8nvEGm1wq3Fn4M92LCM8NU= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=none; spf=softfail (imf04.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154405; a=rsa-sha256; cv=none; b=z4mpGcew2pIPjEIt46W6J2J6XAQQ5A7S03cUfEyRY1yy+b+XLHmB1k7BF7AX3+n4+6kkrI XYELDCUDrjl2+SkcH2JPjEGPuY+h85b/ACWwMpr9qkAQOyk55CMXCKPaKtagkyy8xy/388 4rLcw+PsaOzc6AN4B7SeIMccWaYCz9k= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 21/25] dept: Apply timeout consideration to hashed-waitqueue wait Date: Tue, 31 Jan 2023 17:39:50 +0900 Message-Id: <1675154394-25598-22-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Stat-Signature: n8ni655wuc7aippp36a6i5eh487qb3w7 X-Rspam-User: X-Rspamd-Queue-Id: 8C71940004 X-Rspamd-Server: rspam06 X-HE-Tag: 1675154404-379683 X-HE-Meta: U2FsdGVkX19axQwrQl6ssK0ey0NmdUc2hMGjzYBqHLP74Vf6+xdKX61Bk+d9PPuJTzh2tVcMH9mnkYMQznwOjfKfbhlQngKDHYLjnYKyqJKtvosMWzDEvCXdtzrEQu4uYB1s4oX2nuXU+yYjgWxc0B2g6MHecT84Ri7esGSrBGR0bkw3i2algb6YwGk6+PoyURsajtDSQBZSMvhgiF0GH8YhAK8qk0/qKUE3Levvku2ZFQ6HSBBZwT8SZAe1oUhFEkM4BJ6WpLyidIkdXFAH0GrkhuWpapVRVe1aQkaAJvIYzvXIGoj84cY4qikHfw5xLP9KI3QMbqRAB6RWlpLgJ0XoTTBJ1uHVJXUKkEdcYf+4Gi0UgEulOXympBv10iqxcQX22FAgIkE6V+WyoZi+vR2vw3pRu690ndXwOQSjVxHeGm39YMogwkEgX79uv64YfgBCRdkq4JXUTGGB5183Pe7fD4ezuSS7uQwrRvk8JcPq2xIDZUuVzOHifP8lZi4QP9TqnXjHRjRhLiQtveulwE7Z8G3TgvEMHT4/y95wLWZmbVoOkASVFv5wGLNDaJ0fZcHpYZG2+YaRoLKpIX9s8sBt+VF9wVZKpGC0OffWEJCr4IgZVsBUaOLec5ey07qzUVBH6my2Cbq3CxZgyF0Zt2C92Rq79QrNGlYScRXdkNIckgQVO7etoCZ/oOmO43BgLx3Qvw+1jTJmlYpS3+ftdYygEs1gKf18ag4mO7iNynaQsqwgNAjcAZH144dyTWYkhz1neEYawYJead8qAHaeRGp2S/8UiCKVgfyR1a4U1Tc8ZCzyteMTrhBZj0VO6/i+nE6u71UYHeX/gzFDdMeNkddbou1blTUCEs6IDziOAPe6ORFHLqFW7tRDBhWGI1T5n1caslemGMtNK/aQZ7i0kUzgYr4+kQDDdVkQVayq8GNZ6ji1OyBpyIhYA9Sdr6Ovyxxt7nsZ1bPwlpyKESV kGlIT/Cq USeppoNQoQpxy1r9cRE6bhz1PkfBL1UriyfH8yCzfexg/Nd8NP7No+1Vj4H2LNCNxZ2quXT/134ycgcfAk4yBLPfYbyX7zKs3IdL5WP66njVtvhe0ZkqSmxz0VZViJIHy9ZXv5CjadLxk565sepBqp2e8l5gJ+/4zeqU0+G8aHRCh2KMFYW+YMpA+WPl3gncMGPIvWhFBxO6svwLGUZjkR8SbJkejbrJw5DfLbufIX1OoN1ldviYKFLRW6D3T9a9t/ECvkEvDHx1Dp6NSi3yTHxqXHJWlnNW2rAKKAIPsAfbbgKwngo/zH2ubsq0BaZ131ATB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to hashed-waitqueue wait, assuming an input 'ret' in ___wait_var_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index fe89282..3ef450d 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -247,7 +247,7 @@ struct wait_bit_queue_entry { struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ From patchwork Tue Jan 31 08:39:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4A44C38142 for ; Tue, 31 Jan 2023 08:40:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 614716B0095; Tue, 31 Jan 2023 03:40:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 509A76B009C; Tue, 31 Jan 2023 03:40:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E2EB6B0095; Tue, 31 Jan 2023 03:40:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id ECFDA6B009C for ; Tue, 31 Jan 2023 03:40:05 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D3F1FC058A for ; Tue, 31 Jan 2023 08:40:05 +0000 (UTC) X-FDA: 80414446770.26.4E6199A Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf20.hostedemail.com (Postfix) with ESMTP id A4A8F1C0013 for ; Tue, 31 Jan 2023 08:40:03 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf20.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154404; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=Of4d+Ta7WL1sXWE7irBEC9bLj6kWAVI8eIFLuPC86NQ=; b=bNcLlOZ+vwtXjleSvGdFgj/J53Pf3tuRKxxK9cBYzCpwa44CYOftIaL1mpNKhg7Cn/jBeH vJi1io/hHdLY5zbawMv3nn8PrgqVBhfSr1LcQwLaWqCHMLozv9Ol27PeWXSLDwFMPcUL7g 5FThnXevmCq5v1e3+5uViz/MZPDGsXs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf20.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154404; a=rsa-sha256; cv=none; b=jTbXmh4uSmDlXdq6C5tse9LPfA0KfGo238ongV6M5kxynNgbPkEXFvKpY6wlLQLlP7Tmqi LANzh20AI+g20dvDHfZ006UHGbnJPV03NW7wAUpvnkLtQ66A9fdMTQbjbg1wPysA3yDW7v Bm9d44s6+kKVDZwSilLHYvL2Dfky+LA= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 22/25] dept: Apply timeout consideration to dma fence wait Date: Tue, 31 Jan 2023 17:39:51 +0900 Message-Id: <1675154394-25598-23-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A4A8F1C0013 X-Stat-Signature: ekfbci6i7tn8o3pzcmcr96a5jf3eiset X-HE-Tag: 1675154403-147535 X-HE-Meta: U2FsdGVkX18BhVWRjo3hdIBsLTBbXQahc2lmMqIsapr/Ic9j1Z37wuRBepOsWbDXoSHjad4EMhRwTLbbawjYHmnpX9dXYxyVq7SUHU/nPCJCUzLE/ZDh+32uKq9yKWNYK0gxnqM20Eyd0pOMMT9OOg6QxuKFHRKtDAcce+iYS1fxpqKahLdOrDSQOjqWTZwHx3e1OnWlOEqd0zbz0vKpZY28jdf+TpunBqXLngsW5612Smd1P/2V2QATIYubHJ8p7S+29tmvpu0NVagXwOliMrwplC9Be1IiD62wTDo8Jr1K7ZnT0oAn/9lWgW36lFRa69O3K35MnGpebmUOmR0SnWk++z4/LTDFtfP62lxUcjKN0aFD1D2XXBpkqzCA8/VtjcpcO4iwvNKVWKkIjhPYhiF+NWzaSeQ6Jrit4HhTUas6nJAaWIV0ZAdfLH+JWFgUSUz06yrCuq0LOepdJmhpi7yE1YDHhMKyqamD0ElN/HUfyxcm4R7NmHtWs7274H/w8HGfcyM6MZcyd/aPwq3OosZoLekglmZGMeIuWURASk388atL2I6XIkpCI2G08YZDlcs8eahb5wAjqM869OfdOb4D9JRzMsI5ZXcdPp4ihlxoDHiiS/kxCqq5cwCiHOoZjvDL5PlDyKJa7Z5drsOS1Ajc/hfxWHhEDUXNHSN7yqZNb43GkG1BBRCstmuXWgsjF3Nj/5MiOHvAhrdHFKgp3yXX3QxMbND3lwDGn7STBFNjqPFNS1ER7MVATtiKiZcPjnCOiuQM9vxwpEvTGQlqlFHnzcY/n8jpG/AdGWtLi0uL7JkMz3LyxZdwiU+xCFzmCh+hzG0sPsxgayt6pEqOjxnae1Vpqu687gAdZd3/mz5b81BkKIapIufHY+zcTobY4s6P/f/igXEa94LlKA/Soy2n4v6ugFUCV0CG934pb6Chsb1vuuxOy37161DaeXzWMKcy1B05mWDy1A+Qwub YV2sUZT5 YN1K0hHuedR+MUJMo07jL138AosBQQqN9pBFv2t7HChNbWAhst1/XDX8QBL3vFJYZvIs+HB1IYYcP6iNd0HCYf44XFp7LTbC8w7YEawSi2gu5MzQ1GG56Y6cYCnc8x69Jof7Yn5r48oiZ0nNC+MSlemnTOucdjjoN0qA0UJ8x0ykhQY3u7WQf9ITsZDs3kS8jkIycB0OqiCKwU8GjL+rmZEprJteO6Y49ZeH/NtQpkHbM33wfsM705UvIytIbGM3XSuy0npehOZusRZrUmI9H0F+6WzsIR6soWb1Ul5Uv3keEYylesgD8J9mTmhcRQ94ti8nd X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to dma fence wait. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 1db4bc0..a1ede7b46 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -783,7 +783,7 @@ struct default_wait_cb { cb.task = current; list_add(&cb.base.node, &fence->cb_list); - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -887,7 +887,7 @@ struct default_wait_cb { } } - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); From patchwork Tue Jan 31 08:39:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8000AC636CD for ; Tue, 31 Jan 2023 08:40:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0BDEA6B009E; Tue, 31 Jan 2023 03:40:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 06BFC6B009F; Tue, 31 Jan 2023 03:40:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD92F6B00A0; Tue, 31 Jan 2023 03:40:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CB93A6B009E for ; Tue, 31 Jan 2023 03:40:07 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A5AD5AB080 for ; Tue, 31 Jan 2023 08:40:07 +0000 (UTC) X-FDA: 80414446854.16.F997613 Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf26.hostedemail.com (Postfix) with ESMTP id 64547140006 for ; Tue, 31 Jan 2023 08:40:05 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf26.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154406; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=FNqlUmZTp0E++WFmgXno+LGpGRslaoBWEseXFWsLHaA=; b=PvECJcnWzq6oSe1rwvuREFvGt3WFaH5zcb+d4ZAwlYC8g2ZQ8pV9Pg5ovD6Gio2YgPDFVT GsliwDx0ARZE7B92f5KoODSc+ms0k6CP5b8/wz11xNKGTPGTC9fvQyib9JFw6ZGGYa6GnX 5pb5OQJKd1MyPJNx04wLfC0/OHcV4MU= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf26.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154406; a=rsa-sha256; cv=none; b=3W61fsZSflTwsvRkpf/CTZqexp3KrZzI1X/TB+K9xPDx+CQoPBhv2DnaYRC18wke1yrtpW fIG6P0i9hfc1OyCleWOZTE7UiBIWwTQWIhmzOkMWKdSHoUr82sBU7tHfskQoZQwiUd6WXG QK8QaF0/TpXXuboQ++1QpqzTOXNaR80= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 23/25] dept: Record the latest one out of consecutive waits of the same class Date: Tue, 31 Jan 2023 17:39:52 +0900 Message-Id: <1675154394-25598-24-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 64547140006 X-Stat-Signature: j7qioxuh8c6cib3yf6utbrb7o5ygiwtb X-HE-Tag: 1675154405-792328 X-HE-Meta: U2FsdGVkX1+LNOd61ai+lip1xLxvKVupcSu4hobqpqIeeUIiuNPNu8OvR048JW9YLYWt9nfurBf/V3W8cKSTsThCc2cHLWBbe6S6OPkU9V0bdH6OYvWzcjd1HCKffW1luU6SCU5amHyf4ugkWn9YFGiwGn/kvzGgSXa9SzMpPMlvR6g7Fh9cgzPrHWIRatlND0YzDN9qTYbf/uWkB0HySYjebNTdH3/M0qKpBOEq/3Rm46yXOlXpm5IYX3j9EFEh3B29CxElcqTNyotC8tnwBnYSxkmZjYiLaoWfanX/UGN2fotymSBuGShBb8GNXh6tadOmOXZIJaaFGRRucgqJqmgNzDX8JK6rn3J6buWL7Y7ysAAkNFfFVvk/IcWtlvPmvOd2p+Wcqa7fjTy3GB3EeZfUBzQ9mCm5C7sSRIONwFeNYcQcfaiM0FDm61GUhIOXmsq9xUZy6Cz+J7BdNDbC6pAe1vGDJsjMmgDofZimaRn0eiRiyJrE6schxey6zaVe9m957aGzjEdWR9ItTWjQNWC/tzKH8/ZwP53u5qAPDxt2Nc6x4O+s4WmBD6qhAl6SbC/ehsZ8ZW4I7MaO5JTfs4Zsaxx7oMkEw8qHiaY96zFiB5+5QCEe5h3BJ8voiGXppkphsrRJsII1yfxyq2zMTW/+C7hKHKhnV6MoV8Qxggf2622/6HMnVUFYJBE9TmtH1CMfMrOYgjleCI4H6iSEDW+7xj3ZAayyXlzeDs9ePCMzy+81djSf5KWmWv1FlZcPwbrSWMZdP63CFDt6ygoncsUO1ESMjSjnj/2cWasudSUTQ+RtfZFsXbHyX2+HYnrQT9JlwP8wrHOK0lfI2GKU+eAdRXEpxDUUEBB4IbIVjSk7r5EJjVafweIGrQhdAYEkAZneI4Uxk0UYBcFElYPximcDKWVw3BaeHfCF3Kly31tIPAs5mfCo4bkfX4U7Ks6Y2XrCfLrsnd5coGEQh5D xjanxrFA Ht0PnDE2/0oHa6B3/b4NBtANpm8k+3+yxof6lR2FQ1UWUQ70LnNszq0WaxjCEYls5KFw7Luy7smoFboebdTs87HumVDd0nOMWgW7yC3GtM6T5zE+I0oDQiN+qyuh7WGVRB0hAmJfbe42gtv9Ol6/h38Id029hdP9kLJmKKl1163mnj++2ZCd34j0oQjpBBTF+cdY4OBW0tSiAurBIn0PuW/sRQBIJyzmfBxi/CZgdfiyZIEPkeI73SJrEb+HWb9nCHjZAodMkQQBWZxPdkfq9hOeD7QEktkTwQZ1docoGB02oT3dMMYYSxTy1LfeeNrutArqgTRCBsOkDRiimAHzPxUBppFhQD7MMeb4VppB9a67QSjzPrzOH2sm20GY7+y6p5buM1jE6ZPJA/i52T6X5JHy4dHfq/lVMWH4f9lYmOabPSOLosaUIEKGitzhFW49Prp2cm8ydBoSXUHdKie8qx7ySiES9hMj0P971tGB3ACdWFVs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The current code records all the waits for later use to track relation between waits and events in each context. However, since the same class is handled the same way, it'd be okay to record only one on behalf of the others if they all have the same class. Even though it's the ideal to search the whole history buffer for that, since it'd cost too high, alternatively, let's keep the latest one at least when the same class'ed waits consecutively appear. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 3ab999f..2887858 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1522,9 +1522,28 @@ static inline struct dept_wait_hist *new_hist(void) return wh; } +static inline struct dept_wait_hist *last_hist(void) +{ + int pos_n = hist_pos_next(); + struct dept_wait_hist *wh_n = hist(pos_n); + + /* + * This is the first try. + */ + if (!pos_n && !wh_n->wait) + return NULL; + + return hist(pos_n + DEPT_MAX_WAIT_HIST - 1); +} + static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) { - struct dept_wait_hist *wh = new_hist(); + struct dept_wait_hist *wh; + + wh = last_hist(); + + if (!wh || wh->wait->class != w->class || wh->ctxt_id != ctxt_id) + wh = new_hist(); if (likely(wh->wait)) put_wait(wh->wait); From patchwork Tue Jan 31 08:39:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61BC9C636CD for ; Tue, 31 Jan 2023 08:40:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B9BC6B009B; Tue, 31 Jan 2023 03:40:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2676C6B0098; Tue, 31 Jan 2023 03:40:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9E736B009E; Tue, 31 Jan 2023 03:40:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D41776B0098 for ; Tue, 31 Jan 2023 03:40:06 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id AF095806A0 for ; Tue, 31 Jan 2023 08:40:06 +0000 (UTC) X-FDA: 80414446812.20.6D8760F Received: from lgeamrelo11.lge.com (lgeamrelo12.lge.com [156.147.23.52]) by imf18.hostedemail.com (Postfix) with ESMTP id 4D5271C0003 for ; Tue, 31 Jan 2023 08:40:03 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf18.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675154405; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=H655FqK3GVS8tdxPQzBgvLYDOwUyukSMixkdu172QvQ=; b=552drqh1ODx7ShuGLlIyhxP3IezjTG2JsfVEwQG9GfEPXfIVcVgOAFQhcUozDwY5gFQvtY beDCWb3wXpkr/KpW2nec1J3GQ8r1dE1dOb5oSVf4qJ9x4z90P7vNnP8rqVjWx1766iH1vp L7wGjUyU+Mi0k7chPS8h0pEc8ahbCuk= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf18.hostedemail.com: 156.147.23.52 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675154405; a=rsa-sha256; cv=none; b=U4XRAi5fAdwUHoyXPmHBbm2aU+Xh3pdkZGqxOhfogVYZqyfrWtTauKzzJdfZ05LasqSVwM e1SFejYOR/Le2JMXazECqkPWEwGIbdy0VSvXiXDK3iWQvTJo75cx/NiBAIDedjc4teUZuO UaPduXTqXPq8WRNYWz1RUvAlpDbFU64= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.52 with ESMTP; 31 Jan 2023 17:40:03 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:02 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 24/25] dept: Make Dept able to work with an external wgen Date: Tue, 31 Jan 2023 17:39:53 +0900 Message-Id: <1675154394-25598-25-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 4D5271C0003 X-Rspam-User: X-Stat-Signature: qzqzjnfip4ehesw9gy1tptr1hua9nbpm X-HE-Tag: 1675154403-265113 X-HE-Meta: U2FsdGVkX18jPMQUO/NxtJ5BvAj7FTMRp4c/ke/MnQ07iX3oVYoDx6etNz65aolJSF51PrHYirTQCf8XYntiQPmK4EMq3Xz4E+ow8DA2Ib7r1v63xceY5OQjpNMUC5CmNRONjpz0XR/Lvy7LWWxgCKVy0efz6tcPUQdi8mn26UFUBDFrSMotQTcB6XKzS+Gky7l0sp4HWwoMUX46FEnllhfcW4I6R7ixzNFxMywrvGOz9wcPFXzE63MtJWHM6RFy/vDJs2jUarKpeY+R3WPJ4ff5qixEot+KEH5X2hNKOgjv36WfyJ0zoSQQanm52RmXi2r0wYwUY70oZBLiJXPG8rJVMalNlpjqSVxqKvaNHNNgrGnL0r/AT/GyhXL6O2x9yPq33M0dbduPU0/T54IIJ/AF3txvdCz4T6No1PT3c7PZ4KGPsahMrvUJVvUEx/owR9SXbHxnQtGhVqH7960AFFPj+u+C8U7PyWHE6gz9iabKtL6XI6+49yqLs76xJx4xskS39HugURyyDZjdgQh1gZ3JwmHrdif1x3G/d2ps8er1RH7JscwdqMwlb42O3nirR4W6eVLEZUWE8E9Je4BBYyahL/REiZd26AER6twV0ohzq7h7kS9dGMWm5te2pH/S35w8t9N25eU+NOLdWRjjet3sWUiGrB4cGRugdLRb3rP0nTmw3hM8asNn0GdmA1j2U4mg+KpHsPeJLL4UsokUnWBxnTJxqfon6uCkO60HUoeMAmGhgDH4NDuoN5Z12rKPq9zgIR3tsp0maUEULo/8YCL2p7hopITGFHKMvLfbPurtAi8KPihdL/Pmz6rfp+lPkuokAzYUOAAk6w85oHAP5+lD+8tBIi88x7CPDGPhh9xkiKekcLKRBS5MS29BUCJjnELtSsZ3IX2rji8tSn0GWRSBGO/wyRylD/OjnBrngv9Wavyh+lg8I/KopMskmI34uUzbYTf0GZ3jHjGvPSg 5hqPZdjw Wp0PO/wr4LaehmW1rnUZF92udzGugqPQS6v66fBoaNcLIBN2FSsxalLVfII8uJg9fbu7OQ6pFeDm17NVafReV6NILU6b0xm7eR7K0CaCoJ7PkWfp5ZbO7kvskrhXeNLngm8pgIhHUxGXMsmv0zfhnPGxZyDQ2FNzkBgxvKARVkvoQvBUZLaNNPy1SzvUxsKixesD8fHAYOMMAWFINWfAM8Go+Pz8DLWq2h7nfHGQMWELw0QK7I2atbGJ4FCoYszUyIIB+8MxPGb5wniatigbF5YvMLG1JALsjqGnrQUYhVBql8x1ZfCPXILZJaZ521MowV6SV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There is a case where total maps for its wait/event is so large in size. For instance, struct page for PG_locked and PG_writeback is the case. The additional memory size for the maps would be 'the # of pages * sizeof(struct dept_map)' if each struct page keeps its map all the way, which might be too big to accept. It'd be better to keep the minimum data in the case, which is timestamp called 'wgen' that Dept makes use of. So made Dept able to work with an external wgen when needed. Signed-off-by: Byungchul Park --- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 4 ++-- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 37 insertions(+), 15 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 0aa8d90..ad32ea7 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -487,6 +487,13 @@ struct dept_task { bool in_sched; }; +/* + * for subsystems that requires compact use of memory e.g. struct page + */ +struct dept_ext_wgen{ + unsigned int wgen; +}; + #define DEPT_TASK_INITIALIZER(t) \ { \ .wait_hist = { { .wait = NULL, } }, \ @@ -518,6 +525,7 @@ struct dept_task { extern void dept_free_range(void *start, unsigned int sz); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); @@ -527,8 +535,8 @@ struct dept_task { extern void dept_stage_event(struct task_struct *t, unsigned long ip); extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); -extern void dept_request_event(struct dept_map *m); -extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, struct dept_ext_wgen *ewg); extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); @@ -559,6 +567,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) struct dept_key { }; struct dept_map { }; struct dept_task { }; +struct dept_ext_wgen { }; #define DEPT_MAP_INITIALIZER(n, k) { } #define DEPT_TASK_INITIALIZER(t) { } @@ -571,6 +580,7 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) #define dept_free_range(s, sz) do { } while (0) #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_ext_wgen_init(wg) do { } while (0) #define dept_map_copy(t, f) do { } while (0) #define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) @@ -580,8 +590,8 @@ static inline void dept_ecxt_enter_nokeep(struct dept_map *m) #define dept_stage_event(t, ip) do { } while (0) #define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) #define dept_ecxt_holding(m, e_f) false -#define dept_request_event(m) do { } while (0) -#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event(m, wg) do { } while (0) +#define dept_event(m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 21fce52..8cdac79 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -24,7 +24,7 @@ #define sdt_wait_timeout(m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) #define sdt_wait(m) sdt_wait_timeout(m, -1L) @@ -49,7 +49,7 @@ #define sdt_might_sleep_end() dept_clean_stage() #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) -#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL) #define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 2887858..bce3dc8 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2230,6 +2230,11 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, } EXPORT_SYMBOL_GPL(dept_map_reinit); +void dept_ext_wgen_init(struct dept_ext_wgen *ewg) +{ + WRITE_ONCE(ewg->wgen, 0U); +} + void dept_map_copy(struct dept_map *to, struct dept_map *from) { if (unlikely(!dept_working())) { @@ -2415,7 +2420,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map) + bool sched_map, unsigned int *wgp) { struct dept_class *c; struct dept_key *k; @@ -2437,14 +2442,14 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { - do_event(m, c, READ_ONCE(m->wgen), ip); + do_event(m, c, READ_ONCE(*wgp), ip); pop_ecxt(m, c); } exit: /* * Keep the map diabled until the next sleep. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wgp, 0U); } void dept_wait(struct dept_map *m, unsigned long w_f, @@ -2654,7 +2659,7 @@ void dept_stage_event(struct task_struct *t, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, &m.wgen); exit: dept_exit(flags); } @@ -2833,10 +2838,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) } EXPORT_SYMBOL_GPL(dept_ecxt_holding); -void dept_request_event(struct dept_map *m) +void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { unsigned long flags; unsigned int wg; + unsigned int *wgp; if (unlikely(!dept_working())) return; @@ -2849,32 +2855,38 @@ void dept_request_event(struct dept_map *m) */ flags = dept_enter_recursive(); + wgp = ewg ? &ewg->wgen : &m->wgen; + /* * Avoid zero wgen. */ wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); - WRITE_ONCE(m->wgen, wg); + WRITE_ONCE(*wgp, wg); dept_exit_recursive(flags); } EXPORT_SYMBOL_GPL(dept_request_event); void dept_event(struct dept_map *m, unsigned long e_f, - unsigned long ip, const char *e_fn) + unsigned long ip, const char *e_fn, + struct dept_ext_wgen *ewg) { struct dept_task *dt = dept_task(); unsigned long flags; + unsigned int *wgp; if (unlikely(!dept_working())) return; + wgp = ewg ? &ewg->wgen : &m->wgen; + if (dt->recursive) { /* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wgp, 0U); return; } @@ -2883,7 +2895,7 @@ void dept_event(struct dept_map *m, unsigned long e_f, flags = dept_enter(); - __dept_event(m, e_f, ip, e_fn, false); + __dept_event(m, e_f, ip, e_fn, false, wgp); dept_exit(flags); } From patchwork Tue Jan 31 08:39:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13122570 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 725AFC636CD for ; Tue, 31 Jan 2023 09:10:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 168E36B007B; Tue, 31 Jan 2023 04:10:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0F1B16B007D; Tue, 31 Jan 2023 04:10:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E60B86B0080; Tue, 31 Jan 2023 04:10:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D67726B007B for ; Tue, 31 Jan 2023 04:10:09 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B24AE14032A for ; Tue, 31 Jan 2023 09:10:09 +0000 (UTC) X-FDA: 80414522538.20.D5157FA Received: from lgeamrelo11.lge.com (lgeamrelo11.lge.com [156.147.23.51]) by imf01.hostedemail.com (Postfix) with ESMTP id 2F2684000D for ; Tue, 31 Jan 2023 09:10:05 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf01.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675156207; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=30NU9u78xhDWXaHtqQ5YbCjG+v5+/HtqKX9JK6BUsIM=; b=zHFlSI6E0GwCuaQmXC7aJrRL/2bDmNohOXHTv/3af2q785EE1aZuYazKvpSjle0tQJy8c+ 2jX4c+Dvw6qHJ+cj1203ivw9yGOVTf5019o66zhnkq5jDlwIPy4lueC64XwuVJC0ttf9hS 2icSnxPKy9Nr5O+FU46j9s4jdNGQUy0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (imf01.hostedemail.com: 156.147.23.51 is neither permitted nor denied by domain of max.byungchul.park@gmail.com) smtp.mailfrom=max.byungchul.park@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675156207; a=rsa-sha256; cv=none; b=wpa57EEnUgxyuUH+eEHJYwBD/6MKk49cKl7cfvDLGaiIjIPugZc+IteKyNwwjn86YPnGpL vS92YuMHsTfzW8TkyKreAkqN10r600oFIQb2GKSYrGjXa0y7jwx8+hV8rPjE9y4SqklbiK HC/k5KFTVYwWCY+Ic8JUrjVqeHX6WDs= Received: from unknown (HELO lgeamrelo04.lge.com) (156.147.1.127) by 156.147.23.51 with ESMTP; 31 Jan 2023 17:40:03 +0900 X-Original-SENDERIP: 156.147.1.127 X-Original-MAILFROM: max.byungchul.park@gmail.com Received: from unknown (HELO localhost.localdomain) (10.177.244.38) by 156.147.1.127 with ESMTP; 31 Jan 2023 17:40:03 +0900 X-Original-SENDERIP: 10.177.244.38 X-Original-MAILFROM: max.byungchul.park@gmail.com From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, paolo.valente@linaro.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com Subject: [PATCH v9 25/25] dept: Track the potential waits of PG_{locked,writeback} Date: Tue, 31 Jan 2023 17:39:54 +0900 Message-Id: <1675154394-25598-26-git-send-email-max.byungchul.park@gmail.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> References: <1675154394-25598-1-git-send-email-max.byungchul.park@gmail.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 2F2684000D X-Stat-Signature: 3egdfo3fwxdb3rtnpkko9jpfaw3pdstf X-Rspam-User: X-HE-Tag: 1675156205-53731 X-HE-Meta: U2FsdGVkX1+Ny5MoyJ0EMsnMMOtNPdzd/CpN+6ZnGjf+0UcNxtYf0o3CJ+a38gigJNQYEYFZO6gy2VKLP4qeUIh0q8ylhNuxSyiStLZf/G9QWEBWwxMsPCxyK/ZW8TkcSWUuZIPFb6kM6yiGTbT0Fxre6J7/nnRA8zzquTprnUeXxrPpQB6YRggUgCSjZUavSvAE7dLLswvlKm+u5AbwFolPIdRFquAVA4xkOzSHXTxFT5mqACzzYd2SrNM1Duk5d8xY3flRCi8NX8RLfR5/JG1ITQl2jUncYxSuOb6iQDclB9SE4OuijbaogJAH2WUE+SVneFXNPjJCyCowrGQIU6vsdKVbrcsdYtOGMDwOc5ECHaQH8+B2bAtxA1LAU6iXcQ/FtqynySppFn1nKO/hkR5larm3RBAxkrng3Y6CRRFcl9I9yJsjAJU2MnGaf5e8aNgM/aWDKutonINq8GbCswkuWiZ/h27xVEIbdk4beNPFmIJF0Wr39on+7+MvIMhgRb3Dm6TFNnScw9Gm8BVTaPN1d0iVPRKrFb9HNEg0ERPbjK4nx4bvbRFoUe1HBaIt+f4O5Up0TjJ7jVzz2yq7jrCSG1ncBBgFlSTeyQAjykigaqoNshxWHTc3/bnXjtTLhO2nuEtg0aotaeZUBMuWFsMsqlwLkQNcCozLaeWkRm5zSGBGzgYAcINuB/ryC6GsHTCikKTVn5peJsGvvp3Dt8GwII99Jx1/oBnJNOHyQgyx+z8WcUatY+aQUOfhakxueXCQis5pgGi4A6Kqe+7rsHNUK71veTg9RJPS7P8EVVnNR0XJV8tZgHr0pvqkn1e25N6ePWwURcpVOY095iVHAxGov2mN56SjjNxcqneR02FFXSRRUoguFTnosr/OSwpR6RH9X31h0hAEFcbHW+XdGrQ8oKIInWUFS3bbF28buZB/CLGSjtzZS1dC5rtzHMbLirokP+jKXL+5ck0E+7K KGLSUnzj PK9f6MbHrAAkuk1PzhkALie3rip2JTQzAuyFt1QR/Q/yAqjpkzmVRKM4N5Otai0vuhrShMg71zPmbe4dj5JGBxwtY/oVNPwFoGcZoJkW4hLhn/CJSeqKRxF310Ce6IpZJJ6Mrm7GAnDEN8wyjLgBGpQrVoCE7pXLlJJDuaKU8afWu8SlP5U7E8AQQz1Bx9/5Rd/tV2buqkM3ExSre+jTgV7PXBiWs1Rxsgj8kV5XZcr/yaX2/y0uIxKskWv1eJsdiP4/TkNNZNzfKZsV9cZ662+Q7jnobZphhQoNktmOa064WSQ2waH1IEqA9pA1fe9N7bDLEHPxvoCgKJfT0PCKAN2A2fGJA8wZ+bdqRyUh64uWf+vE+TqbAwJEW7kA6gb1wdGwYrH1uQygCOpN3Arh4o9F9RbCTHy9EMue0yAV+5kl5IisgpXDNa998TzjMdYikrR0JdxxtZ6bgH0cbUHCwvJXbbuMlph8PPYw7b9M5epKcqiWvq178dYSrZjI+gLuo2rDBa4HBfUB14nYhC4/3IyM/TVU4MjCENP0lHSI6eo+TXpY3XoNx+gQiGjTe5IYVYGN9kD1N3fNyav/qvAcXLk+36w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, Dept only tracks the real waits of PG_{locked,writeback} that actually happened having gone through __schedule() to avoid false positives. However, it ends in limited capacity for deadlock detection, because anyway there might be still way more potential dependencies by the waits that have yet to happen but may happen in the future so as to cause a deadlock. So let Dept assume that when PG_{locked,writeback} bit gets cleared, there might be waits on the bit to be woken up. Even though false positives may increase with the aggressive tracking, it's worth doing it because it's going to be useful in practice. See the following link for instance: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/ Signed-off-by: Byungchul Park --- include/linux/mm_types.h | 3 ++ include/linux/page-flags.h | 112 ++++++++++++++++++++++++++++++++++++++++----- include/linux/pagemap.h | 7 ++- mm/filemap.c | 11 ++++- mm/page_alloc.c | 3 ++ 5 files changed, 121 insertions(+), 15 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 3b84750..61d982e 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -19,6 +19,7 @@ #include #include #include +#include #include @@ -252,6 +253,8 @@ struct page { #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS int _last_cpupid; #endif + struct dept_ext_wgen PG_locked_wgen; + struct dept_ext_wgen PG_writeback_wgen; } _struct_page_alignment; /* diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 69e93a0..d6ca114 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -202,6 +202,50 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H +#ifdef CONFIG_DEPT +#include +#include + +extern struct dept_map PG_locked_map; +extern struct dept_map PG_writeback_map; + +/* + * Place the following annotations in its suitable point in code: + * + * Annotate dept_page_set_bit() around firstly set_bit*() + * Annotate dept_page_clear_bit() around clear_bit*() + * Annotate dept_page_wait_on_bit() around wait_on_bit*() + */ + +static inline void dept_page_set_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_request_event(&PG_locked_map, &p->PG_locked_wgen); + else if (bit_nr == PG_writeback) + dept_request_event(&PG_writeback_map, &p->PG_writeback_wgen); +} + +static inline void dept_page_clear_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_event(&PG_locked_map, 1UL, _RET_IP_, __func__, &p->PG_locked_wgen); + else if (bit_nr == PG_writeback) + dept_event(&PG_writeback_map, 1UL, _RET_IP_, __func__, &p->PG_writeback_wgen); +} + +static inline void dept_page_wait_on_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_wait(&PG_locked_map, 1UL, _RET_IP_, __func__, 0, -1L); + else if (bit_nr == PG_writeback) + dept_wait(&PG_writeback_map, 1UL, _RET_IP_, __func__, 0, -1L); +} +#else +#define dept_page_set_bit(p, bit_nr) do { } while (0) +#define dept_page_clear_bit(p, bit_nr) do { } while (0) +#define dept_page_wait_on_bit(p, bit_nr) do { } while (0) +#endif + #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); @@ -383,44 +427,88 @@ static unsigned long *folio_flags(struct folio *folio, unsigned n) #define SETPAGEFLAG(uname, lname, policy) \ static __always_inline \ void folio_set_##lname(struct folio *folio) \ -{ set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_set_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void SetPage##uname(struct page *page) \ -{ set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline \ void folio_clear_##lname(struct folio *folio) \ -{ clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_clear_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void ClearPage##uname(struct page *page) \ -{ clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define __SETPAGEFLAG(uname, lname, policy) \ static __always_inline \ void __folio_set_##lname(struct folio *folio) \ -{ __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_set_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void __SetPage##uname(struct page *page) \ -{ __set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define __CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline \ void __folio_clear_##lname(struct folio *folio) \ -{ __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_clear_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void __ClearPage##uname(struct page *page) \ -{ __clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define TESTSETFLAG(uname, lname, policy) \ static __always_inline \ bool folio_test_set_##lname(struct folio *folio) \ -{ return test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + bool ret = test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy));\ + if (!ret) \ + dept_page_set_bit(&folio->page, PG_##lname); \ + return ret; \ +} \ static __always_inline int TestSetPage##uname(struct page *page) \ -{ return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_set_bit(PG_##lname, &policy(page, 1)->flags);\ + if (!ret) \ + dept_page_set_bit(page, PG_##lname); \ + return ret; \ +} #define TESTCLEARFLAG(uname, lname, policy) \ static __always_inline \ bool folio_test_clear_##lname(struct folio *folio) \ -{ return test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + bool ret = test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy));\ + if (ret) \ + dept_page_clear_bit(&folio->page, PG_##lname); \ + return ret; \ +} \ static __always_inline int TestClearPage##uname(struct page *page) \ -{ return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_clear_bit(PG_##lname, &policy(page, 1)->flags);\ + if (ret) \ + dept_page_clear_bit(page, PG_##lname); \ + return ret; \ +} #define PAGEFLAG(uname, lname, policy) \ TESTPAGEFLAG(uname, lname, policy) \ diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 29e1f9e..2843619 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -898,7 +898,12 @@ bool __folio_lock_or_retry(struct folio *folio, struct mm_struct *mm, */ static inline bool folio_trylock(struct folio *folio) { - return likely(!test_and_set_bit_lock(PG_locked, folio_flags(folio, 0))); + bool ret = !test_and_set_bit_lock(PG_locked, folio_flags(folio, 0)); + + if (ret) + dept_page_set_bit(&folio->page, PG_locked); + + return likely(ret); } /* diff --git a/mm/filemap.c b/mm/filemap.c index adc49cb..b80c8e2 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1097,6 +1097,7 @@ static int wake_page_function(wait_queue_entry_t *wait, unsigned mode, int sync, if (flags & WQ_FLAG_CUSTOM) { if (test_and_set_bit(key->bit_nr, &key->folio->flags)) return -1; + dept_page_set_bit(&key->folio->page, key->bit_nr); flags |= WQ_FLAG_DONE; } } @@ -1206,6 +1207,7 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, if (wait->flags & WQ_FLAG_EXCLUSIVE) { if (test_and_set_bit(bit_nr, &folio->flags)) return false; + dept_page_set_bit(&folio->page, bit_nr); } else if (test_bit(bit_nr, &folio->flags)) return false; @@ -1216,8 +1218,10 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, /* How many times do we accept lock stealing from under a waiter? */ int sysctl_page_lock_unfairness = 5; -static struct dept_map __maybe_unused PG_locked_map = DEPT_MAP_INITIALIZER(PG_locked_map, NULL); -static struct dept_map __maybe_unused PG_writeback_map = DEPT_MAP_INITIALIZER(PG_writeback_map, NULL); +struct dept_map __maybe_unused PG_locked_map = DEPT_MAP_INITIALIZER(PG_locked_map, NULL); +struct dept_map __maybe_unused PG_writeback_map = DEPT_MAP_INITIALIZER(PG_writeback_map, NULL); +EXPORT_SYMBOL(PG_locked_map); +EXPORT_SYMBOL(PG_writeback_map); static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, int state, enum behavior behavior) @@ -1230,6 +1234,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, unsigned long pflags; bool in_thrashing; + dept_page_wait_on_bit(&folio->page, bit_nr); if (bit_nr == PG_locked) sdt_might_sleep_start(&PG_locked_map); else if (bit_nr == PG_writeback) @@ -1327,6 +1332,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, wait->flags |= WQ_FLAG_DONE; break; } + dept_page_set_bit(&folio->page, bit_nr); /* * If a signal happened, this 'finish_wait()' may remove the last @@ -1534,6 +1540,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters != 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + dept_page_clear_bit(&folio->page, PG_locked); if (clear_bit_unlock_is_negative_byte(PG_locked, folio_flags(folio, 0))) folio_wake_bit(folio, PG_locked); } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0745aed..57d6c82 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -76,6 +76,7 @@ #include #include #include +#include #include #include #include @@ -1626,6 +1627,8 @@ static void __meminit __init_single_page(struct page *page, unsigned long pfn, page_mapcount_reset(page); page_cpupid_reset_last(page); page_kasan_tag_reset(page); + dept_ext_wgen_init(&page->PG_locked_wgen); + dept_ext_wgen_init(&page->PG_writeback_wgen); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL