From patchwork Wed Mar 6 08:54:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583562 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EA3FE5DF13; Wed, 6 Mar 2024 08:55:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; cv=none; b=CMjEGW/ygG3DJa+NqAFiVCpU29C5qmpgbXt87wcCTWfkrMGFN26NYjsqwimq3dOwr30ZqIpbWkKXzqkHmNgD5XouYCxDGr4IezQB/7g4vHoAsn7VB3x186lDGSFPOz9fRIFFd0ITohVrSePALpEgLTIF3WAVT+wAltZcTx1oXoY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; c=relaxed/simple; bh=3z6wcnzZigmy1AnwKW6EWYXgb3eqsvFGY6rKaYlxPHE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=uVGNqhyMifgx+jLPRRr4X8o5EB8IC8Cv+IkGW2bwWbHugjjsGvFCDRik2ralUXslF7/nTe+Fq9KEgDaTgKBTOHUJVPMTQQz11e/drIsgWeoLtoR8uKs2WSay79qyWHTHfwr0A97stSKhrM+Sy9jq2Rk+PRvWdk6E2A4ATr/3tiM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-38-65e82f7c082d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 01/27] llist: Move llist_{head,node} definition to types.h Date: Wed, 6 Mar 2024 17:54:47 +0900 Message-Id: <20240306085513.41482-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjHfT7fn3dz9t2x+cqIM7/ph8kem9H8wfcPrMk/+IObvtNRx+5U smxxlZ+VkMg5d1dOKpWrJtTtZMpJV3LqUI00aq6yuOiH4y788+y153k/rz1/PCwhr6aCWJX6 iKhRK+MUtJSUDkw1rUwJ7RPD0vuWQs75MPB+P02CvryUhtayEgSlVScw9D/dDB0jHgTjzS0E 5OW2IjB96CKgqqEbQV3RSRpe9U4Dl3eIBkfuORp0BeU0vPwygaHzykUMJdat0HTBjME++pmE vH4arufpsL/0YRi1FDNgSV0IPUX5DEx8CAdHdzsFdW+XwzVDJw21dQ4SGmp6MLx6qKehu/Q3 BU0Nz0hozcmk4O6gmYYvIxYCLN4hBtrsRgwVaX5RxjcfBY2ZdgwZhfcwuN48QmA7/R6DtbSd hideD4ZKay4BY7efIujJGmAg/fwoA9dPZCE4l36FhLTOCBj/qacj1wpPPEOEkFaZJNSNGEnh uZkXHuR3MUKa7S0jGK0JQmXRMqGgth8LpmEvJViLz9CCdfgiI5wdcGFh0OlkhGdXx0mh15WH o4J2SdfFiHGqRFETun6vNDbDaWAO35AeNd6qplORjT2LJCzPreabcrqY/3y3rYAKMM0t5t3u USLAM7h5fGXmp8k+wXmkfKFzU4Cnc9t4Z+6byV2SW8i/KDOjAMu4CN5w34P/OoP5kgr7pEfC reGzB7PpAMv9mWadyc9Sf+Yny5vtLf+OmMU/LnKTF5DMiKYUI7lKnRivVMWtDolNVquOhuw7 FG9F/meyHJ/YXYOGW6PrEccixVRZpOSzKKeUidrk+HrEs4RihixlrFeUy2KUycdEzaE9moQ4 UVuPZrOkYqZs1UhSjJzbrzwiHhTFw6Lm/xSzkqBUJAlll/yOd3y1Xf5xM6Ukq1zbsaisW/da krTv1m5fVNeWOYamtWEbfmUHGwoz5Y13DhRH5+/95HxXtcDXQQ3pdwZXu+eeyh9bUBv9+nZE SMNWn06vvhkud0Xe275xPmFzG8/UmDpXNF+61ubYEeKLUqf3i96v69ujG2+c+jieoGmpUJDa WGX4MkKjVf4BXK6NJEgDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTYRgH8N73nPOe42hxmlKH+lAsJLJSuxgPdiWiDlERQVRC1KGOuZqr NrOMEu+UOs1orovK1Fqmlja1q5poWitblsMurFEmLXEqmLNMqVzRl4cfz//h/+nhKJWZmcFp dHGyXidp1URBK7YsT114KuyrHF5xQQV52eHgGz5DQ0FVJYGOWxUIKmuTMfS2boA3I14EYy9e UmA2dSAo/vSBgto2N4KGshQCnT1TwOkbJGA3ZRFILa0i8KpvHIMr/zyGCttmeH6uBEPTqIcG cy+BK+ZUPDG+Yhi1lrNgTQqG7rLLLIx/WgR2dxcDLYV2Bhrez4dLRS4C9Q12GtrudWPofFBA wF35m4HnbU9p6MgzMnBzoIRA34iVAqtvkIXXTRYM1WkTbRnffjHwxNiEIePqbQzOdw8RNJ75 iMFW2UWgxefFUGMzUfDzeiuC7px+FtKzR1m4kpyDICs9n4Y0VwSM/SggayLFFu8gJabVHBcb Riy0+KxEEO9f/sCKaY3vWdFiOybWlIWIpfW9WCwe8jGirfwsEW1D51kxs9+JxQGHgxWfXhyj xR6nGW+dGaVYsV/WauJlfdiqvYqYDEcRe6RQccJyrY4koUYuEwVwAr9UuPm6lPGb8HOFt29H Kb+D+NlCjfHL3z3FexXCVcd6vwP5LYLD9I71m+aDhfZbJchvJR8hFN314n+ds4SK6qa/PQH8 MiF3IJf4rZq4eZFaTM4hhQVNKkdBGl18rKTRRoQaDsUk6DQnQvcdjrWhiXexnh7Pu4eGOzc0 I55D6snKNQEeWcVI8YaE2GYkcJQ6SHnqZ4+sUu6XEk7K+sN79Me0sqEZzeRo9XTlxh3yXhV/ QIqTD8nyEVn/P8VcwIwk5L6z2zCttSinw7vrV2RidF92VLUHO92bLO3Xt//gIzqjjVTstMdC +sl1L135VnvX4sIbkWvrAo/Wrs5NmX86dImkO/jRE8lqjVGthsxHgceFRFPgPOwa2bY9LDEd L7iYklg2uZ2bY26XZu2M84Tos6Z+7mkOVVUFb7w/aeX3nClq2hAjLQqh9AbpD3/EcVYqAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: llist_head and llist_node can be used by very primitives. For example, Dept for tracking dependency uses llist things in its header. To avoid header dependency, move those to types.h. Signed-off-by: Byungchul Park --- include/linux/llist.h | 8 -------- include/linux/types.h | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/llist.h b/include/linux/llist.h index 2c982ff7475a..3ac071857612 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -53,14 +53,6 @@ #include #include -struct llist_head { - struct llist_node *first; -}; - -struct llist_node { - struct llist_node *next; -}; - #define LLIST_HEAD_INIT(name) { NULL } #define LLIST_HEAD(name) struct llist_head name = LLIST_HEAD_INIT(name) diff --git a/include/linux/types.h b/include/linux/types.h index 253168bb3fe1..10d94b7f9e5d 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -199,6 +199,14 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +struct llist_head { + struct llist_node *first; +}; + +struct llist_node { + struct llist_node *next; +}; + struct ustat { __kernel_daddr_t f_tfree; #ifdef CONFIG_ARCH_32BIT_USTAT_F_TINODE From patchwork Wed Mar 6 08:54:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583572 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0794D5DF19; Wed, 6 Mar 2024 08:55:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715341; cv=none; b=shHzCjzn0PzkdhXR8kCs/Bv8FWxzW85AFkzwOTCwDZVG47pRvCCpcPzEJ8RUCbN8U73ITFR0avqXM+az/zjc3wiBAV4kj6mURH6VQGMC4IJV37YrAuIYw0/HVfRWCRSuMs8ZE4/u6b+3MiSGurRIEiCy6HfHLzDATnVvnURVaN0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715341; c=relaxed/simple; bh=asSKRbIQNcB08UYPRwv3bUSMHrCIvUqTNtShomc8xTg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=IVOEvfX77Yn3apHmB8bfCQKTebsLcuZSFkR5pLb2ENS3C3m2zwkhF/EJ8ilu6tPMtTdV1ajIDVy3epPi1nt32N6LU512M2Vs9TYSqk+n5DGR8zDYK/M0D+tSoh2Y+4vBULJd1VPGUBwluo8AZg7wFZdX58TUlmPkpgte8gdmZ3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-48-65e82f7c619a From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 02/27] dept: Implement Dept(Dependency Tracker) Date: Wed, 6 Mar 2024 17:54:48 +0900 Message-Id: <20240306085513.41482-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSbUxTVxgHcM+5595bGqs3FfSKUVkXskTjCwjLs2RZiCZ6jDGaGT8oLlsz rmu3Uk0LSJ0mIAXxBUUJ4gtubWEdgU5ZC6IgDHAw8AVBCEMGRAlRsIUattuBELEl+uXkl/P8 88/z4VEw6ltspEJvTJFMRq1BwymJcnyhY93xDaPSRvtSuHB2I8j/5RIovunioPNGBQJXVSaG sZZt8HfAh2Dm0WMGigo7EdifDzJQ1TqEoL7sBAfdI4ugR/Zz0F54hoOskpscdHlnMQxcuoih wr0THuQ7MDROvyRQNMbBtaIsHHxGMUw7y3lwZkTDcNlVHmafx0D7UC8L9f1r4cpPAxzcrW8n 0Hp7GEN3bTEHQ645Fh60thHovJDHwm8TDg68AScDTtnPw5NGG4ZKa7Ao59+3LPyV14ghp/R3 DD1P6xA05D7D4Hb1cnBP9mHwuAsZePNrC4Lhc+M8ZJ+d5uFa5jkEZ7IvEbAOxMPMVDGX8Bm9 5/Mz1Oo5QusDNkLvO0R65+ogT60N/Ty1uVOpp2wNLbk7hql9Umapu/wUR92TF3l6erwH04mO Dp62XZ4hdKSnCO+O3K/8PEky6NMk04YvvlHqvIGn7OFZP5secLWTDDTXSk6jMIUoxIm/TBQw H1xjt+GQOeETsa9vev4/XIgSPXkv2JAZwacUSzu2hrxE2CLKOVN8yESIFht8JVzIKiFerJzL ft+/WqyobJzvCRM+Fc9PnJ/PqIOZR1n2oJXBzJRCLL1R/X6J5WJTWR/JRyobWlCO1HpjWrJW b4hbr7MY9enrvz2U7EbBa3Ien028jSY79zQjQYE0C1UJYS8lNatNM1uSm5GoYDThqmNvRiS1 KklrOSqZDn1tSjVI5ma0QkE0y1SxgSNJauE7bYr0gyQdlkwfplgRFpmBctzeXW0WciDKINJ9 daPezD/lAl5eHJ+4PGrl5p+36XYlHUw10/u6dR85Tu5NXRIR0Wf1vYoZ1M3FHa19RidX7SFR +CvPQOZ2IuXG+ZsqY7Xf/5h+JeP19a4dr+DjhD/2PUQr5KovLf//U1xirF79lok+kFiTbbSj /ISCmP7uTRpi1mlj1jAms/YdI8+PmUkDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTcRgG8P7/c9tWi8MUOhRUjEoqckpaLxXRBfIQJfWhhPyQKw85vK1N LSvD3BS7aK5Qu1htakt0lU6jmw5RXC7JLIepTckxKnG5LDcyLXNRX15+8Dw8n14RIbtOLRap 0jIETZoyRU5LSEnsZt26M4rPQsT3iaVguBQB/slCEioeWmjoeVCHwNJ0DsNoRwy8C3gRTL96 TUB5aQ8C08gQAU32YQQtNXk09HoWgtPvo8FRepEGXdVDGt6MzWBwlV3BUGfdC10llRhapz6R UD5Kw81yHZ47nzFMmWsZMOeuBHfNDQZmRiLBMdxHQfstBwUtg2vh+m0XDc0tDhLsT9wYep9V 0DBsmaWgy95JQo+hiIL745U0jAXMBJj9Pgbethox1Ovn1gq+/6bgRVErhoLqBgzOgecIbIUf MFgtfTS0+70YGq2lBPy814HAXfyFgfxLUwzcPFeM4GJ+GQl6VzRM/6igt23i270+gtc3nuBb AkaSf1nJ8U9vDDG83jbI8EZrJt9Ys4avah7FvOmbn+Kttedp3vrtCsNf+OLE/Hh3N8N3Xpsm eY+zHO9bckiyJVFIUWUJGsXWBEnSWGCAUs/4qJMBi4PMRbN28gISizg2intsMuKgaTaM6++f IoIOZZdzjUUfqaAJ1ivhqrt3BR3C7uT8BT+YoEl2JWfzVtFBS9lorn42/9/mMq6uvvXvjpjd wF0ev/y3I5vrvNKZ6BIkMaJ5tShUlZaVqlSlRIdrk5Oy01Qnw4+mp1rR3MOYc2YMT9Bkb0wb YkVIvkC6TfxJkFHKLG12ahviRIQ8VHrmp0eQSROV2acETfphTWaKoG1DS0SkfJF0d5yQIGOP KTOEZEFQC5r/KRaJF+ciXR7nWnREXdJwuijM+NUzIImyJ8fuaVKs6l7hKds+v9+hDTl4oN+S /DbzuOJsk0HflRS3cOKO+kD8Wpd6S87Q/vrCnQkV8Z3VGfOKuT75yNVHNrHTsLHQLfWEbfql j80x7hhsnoyI9z2Nza81rW8Q5U08GL/7K/H9gNSliLGtdstJbZIycg2h0Sr/AFTGzHQsAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: CURRENT STATUS -------------- Lockdep tracks acquisition order of locks in order to detect deadlock, and IRQ and IRQ enable/disable state as well to take accident acquisitions into account. Lockdep should be turned off once it detects and reports a deadlock since the data structure and algorithm are not reusable after detection because of the complex design. PROBLEM ------- *Waits* and their *events* that never reach eventually cause deadlock. However, Lockdep is only interested in lock acquisition order, forcing to emulate lock acqusition even for just waits and events that have nothing to do with real lock. Even worse, no one likes Lockdep's false positive detection because that prevents further one that might be more valuable. That's why all the kernel developers are sensitive to Lockdep's false positive. Besides those, by tracking acquisition order, it cannot correctly deal with read lock and cross-event e.g. wait_for_completion()/complete() for deadlock detection. Lockdep is no longer a good tool for that purpose. SOLUTION -------- Again, *waits* and their *events* that never reach eventually cause deadlock. The new solution, Dept(DEPendency Tracker), focuses on waits and events themselves. Dept tracks waits and events and report it if any event would be never reachable. Dept does: . Works with read lock in the right way. . Works with any wait and event e.i. cross-event. . Continue to work even after reporting multiple times. . Provides simple and intuitive APIs. . Does exactly what dependency checker should do. Q & A ----- Q. Is this the first try ever to address the problem? A. No. Cross-release feature (b09be676e0ff2 locking/lockdep: Implement the 'crossrelease' feature) addressed it 2 years ago that was a Lockdep extension and merged but reverted shortly because: Cross-release started to report valuable hidden problems but started to give report false positive reports as well. For sure, no one likes Lockdep's false positive reports since it makes Lockdep stop, preventing reporting further real problems. Q. Why not Dept was developed as an extension of Lockdep? A. Lockdep definitely includes all the efforts great developers have made for a long time so as to be quite stable enough. But I had to design and implement newly because of the following: 1) Lockdep was designed to track lock acquisition order. The APIs and implementation do not fit on wait-event model. 2) Lockdep is turned off on detection including false positive. Which is terrible and prevents developing any extension for stronger detection. Q. Do you intend to totally replace Lockdep? A. No. Lockdep also checks if lock usage is correct. Of course, the dependency check routine should be replaced but the other functions should be still there. Q. Do you mean the dependency check routine should be replaced right away? A. No. I admit Lockdep is stable enough thanks to great efforts kernel developers have made. Lockdep and Dept, both should be in the kernel until Dept gets considered stable. Q. Stronger detection capability would give more false positive report. Which was a big problem when cross-release was introduced. Is it ok with Dept? A. It's ok. Dept allows multiple reporting thanks to simple and quite generalized design. Of course, false positive reports should be fixed anyway but it's no longer as a critical problem as it was. Signed-off-by: Byungchul Park --- include/linux/dept.h | 567 ++++++ include/linux/hardirq.h | 3 + include/linux/sched.h | 3 + init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/dependency/Makefile | 3 + kernel/dependency/dept.c | 2966 +++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_object.h | 13 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/module/main.c | 4 + kernel/sched/core.c | 10 + lib/Kconfig.debug | 27 + lib/locking-selftest.c | 2 + 16 files changed, 3616 insertions(+) create mode 100644 include/linux/dept.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_object.h diff --git a/include/linux/dept.h b/include/linux/dept.h new file mode 100644 index 000000000000..c6e2291dd843 --- /dev/null +++ b/include/linux/dept.h @@ -0,0 +1,567 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DEPT(DEPendency Tracker) - runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_H +#define __LINUX_DEPT_H + +#ifdef CONFIG_DEPT + +#include + +struct task_struct; + +#define DEPT_MAX_STACK_ENTRY 16 +#define DEPT_MAX_WAIT_HIST 64 +#define DEPT_MAX_ECXT_HELD 48 + +#define DEPT_MAX_SUBCLASSES 16 +#define DEPT_MAX_SUBCLASSES_EVT 2 +#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) +#define DEPT_MAX_SUBCLASSES_CACHE 2 + +#define DEPT_SIRQ 0 +#define DEPT_HIRQ 1 +#define DEPT_IRQS_NR 2 +#define DEPT_SIRQF (1UL << DEPT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_HIRQ) + +struct dept_ecxt; +struct dept_iecxt { + struct dept_ecxt *ecxt; + int enirq; + /* + * for preventing to add a new ecxt + */ + bool staled; +}; + +struct dept_wait; +struct dept_iwait { + struct dept_wait *wait; + int irq; + /* + * for preventing to add a new wait + */ + bool staled; + bool touched; +}; + +struct dept_class { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * unique information about the class + */ + const char *name; + unsigned long key; + int sub_id; + + /* + * for BFS + */ + unsigned int bfs_gen; + int bfs_dist; + struct dept_class *bfs_parent; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking all classes + */ + struct list_head all_node; + + /* + * for associating its dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for tracking IRQ dependencies + */ + struct dept_iecxt iecxt[DEPT_IRQS_NR]; + struct dept_iwait iwait[DEPT_IRQS_NR]; + + /* + * classified by a map embedded in task_struct, + * not an explicit map + */ + bool sched_map; + }; + }; +}; + +struct dept_key { + union { + /* + * Each byte-wise address will be used as its key. + */ + char base[DEPT_MAX_SUBCLASSES]; + + /* + * for caching the main class pointer + */ + struct dept_class *classes[DEPT_MAX_SUBCLASSES_CACHE]; + }; +}; + +struct dept_map { + const char *name; + struct dept_key *keys; + + /* + * subclass that can be set from user + */ + int sub_u; + + /* + * It's local copy for fast access to the associated classes. + * Also used for dept_key for static maps. + */ + struct dept_key map_key; + + /* + * wait timestamp associated to this map + */ + unsigned int wgen; + + /* + * whether this map should be going to be checked or not + */ + bool nocheck; +}; + +#define DEPT_MAP_INITIALIZER(n, k) \ +{ \ + .name = #n, \ + .keys = (struct dept_key *)(k), \ + .sub_u = 0, \ + .map_key = { .classes = { NULL, } }, \ + .wgen = 0U, \ + .nocheck = false, \ +} + +struct dept_stack { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * backtrace entries + */ + unsigned long raw[DEPT_MAX_STACK_ENTRY]; + int nr; + }; + }; +}; + +struct dept_ecxt { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function that entered to this ecxt + */ + const char *ecxt_fn; + + /* + * event function + */ + const char *event_fn; + + /* + * associated class + */ + struct dept_class *class; + + /* + * flag indicating which IRQ has been + * enabled within the event context + */ + unsigned long enirqf; + + /* + * where the IRQ-enabled happened + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + + /* + * where the event context started + */ + unsigned long ecxt_ip; + struct dept_stack *ecxt_stack; + + /* + * where the event triggered + */ + unsigned long event_ip; + struct dept_stack *event_stack; + }; + }; +}; + +struct dept_wait { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function causing this wait + */ + const char *wait_fn; + + /* + * the associated class + */ + struct dept_class *class; + + /* + * which IRQ the wait was placed in + */ + unsigned long irqf; + + /* + * where the IRQ wait happened + */ + unsigned long irq_ip[DEPT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_IRQS_NR]; + + /* + * where the wait happened + */ + unsigned long wait_ip; + struct dept_stack *wait_stack; + + /* + * whether this wait is for commit in scheduler + */ + bool sched_sleep; + }; + }; +}; + +struct dept_dep { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * key data of dependency + */ + struct dept_ecxt *ecxt; + struct dept_wait *wait; + + /* + * This object can be referred without dept_lock + * held but with IRQ disabled, e.g. for hash + * lookup. So deferred deletion is needed. + */ + struct rcu_head rh; + + /* + * for BFS + */ + struct list_head bfs_node; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking to a class object + */ + struct list_head dep_node; + struct list_head dep_rev_node; + }; + }; +}; + +struct dept_hash { + /* + * hash table + */ + struct hlist_head *table; + + /* + * size of the table e.i. 2^bits + */ + int bits; +}; + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +struct dept_ecxt_held { + /* + * associated event context + */ + struct dept_ecxt *ecxt; + + /* + * unique key for this dept_ecxt_held + */ + struct dept_map *map; + + /* + * class of the ecxt of this dept_ecxt_held + */ + struct dept_class *class; + + /* + * the wgen when the event context started + */ + unsigned int wgen; + + /* + * subclass that only works in the local context + */ + int sub_l; +}; + +struct dept_wait_hist { + /* + * associated wait + */ + struct dept_wait *wait; + + /* + * unique id of all waits system-wise until wrapped + */ + unsigned int wgen; + + /* + * local context id to identify IRQ context + */ + unsigned int ctxt_id; +}; + +struct dept_task { + /* + * all event contexts that have entered and before exiting + */ + struct dept_ecxt_held ecxt_held[DEPT_MAX_ECXT_HELD]; + int ecxt_held_pos; + + /* + * ring buffer holding all waits that have happened + */ + struct dept_wait_hist wait_hist[DEPT_MAX_WAIT_HIST]; + int wait_hist_pos; + + /* + * sequential id to identify each IRQ context + */ + unsigned int irq_id[DEPT_IRQS_NR]; + + /* + * for tracking IRQ-enabled points with cross-event + */ + unsigned int wgen_enirq[DEPT_IRQS_NR]; + + /* + * for keeping up-to-date IRQ-enabled points + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + + /* + * for reserving a current stack instance at each operation + */ + struct dept_stack *stack; + + /* + * for preventing recursive call into DEPT engine + */ + int recursive; + + /* + * for staging data to commit a wait + */ + struct dept_map stage_m; + bool stage_sched_map; + const char *stage_w_fn; + unsigned long stage_ip; + + /* + * the number of missing ecxts + */ + int missing_ecxt; + + /* + * for tracking IRQ-enable state + */ + bool hardirqs_enabled; + bool softirqs_enabled; + + /* + * whether the current is on do_exit() + */ + bool task_exit; + + /* + * whether the current is running __schedule() + */ + bool in_sched; +}; + +#define DEPT_TASK_INITIALIZER(t) \ +{ \ + .wait_hist = { { .wait = NULL, } }, \ + .ecxt_held_pos = 0, \ + .wait_hist_pos = 0, \ + .irq_id = { 0U }, \ + .wgen_enirq = { 0U }, \ + .enirq_ip = { 0UL }, \ + .stack = NULL, \ + .recursive = 0, \ + .stage_m = DEPT_MAP_INITIALIZER((t)->stage_m, NULL), \ + .stage_sched_map = false, \ + .stage_w_fn = NULL, \ + .stage_ip = 0UL, \ + .missing_ecxt = 0, \ + .hardirqs_enabled = false, \ + .softirqs_enabled = false, \ + .task_exit = false, \ + .in_sched = false, \ +} + +extern void dept_on(void); +extern void dept_off(void); +extern void dept_init(void); +extern void dept_task_init(struct task_struct *t); +extern void dept_task_exit(struct task_struct *t); +extern void dept_free_range(void *start, unsigned int sz); +extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_copy(struct dept_map *to, struct dept_map *from); + +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_request_event_wait_commit(void); +extern void dept_clean_stage(void); +extern void dept_stage_event(struct task_struct *t, unsigned long ip); +extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); +extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); +extern void dept_request_event(struct dept_map *m); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); +extern void dept_sched_enter(void); +extern void dept_sched_exit(void); + +static inline void dept_ecxt_enter_nokeep(struct dept_map *m) +{ + dept_ecxt_enter(m, 0UL, 0UL, NULL, NULL, 0); +} + +/* + * for users who want to manage external keys + */ +extern void dept_key_init(struct dept_key *k); +extern void dept_key_destroy(struct dept_key *k); +extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, struct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const char *new_c_fn, const char *new_e_fn, int new_sub_l); + +extern void dept_softirq_enter(void); +extern void dept_hardirq_enter(void); +extern void dept_softirqs_on_ip(unsigned long ip); +extern void dept_hardirqs_on(void); +extern void dept_softirqs_off(void); +extern void dept_hardirqs_off(void); +#else /* !CONFIG_DEPT */ +struct dept_key { }; +struct dept_map { }; +struct dept_task { }; + +#define DEPT_MAP_INITIALIZER(n, k) { } +#define DEPT_TASK_INITIALIZER(t) { } + +#define dept_on() do { } while (0) +#define dept_off() do { } while (0) +#define dept_init() do { } while (0) +#define dept_task_init(t) do { } while (0) +#define dept_task_exit(t) do { } while (0) +#define dept_free_range(s, sz) do { } while (0) +#define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_copy(t, f) do { } while (0) + +#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_request_event_wait_commit() do { } while (0) +#define dept_clean_stage() do { } while (0) +#define dept_stage_event(t, ip) do { } while (0) +#define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) +#define dept_ecxt_holding(m, e_f) false +#define dept_request_event(m) do { } while (0) +#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_ecxt_exit(m, e_f, ip) do { } while (0) +#define dept_sched_enter() do { } while (0) +#define dept_sched_exit() do { } while (0) +#define dept_ecxt_enter_nokeep(m) do { } while (0) +#define dept_key_init(k) do { (void)(k); } while (0) +#define dept_key_destroy(k) do { (void)(k); } while (0) +#define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_sl) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) + +#define dept_softirq_enter() do { } while (0) +#define dept_hardirq_enter() do { } while (0) +#define dept_softirqs_on_ip(ip) do { } while (0) +#define dept_hardirqs_on() do { } while (0) +#define dept_softirqs_off() do { } while (0) +#define dept_hardirqs_off() do { } while (0) +#endif +#endif /* __LINUX_DEPT_H */ diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index d57cab4d4c06..bb279dbbe748 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ void irq_exit_rcu(void); */ #define __nmi_enter() \ do { \ + dept_off(); \ lockdep_off(); \ arch_nmi_enter(); \ BUG_ON(in_nmi() == NMI_MASK); \ @@ -128,6 +130,7 @@ void irq_exit_rcu(void); __preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \ arch_nmi_exit(); \ lockdep_on(); \ + dept_on(); \ } while (0) #define nmi_exit() \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 292c31697248..6dafdc93b462 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -38,6 +38,7 @@ #include #include #include +#include /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -1178,6 +1179,8 @@ struct task_struct { struct held_lock held_locks[MAX_LOCK_DEPTH]; #endif + struct dept_task dept_task; + #if defined(CONFIG_UBSAN) && !defined(CONFIG_UBSAN_TRAP) unsigned int in_ubsan; #endif diff --git a/init/init_task.c b/init/init_task.c index 5727d42149c3..171572fbdb43 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -12,6 +12,7 @@ #include #include #include +#include #include @@ -194,6 +195,7 @@ struct task_struct init_task .curr_chain_key = INITIAL_CHAIN_KEY, .lockdep_recursion = 0, #endif + .dept_task = DEPT_TASK_INITIALIZER(init_task), #ifdef CONFIG_FUNCTION_GRAPH_TRACER .ret_stack = NULL, .tracing_graph_pause = ATOMIC_INIT(0), diff --git a/init/main.c b/init/main.c index e24b0780fdff..39dac78fafc1 100644 --- a/init/main.c +++ b/init/main.c @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -1011,6 +1012,7 @@ void start_kernel(void) panic_param); lockdep_init(); + dept_init(); /* * Need to run this when irqs are enabled, because it wants diff --git a/kernel/Makefile b/kernel/Makefile index 3947122d618b..e864b66d6470 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -51,6 +51,7 @@ obj-y += livepatch/ obj-y += dma/ obj-y += entry/ obj-$(CONFIG_MODULES) += module/ +obj-y += dependency/ obj-$(CONFIG_KCMP) += kcmp.o obj-$(CONFIG_FREEZER) += freezer.o diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile new file mode 100644 index 000000000000..b5cfb8a03c0c --- /dev/null +++ b/kernel/dependency/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DEPT) += dept.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c new file mode 100644 index 000000000000..a3e774479f94 --- /dev/null +++ b/kernel/dependency/dept.c @@ -0,0 +1,2966 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DEPT(DEPendency Tracker) - Runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * + * DEPT provides a general way to detect deadlock possibility in runtime + * and the interest is not limited to typical lock but to every + * syncronization primitives. + * + * The following ideas were borrowed from LOCKDEP: + * + * 1) Use a graph to track relationship between classes. + * 2) Prevent performance regression using hash. + * + * The following items were enhanced from LOCKDEP: + * + * 1) Cover more deadlock cases. + * 2) Allow muliple reports. + * + * TODO: Both LOCKDEP and DEPT should co-exist until DEPT is considered + * stable. Then the dependency check routine should be replaced with + * DEPT after. It should finally look like: + * + * + * + * As is: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | +-------------------------------------+ | + * | | Dependency check | | + * | | (by tracking lock acquisition order)| | + * | +-------------------------------------+ | + * | | + * +-----------------------------------------+ + * + * DEPT + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + * + * + * + * To be: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | (Request dependency check) | + * | T | + * +--------------------|--------------------+ + * | + * DEPT V + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static int dept_stop; +static int dept_per_cpu_ready; + +#define DEPT_READY_WARN (!oops_in_progress) + +/* + * Make all operations using DEPT_WARN_ON() fail on oops_in_progress and + * prevent warning message. + */ +#define DEPT_WARN_ON_ONCE(c) \ + ({ \ + int __ret = 0; \ + \ + if (likely(DEPT_READY_WARN)) \ + __ret = WARN_ONCE(c, "DEPT_WARN_ON_ONCE: " #c); \ + __ret; \ + }) + +#define DEPT_WARN_ONCE(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) \ + WARN_ONCE(1, "DEPT_WARN_ONCE: " s); \ + }) + +#define DEPT_WARN_ON(c) \ + ({ \ + int __ret = 0; \ + \ + if (likely(DEPT_READY_WARN)) \ + __ret = WARN(c, "DEPT_WARN_ON: " #c); \ + __ret; \ + }) + +#define DEPT_WARN(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) \ + WARN(1, "DEPT_WARN: " s); \ + }) + +#define DEPT_STOP(s...) \ + ({ \ + WRITE_ONCE(dept_stop, 1); \ + if (likely(DEPT_READY_WARN)) \ + WARN(1, "DEPT_STOP: " s); \ + }) + +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) + +static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; + +/* + * DEPT internal engine should be careful in using outside functions + * e.g. printk at reporting since that kind of usage might cause + * untrackable deadlock. + */ +static atomic_t dept_outworld = ATOMIC_INIT(0); + +static void dept_outworld_enter(void) +{ + atomic_inc(&dept_outworld); +} + +static void dept_outworld_exit(void) +{ + atomic_dec(&dept_outworld); +} + +static bool dept_outworld_entered(void) +{ + return atomic_read(&dept_outworld); +} + +static bool dept_lock(void) +{ + while (!arch_spin_trylock(&dept_spin)) + if (unlikely(dept_outworld_entered())) + return false; + return true; +} + +static void dept_unlock(void) +{ + arch_spin_unlock(&dept_spin); +} + +/* + * whether to stack-trace on every wait or every ecxt + */ +static bool rich_stack = true; + +enum bfs_ret { + BFS_CONTINUE, + BFS_CONTINUE_REV, + BFS_DONE, + BFS_SKIP, +}; + +static bool after(unsigned int a, unsigned int b) +{ + return (int)(b - a) < 0; +} + +static bool before(unsigned int a, unsigned int b) +{ + return (int)(a - b) < 0; +} + +static bool valid_stack(struct dept_stack *s) +{ + return s && s->nr > 0; +} + +static bool valid_class(struct dept_class *c) +{ + return c->key; +} + +static void invalidate_class(struct dept_class *c) +{ + c->key = 0UL; +} + +static struct dept_ecxt *dep_e(struct dept_dep *d) +{ + return d->ecxt; +} + +static struct dept_wait *dep_w(struct dept_dep *d) +{ + return d->wait; +} + +static struct dept_class *dep_fc(struct dept_dep *d) +{ + return dep_e(d)->class; +} + +static struct dept_class *dep_tc(struct dept_dep *d) +{ + return dep_w(d)->class; +} + +static const char *irq_str(int irq) +{ + if (irq == DEPT_SIRQ) + return "softirq"; + if (irq == DEPT_HIRQ) + return "hardirq"; + return "(unknown)"; +} + +static inline struct dept_task *dept_task(void) +{ + return ¤t->dept_task; +} + +/* + * Dept doesn't work either when it's stopped by DEPT_STOP() or in a nmi + * context. + */ +static bool dept_working(void) +{ + return !READ_ONCE(dept_stop) && !in_nmi(); +} + +/* + * Even k == NULL is considered as a valid key because it would use + * &->map_key as the key in that case. + */ +struct dept_key __dept_no_validate__; +static bool valid_key(struct dept_key *k) +{ + return &__dept_no_validate__ != k; +} + +/* + * Pool + * ===================================================================== + * DEPT maintains pools to provide objects in a safe way. + * + * 1) Static pool is used at the beginning of booting time. + * 2) Local pool is tried first before the static pool. Objects that + * have been freed will be placed. + */ + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +#define OBJECT(id, nr) \ +static struct dept_##id spool_##id[nr]; \ +static DEFINE_PER_CPU(struct llist_head, lpool_##id); + #include "dept_object.h" +#undef OBJECT + +static struct dept_pool pool[OBJECT_NR] = { +#define OBJECT(id, nr) { \ + .name = #id, \ + .obj_sz = sizeof(struct dept_##id), \ + .obj_nr = ATOMIC_INIT(nr), \ + .node_off = offsetof(struct dept_##id, pool_node), \ + .spool = spool_##id, \ + .lpool = &lpool_##id, }, + #include "dept_object.h" +#undef OBJECT +}; + +/* + * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is + * enabled or not because NMI and other contexts in the same CPU never + * run inside of DEPT concurrently by preventing reentrance. + */ +static void *from_pool(enum object_t t) +{ + struct dept_pool *p; + struct llist_head *h; + struct llist_node *n; + + /* + * llist_del_first() doesn't allow concurrent access e.g. + * between process and IRQ context. + */ + if (DEPT_WARN_ON(!irqs_disabled())) + return NULL; + + p = &pool[t]; + + /* + * Try local pool first. + */ + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + n = llist_del_first(h); + if (n) + return (void *)n - p->node_off; + + /* + * Try static pool. + */ + if (atomic_read(&p->obj_nr) > 0) { + int idx = atomic_dec_return(&p->obj_nr); + + if (idx >= 0) + return p->spool + (idx * p->obj_sz); + } + + DEPT_INFO_ONCE("---------------------------------------------\n" + " Some of Dept internal resources are run out.\n" + " Dept might still work if the resources get freed.\n" + " However, the chances are Dept will suffer from\n" + " the lack from now. Needs to extend the internal\n" + " resource pools. Ask max.byungchul.park@gmail.com\n"); + return NULL; +} + +static void to_pool(void *o, enum object_t t) +{ + struct dept_pool *p = &pool[t]; + struct llist_head *h; + + preempt_disable(); + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + llist_add(o + p->node_off, h); + preempt_enable(); +} + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); \ +static void (*dtor_##id)(struct dept_##id *a); \ +static struct dept_##id *new_##id(void) \ +{ \ + struct dept_##id *a; \ + \ + a = (struct dept_##id *)from_pool(OBJECT_##id); \ + if (unlikely(!a)) \ + return NULL; \ + \ + atomic_set(&a->ref, 1); \ + \ + if (ctor_##id) \ + ctor_##id(a); \ + \ + return a; \ +} \ + \ +static struct dept_##id *get_##id(struct dept_##id *a) \ +{ \ + atomic_inc(&a->ref); \ + return a; \ +} \ + \ +static void put_##id(struct dept_##id *a) \ +{ \ + if (!atomic_dec_return(&a->ref)) { \ + if (dtor_##id) \ + dtor_##id(a); \ + to_pool(a, OBJECT_##id); \ + } \ +} \ + \ +static void del_##id(struct dept_##id *a) \ +{ \ + put_##id(a); \ +} \ + \ +static bool __maybe_unused id##_consumed(struct dept_##id *a) \ +{ \ + return a && atomic_read(&a->ref) > 1; \ +} +#include "dept_object.h" +#undef OBJECT + +#define SET_CONSTRUCTOR(id, f) \ +static void (*ctor_##id)(struct dept_##id *a) = f + +static void initialize_dep(struct dept_dep *d) +{ + INIT_LIST_HEAD(&d->bfs_node); + INIT_LIST_HEAD(&d->dep_node); + INIT_LIST_HEAD(&d->dep_rev_node); +} +SET_CONSTRUCTOR(dep, initialize_dep); + +static void initialize_class(struct dept_class *c) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iecxt *ie = &c->iecxt[i]; + struct dept_iwait *iw = &c->iwait[i]; + + ie->ecxt = NULL; + ie->enirq = i; + ie->staled = false; + + iw->wait = NULL; + iw->irq = i; + iw->staled = false; + iw->touched = false; + } + c->bfs_gen = 0U; + + INIT_LIST_HEAD(&c->all_node); + INIT_LIST_HEAD(&c->dep_head); + INIT_LIST_HEAD(&c->dep_rev_head); +} +SET_CONSTRUCTOR(class, initialize_class); + +static void initialize_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + e->enirq_stack[i] = NULL; + e->enirq_ip[i] = 0UL; + } + e->ecxt_ip = 0UL; + e->ecxt_stack = NULL; + e->enirqf = 0UL; + e->event_ip = 0UL; + e->event_stack = NULL; +} +SET_CONSTRUCTOR(ecxt, initialize_ecxt); + +static void initialize_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + w->irq_stack[i] = NULL; + w->irq_ip[i] = 0UL; + } + w->wait_ip = 0UL; + w->wait_stack = NULL; + w->irqf = 0UL; +} +SET_CONSTRUCTOR(wait, initialize_wait); + +static void initialize_stack(struct dept_stack *s) +{ + s->nr = 0; +} +SET_CONSTRUCTOR(stack, initialize_stack); + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_CONSTRUCTOR + +#define SET_DESTRUCTOR(id, f) \ +static void (*dtor_##id)(struct dept_##id *a) = f + +static void destroy_dep(struct dept_dep *d) +{ + if (dep_e(d)) + put_ecxt(dep_e(d)); + if (dep_w(d)) + put_wait(dep_w(d)); +} +SET_DESTRUCTOR(dep, destroy_dep); + +static void destroy_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (e->enirq_stack[i]) + put_stack(e->enirq_stack[i]); + if (e->class) + put_class(e->class); + if (e->ecxt_stack) + put_stack(e->ecxt_stack); + if (e->event_stack) + put_stack(e->event_stack); +} +SET_DESTRUCTOR(ecxt, destroy_ecxt); + +static void destroy_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (w->irq_stack[i]) + put_stack(w->irq_stack[i]); + if (w->class) + put_class(w->class); + if (w->wait_stack) + put_stack(w->wait_stack); +} +SET_DESTRUCTOR(wait, destroy_wait); + +#define OBJECT(id, nr) \ +static void (*dtor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_DESTRUCTOR + +/* + * Caching and hashing + * ===================================================================== + * DEPT makes use of caching and hashing to improve performance. Each + * object can be obtained in O(1) with its key. + * + * NOTE: Currently we assume all the objects in the hashs will never be + * removed. Implement it when needed. + */ + +/* + * Some information might be lost but it's only for hashing key. + */ +static unsigned long mix(unsigned long a, unsigned long b) +{ + int halfbits = sizeof(unsigned long) * 8 / 2; + unsigned long halfmask = (1UL << halfbits) - 1UL; + + return (a << halfbits) | (b & halfmask); +} + +static bool cmp_dep(struct dept_dep *d1, struct dept_dep *d2) +{ + return dep_fc(d1)->key == dep_fc(d2)->key && + dep_tc(d1)->key == dep_tc(d2)->key; +} + +static unsigned long key_dep(struct dept_dep *d) +{ + return mix(dep_fc(d)->key, dep_tc(d)->key); +} + +static bool cmp_class(struct dept_class *c1, struct dept_class *c2) +{ + return c1->key == c2->key; +} + +static unsigned long key_class(struct dept_class *c) +{ + return c->key; +} + +#define HASH(id, bits) \ +static struct hlist_head table_##id[1 << (bits)]; \ + \ +static struct hlist_head *head_##id(struct dept_##id *a) \ +{ \ + return table_##id + hash_long(key_##id(a), bits); \ +} \ + \ +static struct dept_##id *hash_lookup_##id(struct dept_##id *a) \ +{ \ + struct dept_##id *b; \ + \ + hlist_for_each_entry_rcu(b, head_##id(a), hash_node) \ + if (cmp_##id(a, b)) \ + return b; \ + return NULL; \ +} \ + \ +static void hash_add_##id(struct dept_##id *a) \ +{ \ + get_##id(a); \ + hlist_add_head_rcu(&a->hash_node, head_##id(a)); \ +} \ + \ +static void hash_del_##id(struct dept_##id *a) \ +{ \ + hlist_del_rcu(&a->hash_node); \ + put_##id(a); \ +} +#include "dept_hash.h" +#undef HASH + +static struct dept_dep *lookup_dep(struct dept_class *fc, + struct dept_class *tc) +{ + struct dept_ecxt onetime_e = { .class = fc }; + struct dept_wait onetime_w = { .class = tc }; + struct dept_dep onetime_d = { .ecxt = &onetime_e, + .wait = &onetime_w }; + return hash_lookup_dep(&onetime_d); +} + +static struct dept_class *lookup_class(unsigned long key) +{ + struct dept_class onetime_c = { .key = key }; + + return hash_lookup_class(&onetime_c); +} + +/* + * Report + * ===================================================================== + * DEPT prints useful information to help debuging on detection of + * problematic dependency. + */ + +static void print_ip_stack(unsigned long ip, struct dept_stack *s) +{ + if (ip) + print_ip_sym(KERN_WARNING, ip); + + if (valid_stack(s)) { + pr_warn("stacktrace:\n"); + stack_trace_print(s->raw, s->nr, 5); + } + + if (!ip && !valid_stack(s)) + pr_warn("(N/A)\n"); +} + +#define print_spc(spc, fmt, ...) \ + pr_warn("%*c" fmt, (spc) * 4, ' ', ##__VA_ARGS__) + +static void print_diagram(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + bool firstline = true; + int spc = 1; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + if (!firstline) + pr_warn("\nor\n\n"); + firstline = false; + + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, " <%s interrupt>\n", irq_str(irq)); + print_spc(spc + 1, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } + + if (!irqf) { + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } +} + +static void print_dep(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + pr_warn("%s has been enabled:\n", irq_str(irq)); + print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); + pr_warn("\n"); + + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d) in %s context:\n", + w_fn, tc_n, tc->sub_id, irq_str(irq)); + print_ip_stack(w->irq_ip[irq], w->irq_stack[irq]); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } + + if (!irqf) { + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d):\n", w_fn, tc_n, tc->sub_id); + print_ip_stack(w->wait_ip, w->wait_stack); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } +} + +static void save_current_stack(int skip); + +/* + * Print all classes in a circle. + */ +static void print_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + int i; + + dept_outworld_enter(); + save_current_stack(6); + + pr_warn("===================================================\n"); + pr_warn("DEPT: Circular dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + pr_warn("summary\n"); + pr_warn("---------------------------------------------------\n"); + + if (fc == tc) + pr_warn("*** AA DEADLOCK ***\n\n"); + else + pr_warn("*** DEADLOCK ***\n\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + if (fc != c) + pr_warn("\n"); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("\n"); + pr_warn("[S]: start of the event context\n"); + pr_warn("[W]: the wait blocked\n"); + pr_warn("[E]: the event not reachable\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c's detail\n", 'A' + i); + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + pr_warn("\n"); + print_dep(d); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +/* + * BFS(Breadth First Search) + * ===================================================================== + * Whenever a new dependency is added into the graph, search the graph + * for a new circular dependency. + */ + +static void enqueue(struct list_head *h, struct dept_dep *d) +{ + list_add_tail(&d->bfs_node, h); +} + +static struct dept_dep *dequeue(struct list_head *h) +{ + struct dept_dep *d; + + d = list_first_entry(h, struct dept_dep, bfs_node); + list_del(&d->bfs_node); + return d; +} + +static bool empty(struct list_head *h) +{ + return list_empty(h); +} + +static void extend_queue(struct list_head *h, struct dept_class *cur) +{ + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_head, dep_node) { + struct dept_class *next = dep_tc(d); + + if (cur->bfs_gen == next->bfs_gen) + continue; + next->bfs_gen = cur->bfs_gen; + next->bfs_dist = cur->bfs_dist + 1; + next->bfs_parent = cur; + enqueue(h, d); + } +} + +static void extend_queue_rev(struct list_head *h, struct dept_class *cur) +{ + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_rev_head, dep_rev_node) { + struct dept_class *next = dep_fc(d); + + if (cur->bfs_gen == next->bfs_gen) + continue; + next->bfs_gen = cur->bfs_gen; + next->bfs_dist = cur->bfs_dist + 1; + next->bfs_parent = cur; + enqueue(h, d); + } +} + +typedef enum bfs_ret bfs_f(struct dept_dep *d, void *in, void **out); +static unsigned int bfs_gen; + +/* + * NOTE: Must be called with dept_lock held. + */ +static void bfs(struct dept_class *c, bfs_f *cb, void *in, void **out) +{ + LIST_HEAD(q); + enum bfs_ret ret; + + if (DEPT_WARN_ON(!cb)) + return; + + /* + * Avoid zero bfs_gen. + */ + bfs_gen = bfs_gen + 1 ?: 1; + + c->bfs_gen = bfs_gen; + c->bfs_dist = 0; + c->bfs_parent = c; + + ret = cb(NULL, in, out); + if (ret == BFS_DONE) + return; + if (ret == BFS_SKIP) + return; + if (ret == BFS_CONTINUE) + extend_queue(&q, c); + if (ret == BFS_CONTINUE_REV) + extend_queue_rev(&q, c); + + while (!empty(&q)) { + struct dept_dep *d = dequeue(&q); + + ret = cb(d, in, out); + if (ret == BFS_DONE) + break; + if (ret == BFS_SKIP) + continue; + if (ret == BFS_CONTINUE) + extend_queue(&q, dep_tc(d)); + if (ret == BFS_CONTINUE_REV) + extend_queue_rev(&q, dep_fc(d)); + } + + while (!empty(&q)) + dequeue(&q); +} + +/* + * Main operations + * ===================================================================== + * Add dependencies - Each new dependency is added into the graph and + * checked if it forms a circular dependency. + * + * Track waits - Waits are queued into the ring buffer for later use to + * generate appropriate dependencies with cross-event. + * + * Track event contexts(ecxt) - Event contexts are pushed into local + * stack for later use to generate appropriate dependencies with waits. + */ + +static unsigned long cur_enirqf(void); +static int cur_irq(void); +static unsigned int cur_ctxt_id(void); + +static struct dept_iecxt *iecxt(struct dept_class *c, int irq) +{ + return &c->iecxt[irq]; +} + +static struct dept_iwait *iwait(struct dept_class *c, int irq) +{ + return &c->iwait[irq]; +} + +static void stale_iecxt(struct dept_iecxt *ie) +{ + if (ie->ecxt) + put_ecxt(ie->ecxt); + + WRITE_ONCE(ie->ecxt, NULL); + WRITE_ONCE(ie->staled, true); +} + +static void set_iecxt(struct dept_iecxt *ie, struct dept_ecxt *e) +{ + /* + * ->ecxt will never be updated once getting set until the class + * gets removed. + */ + if (ie->ecxt) + DEPT_WARN_ON(1); + else + WRITE_ONCE(ie->ecxt, get_ecxt(e)); +} + +static void stale_iwait(struct dept_iwait *iw) +{ + if (iw->wait) + put_wait(iw->wait); + + WRITE_ONCE(iw->wait, NULL); + WRITE_ONCE(iw->staled, true); +} + +static void set_iwait(struct dept_iwait *iw, struct dept_wait *w) +{ + /* + * ->wait will never be updated once getting set until the class + * gets removed. + */ + if (iw->wait) + DEPT_WARN_ON(1); + else + WRITE_ONCE(iw->wait, get_wait(w)); + + iw->touched = true; +} + +static void touch_iwait(struct dept_iwait *iw) +{ + iw->touched = true; +} + +static void untouch_iwait(struct dept_iwait *iw) +{ + iw->touched = false; +} + +static struct dept_stack *get_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + return s ? get_stack(s) : NULL; +} + +static void prepare_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + /* + * The dept_stack is already ready. + */ + if (s && !stack_consumed(s)) { + s->nr = 0; + return; + } + + if (s) + put_stack(s); + + s = dept_task()->stack = new_stack(); + if (!s) + return; + + get_stack(s); + del_stack(s); +} + +static void save_current_stack(int skip) +{ + struct dept_stack *s = dept_task()->stack; + + if (!s) + return; + if (valid_stack(s)) + return; + + s->nr = stack_trace_save(s->raw, DEPT_MAX_STACK_ENTRY, skip); +} + +static void finish_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + if (stack_consumed(s)) + save_current_stack(2); +} + +/* + * FIXME: For now, disable LOCKDEP while DEPT is working. + * + * Both LOCKDEP and DEPT report it on a deadlock detection using + * printk taking the risk of another deadlock that might be caused by + * locks of console or printk between inside and outside of them. + * + * For DEPT, it's no problem since multiple reports are allowed. But it + * would be a bad idea for LOCKDEP since it will stop even on a singe + * report. So we need to prevent LOCKDEP from its reporting the risk + * DEPT would take when reporting something. + */ +#include + +void noinstr dept_off(void) +{ + dept_task()->recursive++; + lockdep_off(); +} + +void noinstr dept_on(void) +{ + dept_task()->recursive--; + lockdep_on(); +} + +static unsigned long dept_enter(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + dept_off(); + prepare_current_stack(); + return flags; +} + +static void dept_exit(unsigned long flags) +{ + finish_current_stack(); + dept_on(); + arch_local_irq_restore(flags); +} + +static unsigned long dept_enter_recursive(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + return flags; +} + +static void dept_exit_recursive(unsigned long flags) +{ + arch_local_irq_restore(flags); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static struct dept_dep *__add_dep(struct dept_ecxt *e, + struct dept_wait *w) +{ + struct dept_dep *d; + + if (DEPT_WARN_ON(!valid_class(e->class))) + return NULL; + + if (DEPT_WARN_ON(!valid_class(w->class))) + return NULL; + + if (lookup_dep(e->class, w->class)) + return NULL; + + d = new_dep(); + if (unlikely(!d)) + return NULL; + + d->ecxt = get_ecxt(e); + d->wait = get_wait(w); + + /* + * Add the dependency into hash and graph. + */ + hash_add_dep(d); + list_add(&d->dep_node, &dep_fc(d)->dep_head); + list_add(&d->dep_rev_node, &dep_tc(d)->dep_rev_head); + return d; +} + +static enum bfs_ret cb_check_dl(struct dept_dep *d, + void *in, void **out) +{ + struct dept_dep *new = (struct dept_dep *)in; + + /* + * initial condition for this BFS search + */ + if (!d) { + dep_tc(new)->bfs_parent = dep_fc(new); + + if (dep_tc(new) != dep_fc(new)) + return BFS_CONTINUE; + + /* + * AA circle does not make additional deadlock. We don't + * have to continue this BFS search. + */ + print_circle(dep_tc(new)); + return BFS_DONE; + } + + /* + * Allow multiple reports. + */ + if (dep_tc(d) == dep_fc(new)) + print_circle(dep_tc(new)); + + return BFS_CONTINUE; +} + +/* + * This function is actually in charge of reporting. + */ +static void check_dl_bfs(struct dept_dep *d) +{ + bfs(dep_tc(d), cb_check_dl, (void *)d, NULL); +} + +static enum bfs_ret cb_find_iw(struct dept_dep *d, void *in, void **out) +{ + int irq = *(int *)in; + struct dept_class *fc; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + /* + * initial condition for this BFS search + */ + if (!d) + return BFS_CONTINUE_REV; + + fc = dep_fc(d); + iw = iwait(fc, irq); + + /* + * If any parent's ->wait was set, then the children would've + * been touched. + */ + if (!iw->touched) + return BFS_SKIP; + + if (!iw->wait) + return BFS_CONTINUE_REV; + + *out = iw; + return BFS_DONE; +} + +static struct dept_iwait *find_iw_bfs(struct dept_class *c, int irq) +{ + struct dept_iwait *iw = iwait(c, irq); + struct dept_iwait *found = NULL; + + if (iw->wait) + return iw; + + /* + * '->touched == false' guarantees there's no parent that has + * been set ->wait. + */ + if (!iw->touched) + return NULL; + + bfs(c, cb_find_iw, (void *)&irq, (void **)&found); + + if (found) + return found; + + untouch_iwait(iw); + return NULL; +} + +static enum bfs_ret cb_touch_iw_find_ie(struct dept_dep *d, void *in, + void **out) +{ + int irq = *(int *)in; + struct dept_class *tc; + struct dept_iecxt *ie; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + /* + * initial condition for this BFS search + */ + if (!d) + return BFS_CONTINUE; + + tc = dep_tc(d); + ie = iecxt(tc, irq); + iw = iwait(tc, irq); + + touch_iwait(iw); + + if (!ie->ecxt) + return BFS_CONTINUE; + + if (!*out) + *out = ie; + + return BFS_CONTINUE; +} + +static struct dept_iecxt *touch_iw_find_ie_bfs(struct dept_class *c, + int irq) +{ + struct dept_iecxt *ie = iecxt(c, irq); + struct dept_iwait *iw = iwait(c, irq); + struct dept_iecxt *found = ie->ecxt ? ie : NULL; + + touch_iwait(iw); + bfs(c, cb_touch_iw_find_ie, (void *)&irq, (void **)&found); + return found; +} + +/* + * Should be called with dept_lock held. + */ +static void __add_idep(struct dept_iecxt *ie, struct dept_iwait *iw) +{ + struct dept_dep *new; + + /* + * There's nothing to do. + */ + if (!ie || !iw || !ie->ecxt || !iw->wait) + return; + + new = __add_dep(ie->ecxt, iw->wait); + + /* + * Deadlock detected. Let check_dl_bfs() report it. + */ + if (new) { + check_dl_bfs(new); + stale_iecxt(ie); + stale_iwait(iw); + } + + /* + * If !new, it would be the case of lack of object resource. + * Just let it go and get checked by other chances. Retrying is + * meaningless in that case. + */ +} + +static void set_check_iecxt(struct dept_class *c, int irq, + struct dept_ecxt *e) +{ + struct dept_iecxt *ie = iecxt(c, irq); + + set_iecxt(ie, e); + __add_idep(ie, find_iw_bfs(c, irq)); +} + +static void set_check_iwait(struct dept_class *c, int irq, + struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + set_iwait(iw, w); + __add_idep(touch_iw_find_ie_bfs(c, irq), iw); +} + +static void add_iecxt(struct dept_class *c, int irq, struct dept_ecxt *e, + bool stack) +{ + /* + * This access is safe since we ensure e->class has set locally. + */ + struct dept_task *dt = dept_task(); + struct dept_iecxt *ie = iecxt(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(ie->staled))) + return; + + /* + * Skip add_iecxt() if ie->ecxt has ever been set at least once. + * Which means it has a valid ->ecxt or been staled. + */ + if (READ_ONCE(ie->ecxt)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(ie->staled)) + goto unlock; + if (ie->ecxt) + goto unlock; + + e->enirqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * enirq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(e->enirq_ip[irq]); + DEPT_WARN_ON(e->enirq_stack[irq]); + + e->enirq_ip[irq] = dt->enirq_ip[irq]; + e->enirq_stack[irq] = stack ? get_current_stack() : NULL; + + set_check_iecxt(c, irq, e); +unlock: + dept_unlock(); +} + +static void add_iwait(struct dept_class *c, int irq, struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(iw->staled))) + return; + + /* + * Skip add_iwait() if iw->wait has ever been set at least once. + * Which means it has a valid ->wait or been staled. + */ + if (READ_ONCE(iw->wait)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(iw->staled)) + goto unlock; + if (iw->wait) + goto unlock; + + w->irqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * irq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(w->irq_ip[irq]); + DEPT_WARN_ON(w->irq_stack[irq]); + + w->irq_ip[irq] = w->wait_ip; + w->irq_stack[irq] = get_current_stack(); + + set_check_iwait(c, irq, w); +unlock: + dept_unlock(); +} + +static struct dept_wait_hist *hist(int pos) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist + (pos % DEPT_MAX_WAIT_HIST); +} + +static int hist_pos_next(void) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist_pos % DEPT_MAX_WAIT_HIST; +} + +static void hist_advance(void) +{ + struct dept_task *dt = dept_task(); + + dt->wait_hist_pos++; + dt->wait_hist_pos %= DEPT_MAX_WAIT_HIST; +} + +static struct dept_wait_hist *new_hist(void) +{ + struct dept_wait_hist *wh = hist(hist_pos_next()); + + hist_advance(); + return wh; +} + +static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) +{ + struct dept_wait_hist *wh = new_hist(); + + if (likely(wh->wait)) + put_wait(wh->wait); + + wh->wait = get_wait(w); + wh->wgen = wg; + wh->ctxt_id = ctxt_id; +} + +/* + * Should be called after setting up e's iecxt and w's iwait. + */ +static void add_dep(struct dept_ecxt *e, struct dept_wait *w) +{ + struct dept_class *fc = e->class; + struct dept_class *tc = w->class; + struct dept_dep *d; + int i; + + if (lookup_dep(fc, tc)) + return; + + if (unlikely(!dept_lock())) + return; + + /* + * __add_dep() will lookup_dep() again with lock held. + */ + d = __add_dep(e, w); + if (d) { + check_dl_bfs(d); + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iwait *fiw = iwait(fc, i); + struct dept_iecxt *found_ie; + struct dept_iwait *found_iw; + + /* + * '->touched == false' guarantees there's no + * parent that has been set ->wait. + */ + if (!fiw->touched) + continue; + + /* + * find_iw_bfs() will untouch the iwait if + * not found. + */ + found_iw = find_iw_bfs(fc, i); + + if (!found_iw) + continue; + + found_ie = touch_iw_find_ie_bfs(tc, i); + __add_idep(found_ie, found_iw); + } + } + dept_unlock(); +} + +static atomic_t wgen = ATOMIC_INIT(1); + +static void add_wait(struct dept_class *c, unsigned long ip, + const char *w_fn, int sub_l, bool sched_sleep) +{ + struct dept_task *dt = dept_task(); + struct dept_wait *w; + unsigned int wg = 0U; + int irq; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + w = new_wait(); + if (unlikely(!w)) + return; + + WRITE_ONCE(w->class, get_class(c)); + w->wait_ip = ip; + w->wait_fn = w_fn; + w->wait_stack = get_current_stack(); + w->sched_sleep = sched_sleep; + + irq = cur_irq(); + if (irq < DEPT_IRQS_NR) + add_iwait(c, irq, w); + + /* + * Avoid adding dependency between user aware nested ecxt and + * wait. + */ + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + + /* + * the case of invalid key'ed one + */ + if (!eh->ecxt) + continue; + + if (eh->ecxt->class != c || eh->sub_l == sub_l) + add_dep(eh->ecxt, w); + } + + if (!wait_consumed(w) && !rich_stack) { + if (w->wait_stack) + put_stack(w->wait_stack); + w->wait_stack = NULL; + } + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + add_hist(w, wg, cur_ctxt_id()); + + del_wait(w); +} + +static bool add_ecxt(struct dept_map *m, struct dept_class *c, + unsigned long ip, const char *c_fn, + const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + unsigned long irqf; + int irq; + + if (DEPT_WARN_ON(!valid_class(c))) + return false; + + if (DEPT_WARN_ON_ONCE(dt->ecxt_held_pos >= DEPT_MAX_ECXT_HELD)) + return false; + + if (m->nocheck) { + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = NULL; + eh->map = m; + eh->class = get_class(c); + eh->wgen = atomic_read(&wgen); + eh->sub_l = sub_l; + + return true; + } + + e = new_ecxt(); + if (unlikely(!e)) + return false; + + e->class = get_class(c); + e->ecxt_ip = ip; + e->ecxt_stack = ip && rich_stack ? get_current_stack() : NULL; + e->event_fn = e_fn; + e->ecxt_fn = c_fn; + + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = get_ecxt(e); + eh->map = m; + eh->class = get_class(c); + eh->wgen = atomic_read(&wgen); + eh->sub_l = sub_l; + + irqf = cur_enirqf(); + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + add_iecxt(c, irq, e, false); + + del_ecxt(e); + return true; +} + +static int find_ecxt_pos(struct dept_map *m, struct dept_class *c, + bool newfirst) +{ + struct dept_task *dt = dept_task(); + int i; + + if (newfirst) { + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } else { + for (i = 0; i < dt->ecxt_held_pos; i++) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } + return -1; +} + +static bool pop_ecxt(struct dept_map *m, struct dept_class *c) +{ + struct dept_task *dt = dept_task(); + int pos; + int i; + + pos = find_ecxt_pos(m, c, true); + if (pos == -1) + return false; + + if (dt->ecxt_held[pos].class) + put_class(dt->ecxt_held[pos].class); + + if (dt->ecxt_held[pos].ecxt) + put_ecxt(dt->ecxt_held[pos].ecxt); + + dt->ecxt_held_pos--; + + for (i = pos; i < dt->ecxt_held_pos; i++) + dt->ecxt_held[i] = dt->ecxt_held[i + 1]; + return true; +} + +static bool good_hist(struct dept_wait_hist *wh, unsigned int wg) +{ + return wh->wait != NULL && before(wg, wh->wgen); +} + +/* + * Binary-search the ring buffer for the earliest valid wait. + */ +static int find_hist_pos(unsigned int wg) +{ + int oldest; + int l; + int r; + int pos; + + oldest = hist_pos_next(); + if (unlikely(good_hist(hist(oldest), wg))) { + DEPT_INFO_ONCE("Need to expand the ring buffer.\n"); + return oldest; + } + + l = oldest + 1; + r = oldest + DEPT_MAX_WAIT_HIST - 1; + for (pos = (l + r) / 2; l <= r; pos = (l + r) / 2) { + struct dept_wait_hist *p = hist(pos - 1); + struct dept_wait_hist *wh = hist(pos); + + if (!good_hist(p, wg) && good_hist(wh, wg)) + return pos % DEPT_MAX_WAIT_HIST; + if (good_hist(wh, wg)) + r = pos - 1; + else + l = pos + 1; + } + return -1; +} + +static void do_event(struct dept_map *m, struct dept_class *c, + unsigned int wg, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + struct dept_wait_hist *wh; + struct dept_ecxt_held *eh; + unsigned int ctxt_id; + int end; + int pos; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (m->nocheck) + return; + + /* + * The event was triggered before wait. + */ + if (!wg) + return; + + pos = find_ecxt_pos(m, c, false); + if (pos == -1) + return; + + eh = dt->ecxt_held + pos; + + if (DEPT_WARN_ON(!eh->ecxt)) + return; + + eh->ecxt->event_ip = ip; + eh->ecxt->event_stack = get_current_stack(); + + /* + * The ecxt already has done what it needs. + */ + if (!before(wg, eh->wgen)) + return; + + pos = find_hist_pos(wg); + if (pos == -1) + return; + + ctxt_id = cur_ctxt_id(); + end = hist_pos_next(); + end = end > pos ? end : end + DEPT_MAX_WAIT_HIST; + for (wh = hist(pos); pos < end; wh = hist(++pos)) { + if (after(wh->wgen, eh->wgen)) + break; + + if (dt->in_sched && wh->wait->sched_sleep) + continue; + + if (wh->ctxt_id == ctxt_id) + add_dep(eh->ecxt, wh->wait); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_ecxt *e; + + if (before(dt->wgen_enirq[i], wg)) + continue; + + e = eh->ecxt; + add_iecxt(e->class, i, e, false); + } +} + +static void del_dep_rcu(struct rcu_head *rh) +{ + struct dept_dep *d = container_of(rh, struct dept_dep, rh); + + preempt_disable(); + del_dep(d); + preempt_enable(); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_class(struct dept_class *c) +{ + struct dept_dep *d, *n; + int i; + + list_for_each_entry_safe(d, n, &c->dep_head, dep_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + list_for_each_entry_safe(d, n, &c->dep_rev_head, dep_rev_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + stale_iecxt(iecxt(c, i)); + stale_iwait(iwait(c, i)); + } +} + +/* + * Context control + * ===================================================================== + * Whether a wait is in {hard,soft}-IRQ context or whether + * {hard,soft}-IRQ has been enabled on the way to an event is very + * important to check dependency. All those things should be tracked. + */ + +static unsigned long cur_enirqf(void) +{ + struct dept_task *dt = dept_task(); + int he = dt->hardirqs_enabled; + int se = dt->softirqs_enabled; + + if (he) + return DEPT_HIRQF | (se ? DEPT_SIRQF : 0UL); + return 0UL; +} + +static int cur_irq(void) +{ + if (lockdep_softirq_context(current)) + return DEPT_SIRQ; + if (lockdep_hardirq_context()) + return DEPT_HIRQ; + return DEPT_IRQS_NR; +} + +static unsigned int cur_ctxt_id(void) +{ + struct dept_task *dt = dept_task(); + int irq = cur_irq(); + + /* + * Normal process context + */ + if (irq == DEPT_IRQS_NR) + return 0U; + + return dt->irq_id[irq] | (1UL << irq); +} + +static void enirq_transition(int irq) +{ + struct dept_task *dt = dept_task(); + int i; + + /* + * IRQ can cut in on the way to the event. Used for cross-event + * detection. + * + * wait context event context(ecxt) + * ------------ ------------------- + * wait event + * UPDATE wgen + * observe IRQ enabled + * UPDATE wgen + * keep the wgen locally + * + * on the event + * check the wgen kept + */ + + /* + * Avoid zero wgen. + */ + dt->wgen_enirq[irq] = atomic_inc_return(&wgen) ?: + atomic_inc_return(&wgen); + + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + + eh = dt->ecxt_held + i; + e = eh->ecxt; + if (e) + add_iecxt(e->class, irq, e, true); + } +} + +static void dept_enirq(unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long irqf = cur_enirqf(); + int irq; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * IRQ ON/OFF transition might happen while Dept is working. + * We cannot handle recursive entrance. Just ingnore it. + * Only transitions outside of Dept will be considered. + */ + if (dt->recursive) + return; + + flags = dept_enter(); + + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + dt->enirq_ip[irq] = ip; + enirq_transition(irq); + } + + dept_exit(flags); +} + +void dept_softirqs_on_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = true; + dept_enirq(ip); +} + +void dept_hardirqs_on(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = true; + dept_enirq(_RET_IP_); +} + +void dept_softirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = false; +} + +void dept_hardirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = false; +} + +/* + * Ensure it's the outmost softirq context. + */ +void dept_softirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; +} + +/* + * Ensure it's the outmost hardirq context. + */ +void dept_hardirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; +} + +void dept_sched_enter(void) +{ + dept_task()->in_sched = true; +} + +void dept_sched_exit(void) +{ + dept_task()->in_sched = false; +} + +/* + * Exposed APIs + * ===================================================================== + */ + +static void clean_classes_cache(struct dept_key *k) +{ + int i; + + for (i = 0; i < DEPT_MAX_SUBCLASSES_CACHE; i++) { + if (!READ_ONCE(k->classes[i])) + continue; + + WRITE_ONCE(k->classes[i], NULL); + } +} + +void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u < 0)) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u >= DEPT_MAX_SUBCLASSES_USR)) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + clean_classes_cache(&m->map_key); + + m->keys = k; + m->sub_u = sub_u; + m->name = n; + m->wgen = 0U; + m->nocheck = !valid_key(k); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_init); + +void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + if (k) { + clean_classes_cache(&m->map_key); + m->keys = k; + m->nocheck = !valid_key(k); + } + + if (sub_u >= 0 && sub_u < DEPT_MAX_SUBCLASSES_USR) + m->sub_u = sub_u; + + if (n) + m->name = n; + + m->wgen = 0U; + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_reinit); + +void dept_map_copy(struct dept_map *to, struct dept_map *from) +{ + if (unlikely(!dept_working())) { + to->nocheck = true; + return; + } + + *to = *from; + + /* + * XXX: 'to' might be in a stack or something. Using the address + * in a stack segment as a key is meaningless. Just ignore the + * case for now. + */ + if (!to->keys) { + to->nocheck = true; + return; + } + + /* + * Since the class cache can be modified concurrently we could + * observe half pointers (64bit arch using 32bit copy insns). + * Therefore clear the caches and take the performance hit. + * + * XXX: Doesn't work well with lockdep_set_class_and_subclass() + * since that relies on cache abuse. + */ + clean_classes_cache(&to->map_key); +} + +static LIST_HEAD(classes); + +static bool within(const void *addr, void *start, unsigned long size) +{ + return addr >= start && addr < start + size; +} + +void dept_free_range(void *start, unsigned int sz) +{ + struct dept_task *dt = dept_task(); + struct dept_class *c, *n; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Failed to successfully free Dept objects.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_free_range() should not fail. + * + * FIXME: Should be fixed if dept_free_range() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + list_for_each_entry_safe(c, n, &classes, all_node) { + if (!within((void *)c->key, start, sz) && + !within(c->name, start, sz)) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} + +static int sub_id(struct dept_map *m, int e) +{ + return (m ? m->sub_u : 0) + e * DEPT_MAX_SUBCLASSES_USR; +} + +static struct dept_class *check_new_class(struct dept_key *local, + struct dept_key *k, int sub_id, + const char *n, bool sched_map) +{ + struct dept_class *c = NULL; + + if (DEPT_WARN_ON(sub_id >= DEPT_MAX_SUBCLASSES)) + return NULL; + + if (DEPT_WARN_ON(!k)) + return NULL; + + /* + * XXX: Assume that users prevent the map from using if any of + * the cached keys has been invalidated. If not, the cache, + * local->classes should not be used because it would be racy + * with class deletion. + */ + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + c = READ_ONCE(local->classes[sub_id]); + + if (c) + return c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (c) + goto caching; + + if (unlikely(!dept_lock())) + return NULL; + + c = lookup_class((unsigned long)k->base + sub_id); + if (unlikely(c)) + goto unlock; + + c = new_class(); + if (unlikely(!c)) + goto unlock; + + c->name = n; + c->sched_map = sched_map; + c->sub_id = sub_id; + c->key = (unsigned long)(k->base + sub_id); + hash_add_class(c); + list_add(&c->all_node, &classes); +unlock: + dept_unlock(); +caching: + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + WRITE_ONCE(local->classes[sub_id], c); + + return c; +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool sched_map) +{ + int e; + + /* + * Be as conservative as possible. In case of mulitple waits for + * a single dept_map, we are going to keep only the last wait's + * wgen for simplicity - keeping all wgens seems overengineering. + * + * Of course, it might cause missing some dependencies that + * would rarely, probabily never, happen but it helps avoid + * false positive report. + */ + for_each_set_bit(e, &w_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, sched_map); + if (!c) + continue; + + add_wait(c, ip, w_fn, sub_l, sched_sleep); + } +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn, + bool sched_map) +{ + struct dept_class *c; + struct dept_key *k; + int e; + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (DEPT_WARN_ON(e >= DEPT_MAX_SUBCLASSES_EVT)) + return; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); + + if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { + do_event(m, c, READ_ONCE(m->wgen), ip); + pop_ecxt(m, c); + } +} + +void dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (m->nocheck) + return; + + flags = dept_enter(); + + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_wait); + +void dept_stage_wait(struct dept_map *m, struct dept_key *k, + unsigned long ip, const char *w_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m && m->nocheck) + return; + + /* + * Either m or k should be passed. Which means Dept relies on + * either its own map or the caller's position in the code when + * determining its class. + */ + if (DEPT_WARN_ON(!m && !k)) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + /* + * Ensure the outmost dept_stage_wait() works. + */ + if (dt->stage_m.keys) + goto exit; + + if (m) { + dt->stage_m = *m; + + /* + * Ensure dt->stage_m.keys != NULL and it works with the + * map's map_key, not stage_m's one when ->keys == NULL. + */ + if (!m->keys) + dt->stage_m.keys = &m->map_key; + } else { + dt->stage_m.name = w_fn; + dt->stage_sched_map = true; + } + + /* + * dept_map_reinit() includes WRITE_ONCE(->wgen, 0U) that + * effectively disables the map just in case real sleep won't + * happen. dept_request_event_wait_commit() will enable it. + */ + dept_map_reinit(&dt->stage_m, k, -1, NULL); + + dt->stage_w_fn = w_fn; + dt->stage_ip = ip; +exit: + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_stage_wait); + +static void __dept_clean_stage(struct dept_task *dt) +{ + memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); + dt->stage_sched_map = false; + dt->stage_w_fn = NULL; + dt->stage_ip = 0UL; +} + +void dept_clean_stage(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + __dept_clean_stage(dt); + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_clean_stage); + +/* + * Always called from __schedule(). + */ +void dept_request_event_wait_commit(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + unsigned int wg; + unsigned long ip; + const char *w_fn; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + /* + * It's impossible that __schedule() is called while Dept is + * working that already disabled IRQ at the entrance. + */ + if (DEPT_WARN_ON(dt->recursive)) + return; + + flags = dept_enter(); + + /* + * Checks if current has staged a wait. + */ + if (!dt->stage_m.keys) + goto exit; + + w_fn = dt->stage_w_fn; + ip = dt->stage_ip; + sched_map = dt->stage_sched_map; + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + WRITE_ONCE(dt->stage_m.wgen, wg); + + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); +exit: + dept_exit(flags); +} + +/* + * Always called from try_to_wake_up(). + */ +void dept_stage_event(struct task_struct *requestor, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + struct dept_task *dt_req = &requestor->dept_task; + unsigned long flags; + struct dept_map m; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + flags = dept_enter(); + + /* + * Serializing is unnecessary as long as it always comes from + * try_to_wake_up(). + */ + m = dt_req->stage_m; + sched_map = dt_req->stage_sched_map; + __dept_clean_stage(dt_req); + + /* + * ->stage_m.keys should not be NULL if it's in use. Should + * make sure that it's not NULL when staging a valid map. + */ + if (!m.keys) + goto exit; + + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); +exit: + dept_exit(flags); +} + +/* + * Modifies the latest ecxt corresponding to m and e_f. + */ +void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, + struct dept_key *new_k, unsigned long new_e_f, + unsigned long new_ip, const char *new_c_fn, + const char *new_e_fn, int new_sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_class *c; + struct dept_key *k; + unsigned long flags; + int pos = -1; + int new_e; + int e; + + if (unlikely(!dept_working())) + return; + + /* + * XXX: Couldn't handle re-enterance cases. Ingore it for now. + */ + if (dt->recursive) + return; + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + pos = find_ecxt_pos(m, c, true); + if (pos != -1) + break; + } + + if (unlikely(pos == -1)) + goto exit; + + eh = dt->ecxt_held + pos; + new_sub_l = new_sub_l >= 0 ? new_sub_l : eh->sub_l; + + new_e = find_first_bit(&new_e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (new_e < DEPT_MAX_SUBCLASSES_EVT) + /* + * Let it work with the first bit anyway. + */ + DEPT_WARN_ON(1UL << new_e != new_e_f); + else + new_e = e; + + pop_ecxt(m, c); + + /* + * Apply the key to the map. + */ + if (new_k) + dept_map_reinit(m, new_k, -1, NULL); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); + + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + goto exit; + + /* + * Successfully pop_ecxt()ed but failed to add_ecxt(). + */ + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_map_ecxt_modify); + +void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, + const char *c_fn, const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + struct dept_class *c; + struct dept_key *k; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt++; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (e >= DEPT_MAX_SUBCLASSES_EVT) + goto missing_ecxt; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); + + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + goto exit; +missing_ecxt: + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_enter); + +bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + bool ret = false; + int e; + + if (unlikely(!dept_working())) + return false; + + if (dt->recursive) + return false; + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + if (find_ecxt_pos(m, c, true) != -1) { + ret = true; + break; + } + } + + dept_exit(flags); + + return ret; +} +EXPORT_SYMBOL_GPL(dept_ecxt_holding); + +void dept_request_event(struct dept_map *m) +{ + unsigned long flags; + unsigned int wg; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + WRITE_ONCE(m->wgen, wg); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_request_event); + +void dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + if (dt->recursive) { + /* + * Dept won't work with this even though an event + * context has been asked. Don't make it confused at + * handling the event. Disable it until the next. + */ + WRITE_ONCE(m->wgen, 0U); + return; + } + + flags = dept_enter(); + + __dept_event(m, e_f, ip, e_fn, false); + + /* + * Keep the map diabled until the next sleep. + */ + WRITE_ONCE(m->wgen, 0U); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_event); + +void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, + unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt--; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + if (pop_ecxt(m, c)) + goto exit; + } + + dt->missing_ecxt--; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_exit); + +void dept_task_exit(struct task_struct *t) +{ + struct dept_task *dt = &t->dept_task; + int i; + + if (unlikely(!dept_working())) + return; + + raw_local_irq_disable(); + + if (dt->stack) + put_stack(dt->stack); + + for (i = 0; i < dt->ecxt_held_pos; i++) { + if (dt->ecxt_held[i].class) + put_class(dt->ecxt_held[i].class); + if (dt->ecxt_held[i].ecxt) + put_ecxt(dt->ecxt_held[i].ecxt); + } + + for (i = 0; i < DEPT_MAX_WAIT_HIST; i++) + if (dt->wait_hist[i].wait) + put_wait(dt->wait_hist[i].wait); + + dt->task_exit = true; + dept_off(); + + raw_local_irq_enable(); +} + +void dept_task_init(struct task_struct *t) +{ + memset(&t->dept_task, 0x0, sizeof(struct dept_task)); +} + +void dept_key_init(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Key initialization fails.\n"); + return; + } + + flags = dept_enter(); + + clean_classes_cache(k); + + /* + * dept_key_init() should not fail. + * + * FIXME: Should be fixed if dept_key_init() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + DEPT_STOP("The class(%s/%d) has not been removed.\n", + c->name, sub_id); + break; + } + + dept_unlock(); + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_key_init); + +void dept_key_destroy(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive == 1 && dt->task_exit) { + /* + * Need to allow to go ahead in this case where + * ->recursive has been set to 1 by dept_off() in + * dept_task_exit() and ->task_exit has been set to + * true in dept_task_exit(). + */ + } else if (dt->recursive) { + DEPT_STOP("Key destroying fails.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_key_destroy() should not fail. + * + * FIXME: Should be fixed if dept_key_destroy() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(dept_key_destroy); + +static void move_llist(struct llist_head *to, struct llist_head *from) +{ + struct llist_node *first = llist_del_all(from); + struct llist_node *last; + + if (!first) + return; + + for (last = first; last->next; last = last->next); + llist_add_batch(first, last, to); +} + +static void migrate_per_cpu_pool(void) +{ + const int boot_cpu = 0; + int i; + + /* + * The boot CPU has been using the temperal local pool so far. + * From now on that per_cpu areas have been ready, use the + * per_cpu local pool instead. + */ + DEPT_WARN_ON(smp_processor_id() != boot_cpu); + for (i = 0; i < OBJECT_NR; i++) { + struct llist_head *from; + struct llist_head *to; + + from = &pool[i].boot_pool; + to = per_cpu_ptr(pool[i].lpool, boot_cpu); + move_llist(to, from); + } +} + +#define B2KB(B) ((B) / 1024) + +/* + * Should be called after setup_per_cpu_areas() and before no non-boot + * CPUs have been on. + */ +void __init dept_init(void) +{ + size_t mem_total = 0; + + local_irq_disable(); + dept_per_cpu_ready = 1; + migrate_per_cpu_pool(); + local_irq_enable(); + +#define HASH(id, bits) BUILD_BUG_ON(1 << (bits) <= 0); + #include "dept_hash.h" +#undef HASH +#define OBJECT(id, nr) mem_total += sizeof(struct dept_##id) * nr; + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) mem_total += sizeof(struct hlist_head) * (1 << (bits)); + #include "dept_hash.h" +#undef HASH + + pr_info("DEPendency Tracker: Copyright (c) 2020 LG Electronics, Inc., Byungchul Park\n"); + pr_info("... DEPT_MAX_STACK_ENTRY: %d\n", DEPT_MAX_STACK_ENTRY); + pr_info("... DEPT_MAX_WAIT_HIST : %d\n", DEPT_MAX_WAIT_HIST); + pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); + pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); +#define OBJECT(id, nr) \ + pr_info("... memory used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct dept_##id) * nr)); + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) \ + pr_info("... hash list head used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); + #include "dept_hash.h" +#undef HASH + pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); +} diff --git a/kernel/dependency/dept_hash.h b/kernel/dependency/dept_hash.h new file mode 100644 index 000000000000..fd85aab1fdfb --- /dev/null +++ b/kernel/dependency/dept_hash.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * HASH(id, bits) + * + * id : Id for the object of struct dept_##id. + * bits: 1UL << bits is the hash table size. + */ + +HASH(dep, 12) +HASH(class, 12) diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h new file mode 100644 index 000000000000..0b7eb16fe9fb --- /dev/null +++ b/kernel/dependency/dept_object.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * OBJECT(id, nr) + * + * id: Id for the object of struct dept_##id. + * nr: # of the object that should be kept in the pool. + */ + +OBJECT(dep, 1024 * 8) +OBJECT(class, 1024 * 8) +OBJECT(stack, 1024 * 32) +OBJECT(ecxt, 1024 * 16) +OBJECT(wait, 1024 * 32) diff --git a/kernel/exit.c b/kernel/exit.c index aedc0832c9f4..32b7f551674b 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -917,6 +917,7 @@ void __noreturn do_exit(long code) exit_tasks_rcu_finish(); lockdep_free_task(tsk); + dept_task_exit(tsk); do_task_dead(); } diff --git a/kernel/fork.c b/kernel/fork.c index 10917c3e1f03..2f1f5dbec154 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -99,6 +99,7 @@ #include #include #include +#include #include #include @@ -2455,6 +2456,7 @@ __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_LOCKDEP lockdep_init_task(p); #endif + dept_task_init(p); #ifdef CONFIG_DEBUG_MUTEXES p->blocked_on = NULL; /* not blocked yet */ diff --git a/kernel/module/main.c b/kernel/module/main.c index 98fedfdb8db5..507c73b10258 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1228,12 +1228,14 @@ static void free_mod_mem(struct module *mod) /* Free lock-classes; relies on the preceding sync_rcu(). */ lockdep_free_key_range(mod_mem->base, mod_mem->size); + dept_free_range(mod_mem->base, mod_mem->size); if (mod_mem->size) module_memory_free(mod_mem->base, type); } /* MOD_DATA hosts mod, so free it at last */ lockdep_free_key_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); + dept_free_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); module_memory_free(mod->mem[MOD_DATA].base, MOD_DATA); } @@ -3020,6 +3022,8 @@ static int load_module(struct load_info *info, const char __user *uargs, for_class_mod_mem_type(type, core_data) { lockdep_free_key_range(mod->mem[type].base, mod->mem[type].size); + dept_free_range(mod->mem[type].base, + mod->mem[type].size); } module_deallocate(mod, info); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a708d225c28e..e2a7ee076a58 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -64,6 +64,7 @@ #include #include #include +#include #ifdef CONFIG_PREEMPT_DYNAMIC # ifdef CONFIG_GENERIC_ENTRY @@ -4197,6 +4198,8 @@ int try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) guard(preempt)(); int cpu, success = 0; + dept_stage_event(p, _RET_IP_); + if (p == current) { /* * We're waking current, this means 'p->on_rq' and 'task_cpu(p) @@ -6578,6 +6581,12 @@ static void __sched notrace __schedule(unsigned int sched_mode) rq = cpu_rq(cpu); prev = rq->curr; + prev_state = READ_ONCE(prev->__state); + if (sched_mode != SM_PREEMPT && prev_state & TASK_NORMAL) + dept_request_event_wait_commit(); + + dept_sched_enter(); + schedule_debug(prev, !!sched_mode); if (sched_feat(HRTICK) || sched_feat(HRTICK_DL)) @@ -6691,6 +6700,7 @@ static void __sched notrace __schedule(unsigned int sched_mode) __balance_callbacks(rq); raw_spin_rq_unlock_irq(rq); } + dept_sched_exit(); } void __noreturn do_task_dead(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 4405f81248fb..9602f41ad8e8 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1285,6 +1285,33 @@ config DEBUG_PREEMPT menu "Lock Debugging (spinlocks, mutexes, etc...)" +config DEPT + bool "Dependency tracking (EXPERIMENTAL)" + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + select DEBUG_SPINLOCK + select DEBUG_MUTEXES + select DEBUG_RT_MUTEXES if RT_MUTEXES + select DEBUG_RWSEMS + select DEBUG_WW_MUTEX_SLOWPATH + select DEBUG_LOCK_ALLOC + select TRACE_IRQFLAGS + select STACKTRACE + select FRAME_POINTER if !MIPS && !PPC && !ARM && !S390 && !MICROBLAZE && !ARC && !X86 + select KALLSYMS + select KALLSYMS_ALL + select PROVE_LOCKING + default n + help + Check dependencies between wait and event and report it if + deadlock possibility has been detected. Multiple reports are + allowed if there are more than a single problem. + + This feature is considered EXPERIMENTAL that might produce + false positive reports because new dependencies start to be + tracked, that have never been tracked before. It's worth + noting, to mitigate the impact by the false positives, multi + reporting has been supported. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c index 6f6a5fc85b42..2558ae57b117 100644 --- a/lib/locking-selftest.c +++ b/lib/locking-selftest.c @@ -1398,6 +1398,8 @@ static void reset_locks(void) local_irq_disable(); lockdep_free_key_range(&ww_lockdep.acquire_key, 1); lockdep_free_key_range(&ww_lockdep.mutex_key, 1); + dept_free_range(&ww_lockdep.acquire_key, 1); + dept_free_range(&ww_lockdep.mutex_key, 1); I1(A); I1(B); I1(C); I1(D); I1(X1); I1(X2); I1(Y1); I1(Y2); I1(Z1); I1(Z2); From patchwork Wed Mar 6 08:54:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583564 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 04EF05DF18; Wed, 6 Mar 2024 08:55:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; cv=none; b=LiB+HJuvpcuDc0OoIwI3WuoOo8oXvtpLf3PvyQAsMqQ1+ngFMHJ8YXvofO7KJP/95C5Knppt/VLadQo5374Y0XWLvImhcd5Jp+9mJy3dABd5GUkjiQrJiXWeKf24Tyegh5lVc/sR/hJYQ/OFJBNlU6QEkiVyPgrdbk/TYZXbTf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; c=relaxed/simple; bh=yYyo9J9LbrfncpB2CWVjqJhM6aD/eyXV6B0l1bR9NkI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=la+Fs5wwXVjfW5H18u1mY6wgQ3AeZ6LmxpwU6j9bxaXY12alza3TO+T1jt10oda5RP94PV7mr+urvteWj26m02NecgZEyeoaQ14hXyG/0wX+ZTeQcrCCpfmMcONemMdTGTXKGh6OKZpfLreF5M3IffIP/abjzzN+XKfMcY00w0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-59-65e82f7c4dfc From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 03/27] dept: Add single event dependency tracker APIs Date: Wed, 6 Mar 2024 17:54:49 +0900 Message-Id: <20240306085513.41482-4-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0xTZxgH8L3vOec9h8YuJ5XEIyZiqmSLxguLLo/xmizTVxPNksXEzXhp 4EQaC2qhXBQJyiV4waIJVoFhQe0qraKn9W5NrQNFB8PRKCKiICqEa9BWKkRt3fzy5Jc8/+f/ 6REYjZuLEfQpabIxRWfQEhWrGphQNXvP3B553usiAY4cmgeBd0UsVNQ6CTSfdyBwuvdi6K1b CY+D/QjGGv9hwFLajKCq8xkD7voOBB77PgIt3d+CPzBEoKH0IIG8U7UEHvaNY2g/dhSDQ1kD D0qqMXhDb1iw9BIot+Th8OjBELLV8GDLjYMuexkP453x0NDxiANP2yw4UdlO4KangYX6q10Y Wq5XEOhwfuLgQf09FpqPFHNwbrCaQF/QxoAtMMTDv14rhgv54aLCtx85uFvsxVB4+iIG/5Mb CG4VvcCgOB8RuBPox+BSShn48Gcdgq7DAzwUHArxUL73MIKDBcdYyG9fAGOjFWT5Qnqnf4ih +a4M6glaWXq/WqLXyp7xNP9WG0+tiom67DPpqZu9mFaNBDiq1OwnVBk5ytMDA35MB5uaeHrv +BhLu/0W/EvM76rFibJBny4b5y7dokrylZRxOx5PygxaYnORX3MARQmSOF8687aQ/2qzdRhH TMTvpNbWEBNxtDhNchW/5iJmxH6VdLppRcQTxVWSx+lDEbNinDTcV/PlVi0ukJ5evvZ/Z6zk uOD90hMl/iiZB80kYk0405hXFbYqnBkVpM6Xn5j/DiZLt+2tbAlSW9E3NUijT0lP1ukN8+ck ZaXoM+ckbE9WUPiXbHvGN1xFI82/+pAoIO0E9fKoN7KG06WnZiX7kCQw2mh19oduWaNO1GXt ko3bNxtNBjnVh6YIrHaS+odgRqJG3KpLk7fJ8g7Z+HWLhaiYXOQwLTQtm+6j56fvz3FcqjyR dfenuoy/05Jv/HVmNPOJs2Ho/voR81mk+Ozv27zujc+NNIfd2WJ6uM486vqDlC9S4EoLp4tf Gtfdl3AlgS+Qvl8dm/1btC/0s2H909rGk2tVlYVNsUtmbFpbv3KqmwzmTO7ZefzFq+HdJNd+ OTsjW8umJuniZzLGVN1n2ZqmZUcDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTYRgH8N73nPOeOVydluCpiGRlRdGNWjxUREXYW1YEQTeCWnVyo2mx pWUXsNQw8066WhrTYomumltEZQuZZM3QvIzUWiPNLqapmLNMu7iiLw8/eP7P/9MjY5QF3BSZ Lu6YZIjT6FVEzsq3rEief3rhJ2lRRsYsyM1YBIHBNBYK79gINNwuR2C7exZD15P10DLUg2Ck 7gUDpvwGBMXtbxi4W+NH4Co9R6C5czx4A30EPPkXCSRfv0OgsXsUg68gD0O5YzM8zynBUDX8 kQVTF4GrpmQ8Nj5hGLaW8WBNioSOUjMPo+2LweN/yUF1kYcD16t5cOWaj8Ajl4eFmvsdGJof FhLw235z8LzmGQsNuZkc3OotIdA9ZGXAGujjoanKgsGeMtZ2/usvDp5mVmE4f6MCg7etEsHj tLcYHLaXBKoDPRicjnwGftx8gqAj6wsPqRnDPFw9m4XgYmoBCyk+NYx8LySrl9Pqnj6GpjiP U9eQhaW1JSJ9YH7D05THr3hqccRTZ+lcev1RF6bFAwGOOsouEOoYyONp+hcvpr319Tx9dnmE pZ1eE946dbd85UFJr0uQDAtX7ZNr3Tlm7mhL+Ikh0/Qk5FWmoxCZKCwVsy39OGgizBZbW4eZ oMOECNGZ+YELmhF65OKN+qigJwkbRJfNjYJmhUixv7vs761CUIuv7z3g/3VOF8vtVX97QoRl YnZvNglaOZapSy4mOUhuQePKUJguLiFWo9OrFxgPaxPjdCcWHDgS60Bj32I9M5p7Hw02r3cj QYZUoYrVIR8lJadJMCbGupEoY1RhitM/OiWl4qAm8aRkOLLXEK+XjG40VcaqwhUbd0j7lEKM 5ph0WJKOSob/WywLmZKEZk4wpw2m46b9FDYfil6zvetSY21UtDBR1xpR/tWSX+mMMNeN37Jz Gv08LnXCFUNFY873pj3hFalZdu/8dz/ntHkEMbL3wuRTXn9MYIk/ets6+/uZ2la2tu3MW1O7 L7TSvlZu/1bqK2r5dVm9VR3J7nqhjZpRvypmE4nPOxWqT1SxRq1m8VzGYNT8AQw0yjwpAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Wrapped the base APIs for easier annotation on wait and event. Start with supporting waiters on each single event. More general support for multiple events is a future work. Do more when the need arises. How to annotate (the simplest way): 1. Initaialize a map for the interesting wait. /* * Recommand to place along with the wait instance. */ struct dept_map my_wait; /* * Recommand to place in the initialization code. */ sdt_map_init(&my_wait); 2. Place the following at the wait code. sdt_wait(&my_wait); 3. Place the following at the event code. sdt_event(&my_wait); That's it! Signed-off-by: Byungchul Park --- include/linux/dept_sdt.h | 62 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 include/linux/dept_sdt.h diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h new file mode 100644 index 000000000000..12a793b90c7e --- /dev/null +++ b/include/linux/dept_sdt.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Single-event Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_SDT_H +#define __LINUX_DEPT_SDT_H + +#include +#include + +#ifdef CONFIG_DEPT +#define sdt_map_init(m) \ + do { \ + static struct dept_key __key; \ + dept_map_init(m, &__key, 0, #m); \ + } while (0) + +#define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) + +#define sdt_wait(m) \ + do { \ + dept_request_event(m); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + } while (0) + +/* + * sdt_might_sleep() and its family will be committed in __schedule() + * when it actually gets to __schedule(). Both dept_request_event() and + * dept_wait() will be performed on the commit. + */ + +/* + * Use the code location as the class key if an explicit map is not used. + */ +#define sdt_might_sleep_start(m) \ + do { \ + struct dept_map *__m = m; \ + static struct dept_key __key; \ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + } while (0) + +#define sdt_might_sleep_end() dept_clean_stage() + +#define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) +#else /* !CONFIG_DEPT */ +#define sdt_map_init(m) do { } while (0) +#define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start(m) do { } while (0) +#define sdt_might_sleep_end() do { } while (0) +#define sdt_ecxt_enter(m) do { } while (0) +#define sdt_event(m) do { } while (0) +#define sdt_ecxt_exit(m) do { } while (0) +#endif +#endif /* __LINUX_DEPT_SDT_H */ From patchwork Wed Mar 6 08:54:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583566 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 04EAF5DF17; Wed, 6 Mar 2024 08:55:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; cv=none; b=ZSSKevU4neT9Vx9b2K/XQXXyqLrHa+IYi3/IoPxnitLG8noL0YD70J36TPtQiV1evBK0cjFfzJOH2XPdvVEhhziX45i2UfmIl50J6QSwE+NLG53eWayZIKWBNqq+J8QiZOlaTJ2NfQmS97jKbxB7HH+q8JN3D+DJvpGLPT3eoJ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; c=relaxed/simple; bh=ZnDomY68etRURkOftpc7pgxDDdwCW9QdH8k+FPh35/w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=bbXUeq3A0wK/iE7CGh5yIYxGACHJQAYIRWDPJ1Uy1zGcr3pEQ09+1gkb0NlToeM2h3zJgun42d1YVUcM0iQLMfsgCcGfxwDi1GBs0YinfBNT0H+RthvzKBum8alpySoSu70+VGHPTjc2DOjjNlg8TAW8cdOzHl38Y1E1STqunnI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-68-65e82f7c9f1f From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 04/27] dept: Add lock dependency tracker APIs Date: Wed, 6 Mar 2024 17:54:50 +0900 Message-Id: <20240306085513.41482-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/p+s4fjusrzDc1qwsYuFjzPP4mdmYfzxtOvpNN1dxJTK2 S5dF5SEqEusq51Qcd1EeyinSeUgPtyR1Uxqlqyzu9DS5zvzz2Wvvz/vz3uezfSSU/AHjJ1FF xoiaSKVawUppac+E3KCTCzrF4PjUiXAxJRhcv5JoyDYVsVB7txBBUXE8hq6XG+GD24lg+N17 CjLTaxHo21opKK5yICgznmKhoWMi2F19LNjSk1lIyDOxUNc9gqElIw1DoXkLvLmQi8E6+I2G zC4WrmUmYE/pxDBoKODAoPWHdmMWByNtC8HmaGSgrHkeXL3RwsLTMhsNVaXtGBoeZ7PgKBpl 4E1VNQ21F1MZuNOby0K320CBwdXHQb01B8M9nSfo9M8/DLxKtWI4nX8fg/3jEwTlSZ8xmIsa Wah0OTFYzOkUDN16iaD9XA8HiSmDHFyLP4cgOTGDBl3LYhgeyGZXLxMqnX2UoLMcFcrcObTw OpcIj7JaOUFX3swJOeYjgsUYKOQ97cKCvt/FCOaCM6xg7k/jhLM9diz01tRwQvWVYVrosGfi rX67pCvCRLUqVtQsWBkqDdd5FjvU6XvsplWLtKhQfhb5SAgfQqy6euY/l5oyvMzyc0lT0yA1 xlP42cSS+tWrU7xTSvJrNozxZH4NKS9569Vp3p8MmJK8LOMXE8/V1L/MWaTwntXLPvwScr73 PDvGco/nXYLew1KPZ0BC9Hnd7L+BaeS5sYm+gGQ5aFwBkqsiYyOUKnXI/PC4SNWx+fujIszI 80yGkyO7S1F/7fYKxEuQYoJstc83Uc4oY6PjIioQkVCKKbITQx2iXBamjDsuaqL2ao6oxegK NF1CK3xli9xHw+T8AWWMeFAUD4ma/10s8fHTopmbH26aqXYGVSc63Fr/+MOj6y5l4R0W6apt M4Z826I66e2hfT8+2Fr0QbuWhu20ZMH3ZyVrr9Nxj5fX2ZuX/5m0otXREJOc8OVZa9W4gBft l2//DpjzaRXz5Uob2bRUJydp6+tT5t0PsDBTtX4lzgYyfs/kocDG96NG3RPT630fbQo6Oly5 MJDSRCv/ArCRjz1IAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRTH+93nNlpeltgtIWv0ACM1yjj0DikvQS8ilP6pUZccTq1NTc1K 3RTzmZXO0mK+5rNWs4eVk6VozseylMpSK9HUnBtYk3xQTaN/Dh++5/Dl+4UjwCVacoVAHh7J K8NlCiklIkSHtqs3xvuO8n6DZh/IyfAD589UAgoNNRR03a9GUPMoEYOx5kB4P2VDMNv5Ggdt bheCoq/9ODxqGUBgqkiioHtoCfQ4HRRYctMpUJcYKHgzPodBX951DKqNB6H9WjEG5ukRArRj FBRo1ZhrjGIwra+iQZ+wFgYrbtMw93UTWAbekdB0x0KC6eMGuHW3j4J6k4WAlrpBDLqfF1Iw UPOHhPaWVgK6cjJJuGcvpmB8So+D3umg4a1Zh8EDjcst5cdvEl5lmjFIKX2IQU/vCwQNqV8w MNa8o6DJacOg1piLw0x5M4LBrAkakjOmaShIzEKQnpxHgKbPH2Z/FVJ7tnFNNgfOaWovcKYp HcG1FbPcs9v9NKdp+EhzOmMUV1vhzZXUj2Fc0aST5IxVVynOOHmd5tImejDObrXSXGv+LMEN 9WixI54nRDvO8Ap5NK/03XVKFKJxBTs3uiymzJyAElC1JA0JBSyzha0z5JHzTDHr2Q8fpvF5 dmdWsbWZ3xZ0nLGJ2FLr/nleyuxlG552LOgEs5b9ZUhdYDHjz7qa4v88vdjqB+YFFjJb2Wx7 NjXPEtdNp7qIuoZEOrSoCrnLw6PDZHKFv48qNCQ2XB7jczoizIhc76K/NJdTh352BzYiRoCk i8V7hCO8hJRFq2LDGhErwKXu4viZIV4iPiOLjeOVESeVUQpe1Yg8BYR0mfhAEH9KwpyVRfKh PH+OV/7fYgLhigTUa/MPOD8X3N/r+Hxj3/DExeN/HMHRdWl4h2ndneCBfLcRj6VlDet8PXW6 1pvCzCv2pMfu1jB1ycupAGtl2fLhsdnlM5Vxb27uXtXW2lF1rP1753dFxtn8/PIfufUlm8sD 35au4Y9avMwed/VBblsj7DvTycvto58MhicrV7e9OhzUIiVUIbJN3rhSJfsLvl4JyioDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Wrapped the base APIs for easier annotation on typical lock. Signed-off-by: Byungchul Park --- include/linux/dept_ldt.h | 77 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 include/linux/dept_ldt.h diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h new file mode 100644 index 000000000000..062613e89fc3 --- /dev/null +++ b/include/linux/dept_ldt.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Lock Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_LDT_H +#define __LINUX_DEPT_LDT_H + +#include + +#ifdef CONFIG_DEPT +#define LDT_EVT_L 1UL +#define LDT_EVT_R 2UL +#define LDT_EVT_W 1UL +#define LDT_EVT_RW (LDT_EVT_R | LDT_EVT_W) +#define LDT_EVT_ALL (LDT_EVT_L | LDT_EVT_RW) + +#define ldt_init(m, k, su, n) dept_map_init(m, k, su, n) +#define ldt_lock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ + } \ + } while (0) + +#define ldt_rlock(m, sl, t, n, i, q) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ + else { \ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ + } \ + } while (0) + +#define ldt_wlock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ + } \ + } while (0) + +#define ldt_unlock(m, i) dept_ecxt_exit(m, LDT_EVT_ALL, i) + +#define ldt_downgrade(m, i) \ + do { \ + if (dept_ecxt_holding(m, LDT_EVT_W)) \ + dept_map_ecxt_modify(m, LDT_EVT_W, NULL, LDT_EVT_R, i, "downgrade", "read_unlock", -1);\ + } while (0) + +#define ldt_set_class(m, n, k, sl, i) dept_map_ecxt_modify(m, LDT_EVT_ALL, k, 0UL, i, "lock_set_class", "(any)unlock", sl) +#else /* !CONFIG_DEPT */ +#define ldt_init(m, k, su, n) do { (void)(k); } while (0) +#define ldt_lock(m, sl, t, n, i) do { } while (0) +#define ldt_rlock(m, sl, t, n, i, q) do { } while (0) +#define ldt_wlock(m, sl, t, n, i) do { } while (0) +#define ldt_unlock(m, i) do { } while (0) +#define ldt_downgrade(m, i) do { } while (0) +#define ldt_set_class(m, n, k, sl, i) do { } while (0) +#endif +#endif /* __LINUX_DEPT_LDT_H */ From patchwork Wed Mar 6 08:54:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583565 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EA4485DF14; Wed, 6 Mar 2024 08:55:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; cv=none; b=JrCsnc6fyLMNsS+fYDlUrhNaRKZJLP/IxRULGuXk8LQkH3mNB3NbNERcRRTrCmZxob0w41VIGBNSq1S0XUIYF+j0pHsLFxbXdFgt9daScm8AJoS6F1we+PNVBmA2H6M3nqN3OfK0nVM9McxyXdw7m0i/PN4WcXv6qHXeOQK4a6c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715337; c=relaxed/simple; bh=QWHRCvl6WHVJCxC3TSQ+F2P9dkdR+c40EtoDliptNOM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=GHWimIRWeKS0KXRqZyShyTEbAqtHuAgv6/2CJpVcPoCEFKeFC0uZdVVnwtuIBfdJLHXdL/I3shMJOl6JtaQWAKzUjr3tKTG1u9pJmNLGUMW8GQnGdNE8g20dK+hAglmakHnc4rDoySkicgu9thTvOcpxds88z2eFvV2P9hOXbw0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-78-65e82f7cb1ab From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 05/27] dept: Tie to Lockdep and IRQ tracing Date: Wed, 6 Mar 2024 17:54:51 +0900 Message-Id: <20240306085513.41482-6-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUyMcRzA/Z73bs6enTZPsbGbZmNIKV8bMYZnNmTmZfyRhx46erFLpyJK aUnpZXIkdp1zvR3VnRF1lqyo04vcUi3Ri5d01RzXusrLXeaf7z7b9/P9/PVlcJmJ9GYUkadF ZaQQLqckhGRktnZ5wsqvom97tRfkZPiC42caAQXlBgraHpQhMDxMwmCofhu8G7chmGpuxUGd 14agsO89Dg8behGYiy9S8HZwDlgdYxQ05l2hIPluOQVvhqcx6Lmei0GZcQdYsrUY1Dq/EKAe ouCWOhlzja8YOPWlNOgTfaC/OJ+G6b5V0NjbQYK5exncvNNDQY25kYCGqn4M3j4toKDX8IcE S8MrAtpyMkm4P6qlYHhcj4PeMUZDe60Gg4oUVyj1x28SXmbWYpCqq8TA2lWN4FnaRwyMhg4K XjhsGJiMeThMFtUj6L86QsOlDCcNt5KuIrhy6ToBKT0BMDVRQG1cy7+wjeF8iukMbx7XEHyT luOf5L+n+ZRn3TSvMcbwpuKl/N2aIYwvtDtI3lh6meKN9lyaTx+xYvxoSwvNv7oxRfCDVjUW 7H1Qsi5UDFeoROXKoMOSsJJfhcSpa2dju4w/6ET0XUhHDMOxq7nb1dvTkccM9tVPUG6m2CVc Z6cTd7Mnu4gzZX4m3YyzNgmna9nq5rnsBs6i1s34BOvDfUrVku6klA3ghjV7/yUXcmUVtTMZ DzaQyxrNmtFlLqU5udDFEpczwXD2pi7634EX97y4k8hGUg2aVYpkikhVhKAIX70iLC5SEbvi aFSEEbleSZ8wfagK2dv21CGWQfLZ0o0eX0QZKaii4yLqEMfgck/puclBUSYNFeLiRWVUiDIm XIyuQ/MZQj5P6jd+JlTGHhdOiydF8ZSo/L/FGA/vRJSUFh//YcHJLTfzgx+1ezk7fE4k7Uss iykvmiwZCNykre8eCFyTpfR+07w+TAiy5IbUHFirW2y70JC+6wIem7BsYHhRz2t2t+rxtxGn oVKnPba3NUC2069/c4a56ETR5iWWs8n770Wl+fsH2a3ZTM75vhKbqmL/6K+xqswjwSES/wVy IjpMWLUUV0YLfwHS9CctRgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/P3Zz9dhq/6g/clowRI33m2Wx8xzz8gcyGTn501Mkd6chW Oo30aOpSsbtcJ3WU32XycGm1Ik1SJ7Q03fJw66op16Q8XJl/Pntt79def304UpFHB3JqzUlR q1HFKBkZJdu2MmVhYuhXcXHn8HLISV8M3u8XKSiqsDHQerccga0qmQB3wyZ4O+JBMPbyFQnG 3FYE5p4PJFQ1diNwlJ5noL13Gji9gww05V5mIOVmBQOv+8YJ6Mq7QkC5tBWas4sJqB39QoHR zUChMYXwna8EjFrLWLAmBYOrtICF8Z4l0NTdQUP99SYaHJ0L4NqNLgaeOJooaKx2EdD+qIiB btsfGpobn1PQmpNBw52BYgb6RqwkWL2DLLTVmgioNPhqqcO/aXiWUUtAquUeAc73jxHUXPxI gGTrYKDe6yHALuWS8PNWAwJXZj8LF9JHWShMzkRw+UIeBYauMBj7UcSsW4HrPYMkNthPY8eI icIvigX8sOADiw01nSw2SaewvXQ+vvnETWDzkJfGUtklBktDV1ic1u8k8EBLC4uf549RuNdp JHYE7ZWtOiTGqONFbeiaSFn07V9mKu7q2YT30jCbhL6p0pAfJ/DLhJ6GH8wEM3yI8O7dKDnB /vxswZ7xmZ5gkvfIBEvLxgmezq8Vmo2WSZ/ig4VPqcU+h+PkfJjQZ9r1LzlLKK+sncz48cuF rIGsSV3hU16mmJlsJDOhKWXIX62Jj1WpY8IW6Y5F6zXqhEVRx2Ml5PsW67nxnGr0vX1THeI5 pJwqX+f3RVTQqnidPrYOCRyp9Jcn/uwVFfJDKv0ZUXv8gPZUjKirQ0EcpZwp3xwhRir4I6qT 4jFRjBO1/1eC8wtMQs7zNkdFWsjIWIn+TVxbxv68T3W0Y9+ekuyjETOmn3iwc0NWesH9/Gt9 Huthxy39/tXh1a7+/L03zN5HluSd/NxMmKe5eufuLOXngIjwcBy4+bqYlmiL3O4eaigzJFgP lqxZ3z1n39PdWy2Ww+4H9hpxxYHwjmdScMgWV0BU0FL1TCWli1YtmU9qdaq/kC36fikDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: How to place Dept this way looks so ugly. But it's inevitable for now. The way should be enhanced gradually. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 7 +- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++------ include/linux/lockdep_types.h | 3 + include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 8 +-- kernel/locking/lockdep.c | 22 ++++++ 14 files changed, 127 insertions(+), 29 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 2b665c32f5fe..672dac1c3059 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -14,6 +14,7 @@ #include #include +#include #include #include @@ -61,8 +62,10 @@ extern void trace_hardirqs_off(void); # define lockdep_softirqs_enabled(p) ((p)->softirqs_enabled) # define lockdep_hardirq_enter() \ do { \ - if (__this_cpu_inc_return(hardirq_context) == 1)\ + if (__this_cpu_inc_return(hardirq_context) == 1) { \ current->hardirq_threaded = 0; \ + dept_hardirq_enter(); \ + } \ } while (0) # define lockdep_hardirq_threaded() \ do { \ @@ -137,6 +140,8 @@ do { \ # define lockdep_softirq_enter() \ do { \ current->softirq_context++; \ + if (current->softirq_context == 1) \ + dept_softirq_enter(); \ } while (0) # define lockdep_softirq_exit() \ do { \ diff --git a/include/linux/local_lock_internal.h b/include/linux/local_lock_internal.h index 975e33b793a7..39f67788fd95 100644 --- a/include/linux/local_lock_internal.h +++ b/include/linux/local_lock_internal.h @@ -21,6 +21,7 @@ typedef struct { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner = NULL, diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index dc2844b071c2..8825f535d36d 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -12,6 +12,7 @@ #include #include +#include #include struct task_struct; @@ -39,6 +40,8 @@ static inline void lockdep_copy_map(struct lockdep_map *to, */ for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) to->class_cache[i] = NULL; + + dept_map_copy(&to->dmap, &from->dmap); } /* @@ -466,7 +469,8 @@ enum xhlock_context_t { * Note that _name must not be NULL. */ #define STATIC_LOCKDEP_MAP_INIT(_name, _key) \ - { .name = (_name), .key = (void *)(_key), } + { .name = (_name), .key = (void *)(_key), \ + .dmap = DEPT_MAP_INITIALIZER(_name, _key) } static inline void lockdep_invariant_state(bool force) {} static inline void lockdep_free_task(struct task_struct *task) {} @@ -548,33 +552,89 @@ extern bool read_lock_is_recursive(void); #define lock_acquire_shared(l, s, t, n, i) lock_acquire(l, s, t, 1, 1, n, i) #define lock_acquire_shared_recursive(l, s, t, n, i) lock_acquire(l, s, t, 2, 1, n, i) -#define spin_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define spin_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define spin_release(l, i) lock_release(l, i) - -#define rwlock_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) +#define spin_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define spin_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define spin_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwlock_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) #define rwlock_acquire_read(l, s, t, i) \ do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, !read_lock_is_recursive());\ if (read_lock_is_recursive()) \ lock_acquire_shared_recursive(l, s, t, NULL, i); \ else \ lock_acquire_shared(l, s, t, NULL, i); \ } while (0) - -#define rwlock_release(l, i) lock_release(l, i) - -#define seqcount_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define seqcount_acquire_read(l, s, t, i) lock_acquire_shared_recursive(l, s, t, NULL, i) -#define seqcount_release(l, i) lock_release(l, i) - -#define mutex_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define mutex_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define mutex_release(l, i) lock_release(l, i) - -#define rwsem_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define rwsem_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define rwsem_acquire_read(l, s, t, i) lock_acquire_shared(l, s, t, NULL, i) -#define rwsem_release(l, i) lock_release(l, i) +#define rwlock_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define seqcount_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_acquire_read(l, s, t, i) \ +do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, false); \ + lock_acquire_shared_recursive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define mutex_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define mutex_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define mutex_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwsem_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define rwsem_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define rwsem_acquire_read(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_shared(l, s, t, NULL, i); \ +} while (0) +#define rwsem_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) #define lock_map_acquire(l) lock_acquire_exclusive(l, 0, 0, NULL, _THIS_IP_) #define lock_map_acquire_try(l) lock_acquire_exclusive(l, 0, 1, NULL, _THIS_IP_) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 2ebc323d345a..aecd65836b2c 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -11,6 +11,7 @@ #define __LINUX_LOCKDEP_TYPES_H #include +#include #define MAX_LOCKDEP_SUBCLASSES 8UL @@ -77,6 +78,7 @@ struct lock_class_key { struct hlist_node hash_entry; struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES]; }; + struct dept_key dkey; }; extern struct lock_class_key __lockdep_no_validate__; @@ -194,6 +196,7 @@ struct lockdep_map { int cpu; unsigned long ip; #endif + struct dept_map dmap; }; struct pin_cookie { unsigned int val; }; diff --git a/include/linux/mutex.h b/include/linux/mutex.h index a33aa9eb9fc3..04c41faace85 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -26,6 +26,7 @@ , .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define __DEP_MAP_MUTEX_INITIALIZER(lockname) diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index 36b942b67b7d..e871aca04645 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -21,7 +21,7 @@ struct percpu_rw_semaphore { }; #ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) }, #else #define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) #endif diff --git a/include/linux/rtmutex.h b/include/linux/rtmutex.h index 7d049883a08a..35889ac5eeae 100644 --- a/include/linux/rtmutex.h +++ b/include/linux/rtmutex.h @@ -81,6 +81,7 @@ do { \ .dep_map = { \ .name = #mutexname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(mutexname, NULL),\ } #else #define __DEP_MAP_RT_MUTEX_INITIALIZER(mutexname) diff --git a/include/linux/rwlock_types.h b/include/linux/rwlock_types.h index 1948442e7750..6e58dfc84997 100644 --- a/include/linux/rwlock_types.h +++ b/include/linux/rwlock_types.h @@ -10,6 +10,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL), \ } #else # define RW_DEP_MAP_INIT(lockname) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 1dd530ce8b45..1fa391e7770a 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -22,6 +22,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, #else # define __RWSEM_DEP_MAP_INIT(lockname) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index e92f9d5577ba..dee83ab183e4 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -81,7 +81,7 @@ static inline void __seqcount_init(seqcount_t *s, const char *name, #ifdef CONFIG_DEBUG_LOCK_ALLOC # define SEQCOUNT_DEP_MAP_INIT(lockname) \ - .dep_map = { .name = #lockname } + .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) } /** * seqcount_init() - runtime initializer for seqcount_t diff --git a/include/linux/spinlock_types_raw.h b/include/linux/spinlock_types_raw.h index 91cb36b65a17..3dcc551ded25 100644 --- a/include/linux/spinlock_types_raw.h +++ b/include/linux/spinlock_types_raw.h @@ -31,11 +31,13 @@ typedef struct raw_spinlock { .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SPIN, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define SPIN_DEP_MAP_INIT(lockname) \ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define LOCAL_SPIN_DEP_MAP_INIT(lockname) \ @@ -43,6 +45,7 @@ typedef struct raw_spinlock { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define RAW_SPIN_DEP_MAP_INIT(lockname) diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 127ef3b2e607..f6b8266a4bfd 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const char *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) -#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name }, +#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name, .dmap = DEPT_MAP_INITIALIZER(srcu_name, NULL) }, #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ int init_srcu_struct(struct srcu_struct *ssp); diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a3e774479f94..7e12e46dc4b7 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -244,10 +244,10 @@ static bool dept_working(void) * Even k == NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__; static bool valid_key(struct dept_key *k) { - return &__dept_no_validate__ != k; + return &__lockdep_no_validate__.dkey != k; } /* @@ -1936,7 +1936,7 @@ void dept_softirqs_off(void) dept_task()->softirqs_enabled = false; } -void dept_hardirqs_off(void) +void noinstr dept_hardirqs_off(void) { /* * Assumes that it's called with IRQ disabled so that accessing @@ -1958,7 +1958,7 @@ void dept_softirq_enter(void) /* * Ensure it's the outmost hardirq context. */ -void dept_hardirq_enter(void) +void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 151bd3de5936..e27cf9d17163 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -1215,6 +1215,8 @@ void lockdep_register_key(struct lock_class_key *key) struct lock_class_key *k; unsigned long flags; + dept_key_init(&key->dkey); + if (WARN_ON_ONCE(static_obj(key))) return; hash_head = keyhashentry(key); @@ -4310,6 +4312,8 @@ static void __trace_hardirqs_on_caller(void) */ void lockdep_hardirqs_on_prepare(void) { + dept_hardirqs_on(); + if (unlikely(!debug_locks)) return; @@ -4430,6 +4434,8 @@ EXPORT_SYMBOL_GPL(lockdep_hardirqs_on); */ void noinstr lockdep_hardirqs_off(unsigned long ip) { + dept_hardirqs_off(); + if (unlikely(!debug_locks)) return; @@ -4474,6 +4480,8 @@ void lockdep_softirqs_on(unsigned long ip) { struct irqtrace_events *trace = ¤t->irqtrace; + dept_softirqs_on_ip(ip); + if (unlikely(!lockdep_enabled())) return; @@ -4512,6 +4520,8 @@ void lockdep_softirqs_on(unsigned long ip) */ void lockdep_softirqs_off(unsigned long ip) { + dept_softirqs_off(); + if (unlikely(!lockdep_enabled())) return; @@ -4859,6 +4869,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, { int i; + ldt_init(&lock->dmap, &key->dkey, subclass, name); + for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) lock->class_cache[i] = NULL; @@ -5630,6 +5642,12 @@ void lock_set_class(struct lockdep_map *lock, const char *name, { unsigned long flags; + /* + * dept_map_(re)init() might be called twice redundantly. But + * there's no choice as long as Dept relies on Lockdep. + */ + ldt_set_class(&lock->dmap, name, &key->dkey, subclass, ip); + if (unlikely(!lockdep_enabled())) return; @@ -5647,6 +5665,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigned long ip) { unsigned long flags; + ldt_downgrade(&lock->dmap, ip); + if (unlikely(!lockdep_enabled())) return; @@ -6447,6 +6467,8 @@ void lockdep_unregister_key(struct lock_class_key *key) unsigned long flags; bool found = false; + dept_key_destroy(&key->dkey); + might_sleep(); if (WARN_ON_ONCE(static_obj(key))) From patchwork Wed Mar 6 08:54:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583567 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 499595DF30; Wed, 6 Mar 2024 08:55:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; cv=none; b=txFn6d4vAn1L0xhWwCWjRBUMHG5UX3XNSPAYQ/WG6N0bAD24C3OxtCZdX+2WqXMArnCqBKW9HSa7QN+bHYGds2rEr9zJHoD1ftlRLonYRJicoxosbd4UcMa4ISiuhK22Iqv66MlkuKI7kr70h4lBJj8DDdDCds1V/RjwtmYyhE8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; c=relaxed/simple; bh=37cSYW8WA/TnfUxZD66nhjfMFS+Rdxdh6OVt7EPfSog=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=TJO9YkS5KYOQkkILPADFV7J13tMlOPrgIhOSn2bj0pOhASPXrA2xKR6FFn/JEkRdWSYUIZ+ctn50yrsq4nKSpJwRkBn+9QDI1iPu8djmgq9eglcO/XTPnZKvUZjbQD3TiLAcu5SSVP+lKXXLBtShElf0wL9rLik1CPaazGtJ/os= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-89-65e82f7df8d6 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 06/27] dept: Add proc knobs to show stats and dependency graph Date: Wed, 6 Mar 2024 17:54:52 +0900 Message-Id: <20240306085513.41482-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRiA+75z9p2z0eKwgk4mVCMJ7GqUvdEF+1GdiC52+VNCjTzlSC12 U7NC06ysmQXLTKnNZA23so4FWs1spaWRrRq2ykSHpdJUsDZaSuWF/rw8vO/D8+tlKZUki2K1 6QZRl65JVRMFreifbFt4YnGvuORGSxRcurAEQj/P0lBe7SLgveNE4Lqfi6GvcSN8CAcRDL9+ Q0GJxYvA1vWFgvtNHQjcjlME3ndPAV9okECz5TyBvJvVBN5+H8HQfuUyBqe0BV4VV2BoiPTQ UNJHoKwkD4+OXgwRexUD9pwYCDiuMTDSFQfNHW0ycH+aD6XX2wk8djfT0FQbwPD+YTmBDtdf GbxqekmD95JZBrcHKgh8D9spsIcGGXjXYMVwN380VPDjjwxemBswFFTew+D7+AhB/dlODJKr jcCzUBBDjWSh4PetRgSBon4GTl+IMFCWW4Tg/OkrNOS3L4fhX+UkYaXwLDhICfk1GYI7bKWF lgpeqLv2hRHy6z8xglUyCjWOWOHm4z4s2IZCMkGqOkcEaegyIxT2+7Aw0NrKCC+vDtNCt68E b4/ao1idLKZqTaJu8dr9ipTI5+yjLfGZxQ88JAc5FxQiOctzy3ipt0D2n0tLz6AxJtw83u+P UGM8jZvN15i/jTsUF1Twla0bxngqt4N31jnG9zQXw/daaplCxLJKbjn/INc4kZzFO+82jGfk XDx/ceAiGWPVqPI6z0YmnDNyvq1+wwTP4J86/HQxUlrRpCqk0qab0jTa1GWLUrLStZmLDhxJ k9DoJ9lPjOytRUPenR7EsUg9WZkg7xFVMo1Jn5XmQTxLqacpj//uFlXKZE3WMVF3ZJ/OmCrq PWgmS6unK5eGM5JV3CGNQTwsikdF3f8rZuVROQjvyvprmlMZDNgyY3VvEr3RtUH/wa9YwtlF rLWz2mweNpYFpBWeypg4V7G7c6smo2jTxiZf4bZZFj1dEP+oYo1irvH4w1gP+Ry9uys76UnS 1cR5/rrDiT2rqjZvTnr+9Z3hW7RJfWrtc8P+hF0n16+a0lgXyghvXWfY8at/NSSpaX2KJi6W 0uk1/wBT0fQRRQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxiGfc/He06bFU8qcSfgMlJjzMChOCFP1DiyTHnVaIxGTfZnnMyT UYVKWkFQNCBFGR8ViAVBIAVJB7QMLbpQBdKViKCCKIiAQICgQqiQqCUrkDnqsj9Prtz3nevX w9NqMxvEa3VnZL1OitdgJaM8uCPz2wubp+UtwzUICvO2gPdjNgPljXYMvX/YENjvZFAw8yAG Xi54ECx1P6WhxNyLoGpilIY7HWMIWmsvYeibCoB+7zyGLnMuhsybjRiezS5TMFJcRIHNcQAe F1RT4PK9ZaBkBsONkkxq5UxT4LPWc2BN3wCTtWUcLE9EQNfYAAvtFV0stA6HQWnlCIaW1i4G OponKei7V45hzP6JhccdnQz0Fuaz0DBXjWF2wUqD1TvPwXOXhYJbxhXb5Q//sPAw30XB5Zrb FPQP3UfQlj1OgcM+gKHd66GgyWGmYfH3BwgmTe84yMrzcXAjw4QgN6uYAeNIJCz9XY6jt5N2 zzxNjE1nSeuChSGPqkXiLBvliLFtmCMWRxJpqg0lN1tmKFL13ssSR/1vmDjeF3Ek510/ReZ6 ejjSeX2JIVP9JdSh4J+UO0/I8dpkWb95V6wyzvfqfOKjqJSCu26cjmybcpCCF4VtYmnpFeRn LGwUBwd9tJ8DhRCxKf8N62da8CjFmp49fl4jHBZtztrPOSNsEKfNzVwO4nmVECnezUj6T/m1 aLvl+qxRCFHi1bmr2M/qlUl3ZhUuQEoLWlWPArW65ARJGx8ZbjgVl6rTpoT/cjrBgVaexXph ubAZfeyLcSOBR5ovVNGKt7KalZINqQluJPK0JlCVtjglq1UnpNRzsv70z/qkeNngRsE8o/lS te+4HKsWfpXOyKdkOVHW/99SvCIoHb1eLaXuaYtwbf0U4AkxVYS5Dq/5zjnuDqibGDh+3ynO rV23vjv4m2v3rmSvq9v3l6M4dOL7I1l8i31UImPWtbFm/qRp/VelF3N9i9cUx5yNYlmd6emL TYkxlr3XG3TuH3uKhso6U3YPvIysDFVm/TlsPNq2f1e0diaq8nZi5w/haU80jCFOigil9Qbp X+s2LokoAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: It'd be useful to show Dept internal stats and dependency graph on runtime via proc for better information. Introduced the knobs. Signed-off-by: Byungchul Park --- kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 24 +++----- kernel/dependency/dept_internal.h | 26 +++++++++ kernel/dependency/dept_proc.c | 95 +++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 15 deletions(-) create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_proc.c diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index b5cfb8a03c0c..92f165400187 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DEPT) += dept.o +obj-$(CONFIG_DEPT) += dept_proc.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 7e12e46dc4b7..19406093103e 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,7 @@ #include #include #include +#include "dept_internal.h" static int dept_stop; static int dept_per_cpu_ready; @@ -260,20 +261,13 @@ static bool valid_key(struct dept_key *k) * have been freed will be placed. */ -enum object_t { -#define OBJECT(id, nr) OBJECT_##id, - #include "dept_object.h" -#undef OBJECT - OBJECT_NR, -}; - #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT -static struct dept_pool pool[OBJECT_NR] = { +struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ @@ -303,7 +297,7 @@ static void *from_pool(enum object_t t) if (DEPT_WARN_ON(!irqs_disabled())) return NULL; - p = &pool[t]; + p = &dept_pool[t]; /* * Try local pool first. @@ -338,7 +332,7 @@ static void *from_pool(enum object_t t) static void to_pool(void *o, enum object_t t) { - struct dept_pool *p = &pool[t]; + struct dept_pool *p = &dept_pool[t]; struct llist_head *h; preempt_disable(); @@ -2092,7 +2086,7 @@ void dept_map_copy(struct dept_map *to, struct dept_map *from) clean_classes_cache(&to->map_key); } -static LIST_HEAD(classes); +LIST_HEAD(dept_classes); static bool within(const void *addr, void *start, unsigned long size) { @@ -2124,7 +2118,7 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); - list_for_each_entry_safe(c, n, &classes, all_node) { + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) continue; @@ -2200,7 +2194,7 @@ static struct dept_class *check_new_class(struct dept_key *local, c->sub_id = sub_id; c->key = (unsigned long)(k->base + sub_id); hash_add_class(c); - list_add(&c->all_node, &classes); + list_add(&c->all_node, &dept_classes); unlock: dept_unlock(); caching: @@ -2915,8 +2909,8 @@ static void migrate_per_cpu_pool(void) struct llist_head *from; struct llist_head *to; - from = &pool[i].boot_pool; - to = per_cpu_ptr(pool[i].lpool, boot_cpu); + from = &dept_pool[i].boot_pool; + to = per_cpu_ptr(dept_pool[i].lpool, boot_cpu); move_llist(to, from); } } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_internal.h new file mode 100644 index 000000000000..007c1eec6bab --- /dev/null +++ b/kernel/dependency/dept_internal.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dept(DEPendency Tracker) - runtime dependency tracker internal header + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __DEPT_INTERNAL_H +#define __DEPT_INTERNAL_H + +#ifdef CONFIG_DEPT + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +extern struct list_head dept_classes; +extern struct dept_pool dept_pool[]; + +#endif +#endif /* __DEPT_INTERNAL_H */ diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c new file mode 100644 index 000000000000..7d61dfbc5865 --- /dev/null +++ b/kernel/dependency/dept_proc.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Procfs knobs for Dept(DEPendency Tracker) + * + * Started by Byungchul Park : + * + * Copyright (C) 2021 LG Electronics, Inc. , Byungchul Park + */ +#include +#include +#include +#include "dept_internal.h" + +static void *l_next(struct seq_file *m, void *v, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_next(v, &dept_classes, pos); +} + +static void *l_start(struct seq_file *m, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_start_head(&dept_classes, *pos); +} + +static void l_stop(struct seq_file *m, void *v) +{ +} + +static int l_show(struct seq_file *m, void *v) +{ + struct dept_class *fc = list_entry(v, struct dept_class, all_node); + struct dept_dep *d; + const char *prefix; + + if (v == &dept_classes) { + seq_puts(m, "All classes:\n\n"); + return 0; + } + + prefix = fc->sched_map ? " " : ""; + seq_printf(m, "[%p] %s%s\n", (void *)fc->key, prefix, fc->name); + + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + list_for_each_entry(d, &fc->dep_head, dep_node) { + struct dept_class *tc = d->wait->class; + + prefix = tc->sched_map ? " " : ""; + seq_printf(m, " -> [%p] %s%s\n", (void *)tc->key, prefix, tc->name); + } + seq_puts(m, "\n"); + + return 0; +} + +static const struct seq_operations dept_deps_ops = { + .start = l_start, + .next = l_next, + .stop = l_stop, + .show = l_show, +}; + +static int dept_stats_show(struct seq_file *m, void *v) +{ + int r; + + seq_puts(m, "Availability in the static pools:\n\n"); +#define OBJECT(id, nr) \ + r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ + if (r < 0) \ + r = 0; \ + seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + #include "dept_object.h" +#undef OBJECT + + return 0; +} + +static int __init dept_proc_init(void) +{ + proc_create_seq("dept_deps", S_IRUSR, NULL, &dept_deps_ops); + proc_create_single("dept_stats", S_IRUSR, NULL, dept_stats_show); + return 0; +} + +__initcall(dept_proc_init); From patchwork Wed Mar 6 08:54:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583569 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9150D5EE83; Wed, 6 Mar 2024 08:55:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; cv=none; b=nBRO9pvJukS+H3yf08yqGG6qHtC4yd+EOTpPuSOtsDj4Y4eR2NoZaiKrZ9Fxj0JWj3g1yRyf/xaLL4nUFjaMV+f6QCaNGxWEyhtTRorsLk4e9tzebGn3sUJ/ej4Dco39fI/PTOd/ld3QBdvIq6mvpGvs1HdL6efGjvxqM/Ma/rg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; c=relaxed/simple; bh=Z/lrsPfinthw0Keb2TOWgQylHJ0tuWxQmHrjSTWrwaY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=gzuoha344C1zMg2zFtFqPHpJ/8CEw7r/sZUpiN0rh3sFcakQ5yVOL/P7Rwz6H9TSJ9TvrZVHi4jWYlG4yLlukC8NNWBNCMpBup2+HLfMpVR7T8GzhAFd8wGZ86mbLpSMI3+kM6bS3wm4ZqXo1z85J11TTIqidi1K6O4vFdAbk5U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-98-65e82f7d9635 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 07/27] dept: Apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() Date: Wed, 6 Mar 2024 17:54:53 +0900 Message-Id: <20240306085513.41482-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRiA/b5zm8PJaRmdjCgHFhiphcYbmNWfOn+CsD9hhK085cgbM2+V aKndNbuopVZTYw0vZZuQptNptLS8zBQzU0kxt6XOtDZarss0+vPywPu8z69XREjrKV+RIv60 oIyXx8poMSme9arYkhFkEYJvl8ng5vVgsH+/TELZ0xoaTE+qEdTUn8dgfbUP3jtmECx29xJQ XGhCUD4+SkC9cQyBXnOBhv5Jbxiwz9HQWXiNhuzKpzT0TbswjBTdwlCt3Q9vCyowGJxmEoqt NJQWZ2P3sGBwqqsYUGf5w4SmhAHX+FboHBukQD+8Ge49GKGhWd9JgrFhAkP/izIaxmr+UPDW 2EGC6WYeBbW2ChqmHWoC1PY5Bt4ZVBjqctyhi99+U/A6z4Dh4qNnGAY+NCFoufwJg7ZmkIaX 9hkMOm0hAT8fv0IwkT/LQO51JwOl5/MRXMstIiFnJBQWf5TRu3fwL2fmCD5Hl8rrHSqSf1PB 8Y0lowyf0zLM8CptMq/TBPCVzVbMly/YKV5bdYXmtQu3GP7q7ADmbT09DN9xd5HkJweK8QHf SHFYtBCrSBGUQeFHxTGdY5+pxNYVaa7fmVmo3vsq8hRxbAhX3mim/vP0YyuxxDS7iRsaci6z D7uB0+VNLTsEOyPmHvXsXeKVbAzXNVqAl5hk/bmHRuOyI2FDudeaPPSvuZ6rrjMsdzzZ7dwN 2w16iaVupzu73M1it/NDxGn7+ol/B2u4Ns0QWYAkKuRRhaSK+JQ4uSI2JDAmPV6RFng8IU6L 3L+kznAdbkALpoPtiBUhmZdkt6dZkFLylKT0uHbEiQiZj+Tcz0lBKomWp58RlAlRyuRYIakd rRWRstWSbY7UaCl7Un5aOCUIiYLy/xaLPH2z0I6d8o2X7sxnOHThx7q8vhYcORPm9zE5aJfY 1h8hnUuQlE6t80fWqMimtm5SX2seP9ab0KzIDF41GG1TWn0srVP+qlpLpt3SRRxa21x758v3 53taE8N95iMbK8N+mc8uOnc+0DFVMpcgVlEnIlR+kpTMNXqrB6PLT7zfkRbWIiOTYuRbAwhl kvwvXp6YTUcDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSaUiTcRwH8P7/55qrxdO0eqjoGEhUdBgqPzosCvJPUNSLinxRrnrIkc7Y zCNamS7xyJqCrsNqHkxRy9oWHTrPWq7DTO1Ercyy4dKoNtKJ5Yze/PjA98v31U9CyQuZORKV OkHUqJWxClZKS7evTV+uW/lVXPWybBrknV0Fnl+ZNBTVVLPQfqMKQbXtNAbXw0h47XUj8D17 ToGxoB1B8cceCmyOXgT2ijQWOvunQ5dnmAVnQQ4L6aU1LLwYHMPQXZiPocqyDZ4YSjA0jgzQ YHSxcNmYjifOVwwj5koOzKnB0FdxiYOxjyHg7H3FQMsVJwP2d8vg4tVuFursThocd/swdN4v YqG3+g8DTxytNLTn5TJwfaiEhUGvmQKzZ5iDjkYThpv6ibWMn+MMPMptxJBRdgtD19taBPWZ HzBYql+x0OJxY7BaCigYLX+IoO/cNw7OnB3h4PLpcwhyzhTSoO8OA9/vInbjGtLiHqaI3ppE 7F4TTR6XCOTepR6O6OvfccRkOUasFUtJaZ0Lk+IfHoZYKrNYYvmRz5Hsb12YDLW1caT1go8m /V1GvGNulHTdITFWlShqVkZES2OcvZ+Zow0zksfGT6Ui2/RsFCAR+FBhsNxF+c3yi4U3b0Ym HcQvFKy5Xxi/Kd4tFcratvgdyMcIT3sM2G+aDxauORyTHRkfJjyqyEX/NhcIVTcbJ3cC+HDh /NB51m/5ROdZejFrQFITmlKJglTqxDilKjZshfZITIpalbziYHycBU18i1k3lncX/eqMbEa8 BCmmyTYGDIhyRpmoTYlrRoKEUgTJToz2i3LZIWXKcVETv19zLFbUNqO5EloxW7Z1jxgt5w8r E8QjonhU1PxPsSRgTiq6zkXsbe/I94bqaMa4GqlpyhFYtK3Ju3s8bfBazXxGT75HRjU9v7Nv Slh9XQ7+Xhv8YEO4/X1TUkP5ZuuWmrwQw50ls3bdjnofl7ReZ3NN1R9ovRFtE73zO4YWjW46 ORAYb8pYY7NkbU/rSSz1zdxpGPV9crVIde554+vqGn7XKmhtjDJkKaXRKv8C2ya79ykDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index fb2915676574..bd2c207481d6 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -10,6 +10,7 @@ */ #include +#include /* * struct completion - structure used to maintain state for a "completion" @@ -26,14 +27,33 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map dmap; }; +#define init_completion(x) \ +do { \ + sdt_map_init(&(x)->dmap); \ + __init_completion(x); \ +} while (0) + +/* + * XXX: No use cases for now. Fill the body when needed. + */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) {} -static inline void complete_release(struct completion *x) {} + +static inline void complete_acquire(struct completion *x) +{ + sdt_might_sleep_start(&x->dmap); +} + +static inline void complete_release(struct completion *x) +{ + sdt_might_sleep_end(); +} #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ + .dmap = DEPT_MAP_INITIALIZER(work, NULL), } #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -75,13 +95,13 @@ static inline void complete_release(struct completion *x) {} #endif /** - * init_completion - Initialize a dynamically allocated completion + * __init_completion - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized * * This inline function will initialize a dynamically created completion * structure. */ -static inline void init_completion(struct completion *x) +static inline void __init_completion(struct completion *x) { x->done = 0; init_swait_queue_head(&x->wait); From patchwork Wed Mar 6 08:54:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583570 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AFB695F488; Wed, 6 Mar 2024 08:55:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; cv=none; b=kSWa9MPeXw568SR7rkuhAf+WzjKJR0SKIqM2nk6pajSkIXyn/ieiVb50DzYrBXCsIegdTN4j+X5rGpYbUpUyUGEdgMhF1VhdDT0FHlg1+/mipVJKk8eVSYUq9zumgSbpYpsqnbEolN4FG5uVJpYB2K5c+mH6xQJaiQP1OzZTucM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; c=relaxed/simple; bh=tg0pA2ZlUNywKtzMfrdbwkG7GYbhIVlK2HrCAbcpiMY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=kGjAPx53zNwpPmq7Z7zzWdqYhn55VD+CQWORpch4FOxy1sKtqeL1GPbD2lbZ6CkaM/bQEdPuI+V+SAKP8OMHiIp4RkQfn0nGSJOfCnnIZRv3KCIvvbijjSglg+SB31ui//Is02v0S/8HW3oB9D4a6pl1iIo3uJgtm4tTarhg9Ro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-a8-65e82f7d71d2 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 08/27] dept: Apply sdt_might_sleep_{start,end}() to swait Date: Wed, 6 Mar 2024 17:54:54 +0900 Message-Id: <20240306085513.41482-9-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0zMcRjH+3x/d5y+O22+sokzbFk/EB6b0V98/7EZ5g9s3PRVR3fs+nVR W1dJSqUsh8J1tZM6ypWJuuTiCOXUjeRKEkpXWVzrFLkO/zx77Xleez/PHw+DS+6QAYxcGSeo lLIYKSUiRCNz9cEpoYNCWPFYEBScDQPXjywCSqqNFNhuVSEw1mkwGHq8Dd5MOBFMtb3EQVtk Q1D6oQeHOmsvAnNFGgWdA/PA7hqjoLUoh4L0smoKXg1PY+C4UIhBlWk7PD+nx6DZ/YUA7RAF xdp0zFMGMXAbKmkwpC6H/orLNEx/WA2tva9JMHevgktXHRQ0mlsJsNb3Y9B5v4SCXuMMCc+t TwmwFeSScHNUT8HwhAEHg2uMho5mHQY1GZ6gzO+/SXiS24xBZvltDOxvGxA0ZfVhYDK+pqDF 5cSg1lSEw8/rjxH0543QcOqsm4ZiTR6CnFMXCMhwrIOpyRIqYiPf4hzD+YzaRN48oSP4Z3qO v3e5h+YzmrppXmeK52srgviyxiGMLx13kbyp8gzFm8YLaT57xI7xo+3tNP/04hTBD9i12I6A vaJNkUKMPEFQhW4+KIrOy3JRx62M2pzqJFLRSyob+TIcG859s1yh/7PjdyE5yxS7kuvqcuOz 7M8u4WpzP3v7OOsUceXtW2d5PrudK5s673UIdjln+DHjdcTsOq49N5/4mxnIVdU0ex1fdj2X P5rv3SvxOG3ppR4WeZwZhmvTlP87aCH3sKKLOIfEOuRTiSRyZYJCJo8JD4lOUsrVIYeOKUzI 80yGlOl99WjctsuCWAZJ54ojfL8IElKWEJuksCCOwaX+4uSfA4JEHClLOiGojh1QxccIsRa0 iCGkC8RrJhIjJWyULE44KgjHBdX/Kcb4BqSiI8kBxprJk1tu+uui1oS4hS3XLvlEbR4MTLvY ZfmqMBToFF+Fpr3Zp5f1+N3QBT9Qq/sWhzN75JlLd6pW6B3fdsfvduzPeaRVc58m0+zkvDCT UjXQdMD/l19ow4vvyxJHu8sjRjo6yw5zGudan7vv+vW2OYF+1cMbF358b12h2ZAiJWKjZauD cFWs7A8OglZcSAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa2xLYRjHve855z1tqZzUcGIfUPfJZsJ4YiKTCCeEkAWLSDjmRGsX0tqY W8o2GbOrTOlGdpFqtrm1c53KbFkZNmONa9exLGxWHbUuagu2ii9PfnmeX/7Pl7+MUhmZSTJt 8j5JlywmqomCVqyLTg8/Mq9bivzybRwUnI4Ef38WDSXXqgm0Xq1CUF1zDENP4yp4PeBBMNj8 nAJjUSuCso/tFNQ43AjsluME2rrGgtPfR6CpKJtAesU1Ai96hzC4zhZiqLKuhaf55RjqAp9p MPYQKDam4+HRjSFgrmTBbJgBnRYTC0Mf50OT+xUDDReaGLC/mwvnL7oI3Lc30eC404mh7V4J AXf1HwaeOh7T0FqQw8AVbzmB3gEzBWZ/Hwsv60oxXM8YTjvx4zcDj3LqMJy4dAOD820tggdZ HzBYq18RaPB7MNisRRT8utyIoDP3KwuZpwMsFB/LRZCdeZaGDFcUDP4sITFLhAZPHyVk2PYL 9oFSWnhSzgt3Te2skPHgHSuUWlMEmyVMqLjfg4Uyn58RrJUniWD1FbLCqa9OLHhbWljh8blB WuhyGvH60C2KpTulRG2qpJu3bLtCk5vlJ3sdsgN2g4c2oOfkFJLLeG4h7/pdyIww4Wbxb94E qBEO4abwtpxPwT3FeRT8pZaVIzyOW8tXDJ4JOjQ3gzf3/wk6Si6Kb8nJo/9lTuarrtcFHTm3 iM/z5gV/qYad5vQyko8UpWhUJQrRJqcmidrEqAh9giYtWXsgIn5PkhUN18V8ZKjgDupvW1WP OBlSj1HGyD9LKkZM1acl1SNeRqlDlId/dUkq5U4x7aCk27NNl5Io6etRqIxWT1Su3ixtV3G7 xH1SgiTtlXT/r1gmn2RAFYHY9tGxC7q3Dhg0s933TG9DFzldCxwbfTVx3vxwy9HlCzFujvdu nD4h72Im7jD5bBFVY8Kv+m4tftR4s6MxjpuG43cffebteH8rJTZyzYeQqeN31R5aMfPhhk0J 0dMfyhWm2+KQZUJW7Y606Js9c76L+i/eyCW7Z+/3OGKyNWFuNa3XiPPDKJ1e/AudERJ6KgMA AA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by swaits. Signed-off-by: Byungchul Park --- include/linux/swait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/swait.h b/include/linux/swait.h index d324419482a0..277ac74f61c3 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -6,6 +6,7 @@ #include #include #include +#include #include /* @@ -161,6 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); struct swait_queue __wait; \ long __ret = ret; \ \ + sdt_might_sleep_start(NULL); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ @@ -176,6 +178,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); cmd; \ } \ finish_swait(&wq, &__wait); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Wed Mar 6 08:54:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583568 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B75385FB8E; Wed, 6 Mar 2024 08:55:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; cv=none; b=LsjcznYzl2RxDV0fl/6ObXdtfDWVzgSBMdBgR9ZodgY1hNAnrvKlnuFyzKdDLVoTz+rniJqebftvcnj/r6bxJEnxyCxe7Jd2EUt4O359WShCZvaM1kA9mLNGOX2xClNeS3itctNlN0g8HvfQVr9TUvVHcP1KK1D4CI+CekRLXbU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715340; c=relaxed/simple; bh=RIc5udGVOneBYojEExyl4QYDrUVZJa4Vo+L4Ic7nXzw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=GsQGC8r1ozAQSu1uWrUMIThLAabOoE7+cjqMSvuU5aVfOX82Xr0ot1qG0i4jYAi7SK311rvjjcb/OgbgOltWwg0HL325m9N6RCrqzt5S4/SoaCDMjxoyCEMflL8xfAdXAFt0CozJVRwYCeSxrNK6+bUKkTHgiWKWSEsf2ejmpWU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-b8-65e82f7dce79 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 09/27] dept: Apply sdt_might_sleep_{start,end}() to waitqueue wait Date: Wed, 6 Mar 2024 17:54:55 +0900 Message-Id: <20240306085513.41482-10-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSV0xTcRSH/d/ZVmtuqtErPmCqBuMCFMgxMcYnvEZNHIkaieMKV2ksqEVm MKIMwUoFEqijQimkrCpahgsQMVSRMFREMAUFUUHKCNhqgagt0ZeTL7/zO9/TEeGyKtJDpAg/ J6jCeaWckhCSkXn56857Dwo+RY83QuZVH7D/SCVAV26ioP1uGQJT5UUMhhq3wXuHDcF0SxsO 2ux2BPl9PThUWnoR1BZfouDtwHzosI9R0JStpiCxoJyC18MzGFhzsjAoM++C5gwDBvXObwRo hyi4pU3EXGMQA6exlAZjwkroL75Jw0yfLzT1dpJQ+2EN3Mi1UlBT20SA5WE/Bm8f6yjoNf0h odnykoD2zHQS7owaKBh2GHEw2sdoeFOvx+BekkuUMvmbhBfp9RikFN7HoKP7CYK61E8YmE2d FDy32zCoMGfjMFXUiKBfM0JD8lUnDbcuahCok3MISLL6w/QvHbV1E/fcNoZzSRXRXK1DT3Cv DCz36GYPzSXVfaA5vTmSqyhezRXUDGFc/oSd5MylaRRnnsiiuSsjHRg32tpKcy+vTxPcQIcW 2+1xSLI5RFAqogSV95ZjktBXP4fIMyXimPGMy1QCKqKvILGIZfzYvMER/D9/tRQQbqYYL7ar yzmbL2SWsRXpX0k344xNwha2Brp5AbOf1ZaOIjcTzEr2uz53tiNlAtjOB/3//J5s2b36WY/Y lV8bvUa5Wcb4sy2J+S6WuDqJYjY3w/DvYAn7rLiLyEBSPZpTimSK8KgwXqH0Wx8aG66IWR98 OsyMXM9kPD8T9BBNtO9rQIwIyedJt4q/CTKSj4qIDWtArAiXL5TGTw0IMmkIHxsnqE4fVUUq hYgGtFREyBdLNziiQ2TMSf6ccEoQzgiq/1tMJPZIQB65db7xjZo4EgKOZ55o6QnWpZQUDi9a hc+1kF/G+ZkjznU6JmuzWnEh+vamuc6P3aazBwOjNfanvisEfkfNTo3m0uRye17al3deAZ7O tPFsz+pypSGmuS/Iuqe6zRsPUX8sU29flvx5zwGrT9X+XUrF2srD3TlBnxW3bY7qvVFyIiKU 912NqyL4vwEA7wJIAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTYRgH8N73nPOebbQ4LatD0oXVsgvds54udCHKl+4QGAVRow651Blb mUaBpdnFLI10Vlbz0jRdZWdFN2dLyZySWYpWqKRYaU0Nc5YXKhf15eHH83/4f3oUjMbCjVEY jAckk1EfpiUqVrVxSeyMo7NapdkVHn9IPjsbvN2nWEi/YydQdTsfgf3eMQxtz4OgrseDoP/l KwYsKVUIMpoaGLhX2ojAmXucQHXLMKjxdhJwpyQQiM26Q+D11wEM9akXMOTLG6AiKRODq/cz C5Y2AlcssXhwtGLoteXxYIvRQXPuZR4GmuaAu7GWg5Krbg6c76fDpWv1BAqdbhZKHzZjqH6c TqDR/puDitIyFqqSEzm41ZFJ4GuPjQGbt5OHNy4rhoK4wbb47784eJHowhCffRdDzbsnCIpO fcAg22sJlHg9GBxyCgN9Oc8RNJ9r5+HE2V4erhw7hyDhRCoLcfWB0P8znaxYTEs8nQyNcxyi zh4rS8szRfrocgNP44re89QqH6SO3Gk0q7AN04wuL0flvNOEyl0XeHqmvQbTjspKnpal9bO0 pcaCN/tvVy3dI4UZIiXTrGW7VCHlP9q4/TeVUd+STpIYlMOfQUqFKMwXP5VmsT4TIUB8+7aX 8dlPmCA6Ej9xPjOCRyVmV67xeYQQLFryOpDPrKATv1iv/b1RCwvE2gfN/zrHi/kFrr89ysH9 +Y7zxGeNECi+jM0gSUhlRUPykJ/BGBmuN4QFzjSHhkQbDVEzd0eEy2jwXWxHB5Ifou7qoGIk KJB2qHqF8rOk4fSR5ujwYiQqGK2f+khfi6RR79FHH5ZMETtNB8MkczHyV7Da0eq1W6VdGmGv /oAUKkn7JdP/FCuUY2KQtsCvemzo6U2vNRPzV92dMXnexaIbuVM/1re+CYqSg+X7U3Z8aLhZ Vtf1fZihb5Xu+kjPo6ejIhw/n4GxOKVowrrhwWMnHSkM1cYvWJ4wvmeLxr0teG6arntNeVPO RG3d6vW04pD8YMeoKvuPhQGLhl7V7TM7g9ovzuWXYFfqyluH28dpWXOIfs40xmTW/wGOP7T+ KgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 3473b663176f..ebeb4678859f 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -303,6 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ @@ -318,6 +320,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); cmd; \ } \ finish_wait(&wq_head, &__wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Wed Mar 6 08:54:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583571 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 464785FDD2; Wed, 6 Mar 2024 08:55:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715341; cv=none; b=pfYRemDs93kw6JBGVikpiGnSg/Q4fyYK0Y+IW1QmZ1Fr4RoRsURu+XkonrhTUsxs12EfVPFhtAnomjEPj4i/lsz7kdNSM9KNqKPy5LWZueA86+Hs73PbqpiTsGC2aSlgo//oDCeCS8ohy5tOe8eOeGm2eBp9vvZ7HrcAAzF9tps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715341; c=relaxed/simple; bh=MruAYr/ze2aZU4Q6DDtEFdBspBpkL05uqzvhsrQG5Gc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=l9oNF9fJLa7+ayKDGgJFgljVzKnZmKD2hAX4Q2G1YKjKtnPV0GOdEGzdz1p5bh2CE6LyILPVfD60BAy0n+rGDSaMBvHTI0Dw6nu5ZHIQy9o0+eMKrXee+i0NbRUN8ZaFKa8faapTbwtdxHgCVYdKx/QxI1FQSGLV1nkClwGxJJE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-c8-65e82f7dd99b From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 10/27] dept: Apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait Date: Wed, 6 Mar 2024 17:54:56 +0900 Message-Id: <20240306085513.41482-11-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/Pz05nz06bRww7a01WZOJjzJoZz5Y2W+bHmXHco051cqUf xkRlSCpbDt1Sp51bXeJipB9SpESKG09WTa2ppjo77kiRu+ifz177fN577fPHmyUUDyh/VqtL EvU6dZySlpGyUV9T8KkVQ+LKzub1kH9pJbi+nyfBWGmloeNOOQLr/TMYhp9vhQ/uEQQTr98Q YCjoQFDS10PA/eZeBHWWszS8G5gDdpeDhtaCbBoyblXS0PllEkP31SsYym2R0JZnwtAwPkiC YZiGQkMG9owhDOPmMgbM6QHQb7nBwGRfKLT2vqeg7uNyuF7UTUNtXSsJzY/6Mbx7bKSh1zpF QVtzCwkd+TkUVIyZaPjiNhNgdjkYeNtQjOFupkd07tsfCl7kNGA4V3oPg72rBkH9+U8YbNb3 NDS5RjBU2QoI+HX7OYL+y6MMZF0aZ6DwzGUE2VlXScjsDoOJn0Y6fJ3QNOIghMyqFKHOXUwK L028UH2jhxEy6z8yQrHtuFBlCRJu1Q5jocTpogRb2QVasDmvMMLFUTsWxtrbGaHl2gQpDNgN eLu/SrZBI8Zpk0X9io0HZDGDPflUgsSm/n5WzaQjib6IfFieW827jDfxDNdLTcjLNBfIS9I4 4WU/bglflfOZ8jLBjcj40vYtXp7L7eNbSiXGyyQXwH8aezLtlHNr+D5bAfrnXMyX322Y9vh4 9rljudMZBRfGv84o8bDMk/nJ8tVNRf8fms8/tUhkHpIXo1llSKHVJcertXGrQ2LSdNrUkENH 423IUybzqcm9j5CzI6oRcSxS+srDfQZFBaVOTkyLb0Q8Syj95Cd/DYgKuUaddkLUH92vPx4n JjaiBSypnCdf5U7RKLhodZIYK4oJon7milkf/3R0+MjutojAYy/nFQXRC+2qV5st3bmpu4Jj 1u5d9uG0ZVuAu+vh5L1WppdrdHwVdcsORvsNBS/S/Og8yTlCJmqORRSm7Pz8TOObMNWZNLso 6mz8pqjoyKV7tlUskLreOHaHPzEHGIxkSReF+q5L4TmzY3cwlppZP5wKVZgq0nXEUKskE2PU oUGEPlH9Fy08XIRIAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5/L/8zV5LCsDvYhGd2jMlJ5yYqKLofI6FOBULnylCs3ZTPT qDA3o4s2FebKS8wLS6ZlbV1WUxOnpolmumyJWi7JJNNuk6Z2cUZfXn48z8Pv0ysipUY6WKRQ JQlqlTxehsWUeG+kdvW5tR+F0An9YsjJDAXvj0sUFFZVYui4U4Gg8v4FAoYbd8Hr8REEk20v SDAaOhAUD/SRcL+pH0FNeTqGrsFAcHnHMLQYrmLQllZhePlpioDevFwCKqxR0JpdQkCdb4gC 4zCGAqOWmD4fCfCZLQyY05aApzyfgamBddDS302Ds6iFhpqeVXDjZi+G6poWCprsHgK6nhRi 6K/8Q0NrUzMFHTlZNNweLcHwadxMgtk7xkBnnYmAu7pp28Xvv2l4llVHwMWyewS43jgQ1F56 R4C1shuD0ztCgM1qIGHiViMCz7XPDGRk+hgouHANwdWMPAp0veEw+bMQb9nAO0fGSF5nO83X jJso/nkJxz/O72N4XW0Pw5usp3hb+Uq+tHqY4Iu/eWnearmMeeu3XIa/8tlF8KPt7QzffH2S 4gddRmLfwmjxxlghXpEsqNdujhHHDfXl0IluUcqvhsdMGnLjKyhAxLFhXK3bifyM2WWc2+0j /RzEhnC2rA+0n0l2RMyVte/081z2INdc5mb8TLFLuHejT2c8EjaCG7Aa0D/nIq7ibt2MJ2A6 14/qZzZSNpxr0xbjbCQ2oVkWFKRQJSvlivjwNZqTcakqRcqaowlKK5p+F/O5qRw7+tG1qx6x IiSbI9kSMCRIaXmyJlVZjzgRKQuSnJ0YFKSSWHnqGUGdcFh9Kl7Q1KOFIkq2QLL7gBAjZY/L k4STgpAoqP+3hCggOA29t+zXb38Vms7I9xkPLQ3btL717SJJcIo2ko5yFF2fOO+URbzkolct 9zT4Oh89nGdPq7ZvC6Se2fRPTjwoflsa44h1tIVsdZ0/VjS3ymBR5pd+ietanD4UudQ2plsx +fXegJkJmR8tMelxmCrSk6HszLMH9jrSjygyA2/u2COZLaM0cfJ1K0m1Rv4X/86YqyoDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by hashed-waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 7725b7579b78..fe89282c3e96 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -6,6 +6,7 @@ * Linux wait-bit related types and methods: */ #include +#include struct wait_bit_key { void *flags; @@ -246,6 +247,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ @@ -263,6 +265,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); cmd; \ } \ finish_wait(__wq_head, &__wbq_entry.wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Wed Mar 6 08:54:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583576 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AED54605BC; Wed, 6 Mar 2024 08:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715344; cv=none; b=I9EcOsynb1W+2gEQ6O6j88Bk0+93q8J6AWR7zOieLjSIyKGT9HDCL3hagKC/Z7eTQSdPQGn5F4UjsPR/JDri91dDgEzH3Yi3BYMJj0ryfOcXwYVBbSpExl99fWFem5Bj9KNZDItiC+wdL0Rh9HCJS/UcA13UdPmHvpofqEK7RPg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715344; c=relaxed/simple; bh=nifTUInVysx7YljtcW2Qa4mywSC4rocnbBoHNOK15ls=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=nuiiAiKO10E1ohzcNSptKS3+DJYBlc97MneBFL0IMsBGb531QYzbBLqfb6Iz5m1PoqnKwoX/HYQSc1JDg6yhpOjWTNkSUMTTKnfieFY1S4X76WugCTZUsL9HipUZOuJjCFr0mWBGXdUYt/FYogo5SGUXYXz78HloKSvTTiEoixc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-d8-65e82f7dab90 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 11/27] dept: Distinguish each syscall context from another Date: Wed, 6 Mar 2024 17:54:57 +0900 Message-Id: <20240306085513.41482-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRjHe99zznvOZovDMjqtzBpIUWQ3qyeo8Et1uklQ0I3KkacceYmZ plFhaWIu0yRbpZWazaF2cTPwzlKyLDQvI63M0lY68gKrjZaSbUVfHn78n//z+/RwlNLMqDht zElJF6OJUhM5LR+ZWrjk3NIhadl9gxyuXl4Grh/pNOQ/KifQ/rAMQXnleQyOZ5uh2z2MYLz1 NQWG3HYEhf0fKKhs7kNQb7pAoMs+DWyuMQItuXoCKfceEej4NoGh93oOhjLzDniVXYTB6hmk weAgkGdIwd4xhMFjLGXBmBwEA6ZbLEz0L4eWvjcM1L9bDDfv9BKoq2+hoblqAENXTT6BvvJJ Bl41v6Ch/WomAw9Giwh8cxspMLrGWOi0FmB4nOoVpX3/zcDzTCuGtOIKDLa3tQga0j9hMJe/ IdDkGsZgMedS8KvkGYKBKyMsXLzsYSHv/BUE+ovXaUjtXQXjP/NJ6FqxaXiMElMtp8R6dwEt viwSxOpbH1gxteEdKxaY40WLaZF4r86BxUKnixHNpZeIaHbmsGLGiA2Lo21trPjixjgt2m0G vFO1X74uQorSJki6pRvC5ZEdyXXkREt4Yoddk4xqt2cgGSfwIcLguB7/Z8tnA+Njwi8Qeno8 lI/9+XmCJfPr35zih+VCcdumDMRx0/kwoalrqw9pPkjILN7tayj41cIdx1P0zxgolD22/rXI vHnWaBbxsZJfJbSmFHpZ7u1MckJxjoP8O5glPDX10NlIUYCmlCKlNiYhWqONCgmOTIrRJgYf iY02I+8nGc9OHKhCzvZdjYjnkHqqIlQ2KCkZTUJcUnQjEjhK7a8488suKRURmqTTki72sC4+ SoprRLM5Wj1TscJ9KkLJH9OclI5L0glJ93+LOZkqGZEti+ce7fefH5r+KXYo60L1x/t+C5SJ sKbEWlIxI6A28ODDxDl5nW9DmOjKPeudtYHTW6tW7moLDpulPR1Enkxu2/d9zsQUY5VpTKpJ eH979PnCtOrujX7qgB0N4R69KvvLTXvrmciK+Hk1qt5D7q5D/gHObtldfWe4be9tzm/G5DU1 HRepWb6I0sVp/gBenWFuRQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/P83yfp5uzx4kev8ZuM1uWMrLPMJr58cyEYbMxdPRMR4W7 ioztcrF+6BdySnElV6uQuzZR2el0uiLlbvlVjRy5ObXhoh9Dx/zz3mvvz2evv94cpbjIzODU 8QmSJl4VqyQyWrZpuT7kVOgnKazOEAp558LA9z2NhqLb1QQ6blUhqK5NweBpXg8vhrwIRp8+ o8CQ34Gg5F0PBbX2XgSNFacJON2TwOUbJODIzySgv36bQOfnMQzdl85jqDJHQltuKQbrcD8N Bg+BKwY9Ho9PGIZNlSyYdPOgr6KQhbF3i8DR28WArdjBQOPrBVBwtZtAQ6ODBntdHwbn/SIC vdW/GWizt9DQkZfFwM2BUgKfh0wUmHyDLDy3GjHUpI7bzn77xcDjLCuGs2V3MLhe1SN4kPYW g7m6i4DN58VgMedTMFLejKAv+wsLZ84Ns3AlJRtB5plLNKR2h8PozyISsUy0eQcpMdVyTGwc MtJia6kg3ivsYcXUB69Z0WhOFC0VweL1Bg8WS776GNFcmU5E89fzrJjxxYXFgfZ2Vmy5PEqL bpcBb5m5U7YiWopVJ0ma0JVRsphOXQM54og63ulW6VD9xgwUwAn8EsHy3sD4mfDzhZcvhyk/ B/JzBUvWx789xXtlQln7ugzEcVP4TYLNucGPND9PyCrb7v+Q80uFq56H6J9xjlBVY/1rCRjv cwZyiJ8VfLjwVF9CcpHMiCZUokB1fFKcSh0bvlB7KCY5Xn184f7DcWY0vhXTqbG8OvTdub4J 8RxSTpRHBPRLCkaVpE2Oa0ICRykD5SdH3JJCHq1KPiFpDu/VJMZK2iY0k6OVQfINO6QoBX9A lSAdkqQjkub/FXMBM3To6L7FQZHtKTDdrm39MM227K1xYkH9r+SC4sLsGGOf/uOFhDvPinNy nfaNEfUjNausi9PuTo1cmqTYNYDT+22TZxffY7fN8rTsKeBv7AsryZzW5vKGnG6NftRTnlLL Xdu6YPNq1+Yn/aWzV63d0xyS/or7scZuX3Mw7uFuV23iC7fujZLWxqgWBVMareoP2JygmScD AAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: It enters kernel mode on each syscall and each syscall handling should be considered independently from the point of view of Dept. Otherwise, Dept may wrongly track dependencies across different syscalls. That might be a real dependency from user mode. However, now that Dept just started to work, conservatively let Dept not track dependencies across different syscalls. Signed-off-by: Byungchul Park --- arch/arm64/kernel/syscall.c | 3 ++ arch/x86/entry/common.c | 4 +++ include/linux/dept.h | 39 ++++++++++++--------- kernel/dependency/dept.c | 67 +++++++++++++++++++------------------ 4 files changed, 64 insertions(+), 49 deletions(-) diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 9a70d9746b66..96c18ad1dbf7 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,8 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, * (Similarly for HVC and SMC elsewhere.) */ + dept_kernel_enter(); + if (flags & _TIF_MTE_ASYNC_FAULT) { /* * Process the asynchronous tag check fault before the actual diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 6356060caaf3..445e70937b38 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -20,6 +20,7 @@ #include #include #include +#include #ifdef CONFIG_XEN_PV #include @@ -75,6 +76,7 @@ static __always_inline bool do_syscall_x32(struct pt_regs *regs, int nr) /* Returns true to return using SYSRET, or false to use IRET */ __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { + dept_kernel_enter(); add_random_kstack_offset(); nr = syscall_enter_from_user_mode(regs, nr); @@ -262,6 +264,7 @@ __visible noinstr void do_int80_syscall_32(struct pt_regs *regs) { int nr = syscall_32_enter(regs); + dept_kernel_enter(); add_random_kstack_offset(); /* * Subtlety here: if ptrace pokes something larger than 2^31-1 into @@ -283,6 +286,7 @@ static noinstr bool __do_fast_syscall_32(struct pt_regs *regs) int nr = syscall_32_enter(regs); int res; + dept_kernel_enter(); add_random_kstack_offset(); /* * This cannot use syscall_enter_from_user_mode() as it has to diff --git a/include/linux/dept.h b/include/linux/dept.h index c6e2291dd843..4e359f76ac3c 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -25,11 +25,16 @@ struct task_struct; #define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) #define DEPT_MAX_SUBCLASSES_CACHE 2 -#define DEPT_SIRQ 0 -#define DEPT_HIRQ 1 -#define DEPT_IRQS_NR 2 -#define DEPT_SIRQF (1UL << DEPT_SIRQ) -#define DEPT_HIRQF (1UL << DEPT_HIRQ) +enum { + DEPT_CXT_SIRQ = 0, + DEPT_CXT_HIRQ, + DEPT_CXT_IRQS_NR, + DEPT_CXT_PROCESS = DEPT_CXT_IRQS_NR, + DEPT_CXTS_NR +}; + +#define DEPT_SIRQF (1UL << DEPT_CXT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_CXT_HIRQ) struct dept_ecxt; struct dept_iecxt { @@ -94,8 +99,8 @@ struct dept_class { /* * for tracking IRQ dependencies */ - struct dept_iecxt iecxt[DEPT_IRQS_NR]; - struct dept_iwait iwait[DEPT_IRQS_NR]; + struct dept_iecxt iecxt[DEPT_CXT_IRQS_NR]; + struct dept_iwait iwait[DEPT_CXT_IRQS_NR]; /* * classified by a map embedded in task_struct, @@ -207,8 +212,8 @@ struct dept_ecxt { /* * where the IRQ-enabled happened */ - unsigned long enirq_ip[DEPT_IRQS_NR]; - struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_CXT_IRQS_NR]; /* * where the event context started @@ -252,8 +257,8 @@ struct dept_wait { /* * where the IRQ wait happened */ - unsigned long irq_ip[DEPT_IRQS_NR]; - struct dept_stack *irq_stack[DEPT_IRQS_NR]; + unsigned long irq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_CXT_IRQS_NR]; /* * where the wait happened @@ -406,19 +411,19 @@ struct dept_task { int wait_hist_pos; /* - * sequential id to identify each IRQ context + * sequential id to identify each context */ - unsigned int irq_id[DEPT_IRQS_NR]; + unsigned int cxt_id[DEPT_CXTS_NR]; /* * for tracking IRQ-enabled points with cross-event */ - unsigned int wgen_enirq[DEPT_IRQS_NR]; + unsigned int wgen_enirq[DEPT_CXT_IRQS_NR]; /* * for keeping up-to-date IRQ-enabled points */ - unsigned long enirq_ip[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; /* * for reserving a current stack instance at each operation @@ -465,7 +470,7 @@ struct dept_task { .wait_hist = { { .wait = NULL, } }, \ .ecxt_held_pos = 0, \ .wait_hist_pos = 0, \ - .irq_id = { 0U }, \ + .cxt_id = { 0U }, \ .wgen_enirq = { 0U }, \ .enirq_ip = { 0UL }, \ .stack = NULL, \ @@ -503,6 +508,7 @@ extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); +extern void dept_kernel_enter(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -552,6 +558,7 @@ struct dept_task { }; #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) +#define dept_kernel_enter() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 19406093103e..9aba9eb22760 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -220,9 +220,9 @@ static struct dept_class *dep_tc(struct dept_dep *d) static const char *irq_str(int irq) { - if (irq == DEPT_SIRQ) + if (irq == DEPT_CXT_SIRQ) return "softirq"; - if (irq == DEPT_HIRQ) + if (irq == DEPT_CXT_HIRQ) return "hardirq"; return "(unknown)"; } @@ -406,7 +406,7 @@ static void initialize_class(struct dept_class *c) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iecxt *ie = &c->iecxt[i]; struct dept_iwait *iw = &c->iwait[i]; @@ -431,7 +431,7 @@ static void initialize_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { e->enirq_stack[i] = NULL; e->enirq_ip[i] = 0UL; } @@ -447,7 +447,7 @@ static void initialize_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { w->irq_stack[i] = NULL; w->irq_ip[i] = 0UL; } @@ -486,7 +486,7 @@ static void destroy_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (e->enirq_stack[i]) put_stack(e->enirq_stack[i]); if (e->class) @@ -502,7 +502,7 @@ static void destroy_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (w->irq_stack[i]) put_stack(w->irq_stack[i]); if (w->class) @@ -651,7 +651,7 @@ static void print_diagram(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { if (!firstline) pr_warn("\nor\n\n"); firstline = false; @@ -684,7 +684,7 @@ static void print_dep(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { pr_warn("%s has been enabled:\n", irq_str(irq)); print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); pr_warn("\n"); @@ -910,7 +910,7 @@ static void bfs(struct dept_class *c, bfs_f *cb, void *in, void **out) */ static unsigned long cur_enirqf(void); -static int cur_irq(void); +static int cur_cxt(void); static unsigned int cur_ctxt_id(void); static struct dept_iecxt *iecxt(struct dept_class *c, int irq) @@ -1458,7 +1458,7 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) if (d) { check_dl_bfs(d); - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iwait *fiw = iwait(fc, i); struct dept_iecxt *found_ie; struct dept_iwait *found_iw; @@ -1494,7 +1494,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, struct dept_task *dt = dept_task(); struct dept_wait *w; unsigned int wg = 0U; - int irq; + int cxt; int i; if (DEPT_WARN_ON(!valid_class(c))) @@ -1510,9 +1510,9 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; - irq = cur_irq(); - if (irq < DEPT_IRQS_NR) - add_iwait(c, irq, w); + cxt = cur_cxt(); + if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) + add_iwait(c, cxt, w); /* * Avoid adding dependency between user aware nested ecxt and @@ -1593,7 +1593,7 @@ static bool add_ecxt(struct dept_map *m, struct dept_class *c, eh->sub_l = sub_l; irqf = cur_enirqf(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) add_iecxt(c, irq, e, false); del_ecxt(e); @@ -1745,7 +1745,7 @@ static void do_event(struct dept_map *m, struct dept_class *c, add_dep(eh->ecxt, wh->wait); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_ecxt *e; if (before(dt->wgen_enirq[i], wg)) @@ -1787,7 +1787,7 @@ static void disconnect_class(struct dept_class *c) call_rcu(&d->rh, del_dep_rcu); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { stale_iecxt(iecxt(c, i)); stale_iwait(iwait(c, i)); } @@ -1812,27 +1812,21 @@ static unsigned long cur_enirqf(void) return 0UL; } -static int cur_irq(void) +static int cur_cxt(void) { if (lockdep_softirq_context(current)) - return DEPT_SIRQ; + return DEPT_CXT_SIRQ; if (lockdep_hardirq_context()) - return DEPT_HIRQ; - return DEPT_IRQS_NR; + return DEPT_CXT_HIRQ; + return DEPT_CXT_PROCESS; } static unsigned int cur_ctxt_id(void) { struct dept_task *dt = dept_task(); - int irq = cur_irq(); + int cxt = cur_cxt(); - /* - * Normal process context - */ - if (irq == DEPT_IRQS_NR) - return 0U; - - return dt->irq_id[irq] | (1UL << irq); + return dt->cxt_id[cxt] | (1UL << cxt); } static void enirq_transition(int irq) @@ -1893,7 +1887,7 @@ static void dept_enirq(unsigned long ip) flags = dept_enter(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { dt->enirq_ip[irq] = ip; enirq_transition(irq); } @@ -1939,6 +1933,13 @@ void noinstr dept_hardirqs_off(void) dept_task()->hardirqs_enabled = false; } +void noinstr dept_kernel_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + /* * Ensure it's the outmost softirq context. */ @@ -1946,7 +1947,7 @@ void dept_softirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_SIRQ] += 1UL << DEPT_CXTS_NR; } /* @@ -1956,7 +1957,7 @@ void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_HIRQ] += 1UL << DEPT_CXTS_NR; } void dept_sched_enter(void) From patchwork Wed Mar 6 08:54:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583573 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BFDD1605CE; Wed, 6 Mar 2024 08:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715343; cv=none; b=WEfixzGfWMKI+GRmr85IwsYeSUV8SHQKiw7s3pOgDh03ItiRX2F4OKXK8osScfAb5bK7MvXfrwbAu6unmZlmUNy4NWC9G6y3yTMG5KDwKYkS1BLp+nKwwVvbPLgNdBUUPSoubv4hz3BYqNeR7EfeJUiFLWV1G/v46yWkJB7CvP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715343; c=relaxed/simple; bh=7Na745h6mqt6szPtcPf56L4nwkzcVctpfLScpiam1ys=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=TV11BrnAUH9ymDKVQDnDmlfIoqcDv/n0GnHbrviboDgq5ysJ6hgp4kWfSLv+nydNArKXxG9sUA31HzDbolwlBtWvJY3/AV7K8ayGvv9s/yM+TdRTAiJOwArcn7rp5ZXsPEGWSGTIvuGpDyxpepTZIuKKIDvlwGpWhWqAtGyo9e0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-ec-65e82f7dcaa5 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 12/27] dept: Distinguish each work from another Date: Wed, 6 Mar 2024 17:54:58 +0900 Message-Id: <20240306085513.41482-13-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x46zn5P5icluw1ZEDfvYPGSMr5mn8QfacNNvuulpF3Ee UyGRrraKq+UudlKXy0XLw3HiUh4qdeO0iloeogeLOz15uDL/fPba+/P6vP/68JTiNuPHq6P3 S5poVaSSldGy7nEFc4/P+yzNz+xXQMb5+eD+kUJDnsXMQv2NYgTmWycxdDrWwBtPF4Khl3UU 5GTVIzC2tVBwq6oVga0wkYXGjvHgdPeyUJN1joWkKxYWXn0dxtCcnYmh2LoenusKMNgHPtGQ 08lCbk4S9o7PGAZMRRyYEmZCe6Geg+G2YKhpfc2ArSkQLuU3s3DfVkNDVUU7hsa7eSy0mv8w 8Lyqmob6jDQGSnoKWPjqMVFgcvdy0GA3YChN9had/v6bgadpdgynr97E4Hx7D8GDlPcYrObX LDx2d2Eos2ZRMHjNgaD9QjcHp84PcJB78gKCc6eyaUhuXghD/Xls6GLyuKuXIsllB4nNY6DJ swKR3NG3cCT5QRNHDNYDpKwwgFy534mJsc/NEGvRWZZY+zI5ktrtxKSntpYj1ReHaNLhzMGb /HbIloRLkep4STNv2W5ZhP5VL4r9MvGQqyKdSkDXhVTkw4vCArE8P4n6z2/N/cwIs8Js0eUa GM19hRliWdrH0ZwSumTi1drVIzxRWCmWXk8cdWhhpqh/ZORGWC4sEvNaE+l/nf5ical91PHx 5uk96ewIK4SF4ssko5dlXqefFzO+5TL/DqaIjwpdtA7JDWhMEVKoo+OjVOrIBUER2mj1oaA9 MVFW5H0m07HhsArUV7+lEgk8Uo6Th/p8khSMKj5OG1WJRJ5S+sqPDnZICnm4SntY0sTs0hyI lOIq0VSeVk6Wh3gOhiuEvar90j5JipU0/7eY9/FLQFvP+LpME8KOOLbXncWBS7WdljpHU0No +awVmz5ot/1c03A5xt79xz/EtdNzpppfN7jMot8abN6wVhUxSfemaeV024ZflsYl+pqQWD53 M3+iJV73xDAnKhwPPnwRZi0n2WtNqYbl9+aUbNSlrH94zP/22HeBVcYKp6MtaNUX6s60o0o6 LkIVHEBp4lR/ARoqQ6JIAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRzF+/3uaxstLtPoYkGxHoaRj8j6UhlFUDd7EEQUFeTKS47UZFPL SFC3wvKxDOZyLp0P5mulzQIrZ+bQMvNRDlNRSZPSWlrmRNMezuifw4dzDuevIyJkBspHpIyO FVTRikg5LSElh7drNiYGjAiBv5O3QlZ6ILgnU0kwVVpp6LhfgcD6MBnDaOM+eDflQjDb2k6A Qd+BoGCwn4CHTQMI7KUpNHQOLwGne5yGZn0aDZqiShrefJnD0Jd9G0OF7RC03CrEUD/ziQTD KA25Bg2elxEMM5ZyBixJa2Go1MjA3GAQNA90UeC420yBvXcD5OT10VBrbyahqWYIQ+cTEw0D 1j8UtDS9JKEjK4OCe2OFNHyZshBgcY8z8LbejKFKO792/cdvCl5k1GO4XvwAg7PnKYK61PcY bNYuGhxuF4Zqm56AnyWNCIYyvzJwLX2GgdzkTARp17JJ0PYFw+y0id61jXe4xgleW32Jt0+Z Sf5VIcc/NvYzvLaul+HNtji+utSPL6odxXzBhJvibeU3aN42cZvhb351Yn6srY3hX96ZJflh pwEfWX5SsiNciFTGC6qAnWGSCOObcRTz2etyd42OSEJl7E0kFnHsZq7HOk15mGZ9ue7uGcLD 3uwqrjrj44JPsC4JV9y218Ne7B6uqixloUOyaznj8wLGw1J2C2caSCH/ba7kKqrqFzrieV83 pqM9LGODuVZNAX0LScxoUTnyVkbHRymUkcH+6gsRCdHKy/7nLkbZ0PxdLIlzWTVosnNfA2JF SL5Yukv8SZBRinh1QlQD4kSE3Ft69eewIJOGKxKuCKqLZ1RxkYK6AS0XkfJl0tDjQpiMPa+I FS4IQoyg+p9ikdgnCa32OZyDH/mWJTpv/Dq2P39wJKb96GiOtvf1ik179ZPTp/JPK1tCmnR+ Z1MDV4XEVp4X5wu9ac80dS3SEx3qkkxk2nryR850e0ledq6jRr/7QKix36hc0/kha124Uhf8 dI3tW216Ea779a7VXuzlUpwKWP/9RMhBR1fRpom8u4lLQ+WkOkIR5Eeo1Iq/XDpKMioDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Workqueue already provides concurrency control. By that, any wait in a work doesn't prevents events in other works with the control enabled. Thus, each work would better be considered a different context. So let Dept assign a different context id to each work. Signed-off-by: Byungchul Park --- include/linux/dept.h | 2 ++ kernel/dependency/dept.c | 10 ++++++++++ kernel/workqueue.c | 3 +++ 3 files changed, 15 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index 4e359f76ac3c..319a5b43df89 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -509,6 +509,7 @@ extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long extern void dept_sched_enter(void); extern void dept_sched_exit(void); extern void dept_kernel_enter(void); +extern void dept_work_enter(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -559,6 +560,7 @@ struct dept_task { }; #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) #define dept_kernel_enter() do { } while (0) +#define dept_work_enter() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 9aba9eb22760..a8e693fd590f 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1933,6 +1933,16 @@ void noinstr dept_hardirqs_off(void) dept_task()->hardirqs_enabled = false; } +/* + * Assign a different context id to each work. + */ +void dept_work_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + void noinstr dept_kernel_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 2989b57e154a..4452864b918b 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -53,6 +53,7 @@ #include #include #include +#include #include "workqueue_internal.h" @@ -2549,6 +2550,8 @@ __acquires(&pool->lock) lockdep_copy_map(&lockdep_map, &work->lockdep_map); #endif + dept_work_enter(); + /* ensure we're on the correct CPU */ WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); From patchwork Wed Mar 6 08:54:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583575 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C498D605D9; Wed, 6 Mar 2024 08:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715344; cv=none; b=WiKJPpNTgfXL4kDNhQ6UEgdkTQzt2/UmUsVnQmtY6mF9Av34cIp6fAKYJdivGY2XEU35l33PncOTyzxG4J60oiy35LSoLdZq5klLnb3F7n3X3AYi4xFOCHkrTrde4T6rhUH8Rq1w8aaQ95wjEwUkhhjWAQPFLN4AeW9DZ5cdrcc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715344; c=relaxed/simple; bh=FoAZA+0jlnYhnPe3f+IjJ7FMJD3ZMq2wQkEX2mpeEjA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=dGOjVnEpI4lWWZNrZyZDU716e0tk6HntelMTZk1d4CXuf11OHuHottlIvocKF46jUtbdFcKV3y1/rS5IlvnOkOM7zj7GmdQpK1MGA68iQQa4TlIeI1keehhSIJWA7Ywy7qIp6g58MtNTxe+wowPDEBPXBGMbUy+X/pGo6+Xkel0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-f8-65e82f7e0839 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 13/27] dept: Add a mechanism to refill the internal memory pools on running out Date: Wed, 6 Mar 2024 17:54:59 +0900 Message-Id: <20240306085513.41482-14-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSfUzMcRwHcN/fczdnv53STwydYZiePOxjy+M8/Gy0jH8w08391E0PXC5C 7VLpQSlZRcW6ykmlh+v+EJUTUlo5VE5LS040V+nhbk7NuTL/fPba5+H914fBJXrSg1GEnxOU 4bJQKSUiRMNzi9bFen8XfL784uBGmg9YJ5MJKKiqoMBYWY6gQh+HwdDLvfDBZkEw1f4Gh9xs IwLN50846Jv7EDSUXqHgvXkedFpHKWjNvkZBfHEVBW9/TGPQm5OFQbnuALRlFmFgsH8jIHeI gvzceMxZvmNg15bRoFWvgIHSPBqmP/tCa183CQ09a+H23V4K6htaCWh+NIDB+8cFFPRVOEho a24hwHgjnYSHI0UU/LBpcdBaR2l4ZyjEoDrBGXR14g8Jr9INGFwtqcGg8+MTBI3J/RjoKrop eG61YFCry8bh9/2XCAauD9OQmGanIT/uOoJriTkEJPRuhKlfBdT2zfxzyyjOJ9Se5xtshQT/ uojj6/I+0XxCYw/NF+pUfG3pGr64fgjjNeNWkteVpVC8bjyL5lOHOzF+pKOD5ltuTRG8uTMX C/Q4KvKXC6GKKEHpvTVIFNLVlk+fMW678LXaRqhRnV8qcmE4dgM32GPG/7tP3Tpril3FmUz2 Wbuyy7ja9EFyxjhrEXElHXtmPJ89yY1eKZ/tE+wK7sFUP52KGEbMbuIK7vj/i1zKlVcbZmNc nO2MkQxqxhJ2I9cer3Fa5NyZZDhzkgP9O1jIPSs1EZlIXIjmlCGJIjwqTKYI3eAVEh2uuOB1 MiJMh5y/pI2ZPvYIjRsPNSGWQdK54u0u3wQJKYuKjA5rQhyDS13Fl3+bBYlYLou+KCgjTihV oUJkE1rEEFJ3sZ/tvFzCBsvOCacF4Yyg/D/FGBcPNdpWfy/ZXfXCNbNmccTPLzuTzh7ZHWBv vOwW1xidovYclA4mXarzk//xC2pJWVk8sU4asBPl68f2v0vz/mnqfhrsiF1tPX6wxBGjD3P1 7zDW3MwinmhYedCSwOUryX3Blosaz8TpLYagLtXRHf2m9Yczxra6TfTsOqaqXIu61KccC6RE ZIjMdw2ujJT9BTcn9k9HAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRyG+5+7o8VhGZ20MCYiKJmayi+LlKB2kIo+ZQWhQw860mmbl8wK c1rmNSu10mRempdZ2uxDN000L2toy1smKilmrpaStZFpFzX68vLwvvB8ehlcUkw6MQplgqBS ymOklIgQHdmj2XFx56zgfdOCoDDXG2zfswgoa2ygwPxAj6Dh0SUMLJ0yeGu3IljqfY1DSZEZ QcXkOA6PuiYQtNSmUzAwvQEGbfMUGItyKNBUNVLw5vMyBmPF1zHQGw6D6VolBm2LHwkosVBQ WqLBVmIWg0VdPQ26NDeYqr1Dw/KkDxgnhknouGskoWXUE26Xj1HwvMVIQNfjKQwGnpZRMNHw hwRTVw8B5sI8Eu7PVVLw2a7DQWebp6G/TYtBU8aK7fK33yR057VhcLn6IQaD754haM16j4Gh YZiCDpsVg2ZDEQ4/azoRTOV/oSEzd5GG0kv5CHIyiwnIGPOHpR9lVHAg32Gdx/mM5mS+xa4l +FeVHP/kzjjNZ7SO0rzWkMg313rwVc8tGF+xYCN5Q/1VijcsXKf57C+DGD/X10fzPbeWCH56 sAQ76nxStDdSiFEkCaqd+8JF0UOmUjreHHT2Q5OdSENPfLORA8OxftxEmhFfZYp150ZGFtfY kd3ONefNkKuMs1YRV913cJU3shHcfLp+rSdYN65u6T2djRhGzAZwZXf3/lO6cPqmtjWNw0pd MFdArbKE9ed6NRXUNSTSonX1yFGhTIqVK2L8vdSno1OUirNeEXGxBrTyFt2F5cLH6PuArB2x DJKuFwc7fBQkpDxJnRLbjjgGlzqKz/+cFiTiSHnKOUEVF6ZKjBHU7ciZIaSbxSGhQriEjZIn CKcFIV5Q/V8xxsEpDV2cqh3Ktx6YHmnStG4zheo6UjdZAriQ4KPF8cfNnppjlq81yeujZnbT n5R1sq0vEvb7HiuXfXU/tfXdFY6o6EuV2INQvueZsRPDbl7OUVW3F4bwwl1RCTmwpdf1QqDe 6HpjtupezyHaL/1H/69u3OOlyVs2PqwNS4z1S3LxnSGkhDpa7uOBq9Tyv2POYkEpAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Dept engine works in a constrained environment. For example, Dept cannot make use of dynamic allocation e.g. kmalloc(). So Dept has been using static pools to keep memory chunks Dept uses. However, Dept would barely work once any of the pools gets run out. So implemented a mechanism for the refill on the lack by any chance, using irq work and workqueue that fits on the contrained environment. Signed-off-by: Byungchul Park --- include/linux/dept.h | 19 ++++-- kernel/dependency/dept.c | 104 +++++++++++++++++++++++++++----- kernel/dependency/dept_object.h | 10 +-- kernel/dependency/dept_proc.c | 8 +-- 4 files changed, 112 insertions(+), 29 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 319a5b43df89..ca1a34be4127 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -336,9 +336,19 @@ struct dept_pool { size_t obj_sz; /* - * the number of the static array + * the remaining number of the object in spool */ - atomic_t obj_nr; + int obj_nr; + + /* + * the number of the object in spool + */ + int tot_nr; + + /* + * accumulated amount of memory used by the object in byte + */ + atomic_t acc_sz; /* * offset of ->pool_node @@ -348,9 +358,10 @@ struct dept_pool { /* * pointer to the pool */ - void *spool; + void *spool; /* static pool */ + void *rpool; /* reserved pool */ struct llist_head boot_pool; - struct llist_head __percpu *lpool; + struct llist_head __percpu *lpool; /* local pool */ }; struct dept_ecxt_held { diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a8e693fd590f..8ca46ad98e10 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,9 @@ #include #include #include +#include +#include +#include #include "dept_internal.h" static int dept_stop; @@ -122,9 +125,11 @@ static int dept_per_cpu_ready; WARN(1, "DEPT_STOP: " s); \ }) -#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t dept_pool_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; /* * DEPT internal engine should be careful in using outside functions @@ -263,6 +268,7 @@ static bool valid_key(struct dept_key *k) #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ +static struct dept_##id rpool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT @@ -271,14 +277,70 @@ struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ - .obj_nr = ATOMIC_INIT(nr), \ + .obj_nr = nr, \ + .tot_nr = nr, \ + .acc_sz = ATOMIC_INIT(sizeof(spool_##id) + sizeof(rpool_##id)), \ .node_off = offsetof(struct dept_##id, pool_node), \ .spool = spool_##id, \ + .rpool = rpool_##id, \ .lpool = &lpool_##id, }, #include "dept_object.h" #undef OBJECT }; +static void dept_wq_work_fn(struct work_struct *work) +{ + int i; + + for (i = 0; i < OBJECT_NR; i++) { + struct dept_pool *p = dept_pool + i; + int sz = p->tot_nr * p->obj_sz; + void *rpool; + bool need; + + arch_spin_lock(&dept_pool_spin); + need = !p->rpool; + arch_spin_unlock(&dept_pool_spin); + + if (!need) + continue; + + rpool = vmalloc(sz); + + if (!rpool) { + DEPT_STOP("Failed to extend internal resources.\n"); + break; + } + + arch_spin_lock(&dept_pool_spin); + if (!p->rpool) { + p->rpool = rpool; + rpool = NULL; + atomic_add(sz, &p->acc_sz); + } + arch_spin_unlock(&dept_pool_spin); + + if (rpool) + vfree(rpool); + else + DEPT_INFO("Dept object(%s) just got refilled successfully.\n", p->name); + } +} + +static DECLARE_WORK(dept_wq_work, dept_wq_work_fn); + +static void dept_irq_work_fn(struct irq_work *w) +{ + schedule_work(&dept_wq_work); +} + +static DEFINE_IRQ_WORK(dept_irq_work, dept_irq_work_fn); + +static void request_rpool_refill(void) +{ + irq_work_queue(&dept_irq_work); +} + /* * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is * enabled or not because NMI and other contexts in the same CPU never @@ -314,19 +376,31 @@ static void *from_pool(enum object_t t) /* * Try static pool. */ - if (atomic_read(&p->obj_nr) > 0) { - int idx = atomic_dec_return(&p->obj_nr); + arch_spin_lock(&dept_pool_spin); + + if (!p->obj_nr) { + p->spool = p->rpool; + p->obj_nr = p->rpool ? p->tot_nr : 0; + p->rpool = NULL; + request_rpool_refill(); + } + + if (p->obj_nr) { + void *ret; + + p->obj_nr--; + ret = p->spool + (p->obj_nr * p->obj_sz); + arch_spin_unlock(&dept_pool_spin); - if (idx >= 0) - return p->spool + (idx * p->obj_sz); + return ret; } + arch_spin_unlock(&dept_pool_spin); - DEPT_INFO_ONCE("---------------------------------------------\n" - " Some of Dept internal resources are run out.\n" - " Dept might still work if the resources get freed.\n" - " However, the chances are Dept will suffer from\n" - " the lack from now. Needs to extend the internal\n" - " resource pools. Ask max.byungchul.park@gmail.com\n"); + DEPT_INFO("------------------------------------------\n" + " Dept object(%s) is run out.\n" + " Dept is trying to refill the object.\n" + " Nevertheless, if it fails, Dept will stop.\n", + p->name); return NULL; } @@ -2957,8 +3031,8 @@ void __init dept_init(void) pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); #define OBJECT(id, nr) \ - pr_info("... memory used by %s: %zu KB\n", \ - #id, B2KB(sizeof(struct dept_##id) * nr)); + pr_info("... memory initially used by %s: %zu KB\n", \ + #id, B2KB(sizeof(spool_##id) + sizeof(rpool_##id))); #include "dept_object.h" #undef OBJECT #define HASH(id, bits) \ @@ -2966,6 +3040,6 @@ void __init dept_init(void) #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); #include "dept_hash.h" #undef HASH - pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... total memory initially used by objects and hashs: %zu KB\n", B2KB(mem_total)); pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); } diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h index 0b7eb16fe9fb..4f936adfa8ee 100644 --- a/kernel/dependency/dept_object.h +++ b/kernel/dependency/dept_object.h @@ -6,8 +6,8 @@ * nr: # of the object that should be kept in the pool. */ -OBJECT(dep, 1024 * 8) -OBJECT(class, 1024 * 8) -OBJECT(stack, 1024 * 32) -OBJECT(ecxt, 1024 * 16) -OBJECT(wait, 1024 * 32) +OBJECT(dep, 1024 * 4 * 2) +OBJECT(class, 1024 * 4) +OBJECT(stack, 1024 * 4 * 8) +OBJECT(ecxt, 1024 * 4 * 2) +OBJECT(wait, 1024 * 4 * 4) diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c index 7d61dfbc5865..f07a512b203f 100644 --- a/kernel/dependency/dept_proc.c +++ b/kernel/dependency/dept_proc.c @@ -73,12 +73,10 @@ static int dept_stats_show(struct seq_file *m, void *v) { int r; - seq_puts(m, "Availability in the static pools:\n\n"); + seq_puts(m, "Accumulated amount of memory used by pools:\n\n"); #define OBJECT(id, nr) \ - r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ - if (r < 0) \ - r = 0; \ - seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + r = atomic_read(&dept_pool[OBJECT_##id].acc_sz); \ + seq_printf(m, "%s\t%d KB\n", #id, r / 1024); #include "dept_object.h" #undef OBJECT From patchwork Wed Mar 6 08:55:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583574 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0AE1160B97; Wed, 6 Mar 2024 08:55:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715344; cv=none; b=nHxe8/pDCuFyIoRdl8DFePw6cjg4RWyhDE2RkzN35ZltWerXOfkMx9xrm4MWZRTtmMm/JyE5JNaSqvTb+6CRsVQCjdQv2elZk7bTYgueG005SMio1bViddJ8c+pi3dhRty9auWTY/MZPijJxcDdcPy8zO6DVutV/71FJ0YfaGdk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715344; c=relaxed/simple; bh=32dhqEgXIybDFaJN1JuIlSUqelCcRZP5tULcFTO0inM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=P0VS1QNQdTGDu2AQ9UxS6YVs38CdVfE/gzrdkp3mJhNthnaExSZoMied92BNKn1XzYy+hnGY6/BVQPvKcJeUA4VcPbh+t2/82O8iyCCK1uimNM9BZwOTA+K4d5H4xS/Il3BUcUxlWplGvUIgoZEl8pj4Q3ByLzFwaKpQ/UGt2b4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-08-65e82f7e84b4 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 14/27] cpu/hotplug: Use a weaker annotation in AP thread Date: Wed, 6 Mar 2024 17:55:00 +0900 Message-Id: <20240306085513.41482-15-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTYRgH8N73XLdaHJbUSStjFIGRN7SeIKIPXU5XovsN6qSHtlKrWatF heWFMjUN5kolvLFETXNGdHGyFM1l2qplVmZqklpTw9pwadlW9OXhx3P5f3pYQllF+bOauBOS Nk6MUdFyUj44pWDR+ZB+KbSlay5kpYWC68clEvIqy2mwV5QhKL97AcNAwxp443YiGGt5ToDR YEdQ0P2BgLuNnQgsJRdpeNU7FRyuYRpshis0JBZV0vDi6ziGjuxrGMrMG6E5sxCD1dNHgnGA hlxjIvaWfgweUykDpoT50FOSw8B4dxjYOtsosLxbCDdudtBQY7GR0Hi/B8Orh3k0dJZPUNDc 2ESCPSudgttDhTR8dZsIMLmGGXhpzcdwJ8kblPL9NwVP0q0YUoqrMDjePkJQe6kLg7m8jYZ6 lxNDtdlAwM9bDQh6MgYZSE7zMJB7IQPBleRsEpI6ImFsNI9esVSodw4TQlL1KcHizieFp4W8 8CDnAyMk1b5jhHzzSaG6JEgoqhnAQsGIixLMpZdpwTxyjRFSBx1YGGptZYSm62Ok0Osw4s3+ e+TLoqUYjU7Shiw/IFcXptnJYx9lp53ZBxPQBJOKZCzPRfAVn7/g/3bbfv3t09wCvr3dQ/js x83lq9M/Uz4TnFPOF7euTkUsO43bwDsMUT6S3Hy+3yL6qOAW81kp8/4FBvJld6x/Q2Te9tWh q7TPSi6Sb0ks8Fru3Rll+eau29S/g5n845J2MhMp8tGkUqTUxOliRU1MRLBaH6c5HRx1NNaM vI9kOje+9z4asW+tQxyLVFMUK2R9kpISdfH62DrEs4TKT3H2Z6+kVESL+jOS9uh+7ckYKb4O BbCkaoYi3H0qWskdEk9IRyTpmKT9P8WszD8BbXpYyanXhwRaB9v69ShAlJLnbXCsXBigu2cM j6xv8NsXGLv3cFVxxer3Ow6c28mHe1L9VLptkWfXve7Z4l472tT0jFxycPKAciLHM2t5Z3DG N491+rbtuyYyu4+XmQ1zQj6t0q/LOF5Uq64h13wMUs/mZgVaMxcrh27s/n3d5ukrUpHxajEs iNDGi38AaAC9E0QDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSWUwTYRDAcb89vt1Wq5tK4gY0aiOaaORIxIxHjA9GV6Pog0fUqFTYQMNp CyieYKFB5BANRQG1HClXvQoPXgUCilYFERpEQRRSFcIKXkU5FKnGl8kvM8n/aVhSeYH2ZDVR saI2Sh2hwnJKHrhKv/Skb5/od69HBtnpfuD6nkpBwQ0LhpbrlQgs1UkE9D/cAC+HJQRjTc9J yM1pQVDY84aE6sZuBLay0xjanNPB4RrCYM85i0FffAPDi4FxArqM5wmotG6Bp+eKCKgb+UhB bj+G/Fw9MTn6CBgxVzBgTvSG3rI8BsZ7/MHe3U5Dw2U7DbbXS+DSlS4M9212Chpv9xLQdrcA Q7dlgoanjY8paMnOoOHaYBGGgWEzCWbXEAOtdSYCbiZP1gzfftPwKKOOAEPJLQIcr+4hqEl9 R4DV0o6hwSURUGXNIWG09CGC3sxPDKSkjzCQn5SJ4GyKkYLkrgAY+1mA164UGqQhUkiuOizY hk2U8KSIF+7kvWGE5JrXjGCyxglVZYuF4vv9hFD41UUL1oozWLB+Pc8IaZ8chDDY3MwIjy+O UYLTkUts89ojXx0iRmjiRa3vmiB5WFF6CxXzVnZEMh5MRBNMGpKxPLeMH7b/+mvMLeI7OkZI tz24eXxVxgfabZKT5HxJ8/o0xLIzuc28IyfYTYrz5vtsajcV3HI+27DgX3AuX3mz7m9ENrnO GszCbiu5AL5JX4jPIbkJTalAHpqo+Ei1JiLARxcelhClOeITHB1pRZOvYj4xnn0bfW/bUI84 FqmmKdbKPopKWh2vS4isRzxLqjwUx0edolIRok44KmqjD2jjIkRdPfJiKdUsxaZdYpCSC1XH iuGiGCNq/18JVuaZiGwpqafiomsnpq72VY22mmaEBjnLY0qqQ6h1pYGBnwt/13RurL9qNDRJ 4f769i7MHWtyOftnHzj1Q5M1tNf0LHraM7/ngYOxu/flfZ7fu/VQ8I7a9+LCjJWZBmPQBUmy LJrjc3ezbWeZPymVl9csK9EOPMArtvOKL0mdmfs7w67lqyhdmNp/ManVqf8AuIaq+yYDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: cb92173d1f0 ("locking/lockdep, cpu/hotplug: Annotate AP thread") was introduced to make lockdep_assert_cpus_held() work in AP thread. However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire() as a potential wait and might report a deadlock by the wait. So replaced it with trylock version of annotation. Signed-off-by: Byungchul Park --- kernel/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index a86972a91991..b708989f789f 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -535,7 +535,7 @@ int lockdep_is_cpus_held(void) static void lockdep_acquire_cpus_lock(void) { - rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 0, _THIS_IP_); + rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 1, _THIS_IP_); } static void lockdep_release_cpus_lock(void) From patchwork Wed Mar 6 08:55:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583577 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BAA7560DEC; Wed, 6 Mar 2024 08:55:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715345; cv=none; b=WRHvJcZE9KozH9D3e1elkeiZSl5Z+hrXyQAkoXV77rF63Dsb4OUpPi/E0FnfSQxYYM8KFurvUJfZgucdv3nW2It8DQnO/fZ1Kd4DnwscWU9ZImRrzFmNWj1SES86/9HfiOYq8CakkpEbU/XOcNjxKsICeOCm3VA8Jk5RsdxKx5U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715345; c=relaxed/simple; bh=t4Qh0s/9NGiebNos8RWbTI6PuT087Z/eTqnpXervcDg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sx2q5z6r9bQh4Nw9R3itYzm1MqpSYK43Fo/nUbAWYujNMz8bMNiihrMD9jz1ixtsX8Z1G/NhlY7eYP7UysNrcgIyCi1f/sVYy+jY1iOuFKNuNkz4VKVzyklBFmkX1Bu3KnBGq8adRw1oZLFj2ehymjETkox/5YRqizMnoKHeews= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-18-65e82f7ece15 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 15/27] dept: Apply sdt_might_sleep_{start,end}() to dma fence wait Date: Wed, 6 Mar 2024 17:55:01 +0900 Message-Id: <20240306085513.41482-16-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+59zds7ZanJYhqf1wVp0M1ILizeKsC/1JxCiKKIkG3poIzWZ eVllaF4wTVNDLbPUJcvUNDfzkk6moWWXZWW11KTESvOW2rzNrGn15eXH8z487/PhZUlZpUjO qoPPCJpgZaCCllCSoSW6TRc8+gTPrLoNkH7ZE2w/EynILS+loa2sBEFpZQwB/c174f3EIAL7 i5ckZGe2ISj4/JGEypZuBKaiizS86XWCdtsIDa2ZyTTE3i6n4dXALAFdWRkElBh84FmajgDz 9DcKsvtpuJEdSzhGHwHT+mIG9NFroKcoh4HZz5uhtfudCEwdG+H6rS4a6k2tFLTU9BDw5mEu Dd2lv0XwrOUJBW3pKSK4N6yjYWBCT4LeNsLAa3M+AffjHEEJ43MieJxiJiChsIKA9g91CBoS PxFgKH1HwyPbIAFGQyYJM3eaEfSkDjEQf3magRsxqQiS47MoiOvaCvapXNp7O340OELiOGME Nk3kU/ipjse1OR8ZHNfQweB8Qxg2Frnh2/X9BC4Ys4mwofgSjQ1jGQxOGmon8LDFwuAn1+wU 7m3PJvbLj0p2BgiB6nBB47HrhESV2mlBITHSSEMeE42+SJKQmOU5L173IIn+z1/zrGieaW4d b7VOk/PszK3kjSlfRfNMcoMSvtCyZ56Xcof5mM6RBQ/FreFHOzsWWMpt4/V1GczfTFe+5L55 QRc79CvDVxZuybit/IvYAgdLHJ4plr/54/m/Esv5xiIrlYak+WhRMZKpg8ODlOpAL3eVNlgd 6e5/OsiAHL+kj5o9VoPG2g42IY5FiiVSb/E3QSZShodqg5oQz5IKZ+n5mV5BJg1Qas8KmtN+ mrBAIbQJrWAphYt0y0REgIw7qTwjnBKEEEHzf0uwYnk0Whbf4llVM9B49S1yc18cNqMSRqN0 xoryHa8nqxLOig9qh9ePfddWHKq2LvWsl93zkZfVxB+Rp5nta1XJ1axv4TlX+7Z9XnNDzXlH Zw4kbDmCN1zwi1Ikenj3/Rq3avx9E532Tr732N3hW+viLF910pJ73LQ6xKVndNzJuVdyl05X UKEq5WY3UhOq/ANcJVTZRwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTYRgH8N73nPOebbQ4LaND0m0glZElZTxUdKHbW1Tkl+6Qow65nBab mnZ1ucIyLQdqqdW0WEPtdtYHyxTRNO2yrFmZTkmxcuWli7NMqZzRl4cf/+fh/+lRMJosbqJC HxMrGWN0Bi1RsaqNi5JnH5vTKc3tezsWMs7OBV9fCgt5t4oJ1N8sQlB814zBW70G3vR3IRh8 9pyB7Mx6BPltLQzcrWlFUOY4QcDdMQYafL0E6jJTCSRfvUXgxechDJ4sK4YieQM8OV+AoWLg IwvZXgK52cl4eHRiGLAX8mBPCoJ2Rw4PQ22hUNf6moOqS3UclDXNgouXPQQelNWxUFPSjsF9 P49Aa/EfDp7U1LJQn5HGwY2eAgKf++0M2H29PLyssGG4bRluO/X9NweP0iownLp2B0PD21IE 5SnvMMjFrwlU+bowOOVMBn5dr0bQnt7Nw8mzAzzkmtMRpJ7MYsHiCYPBn3lk2UJa1dXLUIvz IC3rt7H0cYFI7+W08NRS3sRTmxxHnY5gevWBF9P8bz6OyoWnCZW/WXl6prsB0x6Xi6e1FwZZ 2tGQjTcFblct3iMZ9PGScc6SCFVkerMLHTCrE+QrfBJ6rzqDlApRmC9+uNKI/CbCdLGxcYDx O0CYKjrTPnB+M0KXSrzmWu33OGGzaG7uHblhhSDxa3PTiNXCAtFeauX/dU4Ri25XjOTK4fxc zznit0YIE58l55PzSGVDowpRgD4mPlqnN4SFmKIiE2P0CSG790fLaPhb7EeHMkpQn3tNJRIU SDtavUz5UdJwunhTYnQlEhWMNkB95FeHpFHv0SUekoz7dxnjDJKpEgUqWO0E9botUoRG2KuL laIk6YBk/L/FCuXEJDRj+6SwrARuZVxE6sx3s8dXP3UvHrfTcrjaHJT0aKt0eaXDXY/zwg93 BpNIb/nkT9McX0J3r3rVZs98GC44l7QGquct3RdiDbgZkd8fq7z0eJtn8Af2eo6XlJzOPdpS mlKz1prTTePHrNrXUbvj5/yi8JyoToNNtqzXrhBSlhcaXFrWFKkLDWaMJt1fHuldUikDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dma fence waits. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 8aa8f8cb7071..76dba11f0dab 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -16,6 +16,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -783,6 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.task = current; list_add(&cb.base.node, &fence->cb_list); + sdt_might_sleep_start(NULL); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -796,6 +798,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); if (!list_empty(&cb.base.node)) list_del(&cb.base.node); @@ -885,6 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, } } + sdt_might_sleep_start(NULL); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); @@ -899,6 +903,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); __set_current_state(TASK_RUNNING); From patchwork Wed Mar 6 08:55:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583578 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 3266E60DD6; Wed, 6 Mar 2024 08:55:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715346; cv=none; b=NCMfy2KVusQuPJ9GiruymgiBeLj0tbogfYC34AmQbM/LG/UXUcF9vU6dDLoF616GMyyyhRrH2pZgAsRKTLpRRampiGLfclE0hc4a/i0yoDnoUGph1p6Wi1Ai2raB2Hi+G5comXah5kTBDNzjEGfgTxo9iu7js3pyUAfG7yOpjmQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715346; c=relaxed/simple; bh=LIy0pGzgh1KfCx3V9I4vVrDFzUkgQh3CEYezt1V8iYA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=p+8aLvSXbX1Hcf3zFC2ZAh291uYySIgyvgaW1ntk08FAvO6Pk5xSRmzitu7QSAr8rSX8Bl+5GECwNd7fnEg7MNDDwqBKr/DlhwN8Dn7LvNFbaoetxXFKvuB6z7qIZ10kb76GCRpOZPuM2/MheZ40N5dCSRHdIox1aTfp6wN4lw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-28-65e82f7e8870 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 16/27] dept: Track timeout waits separately with a new Kconfig Date: Wed, 6 Mar 2024 17:55:02 +0900 Message-Id: <20240306085513.41482-17-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0wTaRiG/f+Z+Weo1oxV44gXaxoNiUQEj59ZNcaLdZJV16h4ARpo7ESq gKTVKroaDpW4KAeJ2BUQKZJaAQGnCB5AKwpyiFilKioQJUSpFEjQEgpEBdSbL0/e981z9XGU Smb8OV3sYUkfq4lWEwWt6J9hWXpqWa8UnJS1As6fCwbv1zM05JWXEnCWlSAorUzE4K7fDK+H PQjGnj6jwJztRGD50ElBZUMXglpbEoG2npng8g4SaMo+SyD5ajmB533jGDouZmEokbdCS2Yh BofvEw1mN4FcczKeOL0YfNZiFqwJi6HblsPC+IcQaOp6xUDt20C4lN9BoKa2iYaG290Y2u7m Eegq/c5AS0MjDc7zaQzcGCgk0DdspcDqHWThhaMAQ4VpQpTy5RsDT9IcGFKKbmJwvbmH4P6Z 9xjk0lcEHnk9GOxyNgWj1+oRdKf3s3D6nI+F3MR0BGdPX6TB1LEKxkbyyMa14iPPICWa7EfF 2uECWmwuFMQ7OZ2saLr/lhUL5COi3bZEvFrjxqJlyMuIcvF/RJSHslgxtd+FxYHWVlZs/H+M FntcZrzdP0yxTitF64ySftmGSEVUfZ+dxPVGHjO7uQR0a0sq8uMEfqWQX12Gf/P1RB8zyYQP ENrbfdQkz+EXCva0j1M5xXsUQlHrX6mI42bzO4TOd4bJmOYXC1W+qqmJkl8tFOUWMT+Vfwgl FY4pjd9EnjGQQSZZxa8SniZbyM/Nd04YSf+1ny88tLXTmUhZgKYVI5Uu1hij0UWvDIqKj9Ud C9p3KEZGE59kPTkefhsNOXfWIZ5D6hnKjX6fJBWjMRriY+qQwFHqOcp/R3sklVKriT8u6Q9F 6I9ES4Y6tICj1fOUy4ePalX8fs1h6aAkxUn63y3m/PwTUODuCFTnPzd0kXHbG9OzHTWv3YPB jSFf/34QpHG93J2ZHVaxJzT8zwVZEZXLN7w4kTQrI7BMG9BcXe152X88wDZ79HHUrrbuODlo zV152/TPzaEpCXth03pH2LVFGVbT+JMLzBdjXw1qqQo2XXY1hX90FltC9Vf+ORDgGzJoOx7n jKhpQ5QmZAmlN2h+AO/k9ZdFAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRiA+87lO8fV6rSkDoZUi1C6uISMtwtRZHmICokiqh8565AjNdnM shvaTLyX1lzlBS9jzbtuWXaZiebSIrO0i6WiIuXSJpmTllqp0Z+Xh+d9eX69LCnT0R6sKjxS VIcrQ+VYQkn2btKuuaQYENfW5HpAespacI4mUJBdUYqhtbwEQendWALsjQHwfmwIwfjLVyTo da0I8nu7SLhr60ZgNV3G0NY/F9qdwxiadckYtIUVGF4PThDQmZlBQIl5D7y4VkBAnesLBXo7 hiy9lpgaAwS4jMUMGGNWQJ/pNgMTvb7Q3P2OhoacZhqsH1fBrdxODI+tzRTYavoIaHuYjaG7 9A8NL2xNFLSmp9JQ5ijAMDhmJMHoHGbgTV0eAZVxU7X4H79peJZaR0C8oYqA9o5HCGoTeggw l77D0OAcIsBi1pHw604jgr60bwxcSXExkBWbhiD5SiYFcZ1+MP4zG2/dKDQMDZNCnOWMYB3L o4TnBbzw4HYXI8TVfmSEPPNpwWJaKRQ+thNC/oiTFszFiVgwj2QwQtK3dkJwtLQwQtPNcUro b9cTgYsPSzYfF0NVUaJasSVIEtI4aMERA0Fn9XY2BlXvTkJuLM+t44tiXfQ0Y86L//DBRU6z O7eUt6R+nvEkNyThDS07kxDLLuD28V2fNNOa4lbw91z3Zk6k3HrekGWg/yWX8CWVdTMZtyl/ 1XEVT7OM8+NfavPxNSTJQ7OKkbsqPCpMqQr189GcDIkOV531OXYqzIymnsV4cSK9Bo22BdQj jkXyOdKtbl9EGa2M0kSH1SOeJeXu0gu/+kWZ9Lgy+pyoPnVUfTpU1NSjxSwlXyTddVAMknEn lJHiSVGMENX/twTr5hGDFtiu1wTmN02aDpXceeu5wTRfgaM37YKvktH73eere6qr2hQW72D0 PN1nY6185+qFByIzv3vNcSzbnjiv2PPSjidF/jt6AvYvD7T1d+QkKzr8a4Ojgi9Ycx9NGirL rdt6g2c7tPE6+820I94VBU+lD6W9VSmZtOJ7ubTzxpLGMueInNKEKH1XkmqN8i+cS3f6KAMA AA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Waits with valid timeouts don't actually cause deadlocks. However, Dept has been reporting the cases as well because it's worth informing the circular dependency for some cases where, for example, timeout is used to avoid a deadlock but not meant to be expired. However, yes, there are also a lot of, even more, cases where timeout is used for its clear purpose and meant to be expired. Let Dept report these as an information rather than shouting DEADLOCK. Plus, introduced CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT Kconfig to make it optional so that any reports involving waits with timeouts can be turned on/off depending on the purpose. Signed-off-by: Byungchul Park --- include/linux/dept.h | 15 ++++++--- include/linux/dept_ldt.h | 6 ++-- include/linux/dept_sdt.h | 12 +++++--- kernel/dependency/dept.c | 66 ++++++++++++++++++++++++++++++++++------ lib/Kconfig.debug | 10 ++++++ 5 files changed, 89 insertions(+), 20 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index ca1a34be4127..0280e45cc2af 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -270,6 +270,11 @@ struct dept_wait { * whether this wait is for commit in scheduler */ bool sched_sleep; + + /* + * whether a timeout is set + */ + bool timeout; }; }; }; @@ -453,6 +458,7 @@ struct dept_task { bool stage_sched_map; const char *stage_w_fn; unsigned long stage_ip; + bool stage_timeout; /* * the number of missing ecxts @@ -490,6 +496,7 @@ struct dept_task { .stage_sched_map = false, \ .stage_w_fn = NULL, \ .stage_ip = 0UL, \ + .stage_timeout = false, \ .missing_ecxt = 0, \ .hardirqs_enabled = false, \ .softirqs_enabled = false, \ @@ -507,8 +514,8 @@ extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, con extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); -extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); -extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn, long timeout); extern void dept_request_event_wait_commit(void); extern void dept_clean_stage(void); extern void dept_stage_event(struct task_struct *t, unsigned long ip); @@ -558,8 +565,8 @@ struct dept_task { }; #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_copy(t, f) do { } while (0) -#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) -#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); } while (0) #define dept_request_event_wait_commit() do { } while (0) #define dept_clean_stage() do { } while (0) #define dept_stage_event(t, ip) do { } while (0) diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h index 062613e89fc3..8adf298dfcb8 100644 --- a/include/linux/dept_ldt.h +++ b/include/linux/dept_ldt.h @@ -27,7 +27,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_wait(m, LDT_EVT_L, i, "lock", sl, false); \ dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ } \ } while (0) @@ -39,7 +39,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ else { \ - dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ } \ } while (0) @@ -51,7 +51,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ } \ } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 12a793b90c7e..21fce525f031 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -22,11 +22,12 @@ #define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) -#define sdt_wait(m) \ +#define sdt_wait_timeout(m, t) \ do { \ dept_request_event(m); \ - dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) +#define sdt_wait(m) sdt_wait_timeout(m, -1L) /* * sdt_might_sleep() and its family will be committed in __schedule() @@ -37,12 +38,13 @@ /* * Use the code location as the class key if an explicit map is not used. */ -#define sdt_might_sleep_start(m) \ +#define sdt_might_sleep_start_timeout(m, t) \ do { \ struct dept_map *__m = m; \ static struct dept_key __key; \ - dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__, t);\ } while (0) +#define sdt_might_sleep_start(m) sdt_might_sleep_start_timeout(m, -1L) #define sdt_might_sleep_end() dept_clean_stage() @@ -52,7 +54,9 @@ #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait_timeout(m, t) do { } while (0) #define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start_timeout(m, t) do { } while (0) #define sdt_might_sleep_start(m) do { } while (0) #define sdt_might_sleep_end() do { } while (0) #define sdt_ecxt_enter(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 8ca46ad98e10..1b8fa9f69d73 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -739,6 +739,8 @@ static void print_diagram(struct dept_dep *d) if (!irqf) { print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + if (w->timeout) + print_spc(spc, "--------------- >8 timeout ---------------\n"); print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); } } @@ -792,6 +794,24 @@ static void print_dep(struct dept_dep *d) static void save_current_stack(int skip); +static bool is_timeout_wait_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + + do { + struct dept_dep *d = lookup_dep(fc, tc); + + if (d->wait->timeout) + return true; + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + return false; +} + /* * Print all classes in a circle. */ @@ -814,10 +834,14 @@ static void print_circle(struct dept_class *c) pr_warn("summary\n"); pr_warn("---------------------------------------------------\n"); - if (fc == tc) + if (is_timeout_wait_circle(c)) { + pr_warn("NOT A DEADLOCK BUT A CIRCULAR DEPENDENCY\n"); + pr_warn("CHECK IF THE TIMEOUT IS INTENDED\n\n"); + } else if (fc == tc) { pr_warn("*** AA DEADLOCK ***\n\n"); - else + } else { pr_warn("*** DEADLOCK ***\n\n"); + } i = 0; do { @@ -1563,7 +1587,8 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) static atomic_t wgen = ATOMIC_INIT(1); static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep) + const char *w_fn, int sub_l, bool sched_sleep, + bool timeout) { struct dept_task *dt = dept_task(); struct dept_wait *w; @@ -1583,6 +1608,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_fn = w_fn; w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; + w->timeout = timeout; cxt = cur_cxt(); if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) @@ -2294,7 +2320,7 @@ static struct dept_class *check_new_class(struct dept_key *local, */ static void __dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, - bool sched_sleep, bool sched_map) + bool sched_sleep, bool sched_map, bool timeout) { int e; @@ -2317,7 +2343,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, if (!c) continue; - add_wait(c, ip, w_fn, sub_l, sched_sleep); + add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); } } @@ -2354,14 +2380,23 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, } void dept_wait(struct dept_map *m, unsigned long w_f, - unsigned long ip, const char *w_fn, int sub_l) + unsigned long ip, const char *w_fn, int sub_l, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (dt->recursive) return; @@ -2370,21 +2405,30 @@ void dept_wait(struct dept_map *m, unsigned long w_f, flags = dept_enter(); - __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false, timeout); dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_wait); void dept_stage_wait(struct dept_map *m, struct dept_key *k, - unsigned long ip, const char *w_fn) + unsigned long ip, const char *w_fn, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (m && m->nocheck) return; @@ -2430,6 +2474,7 @@ void dept_stage_wait(struct dept_map *m, struct dept_key *k, dt->stage_w_fn = w_fn; dt->stage_ip = ip; + dt->stage_timeout = timeout; exit: dept_exit_recursive(flags); } @@ -2441,6 +2486,7 @@ static void __dept_clean_stage(struct dept_task *dt) dt->stage_sched_map = false; dt->stage_w_fn = NULL; dt->stage_ip = 0UL; + dt->stage_timeout = false; } void dept_clean_stage(void) @@ -2471,6 +2517,7 @@ void dept_request_event_wait_commit(void) unsigned long ip; const char *w_fn; bool sched_map; + bool timeout; if (unlikely(!dept_working())) return; @@ -2493,6 +2540,7 @@ void dept_request_event_wait_commit(void) w_fn = dt->stage_w_fn; ip = dt->stage_ip; sched_map = dt->stage_sched_map; + timeout = dt->stage_timeout; /* * Avoid zero wgen. @@ -2500,7 +2548,7 @@ void dept_request_event_wait_commit(void) wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(dt->stage_m.wgen, wg); - __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: dept_exit(flags); } diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 9602f41ad8e8..0ec3addef504 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1312,6 +1312,16 @@ config DEPT noting, to mitigate the impact by the false positives, multi reporting has been supported. +config DEPT_AGGRESSIVE_TIMEOUT_WAIT + bool "Aggressively track even timeout waits" + depends on DEPT + default n + help + Timeout wait doesn't contribute to a deadlock. However, + informing a circular dependency might be helpful for cases + that timeout is used to avoid a deadlock. Say N if you'd like + to avoid verbose reports. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT From patchwork Wed Mar 6 08:55:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583579 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0FC35627ED; Wed, 6 Mar 2024 08:55:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715347; cv=none; b=hyEDZFclGJzBtjn2uniiwl0coySOHrxt5HotC68X7LpmLgudCfypjFV3RNlzotN3PaKRqzWS0JHWT+qPGIuw/sydVej9tQbMcyL6pzrIrVJeoDyBqb3pAka1xbxXnDe7PVPuFbW8/n1D6Tclq61JoWCSgPBQGYkqB5EGH+wneYk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715347; c=relaxed/simple; bh=TSPxdh3H9mjhA/wDLLlJUz6YZTY7hYg5ulgpaHBJJSM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=GqiC6+XbGXCb1SU1XZe3RCyuI2UM2O3+C4mgeeJ0dwIs6mtGWEAvp7dHnQp0j2zSH+v+qrNHddS3Bt8TKWFU62NVYTeZImpfYu2ZSpjwkRqMZ3esGpxyqdJzDNNA/Z1qJ2/lsQ7PGo7mCmyalxwMdTpQzonm02bynHxL74fb1Uo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-38-65e82f7efe45 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 17/27] dept: Apply timeout consideration to wait_for_completion()/complete() Date: Wed, 6 Mar 2024 17:55:03 +0900 Message-Id: <20240306085513.41482-18-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUhTYRTHfZ57d+91tLoto5sFyUyEotTKOIJEn+omCEbYB/swR15y5Uts ahlZmi+kpmamlprMqWu6lbZZWDkxTctFNl8wK5My8YWmkrqVKdVW9OXw43/O//fpMITULPJm lAlJgipBESejxKR4do1216WAaSHQoAuB4muB4Fi6SkJVk5EC230DAmNLBoaZ7sPw1mlHsPL6 DQHlpTYENZ8/EtDSM4bAor9CweDEWhhyzFPQW5pPQWZtEwX9X1cxjJbdwGAwhcOr61oMHctT JJTPUFBZnoldYxrDsq6RBl26H4zrK2hY/RwEvWPDIrC83wm3q0cpaLP0ktDTOo5h8EkVBWPG 3yJ41fOSBFtxgQjuzWkp+OrUEaBzzNMw0KHB0JzlEuUs/hLBi4IODDl1DzAMvXuKoP3qJwwm 4zAFXQ47BrOplICfd7sRjBfO0pB9bZmGyoxCBPnZZSRkjQbDyo8q6mAI32WfJ/gs8zne4tSQ vFXL8Y8rPtJ8Vvt7mteYknmzfgdf2zaD+ZoFh4g3NeZSvGnhBs3nzQ5hfq6vj+Zf3loh+Ymh chzhHSUOjRHilCmCKuBAtDjWOHCbPlstPm/O7cfp6BOThxiGY/dxufNkHvL8i9OGfuRmivXn RkaWCTd7sT6cuWBS5GaCtYu5ur5D7uoGNpprehDijknWj+vLzKbdLGH3c1PWYvxPuY0zNHf8 1Xi68qK5IsrNUjaYe51Z42Kx6+Y3wz1eqkb/Cpu5Z/oR8jqSaJBHI5IqE1LiFcq4fbtjUxOU 53efTIw3Idcr6dJWT7SiBduxTsQySLZGctBzSpCKFCnq1PhOxDGEzEty8eeEIJXEKFIvCKpE uSo5TlB3oi0MKdsk2eM8FyNlTymShDOCcFZQ/d9ixtM7Hd2Rby85nRhZ0tAQldYUVhvgc8TL 4/v6t+r6VsChtfqtlYWDz8v8GiLCZZOakozjthWn6Wjao41fKvTdF/da5FZ/+Y8Dd8KFh6PZ kYvYGuPbkndGPm6fvHw0eFhelL/0zbruZv1iGPvNRyY80aVuDAs83RlR4Gcb9qn07dJuKPog I9WxiqAdhEqt+APBghUaRgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRyG+//P1dHiMIUOBhXrbpRKGb8yonsnu9CHqOhDOfSUw6mxmWVY mFPzWio5S1dsq5bpnLUVaKktTUsz0xQzUVGTnHmDcpJpF1f05eXhfeH59LKETEd5s8rIaFEd qVDJaQkpORioXXPJ1yn6NejkkJ3hB66JFBL0pRYamq3FCCyPL2MYqt0DHyZHEEy/fUdAXm4z AmNfNwGP63oQVBYm0NA6MA/aXOM01Oem06C9U0pDy/AMhi5dDoZi2wF4k2XC4JgaJCFviIaC PC2eDSeGKXMRA+b4ZdBfmM/ATJ8/1Pe0U1Bzq56Cys7VcPN2Fw0VlfUk1JX1Y2h9qqehx/Kb gjd1r0lozs6koGTMRMPwpJkAs2ucgfcOA4aHibO25G+/KHiV6cCQfPcRhraPzxBUpfRisFna aahxjWCw23IJ+HG/FkH/1VEGkjKmGCi4fBVBepKOhMSuAJj+rqe3bhJqRsYJIdF+TqicNJBC g4kXyvO7GSGxqpMRDLazgr3QR7hTMYQF41cXJdiKUmnB9jWHEdJG27Aw1tTECK9vTJPCQFse PrTguGRzqKhSxohq3y3BkjDL+5vMmduS8/bUFhyPetk05MHy3HreWdyC3ExzK/iOjinCzV7c Yt6e+ZlyM8GNSPi7TbvTEMt6csF86aON7prklvFN2iTGzVJuAz/YkI3/KRfxxQ8dfzUes/21 sWu0m2VcAP9Wa6SzkMSA5hQhL2VkTIRCqQpYqwkPi41Unl8bEhVhQ7NvMV+cyS5DE617qhHH Ivlc6VaPQVFGKWI0sRHViGcJuZc07seAKJOGKmIviOqok+qzKlFTjRawpHy+NOioGCzjTiui xXBRPCOq/6+Y9fCORymdnoGvrvTtG7YW7IjOOt4RJd35YXd+kL5/07bU9d6yQy+Dcog1jTsi rC9qk0/s3zC6cMmDI8mHv18/Nr2XcZbPUZV82pmQQD05ZaLGtju/RC0t8l1n4Iy6id53w7tK VoUst1asrO3eZXXK9Z1LGxtNz5dfujf3p6PdboyLbmnmcrbISU2Ywt+HUGsUfwBNhV94KQMA AA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 4 ++-- kernel/sched/completion.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index bd2c207481d6..3200b741de28 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -41,9 +41,9 @@ do { \ */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) +static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start(&x->dmap); + sdt_might_sleep_start_timeout(&x->dmap, timeout); } static inline void complete_release(struct completion *x) diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index 3561ab533dd4..499b1fee9dc1 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -110,7 +110,7 @@ __wait_for_common(struct completion *x, { might_sleep(); - complete_acquire(x); + complete_acquire(x, timeout); raw_spin_lock_irq(&x->wait.lock); timeout = do_wait_for_common(x, action, timeout, state); From patchwork Wed Mar 6 08:55:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583581 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 59EFD62814; Wed, 6 Mar 2024 08:55:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715347; cv=none; b=PwUDtci8aZdvL6k7YzbjifEEEvrbdZTelpDkiLPawXTuxlNMLCyG2/wPFQMDO2yJHfkTeQ2LfxCedJBEkx4M80gLX3dQdeAHPF/yzazCPuHVrg0z8xka5XaNjgj5c8ZGP4FVxGdR7RBDl/QNDQeXNapJH6e0oaledOjAcj8P+Po= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715347; c=relaxed/simple; bh=9o9SI0ONdb1WbiIWXrCSb5nX53qir4UGX5Gt0IQgpYM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=DKoe08a0o9UJECfKJb1Fqnh5s7lsAtGm7hXXKVzGKhkne8WZtmXb6X3RTJmPbyNEmku/WiowoZWQil+lq1+11omjHNEEWLBvm1vlLdLWgdQJDHZluejk5LC+S6xJk17Yqd3CDtX+gkku3h/KbKoyQ80UTv9pjP6FzFu/kLOBp6Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-48-65e82f7e60ef From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 18/27] dept: Apply timeout consideration to swait Date: Wed, 6 Mar 2024 17:55:04 +0900 Message-Id: <20240306085513.41482-19-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSXUyTZxTHfZ73sw01b4qJ7/BCU4Mk6hAUl7OEEEycPBeaGIk3aNROXqWx gCmCQtwEKYQVEMXQqiCBipVAJ9AyFbGmtqGCRMYGccwgmaQROvlQoMUCbhbUm5Nf/uefX87F 4SllOxPBazLOSLoMtVbFymn5ZJj52wvbxqWYm/couFIWA/65EhpqWqws9N9tRmBtL8Dg60qC vwITCBaf/06BqaofQf3rVxS0e0YQOBovsjDgXQ2D/mkWeqpKWSi81cLCH2+XMAwbKzE02/ZB 72UzBmdwjAaTj4VqUyEOjXEMQUsTB5b8SBhtvMHB0utY6Bl5wYDj5Ra4XjvMwiNHDw2eB6MY Bh7WsDBi/Z+BXk83Df1Xyhn4dcrMwtuAhQKLf5qDP511GFr1IVHx7H8MPC13YihuaMMw+Hcn gscl/2CwWV+w4PZPYLDbqihYuNOFYPTSJAdFZUEOqgsuISgtMtKgH94Jix9q2MTviXtimiJ6 +1niCNTR5JlZJB03XnFE//glR+ps2cTeuJnceuTDpH7GzxBb0y8ssc1UcsQwOYjJVF8fR7qv LdLEO2jC+yNS5PGpklaTI+m2JRyTp7V4C9DpWfacqauByUfPGAOS8aIQJ5ZWWqiv3NvawS4z K0SJQ0PBlXyNsEG0l79Z6VPChFxs6NtjQDwfLvwg+qoPLse0ECmW2bvxMiuE78RAWxX9Wble bG51rmhkobxiqmJFrxR2is8L60MsD3VKZeIdr/PLPd+ITxqH6MtIUYdWNSGlJiMnXa3RxkWn 5WZozkUfz0y3odAvWX5aOvQAzfQnu5DAI1WYIlE2JikZdU5WbroLiTylWqM4v+CVlIpUdW6e pMs8qsvWSlkutI6nVWsV2wNnU5XCSfUZ6ZQknZZ0X7eYl0Xko/W1xYnBBMOJ4w1U+Km1W5Pn o2TvxiPCXTMfYzNv+lJ+PnA4LjzTEG/f9O/Vk8rftOvcbndUpKdkr6Qn4kFPgbbsSHLemx2J smu1eSey62+39Z6Pqdgt3R83Homs/XF+ZM/ed9HGOavPtUudZDZ2bgwY7toTtiTdV3UOODpS ht7nqeisNHXsZkqXpf4EpmvrJEcDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTWRiGPWdmzgzVmtnS7I41Rm1EsqIg0Zovq1Fjgh41GhOjRv5AtROp XNMqwkZ3QaphkaqYQFXQcGkqgWqh5YfuCkEQ5KKA28YLQbKwiFQQjFqyFaICZv+8efJ+X55f r8CoijiNYEw5IZtS9ElaomAVezfmrPktakReO/jvz1CQvxYCn3JZKHE5CfTcqUbgrMvG4G/Z Ac8nxxBMPelmwFbYg6Bs4BUDda39COorzxLwDi0EX2CCQHvhBQI5FS4CT0enMfQVXcFQ7d4D nZfLMTQG37Bg8xMotuXgmRjBEHRU8eDICoPByus8TA9EQ3v/Mw6ab7RzUN8bAddu9hG4X9/O QuvdQQzeP0sI9Du/ctDZ2sZCT4GVg9vj5QRGJx0MOAITPPzdWIqhxjJjO//xCwePrI0Yzttr Mfhe/oWgIfcfDG7nMwLNgTEMHnchA59vtSAYvPiOh3P5QR6Ksy8iuHCuiAVLnw6m/ishW3+h zWMTDLV4TtH6yVKWdpRL9N71Vzy1NPTytNR9knoqV9GK+35Myz4EOOqu+oNQ94crPM1758N0 vKuLp21Xp1g65LPhfYtjFZsMcpIxXTZFbY5XJLiGslHaR5Jha7FzWaiDy0MhgiSulzpr7pFZ JmK49OJFkJlltbhM8liH534YcUwh2bu25yFBCBVjJH/xgdmaFcOkfE8bnmWluEGarC1kvyuX StU1jXOakJn+0vilOb1K1ElPcsrIZaQoRfOqkNqYkp6sNybpIs2JCZkpxozIo6nJbjSzFseZ 6YK76JN3RxMSBaRdoNwa8kZWcfp0c2ZyE5IERqtWnv48JKuUBn3mr7IpNc50Mkk2N6HFAqv9 SbnrkByvEo/pT8iJspwmm/6/YiFEk4Uipm+kRf2o686N9/deW7Ypltv2uPuI8/VrO7v+97r5 4fk7UfOG7AxvqE/9dngle9b1Rb2iJ/Vg4vCSmNVShWzYrNn9MKxo+fO4mB+O26fiLYtMh2n3 A+vB1dvzNO/3rtjo2m+IvGU9FO3tiH77ONgyMjDCb3GH6radWjd/XDN6zFCgZc0J+uhVjMms /wZTd+HgKQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to swait, assuming an input 'ret' in ___swait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/swait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/swait.h b/include/linux/swait.h index 277ac74f61c3..233acdf55e9b 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -162,7 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); struct swait_queue __wait; \ long __ret = ret; \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ From patchwork Wed Mar 6 08:55:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583580 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8EF16629ED; Wed, 6 Mar 2024 08:55:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715347; cv=none; b=degqj1WglTv7yJGU6asISEY9GgW9ftgAFRuIeKYfTCb1TFCCt5eex5wYg8m2tnaLIFOoeFXBSEW2WdAhXxu8EGbHSleCLKfBXSMWWZomz2P0rF228790BIEZMVIB6lnHHf++9nIshVyolrOnbHDa7E32ohKnCu3Bqoqv/Pse2TY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715347; c=relaxed/simple; bh=RHlJ2GxVpEp+l17KtAmYPgKdeN+t8Sb6ly9lphI8Ui8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=pMbhhEuO9878QIUvmk2KjJQKyAdhwI54ujVEC179APKfOwv0YGsEXXTAjNDlXU/xt9L8QhssXQW9WEatbYnYFAhaIfpUsPzkm1thDPZjTkjVvxqJmbAKEsvXEmKnCqRc2Ax+dtKG7kMhZzjrxFtE+/limGFJ2tEkiNYRiuydj8c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-58-65e82f7e1343 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 19/27] dept: Apply timeout consideration to waitqueue wait Date: Wed, 6 Mar 2024 17:55:05 +0900 Message-Id: <20240306085513.41482-20-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzH+35/j3ccP1fmpxi7aSzTg3n42Kwxk69NxvzHLEc/Oq7YnSIy pQuVHkQX1bhLO7fK04VJDzu1npj0NJKKWh6aq4hr6hruMv989tr78/m8Pv98eEr5iPHlNTHH JV2MWqti5bR8eKZ5xdmgL1JwkmsGXL4UDM6fF2kovFfGQuvdUgRlD5MwDNVvgTfjDgSul68o yMttRWDu76XgYUMfgmrrORY6BmdBp3OUhebcdBaSb91joe3rFIYeYw6GUls4vMguwmCf+ExD 3hALBXnJ2F2+YJiwlHBgSfSHAWs+B1P9IdDc95qB6u7lcP1GDwtV1c00NDwZwNDxtJCFvrI/ DLxoaKKh9XIGA3dGilj4Om6hwOIc5aDdbsJw3+AWnf/xm4HGDDuG88UPMHS+rURQc/EDBlvZ axbqnA4M5bZcCiZv1yMYyBzmIOXSBAcFSZkI0lOMNBh6VoPrVyG7YR2pc4xSxFB+glSPm2jy vEgkFfm9HDHUdHPEZIsl5dYAcqtqCBPzmJMhtpJUltjGcjiSNtyJyUhLC0earrloMtiZh3f4 7pavj5S0mjhJFxS6Tx5lrmmijj3iThZf6EaJyMCmIRkvCqvEqql06j/fHX2HPcwKS8Wuronp 3EdYLJZnfGI8TAkOuVjcEuZhb2G7+Ph9FudhWvAXh/uKkIcVwhrR+s3K/HMuEkvv26c9Mnee NZI1fVcprBZfJpvdLHfP/OHF1Mrv3L+F+eIzaxedjRQm5FWClJqYuGi1RrsqMCo+RnMy8MDR aBtyP5PlzNSeJ2isdVctEnikmqnYIPssKRl1nD4+uhaJPKXyUSRMDkpKRaQ6/pSkOxqhi9VK +lrkx9OqeYqV4ycilcIh9XHpiCQdk3T/u5iX+SaihVfEx7JmubnOD9rb+jXeCRVeSZP1Qlf+ jNP81br6OdduZif8fO/tXN7YA15n5hqllgUubRi3ZtsSx7bwwxvXLvsU4LV/X2r4fH1b5sC5 ZcaakI8jvWMmlZ7fG0bijFs3+6bE5kfsPIjTFmeHbra3364Qr6f4BRhyrJsKGmeH5qpofZQ6 JIDS6dV/AdVQUNRIAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXSe0hTcRQH8H6/e+/vztHitoQuShSjiLQnaB0sKgLz0osiIymobnXLlS82 M40yy/U0X4UtU8sXc7n12ix6OBuaNpPUUixfq8Qy8bEoJ5lSbUX/HD6cA99z/jgySqln/GTq mHhJEyNGqYiclm9anrogedEXaXHexyWQfWkxuEfO05B/10yg+Y4JgbniFIb+2jB4OzqIYPxV EwX6nGYERR+7KaiocyKwGU8TaOmdAq1uF4H6nDQCqSV3CbwemMDQdfUyBpNlIzRkFWOwj/XR oO8nkKdPxZ7yBcOYoZwFQ8oc6DFeZ2HCc0S9s42BmoJ6BmwdgZB7o4tApa2ehrpHPRhanuQT cJp/M9BQ56ChOTudgdvDxQQGRg0UGNwuFt7YCzHc03nSzn7/xcCLdDuGs6X3MbS2P0VQdf4D Bou5jUCNexCD1ZJDwc+yWgQ9GUMsnLk0xkLeqQwEaWeu0qDrCobxH/lkdYhQM+iiBJ31qGAb LaSFl8W88Ph6NyvoqjpYodByRLAaA4SSyn4sFH1zM4Kl/AIRLN8us8LFoVYsDDc2soLj2jgt 9Lbq8Wb/HfIV+6UodYKkWbRyjzyyqMpBxT1gE0vPdaAUpCMXkY+M54L4O65O7DXh5vLv3o1R Xvtys3hr+mfGa4oblPOljWu9nsZt4h++z2S9prk5/JCzGHmt4Jbyxq9G5l/mTN50z/43x8fT zxzO/LtLyQXzr1KLSBaSF6JJ5chXHZMQLaqjghdqD0cmxagTF+6LjbYgz7sYTkxkP0IjLWHV iJMh1WTFap8+ScmICdqk6GrEyyiVr+L4z15JqdgvJh2TNLG7NUeiJG018pfRqumKddulPUru oBgvHZakOEnzf4plPn4p6KQYXRDWvqZvin5SeIBj/tOyypDIJkeftHPDLecVe4Ttc9Wy9U/2 ZuwyW+PKbmZdc22/3bCODX0eSrY2Bj1LCxenblmzoeDDs87jESMVvf6rZuQ2X1hvemlXJX4q C42PHZg9zxRU4Aw8EPijsyG52/mpZEa7NthyqDY3YptzegROXquitZHikgBKoxX/AKRsMwwq AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to waitqueue wait, assuming an input 'ret' in ___wait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index ebeb4678859f..e5e3fb2981f4 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -304,7 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ From patchwork Wed Mar 6 08:55:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583582 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0411B62A17; Wed, 6 Mar 2024 08:55:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715348; cv=none; b=r+gMe/vm0zuoYOHbQl9KgiynPpunWPQWiPWekvpHku7vLbCh/KiAyjM07lyF0VzkEgZea/ytJte5o5Ron0pYa+Y2JCvwTSwE2Th1q4SU+MZQPofFqOuMCWxv5wO41S3/kZb6KbgJ1CICnKvDQiUh9e3/cXOYY50zCLoq+MjyOQY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715348; c=relaxed/simple; bh=oVu1M/z+Be6zJoonqEicwRg8jn1j0IbzUk3u5AE/er0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=fH9fASKNYv+lxHAmtKJaou/E7xXbGpoxTbzkDEA2oPLGDDMijgS3y7ianruMWnIulu5n12QXXmb9igflf76/47CG0Vgy0tq5WSuCuaI3REB4cVZm8iQyFovi1KTmxVO+F2+xjMIJ4Tk5yMuSwi8w5c6skp5klr5XX//gnfoHsmA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-69-65e82f7ebc4d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 20/27] dept: Apply timeout consideration to hashed-waitqueue wait Date: Wed, 6 Mar 2024 17:55:06 +0900 Message-Id: <20240306085513.41482-21-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAz2SeUiTcRjH+/3ec6PV2xR6M+hYRGF3dDxClFHUW1BIxx8lVCNfcqQmM68o UqcdHrNLV7pCp62hlrat25UZWrO05cQspqXZIc1p1kzTjmnUPw8fvs/3+fz1sITcSgWwqqiD ojpKGaGgpaS0e6xh3tEFn8SFmu8T4HTmQvB+O0GCvryMBse1UgRl1mQMXTXr4WW/G8FQ/XMC dDkOBIXtrQRYa9sQ2EwpNDg7x0GTt4cGe04GDZqichpefB7G4Mo9g6HUvAmenjJgqBr8SIKu i4Z8nQb7xicMg8YSBoxJM6HDlMfAcPsisLc1U2B7PQcuXHLRUGmzk1B7uwOD866ehray3xQ8 rX1CguN0FgVXPQYaPvcbCTB6exhorCrAUJHqEx37+ouCx1lVGI4VX8fQ9Ooegvsn3mIwlzXT 8MjrxmAx5xDw40oNgg5tNwNpmYMM5CdrEWSk5ZKQ6loKQwN6OjhIeOTuIYRUS7xg6y8ghToD L9zJa2WE1PuvGaHAHCtYTIFCUWUXFgr7vJRgLjlJC+a+M4yQ3t2EBU9DAyM8OT9ECp1NOhwS sFO6IkyMUMWJ6gUr90jDn2nKUbSWSeh1ZeEk1E6lIwnLc0t4Q/Od/1xa10uMMM3N4ltaBkfZ n5vGW7I+jHYIzi3lixvWjbAft53PSNOjdMSyJDeTH3o8ZSSWccv4uq9J+K9yKl9aUTWqkfjy bE82PcJybilfryn0sdTXyZbw+T8/EH8PJvEPTS3kKSQrQGNKkFwVFRepVEUsmR+eGKVKmL/3 QKQZ+X7JeGQ49Dbqc2ytRhyLFGNlwZKPopxSxsUkRlYjniUU/rLDPzpFuSxMmXhIVB/YrY6N EGOq0WSWVEyULe6PD5Nz+5QHxf2iGC2q/20xKwlIQhtKFL155AbP/rPUKueX2Mp3zRdm33pR k9A4N3TX2pUbx8w43n419Nz53XFpHvP0gUi7ljeluHIz7TNayx07rCHPjix/ExQ8UdeqL7Ks sfm9z413RrfEr65/ZeskZVu03iC/8bEDdONOd7Jl847LKTem3zS8u8UVdznnLW/z32a9+EBB xoQrFwUS6hjlH/DytTJHAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTWRiGPWdmzkybrY5dohMNaipo1Cjqin663mP0xFuMuqvRTbTKuHQt 1bTKysYLbgsKCIKmVBENVFMJRdFCTL0UCRW0uCC7VERTiSARUC7eSkSILmD88+bJ+315fr0C o87kRgk6wz7ZaNDqNUTJKtf+bJ56OKJVnv7ONhcyTkyH4MfjLGQXFhCouepEUFB8FENb+Qp4 0t2OoLfqEQM2aw2C3MbnDBRXNCDw5P1NoLZ5KPiDXQR81hQC5ouFBP5904chkHkKg9O1Bh6m 2zGU9rSwYGsjcM5mxv3RiqHHkc+DIz4cmvKyeOhrnAG+hjoOvOd9HHieTYGzFwIE7nh8LFS4 mzDU3som0FDwlYOHFQ9YqMlI5eBKp53Am24HA45gFw//leZguGbptyV++MLB/dRSDImXrmPw P72NoOT4CwyugjoC3mA7hiKXlYHPl8sRNKV18JBwooeHc0fTEKQkZLJgCURC76dssnge9bZ3 MdRS9Cf1dOewtNIu0ZtZz3lqKXnG0xzXflqUN5levNOGae77IEdd+UmEut6f4mlyhx/Tzupq nj4408vSZr8Nrxu9RTk/StbrYmVjxMLtyuh/zIVobxp/4G0gFcejRi4ZKQRJnCU5K98yA0zE iVJ9fc8gh4jjpKLUV4M/jNiulC5VLx/gH8VfpJSEbJSMBIEVw6Xe+2MGapU4W6r8EI+/KcdK zmulgxpFf3+y8yQZYLUYKVWZc0k6UuagIfkoRGeIjdHq9JHTTLuj4wy6A9N27olxof61OA71 ZbjRx9oVZUgUkOYH1WJFi6zmtLGmuJgyJAmMJkR18HOzrFZFaeP+ko17thn362VTGRotsJqR qpWb5O1q8XftPnm3LO+Vjd+vWFCMikdbzky9d8xTFTAtm1XSOvOP1g23siISwyxJ51/Pnnhh 0t05bl/SzmHFx6La1phvOG/8dnUV2bhLT1PGj/daF6QpRiztOG1tKQ9df3pu88w6mzqM3VFv DxUMw8M1P2UsWbI6/Uj1Uu8j97bHX1/u4BaNsLt/HRcaXD7SX7f19QRb2ObYObs0rClaO2My YzRp/wfZerU2KQMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to hashed-waitqueue wait, assuming an input 'ret' in ___wait_var_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index fe89282c3e96..3ef450d9a7c5 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -247,7 +247,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ From patchwork Wed Mar 6 08:55:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583583 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id ACF216313F; Wed, 6 Mar 2024 08:55:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715349; cv=none; b=LTxq/fpP3wOcCYeOXj+9ZUUflbPhK1vKHdK+ZuLisTl0pPEZ/HRbWCxwxTu5mxmy5w60Slxi8S7VO8kVigW0H52tTVOK2vjFDIP60cuQrbplWpQxtOx1pfnEe/rDTsNyIi0kCiNo53Zvi6gPF96lcZJlSgGll3fjgkwqOUYKMe4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715349; c=relaxed/simple; bh=A0HNCh9MAJFiyY64P3s+ZnF2ZrT06e1PDF08t8YO27Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Z9JpUcH8hmQGoh1XwV09ORm8ogHa8MKEKCnEPUTsz0ckjxxpzwlHswN1pcsnaW2zoClwFocTkKxsKUOwOC1BdMJ+jVixJYxAEdskNHeWXrMT+DNwS09WiuTyf3dDvLIJ+SNn/tN50bOEriZ9IrdikeN5J/aaWLde7woS3+0CG2Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-79-65e82f7fe118 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 21/27] dept: Apply timeout consideration to dma fence wait Date: Wed, 6 Mar 2024 17:55:07 +0900 Message-Id: <20240306085513.41482-22-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa1BMYRgHcO97zp5zdllzZmvGkfvOYMalcp3HuIyZXI4PDGMMwwc2Hdpp 29hNZBilWolKqEhMt1mrlnJ2EVkSUkpKDUmlTSS6zMQuWztll/Hlmd/8n8unhyEUVokfo9ZG CjqtSqOkZKSsb1ze/BMBX4XA2tqJkHY2EBw/E0nILjZTUH+rCIHZGouh5/l6eOfsRTD86jUB men1CHLtbQRYK9sR2EwnKWjsGg9NjgEKqtPPUBCXX0xBw3c3htaM8xiKxI1Qcy4PQ7mrm4TM HgquZMZhT/mKwWUspMEYMxM6TVk0uO0LoLr9rQRsLXPh8rVWCh7aqkmoLO3E0Pggm4J286gE aiqrSKhPS5bAzf48Cr47jQQYHQM0vCnPwVAS7zlk+DEigRfJ5RgMBbcxNL0vQ/AosQODaH5L wVNHLwaLmE7A0PXnCDpT+mhIOOui4UpsCoIzCRkkxLcugeHf2dTqZfzT3gGCj7cc5m3OHJJ/ mcfx97PaaD7+UQvN54iHeItpDp//sAfzuYMOCS8WnqZ4cfA8zSf1NWG+v66O5qsuDZN8V1Mm 3uy3U7YiRNCoowRdwKo9stBfn+KoA43MkaqSZjoGFdFJSMpw7GIuf8iK/7vFflXiNcXO5pqb XYTXvux0zpL85W9OsL0yrqBundc+7Cau4dYImYQYhmRncgXWld5Yzi7lRrOcxL+T07iikvK/ lnry1P5UymsFu4R7FZfrscwzc0rKGd61kf8WJnJPTM3kOSTPQWMKkUKtjQpXqTWL/UOjteoj /nsjwkXk+SXjcfeuUjRYv7UCsQxSjpOvlnYLCokqSh8dXoE4hlD6yo8NdQkKeYgq+qigi9it O6QR9BVoEkMqJ8gXOg+HKNj9qkghTBAOCLr/XcxI/WLQpKCtxSuXJ9qfBa6xLkyaWpM2a8v+ mz8P3tuwbK3av2Mkcv28950B3U98OmpNQQU3GphfYd/2BrV8/HbhpT4/dtGFKRcPXmXGVuzQ dm2oXn5yrb+29G4o3hehyXIVZxh23t5sE0cTNt1xJ4uPZ4QFf2ANr8sub09IvWu2WyanfA52 j92mJPWhqgVzCJ1e9QeZ9LuDRwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSXUxTZxzG956P9xwau5x1oCcQozYxTlSQhOrfDwwXi32jk+AuptFFbPBY GgGxxWqNxroi6VCsGKAOUMtHKgGmeMqFE6sVIoIGRCGIBFAQP4gFEqUgwlSq8ebJL78nea4e nlYVsuG8IT1TMqbrUtVYwSgS1tlWHI9+I61sdzGQd3olBMbtDJRcrcHQfqUaQU3dCQqG72rh yYQfwXTrQxqcBe0ISgf6aKhr6kfgrfwLQ8fQj9AZGMPQUnAKg638KoZHb2co6C08R0G1vAUe nC2jwDf1mgHnMIZip42ajTcUTLmrOHBbF8NgZREHMwMx0NLfxULjhRYWvD3L4J+LvRhuelsY aLo+SEHHjRIM/TWfWXjQ1MxAe14uC/+OlmF4O+GmwR0Y4+Cxz0VBbdbsWvb7Tyzcy/VRkF1x jYLOp/UIbtmfUyDXdGFoDPgp8MgFNHy8fBfB4JkRDk6enuKg+MQZBKdOFjKQ1auB6Q8lOH4t afSP0STLc4h4J1wMuV8mkv+K+jiSdauHIy75IPFURpLym8MUKX0XYIlc9Tcm8rtzHMkZ6aTI aFsbR5rPTzNkqNNJJUbsUKzfI6UazJIxesNuRcrkCxvO6OAPN9d2c1ZUzeWgEF4UYsWegQts kLGwROzunqKDHCosFD25r756WvArxIq2jUH+WUgQH135xOQgnmeExWJFXVxQK4VV4ueiCfrb 5AKxutb3lUNmvWPUgYOsEjRiq60Un0UKF/qhCoUa0s1pOkOqJsq0L8WSbjgclbw/TUazb3Ef m8m7jsY7tA1I4JF6jjI+5LWkYnVmkyWtAYk8rQ5VHv04JKmUe3SWI5Jxf5LxYKpkakARPKOe p9y0TdqtEvS6TGmfJGVIxu8txYeEW9HepEVNf9aH3/5/zeONI31rGTDrtU7Dh/rjkTH6X1Yz 4wqt/U7yDrpI4/Vvnx+9efNSEvWyz1KuCZu+pF9uzs/e6o2djMh3xMXZR2MTw3btdHheDP70 u9xK8h2WA9Z67ZZMq5y37fJWeW5YUmLyjT+e/aY/Hz9pL4yyJuzccMf3q5oxpehiImmjSfcF 3yO57ikDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to dma fence wait. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 76dba11f0dab..95121cbcc6b5 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -784,7 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.task = current; list_add(&cb.base.node, &fence->cb_list); - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -888,7 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, } } - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); From patchwork Wed Mar 6 08:55:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583584 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id B4D2564A; Wed, 6 Mar 2024 08:55:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715350; cv=none; b=i6y3JaaGQ8WxvFwt2OYO9jTamFemfUj24HAgnbywjCBgAvt6OPcIos8YxAhVWcX0zrq+7oSrRjje71zd5UzCn/4SYg0Jvnf6LZZAnPXYckD9RTPVG5a/rikwj+e9gKXIBLemq/y1dYZBwBvTFFqyyYzGqNEgJNAvW1fMaMENeJ4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715350; c=relaxed/simple; bh=rOOuSh+1eZIFPiRqy8Z4jjiN4PmFYfqaD61RmkKO4sk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=OmGHGC5JyNIGuIxyXMT2ELhvy/CsB4eM+Nfrr3K3im95TpuDznGHMzMP15sxNCF7SrJldJYwZK52eGn7+sp3apuiosru35yTa5pz/HkU17oH5OP9g7AULj+A6li/YKB4mbMDDMzr0w5XLN+rx4yYE/dTIojaDM+oAUhr5IM3yns= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-8a-65e82f7feb03 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 22/27] dept: Record the latest one out of consecutive waits of the same class Date: Wed, 6 Mar 2024 17:55:08 +0900 Message-Id: <20240306085513.41482-23-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSaUxTaRSG/b67tlpzp0PiFU00dUskoEyEnIlL0Bj9TNwSxj+aqDdwR8oU NK2iqAQUUERB0UBRcGSzNsC4tDIwDji1bMUFGSWKBogQFCotKNrOFIjaavxz8uQ973l+HZ5S W5lgXpu4X9YnSjoNq6SV7mmloalLhuSlPW/mQd6ZpeD5mEVD8Y1qFjquVyGovn0Mg7N5PTz3 uhBMPHpMgTG/A0FpXw8Ft1t6ETSYj7PwdGA6dHpGWWjLP81CevkNFv4dnsTQXXAeQ5VlEzw4 V4bB5hukwehkociYjv1jCIPPVMmBKW0B9JsvcTDZFw5tvc8YaHgZAhd/72ahvqGNhpa6fgxP 7xSz0Fv9mYEHLQ4aOvJyGPhjpIyFYa+JApNnlIMnthIMNzP8ohMfPjHQmmPDcKLiFobOF38j uJv1CoOl+hkLjR4XBqsln4Lxa80I+nPdHGSe8XFQdCwXwenMAhoyuiNg4v9iNupn0ugapUiG 9SBp8JbQ5H6ZSP661MORjLsvOVJiOUCs5sWkvN6JSemYhyGWylMssYyd50i2uxOTkfZ2jjgK J2gy0GnEW4O3K1fEyjptkqxfsmq3Mu5xzQdmn3Pqodz+AZSGWhXZSMGLwjLxlb2c+s6Nta+5 ALPCIrGry/c1DxLmitacN0yAKcGlFCva12Ujnv9RkES3d34AaWGBWPt+TaChEiLFwquF3Dfj HLHqpu2rReHPz46cZQOsFiLER+mlflb6O5958cX1JvztYKZ4z9xFn0OqEjSlEqm1iUkJkla3 LCwuOVF7KCxmb4IF+V/JlDK5ow6NdUTbkcAjzTRVlGJQVjNSkiE5wY5EntIEqY6OD8hqVayU fFjW792lP6CTDXY0i6c1M1Q/eQ/GqoU90n75N1neJ+u/bzGvCE5D5ui89453yyPjY/45/Eum fWX9tcjNs1fvCabeeULX7Lgc3lzQ9MORq6eGyrIGK2pD+tZxRVLtmKU5h9nZchTZ3Lf+nOXb snH3f16H4Z7DScA1ak2Nj7njStHtCqmIOp56Misl6f6vw89rtkXrV8YXXnh7ZZG89kqE8cm4 6+HrhU0b2jW0IU4KX0zpDdIXRHFUWkYDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5/7lqvDkjwkUYy0MCqFjJduRFT+CQr7UFIfylMecuSltjIX XaytME1N8ZaaeJlLdGZNu6uYo5VFZil2m5ZjlSMvpW61tIsr+vLy43kefp9ejlTm0XM4dcJh SZMgxqkYOSXfukq/5NSyASm0/74fZF8IBfd4KgUl9WYGOq/WIjA3nibA9SACXnoGEUw8fUZC QV4ngvL+XhIabX0ImqvPMNDlnAHd7hEG2vPSGdBX1jPw/PMkAfb8HAJqLVvgycUKAlq9nygo cDFQXKAnps4AAV5TDQumlCBwVBexMNkfBu19PTRYL7fT0PxmMVwqtTPQ1NxOge22g4CuuyUM 9Jl/0/DE9oiCzuwMGuqGKxj47DGRYHKPsPCitYyAa4Yp27mxXzQ8zGgl4JzxOgHdr+8haEl9 T4DF3MOA1T1IQIMlj4QfVx4gcGQOsXD2gpeF4tOZCNLP5lNgsIfDxPcSZt1KbB0cIbGh4Shu 9pRR+HGFgO8U9bLY0PKGxWWWI7ihOgRXNrkIXD7qprGl5jyDLaM5LE4b6ibwcEcHix8VTlDY 2V1ARAbukq+OkeLUSZJm2dpoeeyzG2P0Qdf05EyHE6Wgh7I0JOMEfrlgvfWB9THDLxRevfKS Pvbn5wsNGR9pH5P8oFwwdmxKQxw3ixeFIc8CH1J8kHDr63rfQsGvEAqrCtl/xnlC7bXWvxbZ VJ41nMX4WMmHC0/15cxFJC9D02qQvzohKV5Ux4Uv1R6I1SWok5fuS4y3oKlnMZ2YzL6Nxrsi 2hDPIZWfYp3sk6SkxSStLr4NCRyp8lcc/+GUlIoYUXdM0iTu0RyJk7RtKJCjVAGKzVFStJLf Lx6WDkjSQUnzvyU42ZwUFDpsOmnrP/FuZUvptun6HmHvjp6Za7zW/AD5uCXg5KINxvVVWeK3 XkXupaiwnwO7364w1xW9Tr5cj0McpPGQxz5tH2XfODt100CJ/9xvvcGRjvlIt7P6uGZPYC7X kp1eZUyZ1Wj4Ul4Z4Ay0BY/eXBKxOCPGPOZqKhV3Jm6/qXuporSxYlgIqdGKfwD78fKUKAMA AA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The current code records all the waits for later use to track relation between waits and events in each context. However, since the same class is handled the same way, it'd be okay to record only one on behalf of the others if they all have the same class. Even though it's the ideal to search the whole history buffer for that, since it'd cost too high, alternatively, let's keep the latest one at least when the same class'ed waits consecutively appear. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 1b8fa9f69d73..5c996f11abd5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1521,9 +1521,28 @@ static struct dept_wait_hist *new_hist(void) return wh; } +static struct dept_wait_hist *last_hist(void) +{ + int pos_n = hist_pos_next(); + struct dept_wait_hist *wh_n = hist(pos_n); + + /* + * This is the first try. + */ + if (!pos_n && !wh_n->wait) + return NULL; + + return hist(pos_n + DEPT_MAX_WAIT_HIST - 1); +} + static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) { - struct dept_wait_hist *wh = new_hist(); + struct dept_wait_hist *wh; + + wh = last_hist(); + + if (!wh || wh->wait->class != w->class || wh->ctxt_id != ctxt_id) + wh = new_hist(); if (likely(wh->wait)) put_wait(wh->wait); From patchwork Wed Mar 6 08:55:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583585 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8EE73651AE; Wed, 6 Mar 2024 08:55:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715351; cv=none; b=i56enNiM2f2+P/XVvs8gu+XI2hGXxqCB1rk/Zm65IkAZ9HcggZS1dSszrrxdh7t221dpeHcXRD09ckB1QAVPUJ+fltwOxNJ/t8ke07d2Jq+/C4zmXptcAU9+V3T2YrvEWgqejjRduQKAh+f/VyLOo73/iXIKvSciQfGqqfh9R5o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715351; c=relaxed/simple; bh=Zg3dPmxAeZfdPCpe0eFW6kAfwZet+/NUlIS50hmLxGs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=dnbg2ZHmel5f68OUa1sYXotktR+yPXCylc7QkpHqAl8k8pkhCFuqavNiXWnaGPHhJeR0rNIBXaB1HlN9jw6882WK5OKc4gx87WMMuTB4H5V2qcyElJHRmJajC/KnR6js+Y70SfM9jzH6JOqNfhm/ZlojcLGug+QS/0syvzLa7Fc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-9a-65e82f7f922e From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 23/27] dept: Make Dept able to work with an external wgen Date: Wed, 6 Mar 2024 17:55:09 +0900 Message-Id: <20240306085513.41482-24-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSbUiTaxgH8O77eXW5eFrGebKiGkSHDplJ6hVESVDdEEUUfbGido4PbRxd Madmr5bLaqapoetFSletpTtpc4dTp+Yxw5WZtso6Zmolkg7nBG3Lpb1sRV8uflwv/08XTyn+ ZqJ5jVYv6bSqVCUro2VDkeZFhxcPSLGuslgoPhUL/g8naCivsbHgvlGNwOY4gsHTtBb+D3gR jLc+ocBU6kZQ+a6bAoerB4HTepSF531ToN0/zEJzaT4LuZdrWHg6OIGhq6wEQ7V9PbQUmTE0 BPtpMHlYuGDKxaEygCFoqeLAkjMfeq3nOZh4twSae14y4Oz8Dc5d7GLhrrOZBtetXgzP/y1n ocf2lYEW10Ma3MUFDPzlM7MwGLBQYPEPc/CsoQJDrSEUlDf6hYEHBQ0Y8q7cxND+6g6C+hNv MdhtL1m47/diqLOXUvDpWhOC3sIhDo6dCnJw4UghgvxjZTQYuuJhfKycTVpG7nuHKWKoyyLO QAVNHplFcvt8N0cM9Z0cqbBnkDrrQnL5rgeTyhE/Q+xVJ1liHynhiHGoHRNfWxtHHp4dp0lf uwlvjE6WLU+RUjWZkm7xip0ytWPgBbXHuHKvo+g6lYO8cUYUwYvCUnHM8Bn/tL3/Nhc2KywQ OzqCVNhRwlyxruA9EzYleGXilbY1YU8T1otNwePf92lhvnjm8T902HIhQXR2VjM/MueI1bUN 33MiQv3TvtNs2AohXmzNrQxZFtoZ48WnXUbqx8EM8Z61gy5C8go0qQopNNrMNJUmdWmMOlur 2Rvzx+40Owo9k+XgxNZbaMS9uREJPFJGypMi+iUFo8pMz05rRCJPKaPkBz71SQp5iip7n6Tb vUOXkSqlN6KZPK38RR4XyEpRCLtUeulPSdoj6X5OMR8RnYOSc4v+u+rN02fzm0Y33LwmLHOo hzOmv1/ncTviR1bsPLSq0rbNZ9QWxPzuUX9c3Zr1q8zsSvQNNtou3ggkKgrJVEP968w78aML kvKTZick1h7aOv0Ns2VecbewMfmAPnHy1JIPLbN8HyOf1FxKeBA3KU+vf7Y90G9dO08ftX+a 3kSUdLpatWQhpUtXfQPnzjBMSAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0iTYRQH8J7nvc3V4m0JvWWQDCzo4qUyDt0/5VNg2ZeEPpQzX3I0Z2yl GUWmy0yztJpTs/LGMme3d4MynY2Zll1M00pNLUVScSlUk9akckVfDj/O//D/dGSU0swskml0 R0S9Tq1VsXJavnND5qpTYaNieE9/IBScDwfP92waSu/WstB+x4qg1n4aw1hzFLyfciPwvXpN gdnUjqB8sJ8Ce8sAAkd1Bgudw3OhyzPJQqspl4XMyrssdIxPY+grvITBKkXDi/wKDE7vCA3m MRaumjPxzBjF4LXUcGBJD4Gh6hIOpgcjoHXgHQNN11oZcPSugOLrfSw0OFppaHk4hKHzUSkL A7W/GXjR8oyG9oI8Bm5PVLAwPmWhwOKZ5OCNswzDPeNMW9a3Xww8zXNiyKq6j6Grpx5BY/Yn DFLtOxaaPG4MNslEwc+bzQiGLnzh4Mx5LwdXT19AkHumkAZjXyT4fpSyW9eTJvckRYy2VOKY KqPJ8wqB1JX0c8TY2MuRMukosVUvJ5UNY5iUf/UwRKo5xxLp6yWO5HzpwmSirY0jz4p8NBnu MuOYoL3yjQmiVpMi6sM2x8kT7aNvqcM5W47Z829R6ci9OgcFyAR+rSCN1HF+s/wyobvbS/kd yAcLtrzPjN8U75YLVW3b/J7PRwvN3rN/72k+RLj88gHtt4JfJzh6rcy/ziWC9Z7zb0/AzP7i xEXWbyUfKbzKLGfzkbwMzapBgRpdSpJao40MNRxKTNNpjoUeSE6S0My7WE5OFzxE3zujXIiX IdUcxdaAEVHJqFMMaUkuJMgoVaDixM9hUalIUKcdF/XJ+/VHtaLBhYJktGqBYkesGKfkD6qP iIdE8bCo/59iWcCidBSyMvFWzHHjqfg18bkCLnFZi/YNNmm5KxkdcXOC19z4ZcrSp1Ul58Ym 3aw22z6ERsQWKsNXxJhuj0vbNAnu6E17dvLEt/rgvCsPOsK2Pym5tvhjvWtg3+zdPkVxccrS 4Pye1PWPM87pzjo3bhpcFTS+wz5WV1Rq139qjA/PXtgftUtFGxLVEcspvUH9B2NIzsIqAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: There is a case where total maps for its wait/event is so large in size. For instance, struct page for PG_locked and PG_writeback is the case. The additional memory size for the maps would be 'the # of pages * sizeof(struct dept_map)' if each struct page keeps its map all the way, which might be too big to accept. It'd be better to keep the minimum data in the case, which is timestamp called 'wgen' that Dept makes use of. So made Dept able to work with an external wgen when needed. Signed-off-by: Byungchul Park --- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 4 ++-- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 37 insertions(+), 15 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 0280e45cc2af..dea53ad5b356 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -482,6 +482,13 @@ struct dept_task { bool in_sched; }; +/* + * for subsystems that requires compact use of memory e.g. struct page + */ +struct dept_ext_wgen{ + unsigned int wgen; +}; + #define DEPT_TASK_INITIALIZER(t) \ { \ .wait_hist = { { .wait = NULL, } }, \ @@ -512,6 +519,7 @@ extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); @@ -521,8 +529,8 @@ extern void dept_clean_stage(void); extern void dept_stage_event(struct task_struct *t, unsigned long ip); extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); -extern void dept_request_event(struct dept_map *m); -extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, struct dept_ext_wgen *ewg); extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); @@ -551,6 +559,7 @@ extern void dept_hardirqs_off(void); struct dept_key { }; struct dept_map { }; struct dept_task { }; +struct dept_ext_wgen { }; #define DEPT_MAP_INITIALIZER(n, k) { } #define DEPT_TASK_INITIALIZER(t) { } @@ -563,6 +572,7 @@ struct dept_task { }; #define dept_free_range(s, sz) do { } while (0) #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_ext_wgen_init(wg) do { } while (0) #define dept_map_copy(t, f) do { } while (0) #define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) @@ -572,8 +582,8 @@ struct dept_task { }; #define dept_stage_event(t, ip) do { } while (0) #define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) #define dept_ecxt_holding(m, e_f) false -#define dept_request_event(m) do { } while (0) -#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event(m, wg) do { } while (0) +#define dept_event(m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 21fce525f031..8cdac7982036 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -24,7 +24,7 @@ #define sdt_wait_timeout(m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) #define sdt_wait(m) sdt_wait_timeout(m, -1L) @@ -49,7 +49,7 @@ #define sdt_might_sleep_end() dept_clean_stage() #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) -#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL) #define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 5c996f11abd5..fb33c3758c25 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2186,6 +2186,11 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, } EXPORT_SYMBOL_GPL(dept_map_reinit); +void dept_ext_wgen_init(struct dept_ext_wgen *ewg) +{ + ewg->wgen = 0U; +} + void dept_map_copy(struct dept_map *to, struct dept_map *from) { if (unlikely(!dept_working())) { @@ -2371,7 +2376,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map) + bool sched_map, unsigned int wg) { struct dept_class *c; struct dept_key *k; @@ -2393,7 +2398,7 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { - do_event(m, c, READ_ONCE(m->wgen), ip); + do_event(m, c, wg, ip); pop_ecxt(m, c); } } @@ -2606,7 +2611,7 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); exit: dept_exit(flags); } @@ -2785,10 +2790,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) } EXPORT_SYMBOL_GPL(dept_ecxt_holding); -void dept_request_event(struct dept_map *m) +void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { unsigned long flags; unsigned int wg; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2801,21 +2807,25 @@ void dept_request_event(struct dept_map *m) */ flags = dept_enter_recursive(); + wg_p = ewg ? &ewg->wgen : &m->wgen; + /* * Avoid zero wgen. */ wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); - WRITE_ONCE(m->wgen, wg); + WRITE_ONCE(*wg_p, wg); dept_exit_recursive(flags); } EXPORT_SYMBOL_GPL(dept_request_event); void dept_event(struct dept_map *m, unsigned long e_f, - unsigned long ip, const char *e_fn) + unsigned long ip, const char *e_fn, + struct dept_ext_wgen *ewg) { struct dept_task *dt = dept_task(); unsigned long flags; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2823,24 +2833,26 @@ void dept_event(struct dept_map *m, unsigned long e_f, if (m->nocheck) return; + wg_p = ewg ? &ewg->wgen : &m->wgen; + if (dt->recursive) { /* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); return; } flags = dept_enter(); - __dept_event(m, e_f, ip, e_fn, false); + __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); /* * Keep the map diabled until the next sleep. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); dept_exit(flags); } From patchwork Wed Mar 6 08:55:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583588 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EBE53657C1; Wed, 6 Mar 2024 08:55:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715352; cv=none; b=QkEVobbOds1rqRCeqQ939yo2bvSwNiHY9Hcma3OeUXjwq6CA6x+O5l2oq7MQvZZjTdHHxmtOoNF01cxQTl2R4i/oqwwob73zPxDfIT4ax8X955CEWSVr/GYWZhxbYFC5Qv+KK2fbUgAwGb0E8ejo1s4AWyoGNpR5UsmYfCAgBAk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715352; c=relaxed/simple; bh=tC26hq3Ijka0/rt3bw+rfbu4EsnZw7fNBY7MOAe1vjA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=BU4nlhn6YsKSvshrHWSyjX6T5v7JnPk3Q1RJSbNhkwixwRpn8WUosI8hYYu3RQGpK2gj343wjBFjd33lNblOcWOYJsFTT5W4PKB5/ljhwdftdfC10ORIVbqwKi5DLTgSUV6rftNVjdIfhaFVwIczuPxdo9oRkFwNBKPSkbAxhYc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-aa-65e82f7ffbc0 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 24/27] dept: Track PG_locked with dept Date: Wed, 6 Mar 2024 17:55:10 +0900 Message-Id: <20240306085513.41482-25-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSW0xTaRAHcL/vnH6nNNScVOMe8UFtIuttAS9sBqJGjcr3YmJ2ffAat7En trGAKYhl4wWhGESKigG8IAuItQEEtsWoa9tUDAi4YtGKqNAoIVwUSkRLrDRq6+Vl8svM5J95 GCmjuCGJkmpT0kV9ikqnJDJWNhZZ+dux2GEx7qUrEc4WxIH/Qx4LZQ11BNz1tQjqmo5jGGlJ gmeTowimHj5ioLTYjaDydR8DTa1eBA5LNoEnA9PB4x8n0F58ikDOlQYCXW+DGHpLijDUWjfD gzNVGFyBIRZKRwhcKs3BoTKMIWCu4cCctQD6LRc5CL5eBu3ebgk4XiyBC+W9BOyOdhZab/Vj ePJfGQFv3RcJPGhtY8F91iSB674qAm8nzQyY/eMcPHZVYGg0hoJOvP8sgfsmF4YT1f9i8Dy/ g8CZ9wqDta6bwD3/KAabtZiBT9daEPQXjnGQWxDg4NLxQgSncktYMPbGw9THMrI2gd4bHWeo 0XaIOiYrWNpRJdDbF/s4anS+4GiF9SC1WRbTK/YRTCsn/BJqrTlJqHWiiKP5Yx5MfZ2dHG07 P8XSAU8p3hK1Q7ZKLeq0GaI+ds1fMk2/yUwO2LcairscXBZ6vyEfRUgFfqVw1fIG/3TD/2Yu bML/KvT0BJiwZ/LzBJtpUBI2w4/KhOrOTWHP4BOFl0VNbNgsv0C45r35zXL+d+F6/QX2e+Zc obbR9S0nItQ/7TtNwlbw8cLDnMqQZaGdL1Khz+f9ccRs4a6lhz2D5BVoWg1SaFMyklVa3coY TWaK1hCzNzXZikLPZD4S3HkLTbj/bEa8FCkj5WsjhkSFRJWRlpncjAQpo5wpP/xpQFTI1arM v0V96h79QZ2Y1ozmSFnlL/Llk4fUCn6fKl3cL4oHRP3PKZZGRGWhzbXZBb7c+MLL5TUb86ca g/q8llfMH+VD6pKFxLXo+eP5SvpPdL0OD+ZGp1bzRzu68o3r1q0fDurn7lqY0JY0x+M02Feb NB3RtrjPKxKcs7Jj1OdMu3M05qTbVb1292Fv4vbZRwOD05wnl6Q7ZkyP3WaIjGruePfUsHzI urTb2Klk0zSqZYsZfZrqK4PCUGRIAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXSe0hTcRTA8X738btzNLtNoUsFxeiFlillHCxCovJHZET0gAhq5CVHc9pW lj0tnVlpPsKmaeGL5bPVnfR0YZorM01zmZlZDUnNpWVNMu3hiv45fDgHvn8dGa00sdNlGt1+ Ua9Ta1VYzsg3LE9YdGJxnxiY1KGCjJRAcH9LZiDPUoGh5Vo5goqqkxT014fByxEXgrGmZzSY sloQFLx/Q0OVvRuBreQUhrYeb3C4hzA0ZJ3DkFBkwdA6ME5B18VMCsqlcGhML6SgZrSXAVM/ hlxTAjUx+igYNZdxYI6fC86SSxyMvw+Chu52FuouN7Bg6/SHnCtdGKptDQzYbzspaLubh6G7 4jcLjfbHDLRkpLJQOViIYWDETIPZPcTB85p8Cq4nTtSSvv5i4VFqDQVJxTcocLy6h+B+8jsK pIp2DHVuFwVWKYuGH1frETjPf+LAmDLKQe7J8wjOGS8ykNgVDGPf83BoCKlzDdEk0XqQ2Eby GfKkUCB3Lr3hSOL9To7kSweItcSPFFX3U6Rg2M0SqewMJtJwJkfOfnJQZLC5mSOPs8cY0uMw URtnbJeviBC1mlhRv3jlLnmkM9WMY6o3H8pqtXHx6Ovqs8hLJvBLBctTM+cx5ucLHR2jtMe+ /GzBmvqB9ZjmXXKhuHmtxz58iPA6s4rxmOHnCle7b/21gl8mVF7LYf41Zwnl12v+drwm9mmD adhjJR8sNCUU4HQkz0eTypCvRhcbpdZogwMMeyPjdJpDAbujoyQ08S7mY+MZt9G3trBaxMuQ arIi1KtXVLLqWENcVC0SZLTKV3H0R4+oVESo4w6L+uid+gNa0VCLZsgY1TTFum3iLiW/R71f 3CuKMaL+/5WSeU2PR5ct6jnmYqNR9zZpRd/H9HrFmh3Hs2PygqaNHB//vqXOvsB+9OCXF+GZ 4d7W0LaUm2un+E8tX1MU+5R0li5Z6h2+NYRulFY/zN2wcNhn1ZPPvZUffF94P8+ZaWxdt6pw oH/9rCOlD05nQ7JUeSGiaZ7jWdoDrZ+8/WfGBee+MJfcskmjYgyR6iA/Wm9Q/wGkiCB6KgMA AA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track PG_locked waits and events. It's going to be useful in practice. See the following link that shows dept worked with PG_locked and can detect real issues: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/ Signed-off-by: Byungchul Park --- include/linux/mm_types.h | 2 + include/linux/page-flags.h | 105 ++++++++++++++++++++++++++++++++----- include/linux/pagemap.h | 7 ++- mm/filemap.c | 26 +++++++++ mm/mm_init.c | 2 + 5 files changed, 129 insertions(+), 13 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 957ce38768b2..5c1112bc7a46 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -19,6 +19,7 @@ #include #include #include +#include #include @@ -203,6 +204,7 @@ struct page { struct page *kmsan_shadow; struct page *kmsan_origin; #endif + struct dept_ext_wgen PG_locked_wgen; } _struct_page_alignment; /* diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index a88e64acebfe..0a498f2c4543 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -198,6 +198,43 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H +#ifdef CONFIG_DEPT +#include +#include + +extern struct dept_map PG_locked_map; + +/* + * Place the following annotations in its suitable point in code: + * + * Annotate dept_page_set_bit() around firstly set_bit*() + * Annotate dept_page_clear_bit() around clear_bit*() + * Annotate dept_page_wait_on_bit() around wait_on_bit*() + */ + +static inline void dept_page_set_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_request_event(&PG_locked_map, &p->PG_locked_wgen); +} + +static inline void dept_page_clear_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_event(&PG_locked_map, 1UL, _RET_IP_, __func__, &p->PG_locked_wgen); +} + +static inline void dept_page_wait_on_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_wait(&PG_locked_map, 1UL, _RET_IP_, __func__, 0, -1L); +} +#else +#define dept_page_set_bit(p, bit_nr) do { } while (0) +#define dept_page_clear_bit(p, bit_nr) do { } while (0) +#define dept_page_wait_on_bit(p, bit_nr) do { } while (0) +#endif + #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); @@ -379,44 +416,88 @@ static __always_inline int Page##uname(struct page *page) \ #define SETPAGEFLAG(uname, lname, policy) \ static __always_inline \ void folio_set_##lname(struct folio *folio) \ -{ set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_set_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void SetPage##uname(struct page *page) \ -{ set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline \ void folio_clear_##lname(struct folio *folio) \ -{ clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_clear_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void ClearPage##uname(struct page *page) \ -{ clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define __SETPAGEFLAG(uname, lname, policy) \ static __always_inline \ void __folio_set_##lname(struct folio *folio) \ -{ __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_set_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void __SetPage##uname(struct page *page) \ -{ __set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define __CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline \ void __folio_clear_##lname(struct folio *folio) \ -{ __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_clear_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void __ClearPage##uname(struct page *page) \ -{ __clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define TESTSETFLAG(uname, lname, policy) \ static __always_inline \ bool folio_test_set_##lname(struct folio *folio) \ -{ return test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + bool ret = test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy));\ + if (!ret) \ + dept_page_set_bit(&folio->page, PG_##lname); \ + return ret; \ +} \ static __always_inline int TestSetPage##uname(struct page *page) \ -{ return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_set_bit(PG_##lname, &policy(page, 1)->flags);\ + if (!ret) \ + dept_page_set_bit(page, PG_##lname); \ + return ret; \ +} #define TESTCLEARFLAG(uname, lname, policy) \ static __always_inline \ bool folio_test_clear_##lname(struct folio *folio) \ -{ return test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + bool ret = test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy));\ + if (ret) \ + dept_page_clear_bit(&folio->page, PG_##lname); \ + return ret; \ +} \ static __always_inline int TestClearPage##uname(struct page *page) \ -{ return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_clear_bit(PG_##lname, &policy(page, 1)->flags);\ + if (ret) \ + dept_page_clear_bit(page, PG_##lname); \ + return ret; \ +} #define PAGEFLAG(uname, lname, policy) \ TESTPAGEFLAG(uname, lname, policy) \ diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 06142ff7f9ce..c6683b228b20 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -991,7 +991,12 @@ void folio_unlock(struct folio *folio); */ static inline bool folio_trylock(struct folio *folio) { - return likely(!test_and_set_bit_lock(PG_locked, folio_flags(folio, 0))); + bool ret = !test_and_set_bit_lock(PG_locked, folio_flags(folio, 0)); + + if (ret) + dept_page_set_bit(&folio->page, PG_locked); + + return likely(ret); } /* diff --git a/mm/filemap.c b/mm/filemap.c index ad5b4aa049a3..241a67a363b0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #include "internal.h" @@ -1098,6 +1099,7 @@ static int wake_page_function(wait_queue_entry_t *wait, unsigned mode, int sync, if (flags & WQ_FLAG_CUSTOM) { if (test_and_set_bit(key->bit_nr, &key->folio->flags)) return -1; + dept_page_set_bit(&key->folio->page, key->bit_nr); flags |= WQ_FLAG_DONE; } } @@ -1181,6 +1183,7 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, if (wait->flags & WQ_FLAG_EXCLUSIVE) { if (test_and_set_bit(bit_nr, &folio->flags)) return false; + dept_page_set_bit(&folio->page, bit_nr); } else if (test_bit(bit_nr, &folio->flags)) return false; @@ -1191,6 +1194,9 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, /* How many times do we accept lock stealing from under a waiter? */ int sysctl_page_lock_unfairness = 5; +struct dept_map __maybe_unused PG_locked_map = DEPT_MAP_INITIALIZER(PG_locked_map, NULL); +EXPORT_SYMBOL(PG_locked_map); + static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, int state, enum behavior behavior) { @@ -1202,6 +1208,8 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, unsigned long pflags; bool in_thrashing; + dept_page_wait_on_bit(&folio->page, bit_nr); + if (bit_nr == PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { delayacct_thrashing_start(&in_thrashing); @@ -1295,6 +1303,23 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, break; } + /* + * dept_page_set_bit() might have been called already in + * folio_trylock_flag(), wake_page_function() or somewhere. + * However, call it again to reset the wgen of dept to ensure + * dept_page_wait_on_bit() is called prior to + * dept_page_set_bit(). + * + * Remind dept considers all the waits between + * dept_page_set_bit() and dept_page_clear_bit() as potential + * event disturbers. Ensure the correct sequence so that dept + * can make correct decisions: + * + * wait -> acquire(set bit) -> release(clear bit) + */ + if (wait->flags & WQ_FLAG_DONE) + dept_page_set_bit(&folio->page, bit_nr); + /* * If a signal happened, this 'finish_wait()' may remove the last * waiter from the wait-queues, but the folio waiters bit will remain @@ -1471,6 +1496,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters != 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + dept_page_clear_bit(&folio->page, PG_locked); if (folio_xor_flags_has_waiters(folio, 1 << PG_locked)) folio_wake_bit(folio, PG_locked); } diff --git a/mm/mm_init.c b/mm/mm_init.c index 077bfe393b5e..fc150d7a3686 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -564,6 +565,7 @@ void __meminit __init_single_page(struct page *page, unsigned long pfn, page_mapcount_reset(page); page_cpupid_reset_last(page); page_kasan_tag_reset(page); + dept_ext_wgen_init(&page->PG_locked_wgen); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL From patchwork Wed Mar 6 08:55:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583586 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 06B5C657CC; Wed, 6 Mar 2024 08:55:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715351; cv=none; b=WXMcqkBngTZO/7ZNjx7PIv827mFcrIE2Y5NVaD1sC7oQlsfjfyCIYqcX0FMa5PiSWbeuIn0YzJ4Bnn6NeKqRnOvnz5JPgXUFmh5NfmqRbBATMSeAbn63t5MhgUdB42HcbPn5r5hcSdKRvlew/1kZ0IKzq9ItwzSNpwBLB5uWb9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715351; c=relaxed/simple; bh=ul/wiSzDZit7bOUs1eKkFyN45Ghg8m4Ni94/4VNQ6UQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Bor5lK60N+eg1CjJDfn8UtweFWmFPLrHtFTSRdo3Iuo2FpzgdDchBWTl3YG9X44XGRggqAlCHkD0SN0yShN+g1SPyZkPZKrIQioewShhW3X4xfjYcu0yvPiyFxjvGrjJ3gDXraAlSRfrTMFuRk8Q7PCiaMzr0HL0Pwt80zwnRJc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-ba-65e82f7fca0c From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 25/27] dept: Print event context requestor's stacktrace on report Date: Wed, 6 Mar 2024 17:55:11 +0900 Message-Id: <20240306085513.41482-26-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/Pc9/n7jh7nPDIH9lt5Mf6wcSHmTVbPLMx8+sPxrnpoZvr yl0/zY+40xQhlqjkKk6ro7qaUh0JqShRiKWp9cNxlVV3pBZ1+Oez197v195/fcSUvETkKVZr IwSdVqVRECkt7ZuW7RPn90XwL+0hkHzeH5zDZ2nIKLAQaLqXj8BScgqD/dlGeO9yIBhteEVB akoTgqyOTxSU1LQjsOWeJtDcNR1anAME6lLOETDkFBB4/W0MQ9vVyxjyrZvhxaVsDFUjvTSk 2gmkpxrwxPmCYcScx4A5bgF05qYxMNaxDOra34nA9nEpXM9sI1Bpq6OhpqwTQ3N5BoF2y28R vKippaEpOUkEd/uzCXxzmSkwOwcYeFNlwlBonBiKHxoXwfOkKgzxt4owtHyoQPDw7GcMVss7 Ak+cDgzF1hQKft15hqDzQh8DZ86PMJB+6gKCc2eu0mBsC4DRnxkkcDX/xDFA8cbiaN7mMtF8 fTbHP0j7xPDGhx8Z3mSN5Itzl/A5lXbMZw06Rbw1L4Hw1sHLDJ/Y14L5/sZGhq+9NkrzXS2p eKvnbunaYEGjjhJ0fuv2S0PqbvRR4QlBMd32SjoOXVuViCRijl3BDT1Pp/5zb4VBNMmE9eZa W0fcuQc7nytO6nHnFOuQcrcaN0zyTHYnl1BRQCaZZhdwifXdbl/GruQMPa5/m15cfmGVmyUT +cX+i25fzgZwDYYs8tf5KebsXX5/eS73OLeVvoRkJjQlD8nV2qhQlVqzwjckVquO8T0QFmpF E79kPj62pwwNNm2vRqwYKabJAiW9glykitLHhlYjTkwpPGTHfnUJclmwKvaooAtT6iI1gr4a zRPTijmy5a7oYDl7SBUhHBaEcEH3v8ViiWccMi3cx5XfXDz7lY/+ypaA5tLruPvE1+316+cr j+w1fZ6h9PSOd8BbbsOx5T/ktck246HdLyWPC99oQ55K/C23X3Y8Uo6v9x6eGhOJfnzXbFq4 JsJrypCqE4K8dkmGV31X7uotWjfwdFFQbdis+yk7DmaOB5a8PpmTmeZR7hP9QbOt/LeC1oeo li2hdHrVH4vOInFHAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSSUxTcRjE/b+1bay+FBIfclBrjIlGEEL1E5dwUZ9r9KBGPGiFF6mUxVYQ jAtYVBYB0WCRxRTQSgAtFhJUKCEslSKrFNygCkGxoYCirUKJChgvk19mJnMaAS7RkksFisiz vCpSrpRSIkK0f7Nm3WXfL/z6JtdayLqxHpw/kgnIN5RT0PW4DEF5VSIG9uad8NrlQOBu78RB m92FoHBwAIcqsw2BqeQKBT3Di8DqnKDAkp1GgabYQEH36AwG/XduYVBm3AcvbxZhUD81QoDW TkGeVoPNyhcMpvSlNOgTVsFQSS4NM4N+YLH1kdBYYCHB9G4t3L3XT0GtyUKA+ekQBj3P8ymw lf8h4aW5hYCurHQSHo0XUTDq0uOgd07Q8Kpeh0FF0uzate+/SXiRXo/BtftPMLC+rUFQl/wR A2N5HwWNTgcGlcZsHKYfNiMYyhij4eqNKRryEjMQpF29Q0BSvwzcv/KpoECu0TGBc0mV5ziT S0dwrUUs9yx3gOaS6t7RnM4Yw1WWrOGKa+0YVzjpJDljaQrFGSdv0VzqmBXjxjs6aK4lx01w w1YtdsA7WLQllFcqYnmV77YTojBLwRgenbI97pO9lkhAORtTkVDAMgHsSI2GnGOKWc2+eTOF z7Ens5ytTP887+OMQ8Te79gxxx7MITalxkDNMcGsYlNbP833xcwGVvPZhf/bXMaWVdTPs3DW zxzPnO9LGBnbrimkbiKRDi0oRZ6KyNgIuUIp81GHh8VHKuJ8QqIijGj2LvqLM1lP0Y+enQ2I ESDpQnGQcISXkPJYdXxEA2IFuNRTfGF6mJeIQ+Xx53lV1HFVjJJXNyBvASFdIt59hD8hYU7J z/LhPB/Nq/6nmEC4NAHlfRAevP3+1++wFkPjsa293dWt02mDm93mgDYv2UWl1NL0nnVEZB7O KVYNGTbpHwSHvy4O1nV69IaMnPy4ovq4vzvZP2P3pZ/L9qyMeqg0k19XmM4EDOive5n0L0b3 2LoX37MG2rbv7Tn99hjf5qeVdRlyd9m3HP1WpEKpktJY0XkpoQ6T+63BVWr5XwZeymEqAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Currently, print nothing in place of [S] in report, which means stacktrace of event context's start if the event is not an unlock thing by typical lock but general event because it's not easy to specify the point in a general way, where the event context has started from. However, unfortunately it makes hard to interpret dept's report in that case. So made it print the event requestor's stacktrace instead of the event context's start, in place of [S] in report. Signed-off-by: Byungchul Park --- include/linux/dept.h | 13 +++++++ kernel/dependency/dept.c | 83 ++++++++++++++++++++++++++++++++-------- 2 files changed, 80 insertions(+), 16 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index dea53ad5b356..6db23d77905e 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -145,6 +145,11 @@ struct dept_map { */ unsigned int wgen; + /* + * requestor for the event context to run + */ + struct dept_stack *req_stack; + /* * whether this map should be going to be checked or not */ @@ -486,7 +491,15 @@ struct dept_task { * for subsystems that requires compact use of memory e.g. struct page */ struct dept_ext_wgen{ + /* + * wait timestamp associated to this map + */ unsigned int wgen; + + /* + * requestor for the event context to run + */ + struct dept_stack *req_stack; }; #define DEPT_TASK_INITIALIZER(t) \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index fb33c3758c25..abf1cdab0615 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -129,6 +129,7 @@ static int dept_per_cpu_ready; #define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t dept_req_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; static arch_spinlock_t dept_pool_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; /* @@ -1669,7 +1670,8 @@ static void add_wait(struct dept_class *c, unsigned long ip, static bool add_ecxt(struct dept_map *m, struct dept_class *c, unsigned long ip, const char *c_fn, - const char *e_fn, int sub_l) + const char *e_fn, int sub_l, + struct dept_stack *req_stack) { struct dept_task *dt = dept_task(); struct dept_ecxt_held *eh; @@ -1700,10 +1702,16 @@ static bool add_ecxt(struct dept_map *m, struct dept_class *c, e->class = get_class(c); e->ecxt_ip = ip; - e->ecxt_stack = ip && rich_stack ? get_current_stack() : NULL; e->event_fn = e_fn; e->ecxt_fn = c_fn; + if (req_stack) + e->ecxt_stack = get_stack(req_stack); + else if (ip && rich_stack) + e->ecxt_stack = get_current_stack(); + else + e->ecxt_stack = NULL; + eh = dt->ecxt_held + (dt->ecxt_held_pos++); eh->ecxt = get_ecxt(e); eh->map = m; @@ -2147,6 +2155,7 @@ void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, m->sub_u = sub_u; m->name = n; m->wgen = 0U; + m->req_stack = NULL; m->nocheck = !valid_key(k); dept_exit_recursive(flags); @@ -2181,6 +2190,7 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, m->name = n; m->wgen = 0U; + m->req_stack = NULL; dept_exit_recursive(flags); } @@ -2189,6 +2199,7 @@ EXPORT_SYMBOL_GPL(dept_map_reinit); void dept_ext_wgen_init(struct dept_ext_wgen *ewg) { ewg->wgen = 0U; + ewg->req_stack = NULL; } void dept_map_copy(struct dept_map *to, struct dept_map *from) @@ -2376,7 +2387,8 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map, unsigned int wg) + bool sched_map, unsigned int wg, + struct dept_stack *req_stack) { struct dept_class *c; struct dept_key *k; @@ -2397,7 +2409,7 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); - if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { + if (c && add_ecxt(m, c, 0UL, "(event requestor)", e_fn, 0, req_stack)) { do_event(m, c, wg, ip); pop_ecxt(m, c); } @@ -2506,6 +2518,8 @@ EXPORT_SYMBOL_GPL(dept_stage_wait); static void __dept_clean_stage(struct dept_task *dt) { + if (dt->stage_m.req_stack) + put_stack(dt->stage_m.req_stack); memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); dt->stage_sched_map = false; dt->stage_w_fn = NULL; @@ -2571,6 +2585,7 @@ void dept_request_event_wait_commit(void) */ wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(dt->stage_m.wgen, wg); + dt->stage_m.req_stack = get_current_stack(); __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: @@ -2602,6 +2617,8 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) */ m = dt_req->stage_m; sched_map = dt_req->stage_sched_map; + if (m.req_stack) + get_stack(m.req_stack); __dept_clean_stage(dt_req); /* @@ -2611,8 +2628,12 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen, + m.req_stack); exit: + if (m.req_stack) + put_stack(m.req_stack); + dept_exit(flags); } @@ -2692,7 +2713,7 @@ void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); - if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l, NULL)) goto exit; /* @@ -2744,7 +2765,7 @@ void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); - if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l, NULL)) goto exit; missing_ecxt: dt->missing_ecxt++; @@ -2792,9 +2813,11 @@ EXPORT_SYMBOL_GPL(dept_ecxt_holding); void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { + struct dept_task *dt = dept_task(); unsigned long flags; unsigned int wg; unsigned int *wg_p; + struct dept_stack **req_stack_p; if (unlikely(!dept_working())) return; @@ -2802,12 +2825,18 @@ void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) if (m->nocheck) return; - /* - * Allow recursive entrance. - */ - flags = dept_enter_recursive(); + if (dt->recursive) + return; - wg_p = ewg ? &ewg->wgen : &m->wgen; + flags = dept_enter(); + + if (ewg) { + wg_p = &ewg->wgen; + req_stack_p = &ewg->req_stack; + } else { + wg_p = &m->wgen; + req_stack_p = &m->req_stack; + } /* * Avoid zero wgen. @@ -2815,7 +2844,13 @@ void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(*wg_p, wg); - dept_exit_recursive(flags); + arch_spin_lock(&dept_req_spin); + if (*req_stack_p) + put_stack(*req_stack_p); + *req_stack_p = get_current_stack(); + arch_spin_unlock(&dept_req_spin); + + dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_request_event); @@ -2826,6 +2861,8 @@ void dept_event(struct dept_map *m, unsigned long e_f, struct dept_task *dt = dept_task(); unsigned long flags; unsigned int *wg_p; + struct dept_stack **req_stack_p; + struct dept_stack *req_stack; if (unlikely(!dept_working())) return; @@ -2833,7 +2870,18 @@ void dept_event(struct dept_map *m, unsigned long e_f, if (m->nocheck) return; - wg_p = ewg ? &ewg->wgen : &m->wgen; + if (ewg) { + wg_p = &ewg->wgen; + req_stack_p = &ewg->req_stack; + } else { + wg_p = &m->wgen; + req_stack_p = &m->req_stack; + } + + arch_spin_lock(&dept_req_spin); + req_stack = *req_stack_p; + *req_stack_p = NULL; + arch_spin_unlock(&dept_req_spin); if (dt->recursive) { /* @@ -2842,17 +2890,20 @@ void dept_event(struct dept_map *m, unsigned long e_f, * handling the event. Disable it until the next. */ WRITE_ONCE(*wg_p, 0U); + if (req_stack) + put_stack(req_stack); return; } flags = dept_enter(); - - __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); + __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p), req_stack); /* * Keep the map diabled until the next sleep. */ WRITE_ONCE(*wg_p, 0U); + if (req_stack) + put_stack(req_stack); dept_exit(flags); } From patchwork Wed Mar 6 08:55:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583587 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 876E165BBF; Wed, 6 Mar 2024 08:55:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715351; cv=none; b=B03AsGBAVch92t22byUw9DiBbus3k74k2NjGS5sTwJxBPGbBM99xTcOsofZZ/DFF7LuOpnN6L+KhNQ0Jvss+ZtrZezVNYmgzDu178BYHZh20fToS9FGLTsbbScp9XXEP6l0eTnGsGo/QmyALSBkThDlyBYhIQnT03n1p2x+TLDQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715351; c=relaxed/simple; bh=5yRKL/D4WKMUyz5KF4LYQmab3vQapC/hAllcpi+yp24=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jFJvKkxh+MJoqUiuPD4ef9cp1fgjQKumQwYlm3wONqZObpovz1T9IG2aXMNCmPSnC8o5JcumtVbjqM2EhCOQz5/wmKqVCrSWDO2M1AuS2r7RmXX3V115YeTYSFaOj3EIdFxII6YTP34VecrGiXY4sgDhCyPmNO1V7ddzlfr4v34= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-ca-65e82f7ffbf2 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 26/27] fs/jbd2: Use a weaker annotation in journal handling Date: Wed, 6 Mar 2024 17:55:12 +0900 Message-Id: <20240306085513.41482-27-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa2yLYRQHcM/zXlurvKklXpMgFffYRUzOEqRCeHwQQnwhQWNvrHQjHbsI yVgtdDpDdrENu6lmF5eOmMsutnQ3NmOzbjPDUmbRbWxaujVoJ76c/HLO/5xPh6eUD5ggXhtz TNLHaHQqVk7LhwMKVySFfJFCi20BcOlCKLh+nKMh704ZC+23SxGU3T+NYci2GexuJ4LJ1pcU ZGW0Iyj4+I6C+w39CKosZ1jocMyATtcoC80ZqSwkF91h4dVXL4a+zMsYSq1b4Xl6IYZazyAN WUMs5GYlY1/5gsFjLuHAnLQQBiw5HHg/hkFzfxcDVb3L4er1PhaeVjXT0FA5gKHjcR4L/WV/ GHje0ERD+yUTA+UjhSx8dZspMLtGOXhdm4/hrsF3KGX8NwONploMKcX3MHT2PEFQfe4DBmtZ Fwv1LieGCmsGBRO3bAgG0oY5OHvBw0Hu6TQEqWczaTD0hcPkrzxWHUHqnaMUMVTEkyp3Pk1a CkXyKOcdRwzVvRzJtx4nFZZlpOjpECYFYy6GWEvOs8Q6dpkjxuFOTEba2jjSlD1JE0dnFt4e tFu+JlLSaeMkfci6/fKo259eU0fd0xNabDUoCU3IjEjGi8Iqsf+Zg/7vGqOZ8psVFovd3Z4p BwrzxQrTZ8ZvSnDKxeK2TX7PFLaJxvdG7DctLBRH7capvEJYLZp6bdy/m/PE0ru1U32Zr39x 5CLrt1IIF1uTC3yW+zK/ePGNzcL+W5gtPrN00+lIkY+mlSClNiYuWqPVrQqOSozRJgQfOBJt Rb5nMp/y7qlEY+0765DAI1WAQi0blJSMJi42MboOiTylClScnHBISkWkJvGEpD+yT39cJ8XW oTk8rZqlWOmOj1QKBzXHpMOSdFTS/59iXhaUhEImUsZ/dO00hmtHrm202zfbb8398z3tRfm3 tq4cwbv95Iv0t+qIJa1icEJEx/j5eEevWLp3wYZFV3Ln6x6ailMPD1Yq0nY4mR1eT1wmurHm Z25lWI/JtTTg+pbpQw7+ULORa1E1rs+WB5KHps+qyXT1lZs9aoMraG35AUvJrux4FR0bpQlb RuljNX8BC/lkdUgDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0iTYRQH8J7nvW2rxcsa9aZFMZJIy7RaHCrCiOqlqxEU5Qcd9ZJLZ7ap aSWutKiZlYEt00RN1nLz0uyDXUxTtlLJTEXLWyWWyqYDa9LULs7oy+HH/xz+n46IkBkpP5E6 LkHQxqliFbSElOzfnL4mbe2wEJL3aj1kXw8Bz4+rJORXWGloLbcgsD65iGHEvgu6JlwIpt6+ I8CY04qg6EsfAU8c/QhqzJdoaB+cDx0eNw2NOZk0pD+ooOG9cxpD753bGCy2fdB8qxhDnXeI BOMIDXnGdDwzhjF4TaUMmPQBMGC+x8D0l1Bo7O+koOF+IwU13UGQW9BLw4uaRhIc1QMY2p/l 09Bv/UNBs+MNCa3ZWRSUjRXT4JwwEWDyuBloqyvEUJkx03bl+28KXmfVYbhS8hhDx8fnCF5e /YzBZu2kocHjwlBlyyFg8qEdwcCNUQYuX/cykHfxBoLMy3dIyOhVwtTPfDpsE9/gchN8RtVZ vmaikOSbijn+6b0+hs942c3whbZEvsocyD94MYL5onEPxdtKr9G8bfw2wxtGOzA/1tLC8G/u TpH8YIcRh/sfk2w5IcSqkwTt2q1Rkujyr21E/MTc5CZ7LdKjSbEBiUUcu4GrNZgIn2l2Jffh g3fWcnY5V5X1jfKZYF0SrqRlp88L2AOc4ZMB+0yyAZy7yzB7L2U3clndduZf5zLOUlk3m4tn 8ptjN2mfZaySe5teRN9CkkI0pxTJ1XFJGpU6Vhmsi4lOiVMnBx8/rbGhmXcxpU5nV6Mf7bvq EStCinnSMPGQIKNUSboUTT3iRIRCLr0wOSjIpCdUKecE7elIbWKsoKtH/iJSsUi6+4gQJWNP qhKEGEGIF7T/t1gk9tMj0O85vsomL4p4FBLUHOSKdJCagoPb/kQSqdfeL+v6tSTi83BqpN9o eOjCw0PygEnv0epM2fkC5871/atze0rKTgU83P1scXlt4N7gHmeP9Mw35b7H9vDxV8mOsM1a sdJxeAFLLLW05QqOxB3mR5oovXPRuuQV7u3nnt6F0UPB5jQFqYtWhQYSWp3qL1XXE88qAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: jbd2 journal handling code doesn't want jbd2_might_wait_for_commit() to be placed between start_this_handle() and stop_this_handle(). So it marks the region with rwsem_acquire_read() and rwsem_release(). However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire_read() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire_read() as a potential wait and might report a deadlock by the wait. So replaced it with trylock version of annotation. Signed-off-by: Byungchul Park --- fs/jbd2/transaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index 5f08b5fd105a..2c159a547e15 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -460,7 +460,7 @@ static int start_this_handle(journal_t *journal, handle_t *handle, read_unlock(&journal->j_state_lock); current->journal_info = handle; - rwsem_acquire_read(&journal->j_trans_commit_map, 0, 0, _THIS_IP_); + rwsem_acquire_read(&journal->j_trans_commit_map, 0, 1, _THIS_IP_); jbd2_journal_free_transaction(new_transaction); /* * Ensure that no allocations done while the transaction is open are From patchwork Wed Mar 6 08:55:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13583589 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 270F866B4E; Wed, 6 Mar 2024 08:55:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715354; cv=none; b=Gj5Do2IhkPxaiQ+bfZyJ8HlKFfEIoVtmTSupZGPlGGoQ4EXR0seODiD2iNf6N+mGXcyyC2mDLFmT4K7imNNwERCGEYNkMH+b+gz7mRn4SQzv8OwBxfJZCI6+BA+elwaDQfANtxbRLIAmW0Y8dp8EoICGcyWa3eNPkyJzP3KKosY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709715354; c=relaxed/simple; bh=Smsc/UzDl3/jK/dEYNi0KPrzkpNTVo3i0D8H9H66qB8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=je/bGwKxCzk+UnsOGsWV+jDzLz6zKVQAijRh1+zxJHKkDYwLUK0LCPoUYgaqATPp5l3Md75cN6hIyOr6dFDLPNy4GlF2VVvmNTubz9wVatQGOhVgK7ClJNNlSgqEsi3zjFwG3zOew7dNxsNZ7wC2WsNQpU9LDxK1pPx3uDxuFKo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-da-65e82f803283 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v13 27/27] dept: Add documentation for Dept Date: Wed, 6 Mar 2024 17:55:13 +0900 Message-Id: <20240306085513.41482-28-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240306085513.41482-1-byungchul@sk.com> References: <20240306085513.41482-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSfUzMcRwHcN/v7+nu1vHbYX7lD9xmzEMPlvZpjGwefv7wsPnDPEyOftNN XbmIwnZc5THV6SpJ7sK5dSnumKjLuSbKOkUUPUwPVNRFXJMa7sI/n732+Xz23uePj4iQ3acC RErVQUGtUsTIaQkpGfQzLtYE9QnBX03+kHU+GDzfT5NQUFZCQ0OpBUHJ3RMY+p+sg+aRAQRj 9S8IyNU3IDB2thNwt6YDgd18koZXPZOhyTNEQ63+HA3aa2U0NH4ex9CWo8NgsW6A55lFGByj vSTk9tNwOVeLvaUPw6ipmAGTZi50mfMZGO8MgdqONxTY3y2ES4VtNFTaa0moKe/C8OphAQ0d Jb8peF7zjISGrHQKbrmLaPg8YiLA5Bli4KXDgOF2ijco7dsvCp6mOzCkXb+DoeltBYKq0+8x WEve0FDtGcBgs+oJ+HnzCYKuC4MMpJ4fZeDyiQsIzqXmkJDSthTGfhTQEeF89cAQwafYDvP2 EQPJ1xVx/IP8doZPqXrH8AbrId5mXsBfq+zHvHHYQ/HW4jM0bx3WMfzZwSbMu10uhn+WN0by PU25eHPAdsnyKCFGmSiog1bslkTXZXlQ/FcbOuLIuYg1yKFFZ5FYxLGhXMepavzf9ns2ymea nce1tIwSPk9jZ3O29I8TfYIdkHDXXWt9nsou4y5askmfSXYu12j4MpEpZcM4t1lH/M2cxVlu OyYs9vYz3Bm0zzJ2KVevNXot8e58F3GFefp/B/lzj80tZCaSGtCkYiRTqhJjFcqY0MDoJJXy SODeuFgr8r6T6fj4jnI03LDFiVgRkvtJI8S9goxSJCYkxToRJyLk06THfvYIMmmUIilZUMdF qg/FCAlONFNEymdIl4wcjpKx+xQHhf2CEC+o/0+xSBygQTPMebY5OzdOXpLGZsrz94XrVPq3 r1ddVdY9Wt0v/uR5Yeie3hIWldnnFIeXZ2+0tGZEulJX5nQKRw9E3zDm6bdGJhd8YCuapzup sWadRr4nJHvT00Vh/qWtfu71+2WP1uxOnj+lvXHSLue6bWduZu+Nu/Nyk8sQP6u7NSNikVZ2 JVlOJkQrQhYQ6gTFH09Js91KAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0hTcRgG8P7/c5ujxWEJHeqDMSrL0DSz3jJCyOpQGEJREISOPORoTtnU Mgq1aXdvM12lxrywRFfOrcDKxXBoWmiaolYmKVYT5zRr0tQsZ/Tl5cfzwPPpFRHSe9RakUKV KqhVcqWMFpPio5Ha4MxtTiHUaNoLRbdDwfPzOgnlDSYauh/XIzA9ycYw3noIBmZdCOY73xKg L+lGUDnyiYAnbcMIbLVXaOgdWwV9nikaOkpu0aCtbqChZ2IBw1CpDkO9JQbeFFZhsHu/kaAf p6FMr8VLx4nBa6xjwJi1EUZr7zOwMBIGHcP9FDgqOiiwfdgK9x4M0dBs6yChrWkUQ+/zchqG TX8oeNPWTkJ3UR4Fj9xVNEzMGgkweqYYeGc3YDDnLK1d/bFIwas8O4arNY0Y+t6/QPDy+mcM FlM/DQ6PC4PVUkLA3MNWBKP5kwzk3vYyUJadj+BWbikJOUMRMP+rnI7awztcUwSfYz3P22YN JP+6iuOf3f/E8DkvPzC8wZLGW2uD+OrmccxXzngo3lJ3g+YtMzqGvznZh3l3VxfDt9+dJ/mx Pj2OXXdKvDdBUCrSBfW2ffHixNdFHpTy3You2EuLcRaya9FN5Cfi2B2c7amV8plmA7nBQS/h sz+7nrPmfV3OCdYl5mq6Dvq8mo3kiuvvkD6T7EauxzC9vCNhd3LuWh3xbzOAqzfbl+23lBe4 C2ifpWwE16mtpAuR2IBW1CF/hSo9Sa5QRoRoziVmqBQXQs4kJ1nQ0sMYLy8UNaGfvYdaECtC spWSKL9vgpSSp2sykloQJyJk/pJLc2OCVJIgz7goqJPj1GlKQdOC1olI2RrJ4ZNCvJQ9K08V zglCiqD+32KR39osNCAqBF0P5TAT0aMR239s0JxePHlixFH2W/fRNR/LZJKKuV3XvpjTNgU6 E0ai4xtUce6YsMmtAZQpoDF/Oqp8S/oRxvjUVKEtafdGZofGhmPzsdmJ1APKqF/HZ5qCd7cZ 9TW5m6vFi+F/CvYUb/64yelsDXi15vxk85mB7s/a/TJSkygPCyLUGvlfAVHXCiwDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: This document describes the concept of Dept. Signed-off-by: Byungchul Park --- Documentation/dependency/dept.txt | 735 ++++++++++++++++++++++++++++++ 1 file changed, 735 insertions(+) create mode 100644 Documentation/dependency/dept.txt diff --git a/Documentation/dependency/dept.txt b/Documentation/dependency/dept.txt new file mode 100644 index 000000000000..5dd358b96734 --- /dev/null +++ b/Documentation/dependency/dept.txt @@ -0,0 +1,735 @@ +DEPT(DEPendency Tracker) +======================== + +Started by Byungchul Park + +How lockdep works +----------------- + +Lockdep detects a deadlock by checking lock acquisition order. For +example, a graph to track acquisition order built by lockdep might look +like: + + A -> B - + \ + -> E + / + C -> D - + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep keeps adding each new acquisition order into the graph in +runtime. For example, 'E -> C' will be added when the two locks have +been acquired in the order, E and then C. The graph will look like: + + A -> B - + \ + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +This graph contains a subgraph that demonstrates a loop like: + + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep reports it as a deadlock on detection of a loop and stops its +working. + +CONCLUSION + +Lockdep detects a deadlock by checking if a loop has been created after +adding a new acquisition order into the graph. + + +Limitation of lockdep +--------------------- + +Lockdep deals with a deadlock by typical lock e.g. spinlock and mutex, +that are supposed to be released within the acquisition context. However, +when it comes to a deadlock by folio lock that is not supposed to be +released within the acquisition context or other general synchronization +mechanisms, lockdep doesn't work. + +Can lockdep detect the following deadlock? + + context X context Y context Z + + mutex_lock A + folio_lock B + folio_lock B <- DEADLOCK + mutex_lock A <- DEADLOCK + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +No. What about the following? + + context X context Y + + mutex_lock A + mutex_lock A <- DEADLOCK + wait_for_complete B <- DEADLOCK + complete B + mutex_unlock A + mutex_unlock A + +No. + +CONCLUSION + +Lockdep cannot detect a deadlock by folio lock or other general +synchronization mechanisms. + + +What leads a deadlock +--------------------- + +A deadlock occurs when one or multi contexts are waiting for events that +will never happen. For example: + + context X context Y context Z + + | | | + v | | + 1 wait for A v | + . 2 wait for C v + event C . 3 wait for B + event B . + event A + +Event C cannot be triggered because context X is stuck at 1, event B +cannot be triggered because context Y is stuck at 2, and event A cannot +be triggered because context Z is stuck at 3. All the contexts are stuck. +We call this *deadlock*. + +If an event occurrence is a prerequisite to reaching another event, we +call it *dependency*. In this example: + + Event A occurrence is a prerequisite to reaching event C. + Event C occurrence is a prerequisite to reaching event B. + Event B occurrence is a prerequisite to reaching event A. + +In terms of dependency: + + Event C depends on event A. + Event B depends on event C. + Event A depends on event B. + +Dependency graph reflecting this example will look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +A circular dependency exists. Such a circular dependency leads a +deadlock since no waiters can have desired events triggered. + +CONCLUSION + +A circular dependency of events leads a deadlock. + + +Introduce DEPT +-------------- + +DEPT(DEPendency Tracker) tracks wait and event instead of lock +acquisition order so as to recognize the following situation: + + context X context Y context Z + + | | | + v | | + wait for A v | + . wait for C v + event C . wait for B + event B . + event A + +and builds up a dependency graph in runtime that is similar to lockdep. +The graph might look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT keeps adding each new dependency into the graph in runtime. For +example, 'B -> D' will be added when event D occurrence is a +prerequisite to reaching event B like: + + | + v + wait for D + . + event B + +After the addition, the graph will look like: + + -> D + / + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT is going to report a deadlock on detection of a new loop. + +CONCLUSION + +DEPT works on wait and event so as to theoretically detect all the +potential deadlocks. + + +How DEPT works +-------------- + +Let's take a look how DEPT works with the 1st example in the section +'Limitation of lockdep'. + + context X context Y context Z + + mutex_lock A + folio_lock B + folio_lock B <- DEADLOCK + mutex_lock A <- DEADLOCK + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +Adding comments to describe DEPT's view in terms of wait and event: + + context X context Y context Z + + mutex_lock A + /* wait for A */ + folio_lock B + /* wait for A */ + /* start event A context */ + + folio_lock B + /* wait for B */ <- DEADLOCK + /* start event B context */ + + mutex_lock A + /* wait for A */ <- DEADLOCK + /* start event A context */ + + folio_unlock B + /* event B */ + folio_unlock B + /* event B */ + + mutex_unlock A + /* event A */ + mutex_unlock A + /* event A */ + +Adding more supplementary comments to describe DEPT's view in detail: + + context X context Y context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + + folio_lock B + /* might wait for B */ <- DEADLOCK + /* start to take into account event B's context */ + /* 3 */ + + mutex_lock A + /* might wait for A */ <- DEADLOCK + /* start to take into account + event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +Let's build up dependency graph with this example. Firstly, context X: + + context X + + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + +There are no events to create dependency. Next, context Y: + + context Y + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 3 */ + + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + +There are two events. For event B, folio_unlock B, since there are no +waits between 3 and the event, event B does not create dependency. For +event A, there is a wait, folio_lock B, between 1 and the event. Which +means event A cannot be triggered if event B does not wake up the wait. +Therefore, we can say event A depends on event B, say, 'A -> B'. The +graph will look like after adding the dependency: + + A -> B + + where 'A -> B' means that event A depends on event B. + +Lastly, context Z: + + context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +There are also two events. For event B, folio_unlock B, there is a +wait, mutex_lock A, between 2 and the event - remind 2 is at a very +start and before the wait in timeline. Which means event B cannot be +triggered if event A does not wake up the wait. Therefore, we can say +event B depends on event A, say, 'B -> A'. The graph will look like +after adding the dependency: + + -> A -> B - + / \ + \ / + ----------- + + where 'A -> B' means that event A depends on event B. + +A new loop has been created. So DEPT can report it as a deadlock. For +event A, mutex_unlock A, since there are no waits between 4 and the +event, event A does not create dependency. That's it. + +CONCLUSION + +DEPT works well with any general synchronization mechanisms by focusing +on wait, event and its context. + + +Interpret DEPT report +--------------------- + +The following is the example in the section 'How DEPT works'. + + context X context Y context Z + + mutex_lock A + /* might wait for A */ + /* start to take into account event A's context */ + /* 1 */ + folio_lock B + /* might wait for B */ + /* start to take into account event B's context */ + /* 2 */ + + folio_lock B + /* might wait for B */ <- DEADLOCK + /* start to take into account event B's context */ + /* 3 */ + + mutex_lock A + /* might wait for A */ <- DEADLOCK + /* start to take into account + event A's context */ + /* 4 */ + + folio_unlock B + /* event B that's been valid since 2 */ + folio_unlock B + /* event B that's been valid since 3 */ + + mutex_unlock A + /* event A that's been valid since 1 */ + + mutex_unlock A + /* event A that's been valid since 4 */ + +We can Simplify this by replacing each waiting point with [W], each +point where its event's context starts with [S] and each event with [E]. +This example will look like after the replacement: + + context X context Y context Z + + [W][S] mutex_lock A + [W][S] folio_lock B + [W][S] folio_lock B <- DEADLOCK + + [W][S] mutex_lock A <- DEADLOCK + [E] folio_unlock B + [E] folio_unlock B + [E] mutex_unlock A + [E] mutex_unlock A + +DEPT uses the symbols [W], [S] and [E] in its report as described above. +The following is an example reported by DEPT for a real problem. + + Link: https://lore.kernel.org/lkml/6383cde5-cf4b-facf-6e07-1378a485657d@I-love.SAKURA.ne.jp/#t + Link: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/ + + =================================================== + DEPT: Circular dependency has been detected. + 6.2.0-rc1-00025-gb0c20ebf51ac-dirty #28 Not tainted + --------------------------------------------------- + summary + --------------------------------------------------- + *** DEADLOCK *** + + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S]: start of the event context + [W]: the wait blocked + [E]: the event not reachable + --------------------------------------------------- + context A's detail + --------------------------------------------------- + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + [S] lock(&ni->ni_lock:0): + [] ntfs3_setattr+0x54b/0xd40 + stacktrace: + ntfs3_setattr+0x54b/0xd40 + notify_change+0xcb3/0x1430 + do_truncate+0x149/0x210 + path_openat+0x21a3/0x2a90 + do_filp_open+0x1ba/0x410 + do_sys_openat2+0x16d/0x4e0 + __x64_sys_creat+0xcd/0x120 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + + [W] folio_wait_bit_common(PG_locked_map:0): + [] truncate_inode_pages_range+0x9b0/0xf20 + stacktrace: + folio_wait_bit_common+0x5e0/0xaf0 + truncate_inode_pages_range+0x9b0/0xf20 + truncate_pagecache+0x67/0x90 + ntfs3_setattr+0x55a/0xd40 + notify_change+0xcb3/0x1430 + do_truncate+0x149/0x210 + path_openat+0x21a3/0x2a90 + do_filp_open+0x1ba/0x410 + do_sys_openat2+0x16d/0x4e0 + __x64_sys_creat+0xcd/0x120 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + + [E] unlock(&ni->ni_lock:0): + (N/A) + --------------------------------------------------- + context B's detail + --------------------------------------------------- + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S] (unknown)(PG_locked_map:0): + (N/A) + + [W] lock(&ni->ni_lock:0): + [] attr_data_get_block+0x32c/0x19f0 + stacktrace: + attr_data_get_block+0x32c/0x19f0 + ntfs_get_block_vbo+0x264/0x1330 + __block_write_begin_int+0x3bd/0x14b0 + block_write_begin+0xb9/0x4d0 + ntfs_write_begin+0x27e/0x480 + generic_perform_write+0x256/0x570 + __generic_file_write_iter+0x2ae/0x500 + ntfs_file_write_iter+0x66d/0x1d70 + do_iter_readv_writev+0x20b/0x3c0 + do_iter_write+0x188/0x710 + vfs_iter_write+0x74/0xa0 + iter_file_splice_write+0x745/0xc90 + direct_splice_actor+0x114/0x180 + splice_direct_to_actor+0x33b/0x8b0 + do_splice_direct+0x1b7/0x280 + do_sendfile+0xb49/0x1310 + + [E] folio_unlock(PG_locked_map:0): + [] generic_write_end+0xf2/0x440 + stacktrace: + generic_write_end+0xf2/0x440 + ntfs_write_end+0x42e/0x980 + generic_perform_write+0x316/0x570 + __generic_file_write_iter+0x2ae/0x500 + ntfs_file_write_iter+0x66d/0x1d70 + do_iter_readv_writev+0x20b/0x3c0 + do_iter_write+0x188/0x710 + vfs_iter_write+0x74/0xa0 + iter_file_splice_write+0x745/0xc90 + direct_splice_actor+0x114/0x180 + splice_direct_to_actor+0x33b/0x8b0 + do_splice_direct+0x1b7/0x280 + do_sendfile+0xb49/0x1310 + __x64_sys_sendfile64+0x1d0/0x210 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + --------------------------------------------------- + information that might be helpful + --------------------------------------------------- + CPU: 1 PID: 8060 Comm: a.out Not tainted + 6.2.0-rc1-00025-gb0c20ebf51ac-dirty #28 + Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), + BIOS Bochs 01/01/2011 + Call Trace: + + dump_stack_lvl+0xf2/0x169 + print_circle.cold+0xca4/0xd28 + ? lookup_dep+0x240/0x240 + ? extend_queue+0x223/0x300 + cb_check_dl+0x1e7/0x260 + bfs+0x27b/0x610 + ? print_circle+0x240/0x240 + ? llist_add_batch+0x180/0x180 + ? extend_queue_rev+0x300/0x300 + ? __add_dep+0x60f/0x810 + add_dep+0x221/0x5b0 + ? __add_idep+0x310/0x310 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + ? add_iecxt+0x1bc/0xa60 + __dept_wait+0x600/0x1490 + ? add_iecxt+0x1bc/0xa60 + ? truncate_inode_pages_range+0x9b0/0xf20 + ? check_new_class+0x790/0x790 + ? dept_enirq_transition+0x519/0x9c0 + dept_wait+0x159/0x3b0 + ? truncate_inode_pages_range+0x9b0/0xf20 + folio_wait_bit_common+0x5e0/0xaf0 + ? filemap_get_folios_contig+0xa30/0xa30 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x10e/0x160 + ? lock_is_held_type+0x11e/0x160 + truncate_inode_pages_range+0x9b0/0xf20 + ? truncate_inode_partial_folio+0xba0/0xba0 + ? setattr_prepare+0x142/0xc40 + truncate_pagecache+0x67/0x90 + ntfs3_setattr+0x55a/0xd40 + ? ktime_get_coarse_real_ts64+0x1e5/0x2f0 + ? ntfs_extend+0x5c0/0x5c0 + ? mode_strip_sgid+0x210/0x210 + ? ntfs_extend+0x5c0/0x5c0 + notify_change+0xcb3/0x1430 + ? do_truncate+0x149/0x210 + do_truncate+0x149/0x210 + ? file_open_root+0x430/0x430 + ? process_measurement+0x18c0/0x18c0 + ? ntfs_file_release+0x230/0x230 + path_openat+0x21a3/0x2a90 + ? path_lookupat+0x840/0x840 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x10e/0x160 + do_filp_open+0x1ba/0x410 + ? may_open_dev+0xf0/0xf0 + ? find_held_lock+0x2d/0x110 + ? lock_release+0x43c/0x830 + ? dept_ecxt_exit+0x31a/0x590 + ? _raw_spin_unlock+0x3b/0x50 + ? alloc_fd+0x2de/0x6e0 + do_sys_openat2+0x16d/0x4e0 + ? __ia32_sys_get_robust_list+0x3b0/0x3b0 + ? build_open_flags+0x6f0/0x6f0 + ? dept_enirq_transition+0x519/0x9c0 + ? dept_enirq_transition+0x519/0x9c0 + ? lock_is_held_type+0x4e/0x160 + ? lock_is_held_type+0x4e/0x160 + __x64_sys_creat+0xcd/0x120 + ? __x64_compat_sys_openat+0x1f0/0x1f0 + do_syscall_64+0x41/0xc0 + entry_SYSCALL_64_after_hwframe+0x63/0xcd + RIP: 0033:0x7f8b9e4e4469 + Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 + 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> + 3d 01 f0 ff ff 73 01 c3 48 8b 0d ff 49 2b 00 f7 d8 64 89 01 48 + RSP: 002b:00007f8b9eea4ef8 EFLAGS: 00000202 ORIG_RAX: 0000000000000055 + RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f8b9e4e4469 + RDX: 0000000000737562 RSI: 0000000000000000 RDI: 0000000020000000 + RBP: 00007f8b9eea4f20 R08: 0000000000000000 R09: 0000000000000000 + R10: 0000000000000000 R11: 0000000000000202 R12: 00007fffa75511ee + R13: 00007fffa75511ef R14: 00007f8b9ee85000 R15: 0000000000000003 + + +Let's take a look at the summary that is the most important part. + + --------------------------------------------------- + summary + --------------------------------------------------- + *** DEADLOCK *** + + context A + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + [E] unlock(&ni->ni_lock:0) + + context B + [S] (unknown)(PG_locked_map:0) + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(PG_locked_map:0) + + [S]: start of the event context + [W]: the wait blocked + [E]: the event not reachable + +The summary shows the following scenario: + + context A context B context ?(unknown) + + [S] folio_lock(&f1) + [S] lock(&ni->ni_lock:0) + [W] folio_wait_bit_common(PG_locked_map:0) + + [W] lock(&ni->ni_lock:0) + [E] folio_unlock(&f1) + + [E] unlock(&ni->ni_lock:0) + +Adding supplementary comments to describe DEPT's view in detail: + + context A context B context ?(unknown) + + [S] folio_lock(&f1) + /* start to take into account context + B heading for folio_unlock(&f1) */ + /* 1 */ + [S] lock(&ni->ni_lock:0) + /* start to take into account this context heading for + unlock(&ni->ni_lock:0) */ + /* 2 */ + + [W] folio_wait_bit_common(PG_locked_map:0) (= folio_lock(&f1)) + /* might wait for folio_unlock(&f1) */ + + [W] lock(&ni->ni_lock:0) + /* might wait for unlock(&ni->ni_lock:0) */ + + [E] folio_unlock(&f1) + /* event that's been valid since 1 */ + + [E] unlock(&ni->ni_lock:0) + /* event that's been valid since 2 */ + +Let's build up dependency graph with this report. Firstly, context A: + + context A + + [S] lock(&ni->ni_lock:0) + /* start to take into account this context heading for + unlock(&ni->ni_lock:0) */ + /* 2 */ + + [W] folio_wait_bit_common(PG_locked_map:0) (= folio_lock(&f1)) + /* might wait for folio_unlock(&f1) */ + + [E] unlock(&ni->ni_lock:0) + /* event that's been valid since 2 */ + +There is one interesting event, unlock(&ni->ni_lock:0). There is a +wait, folio_lock(&f1), between 2 and the event. Which means +unlock(&ni->ni_lock:0) is not reachable if folio_unlock(&f1) does not +wake up the wait. Therefore, we can say unlock(&ni->ni_lock:0) depends +on folio_unlock(&f1), say, 'unlock(&ni->ni_lock:0) -> folio_unlock(&f1)'. +The graph will look like after adding the dependency: + + unlock(&ni->ni_lock:0) -> folio_unlock(&f1) + + where 'A -> B' means that event A depends on event B. + +Secondly, context B: + + context B + + [W] lock(&ni->ni_lock:0) + /* might wait for unlock(&ni->ni_lock:0) */ + + [E] folio_unlock(&f1) + /* event that's been valid since 1 */ + +There is also one interesting event, folio_unlock(&f1). There is a +wait, lock(&ni->ni_lock:0), between 1 and the event - remind 1 is at a +very start and before the wait in timeline. Which means folio_unlock(&f1) +is not reachable if unlock(&ni->ni_lock:0) does not wake up the wait. +Therefore, we can say folio_unlock(&f1) depends on unlock(&ni->ni_lock:0), +say, 'folio_unlock(&f1) -> unlock(&ni->ni_lock:0)'. The graph will look +like after adding the dependency: + + -> unlock(&ni->ni_lock:0) -> folio_unlock(&f1) - + / \ + \ / + ------------------------------------------------ + + where 'A -> B' means that event A depends on event B. + +A new loop has been created. So DEPT can report it as a deadlock! Cool! + +CONCLUSION + +DEPT works awesome!