From patchwork Wed Feb 21 09:49:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565306 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1C44F3D3A7; Wed, 21 Feb 2024 09:49:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; cv=none; b=B3ehnE+L8HHmubv6fVl80SVz/+keDdAhyb1MkGtu2Kl0LJNQ5uhi3mXh90JX8xdRYeTDfCoe3hSW3BXQ9jdXDFcA+QVw9KEAyU3pJUoLcXOpyEBUnrzjH2PssN3xRwsn2b/HPgU+M6k8hbcCyugYryj+EoMHnIl4HxXpMYtPlM0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; c=relaxed/simple; bh=3z6wcnzZigmy1AnwKW6EWYXgb3eqsvFGY6rKaYlxPHE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=pd/ia2amR2KX9FI+m9ITsBApz4TLSvyjk6RXADwglwoxdslZUV/jcR5em3qLrQKn0UQ6/IpwWR5z32KN5nMa9gjIgfdbc5Tn8q0dCAY4uAm5DpuIwX4QtzgKkVe97FIYMDm9mXXkRDkWwuZ4f8LjQq6QeOWHObR1gfdgvGgW79w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-96-65d5c7387f8b From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 01/27] llist: Move llist_{head,node} definition to types.h Date: Wed, 21 Feb 2024 18:49:07 +0900 Message-Id: <20240221094933.36348-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0xTZxgHcN5zec+h0nlSyXbELC4Vxft0inuye1zC3kUlZi7RoMZ1cmKr BbVFkCUuRbm4So24ABPQFHC1g6JyKtFN23QwqUjECtUxghWIUZAik63NKkxtNX558stz+X96 eFp1mU3iddk5kiFbo1djBaMYS6hdAt7b0rLOoiQoK10GoX8PM1BzzoHBd7YRgeNCAQUjV7+A P8NBBJM3btJQWe5DUDt4l4YL7QEELvtBDD333wB/aBxDR/kRDIfqz2G4NTpFQX/FcQoa5XXQ eayOAk/kIQOVIxiqKw9R0TJMQcTWwIHNNBeG7FUcTA0uh47AHRZcfYvgxKl+DFdcHQy0Xxqi oOe3GgwBx3MWOtuvMeArs7DQ9LgOw2jYRoMtNM5Bt8dKwfnCaFDxP89Y8Fo8FBSfbqbA/9dl BO7DAxTIjjsY2kJBCpxyOQ1Pz1xFMHR0jIOi0ggH1QVHERwpqmDg5v9eFgr7U2Hyvxr82Qek LThOk0JnHnGFrQy5XieSX6vucqTQ3ccRq7yPOO0LSf2VEYrUToRYIjf8gIk8cZwj5jE/RR53 dXHk2k+TDLnvr6TWz8pQfJQp6XW5kuHdT75RaIu7TnF7Tir2W39uwSbk5s0onheFlaLXbOFe W37S+tJYSBF7eyN0zInCO6LT8oA1IwVPCyXTRPvfN3BsMENIF9uetKOYGWGu2O18ysSsFFLF Zvke/Sp0tth43vPS8cIq8ZfqIBuzKrpz+1YLHQsVhZJ4sbnDz746mCn+bu9ljiGlFcU1IJUu OzdLo9OvXKrNz9btX7p9d5aMoi9lOzC1+RKa8G1oRQKP1AlK7UW/pGI1ucb8rFYk8rQ6Ucnk RVvKTE3+d5Jh9zbDPr1kbEWzeEb9lvK9cF6mStihyZF2SdIeyfB6SvHxSSaUX+PZ7uub83G6 Y/rMjOSAO/F5sXHtl6umhXdgLrJkhVWf8uOWtEdfzclBi60DcV+nxun+2Lxl9rfX530/uMa8 Vrt4IFC1KaGntCR1tXera3hFyuemt+OaCg5+augyZQQe1L1pcY/WB8u6328qWZ+80+Lb+2Ha cPLGBblp5EBL+omh1fPVjFGrWb6QNhg1LwC9bUM1TgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzXSf0yMcRwHcN/v8zzf5+k4e5Y2jx/z47bm16RG9pkMfxgPxowN87ObHu5W l9wlMmflUqSiLKGyKzmpQ56roZTWzeVEHR1iCc2vJkpccxW5Y/757LXP+7P3Xx+OCixkxnPa 2HhJH6uOUREFrVgTYZoNjU+l0KsuDNkZoeD5cYyGgutWAq5r5QislckYuu4th+f93QgGH7VQ kJfrQlD09hUFlY4OBLWlRwi0vhsNbk8PAWfuCQKmi9cJPP48hKH9TA6Gcnk1NJ0qxlDv/UhD XheB/DwT9o1PGLyWMhYsScHQWXqehaG3YeDseMaAvdDJQO3LWXDuQjuBO7VOGhy3OjG0VhcQ 6LAOM9DkuE+DKzuTgatfiwl87rdQYPH0sPCk3oyhIsXXlvr9NwONmfUYUktuYHC/qEFQd+wN Btn6jIDd043BJudSMHD5HoLOrC8sHM3wspCfnIXgxNEzNLT8amQgpT0cBn8WkCURor27hxJT bPvF2n4zLT4oFsTb51+xYkrdS1Y0y/tEW+lM8eKdLiwW9XkYUS47TkS5L4cV07+4sfi1uZkV 758dpMV37jy8duJmxcIoKUabIOnnLIpUaFKbL7BxhYoD5ktVJAnVcekogBP4eYL8rYH1m/DT hLY2L+V3ED9FsGV+YNKRgqP4tJFCae8j4g/G8GsE+zcH8pvmg4UntgHabyUfLtyQX1P/SicL 5RX1fx3Azxeu5Hczfgf6bp4+rqJOIYUZjShDQdrYBJ1aGxMeYojWJMZqD4Ts3KOTke9pLMah 7FvoR+vyBsRzSDVKqbnplgIZdYIhUdeABI5SBSnp/b6VMkqdeFDS79mh3xcjGRrQBI5WjVWu 3ChFBvK71fFStCTFSfr/KeYCxiehu8kObfyy6etqhpWjjORwm77lnHPRgqIJe9dHbtpspgYq pvZmZ6SVlMct3pH50bLJEjLJrK8arMia8XpdY5P1Yc5coZ8aGxzw/sHzjfZVp71jjLs0C8eN q+x1VW+4dNx0syPthVHdpzN1bivZoltx0HgolHxwbd86HBmxZenJGaqouyraoFGHzaT0BvUf NFPz6TADAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: llist_head and llist_node can be used by very primitives. For example, Dept for tracking dependency uses llist things in its header. To avoid header dependency, move those to types.h. Signed-off-by: Byungchul Park --- include/linux/llist.h | 8 -------- include/linux/types.h | 8 ++++++++ 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/include/linux/llist.h b/include/linux/llist.h index 2c982ff7475a..3ac071857612 100644 --- a/include/linux/llist.h +++ b/include/linux/llist.h @@ -53,14 +53,6 @@ #include #include -struct llist_head { - struct llist_node *first; -}; - -struct llist_node { - struct llist_node *next; -}; - #define LLIST_HEAD_INIT(name) { NULL } #define LLIST_HEAD(name) struct llist_head name = LLIST_HEAD_INIT(name) diff --git a/include/linux/types.h b/include/linux/types.h index 253168bb3fe1..10d94b7f9e5d 100644 --- a/include/linux/types.h +++ b/include/linux/types.h @@ -199,6 +199,14 @@ struct hlist_node { struct hlist_node *next, **pprev; }; +struct llist_head { + struct llist_node *first; +}; + +struct llist_node { + struct llist_node *next; +}; + struct ustat { __kernel_daddr_t f_tfree; #ifdef CONFIG_ARCH_32BIT_USTAT_F_TINODE From patchwork Wed Feb 21 09:49:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565311 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D8C53D3B7; Wed, 21 Feb 2024 09:49:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; cv=none; b=svGG2NdhiVd+P6RB638evqoM9HfNA6EvDi1V0e4EAaDM7p3OmrtcHHIOFEaFHvM1DykawGvtIhxh9S9LEP/M7r4jPRqxYMihXVrlFikbbaTz0qoUZVcT/pBwaykxiPBqHlCl+A/TOf8M2m5+3gCgmDx0vAs8E/5O51uUh+0zGyk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; c=relaxed/simple; bh=asSKRbIQNcB08UYPRwv3bUSMHrCIvUqTNtShomc8xTg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=a3mmUTTEBlabOfnJp3VbuXOoeuTVMc98bKcbDyqX7CzO1uf4UgCrZ6CvRNytcj7FOoqdKScjfL2q3/ubbPoflUwzQTEnk5wD5XJ37+ziVMlyFuk/Oax94ZufgU1Ou331l9il5ZgS1Qn6V5gay3llLRvoPTT7Ftg4yInTr4pGiaQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-a6-65d5c738a185 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 02/27] dept: Implement Dept(Dependency Tracker) Date: Wed, 21 Feb 2024 18:49:08 +0900 Message-Id: <20240221094933.36348-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAz2SbUiTaxjHu+/n1dXiYadDT4UU44RgbNXR4kIiggN2fymkAxEn6LTyoY10 ypZv0YualfmGKb4ctVgqO8uttK0XK2fLbGamWU6PlpOSqETNMLda6qmtqC8XP/7/6/p9unhK 0cIs53X6Q5JBr0lQsjJaNrmoVgUd/dK691UqOFuwDnwzuTTUNNpY6L1sRWC7moVh7P5W+M8/ gWC2+zEFFWW9CC689FJw1T2CwGnJZqHv1WLw+KZY6CzLZ+FEXSMLT8bnMAyXl2Cw2rdBV3Et BlfgDQ0VYyxUV5zAwfEWQ8DcwIE5czWMWqo4mHu5HjpHBhhwPlsD/5wfZqHF2UmDu3kUQ9+t GhZGbF8Y6HI/oKH3bCEDl97VsjDuN1Ng9k1x8NRlwtCUExSd+vA/Ax2FLgyn6q9g8AzdRtCa +wKD3TbAwj3fBAaHvYyCz//eRzBaNMnByYIAB9VZRQjyT5bT8Hi+g4Gc4Q0w+6mG3RJD7k1M USTHkUacfhNNHtaK5GaVlyM5rc84YrKnEIclktS1jGFyYdrHEHvDGZbYp0s4kjfpweRdTw9H HlTO0uSVpwLHrfhLtileStClSoa1m/fKtOP+ISZ5bopJ99s66Uz0xU3noTBeFKLFOkfdT/70 dIgLMStEiIODASrES4RVoqPwNZOHZDwlnF4oWt53s6HiF+EPsd1aivMQz9PCarFwPjYUy4UN osebz3x3rhStTa5vnjBho3ixeuJbrgju9D+5RoWconA6TGy9089+P1gm3rUM0sVIbkILGpBC p09N1OgSotXaDL0uXb0/KdGOgj9lPjq3uxlN9/7ZhgQeKRfJtTc8koLRpBozEtuQyFPKJXI6 LRjJ4zUZhyVD0t+GlATJ2IZW8LRyqfx3f1q8QjigOSQdlKRkyfCjxXzY8kx0uJlPv9h+JelR m3of3liSNFMf1fSwSI8j7LvM57I/77l7vDxF5TUxwqOFsbP1WmdfnMrbHq427rSVak091o+/ JQdiDmrbo8/QIwuuu1WxpVlxz+8w1HTU4O3GgY9rcjNKjiRmp8yTrsqtS9Xbfy2u3BGjCC/a vOWYy18QPj+zWEkbtZr1kZTBqPkKweiNwU8DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x47Lb2n8PGy4NFakkH1Gms2WH8MwefyDGz/u1pPdVcrY okN11RQViuVwrjpPd5mis5Qup6XoUKmm5KH1cMgdKXIx/3z23uv93uuvD0t4XaKmscroWFEV LY+U0RJSsmF58gKofSkGVmsmQlZ6IDi/pZBQcMtIQ+PNEgTG0mMYempWw2tXH4Lh+gYC8nIa EVzubCeg1NqBwGI4TkNTtyfYnQ4abDlaGpKv3KLhee8IhrbcbAwlpvVQd1qHoXLoIwl5PTTk 5yVj9/mEYUhfzIA+yRe6DBcYGOkMAlvHKwqqL9oosLT6w/lLbTRUWGwkWMu6MDTdL6ChwzhK QZ31CQmNWRkU3BjQ0dDr0hOgdzoYeFFZiOG2xm07OfibgtqMSgwnr97BYG95gOBhylsMJuMr GqqdfRjMphwCfl6vQdCV2c/AifQhBvKPZSLQnsgloeFXLQWatmAY/lFAr1wuVPc5CEFjPiRY XIWk8FTHC+UX2hlB87CVEQpNcYLZ4CdcqejBwuWvTkowFafSgulrNiOk9duxMPDsGSM8OTdM Ct32PLxxxk5JyD4xUhkvqhaG7pEoel0t1MERB5XgMtrIJDRqJdOQB8tzS/gfL1qYsUxzc/nm 5iFiLHtzs3hzxgcqDUlYgjs1njd8rqfHikncKv5xyRmchliW5Hz5jF9hY1jKBfP2di31zzmT L7ld+dfjwS3li/L7/nIv9+bl87vEaSQpROOKkbcyOj5KrowMDlBHKBKjlQkBe2OiTMj9Nfqj I1ll6FvT6irEsUg2Qaq4Zxe9KHm8OjGqCvEsIfOWkofcSLpPnnhYVMXsVsVFiuoqNJ0lZVOk a7eJe7y4A/JYMUIUD4qq/y1mPaYlodmyd74+hg9n666VZ6qTtGXxxnU11sHmBJ+dwSE5LR07 VrS9C5gz6ciuN0es87oxWlZUH6PY0mSzho1X+Lx/4GlRbYyImDxhMHC+QxtS7r81dHJcY4Ut dcqWGf6yL6VhTO3+bWh7nDk8/Du+nx42dbdDt8Zv08/NOuujrOzF86IWxcpItUIe5Eeo1PI/ E/58hTEDAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: CURRENT STATUS -------------- Lockdep tracks acquisition order of locks in order to detect deadlock, and IRQ and IRQ enable/disable state as well to take accident acquisitions into account. Lockdep should be turned off once it detects and reports a deadlock since the data structure and algorithm are not reusable after detection because of the complex design. PROBLEM ------- *Waits* and their *events* that never reach eventually cause deadlock. However, Lockdep is only interested in lock acquisition order, forcing to emulate lock acqusition even for just waits and events that have nothing to do with real lock. Even worse, no one likes Lockdep's false positive detection because that prevents further one that might be more valuable. That's why all the kernel developers are sensitive to Lockdep's false positive. Besides those, by tracking acquisition order, it cannot correctly deal with read lock and cross-event e.g. wait_for_completion()/complete() for deadlock detection. Lockdep is no longer a good tool for that purpose. SOLUTION -------- Again, *waits* and their *events* that never reach eventually cause deadlock. The new solution, Dept(DEPendency Tracker), focuses on waits and events themselves. Dept tracks waits and events and report it if any event would be never reachable. Dept does: . Works with read lock in the right way. . Works with any wait and event e.i. cross-event. . Continue to work even after reporting multiple times. . Provides simple and intuitive APIs. . Does exactly what dependency checker should do. Q & A ----- Q. Is this the first try ever to address the problem? A. No. Cross-release feature (b09be676e0ff2 locking/lockdep: Implement the 'crossrelease' feature) addressed it 2 years ago that was a Lockdep extension and merged but reverted shortly because: Cross-release started to report valuable hidden problems but started to give report false positive reports as well. For sure, no one likes Lockdep's false positive reports since it makes Lockdep stop, preventing reporting further real problems. Q. Why not Dept was developed as an extension of Lockdep? A. Lockdep definitely includes all the efforts great developers have made for a long time so as to be quite stable enough. But I had to design and implement newly because of the following: 1) Lockdep was designed to track lock acquisition order. The APIs and implementation do not fit on wait-event model. 2) Lockdep is turned off on detection including false positive. Which is terrible and prevents developing any extension for stronger detection. Q. Do you intend to totally replace Lockdep? A. No. Lockdep also checks if lock usage is correct. Of course, the dependency check routine should be replaced but the other functions should be still there. Q. Do you mean the dependency check routine should be replaced right away? A. No. I admit Lockdep is stable enough thanks to great efforts kernel developers have made. Lockdep and Dept, both should be in the kernel until Dept gets considered stable. Q. Stronger detection capability would give more false positive report. Which was a big problem when cross-release was introduced. Is it ok with Dept? A. It's ok. Dept allows multiple reporting thanks to simple and quite generalized design. Of course, false positive reports should be fixed anyway but it's no longer as a critical problem as it was. Signed-off-by: Byungchul Park --- include/linux/dept.h | 567 ++++++ include/linux/hardirq.h | 3 + include/linux/sched.h | 3 + init/init_task.c | 2 + init/main.c | 2 + kernel/Makefile | 1 + kernel/dependency/Makefile | 3 + kernel/dependency/dept.c | 2966 +++++++++++++++++++++++++++++++ kernel/dependency/dept_hash.h | 10 + kernel/dependency/dept_object.h | 13 + kernel/exit.c | 1 + kernel/fork.c | 2 + kernel/module/main.c | 4 + kernel/sched/core.c | 10 + lib/Kconfig.debug | 27 + lib/locking-selftest.c | 2 + 16 files changed, 3616 insertions(+) create mode 100644 include/linux/dept.h create mode 100644 kernel/dependency/Makefile create mode 100644 kernel/dependency/dept.c create mode 100644 kernel/dependency/dept_hash.h create mode 100644 kernel/dependency/dept_object.h diff --git a/include/linux/dept.h b/include/linux/dept.h new file mode 100644 index 000000000000..c6e2291dd843 --- /dev/null +++ b/include/linux/dept.h @@ -0,0 +1,567 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DEPT(DEPendency Tracker) - runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_H +#define __LINUX_DEPT_H + +#ifdef CONFIG_DEPT + +#include + +struct task_struct; + +#define DEPT_MAX_STACK_ENTRY 16 +#define DEPT_MAX_WAIT_HIST 64 +#define DEPT_MAX_ECXT_HELD 48 + +#define DEPT_MAX_SUBCLASSES 16 +#define DEPT_MAX_SUBCLASSES_EVT 2 +#define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) +#define DEPT_MAX_SUBCLASSES_CACHE 2 + +#define DEPT_SIRQ 0 +#define DEPT_HIRQ 1 +#define DEPT_IRQS_NR 2 +#define DEPT_SIRQF (1UL << DEPT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_HIRQ) + +struct dept_ecxt; +struct dept_iecxt { + struct dept_ecxt *ecxt; + int enirq; + /* + * for preventing to add a new ecxt + */ + bool staled; +}; + +struct dept_wait; +struct dept_iwait { + struct dept_wait *wait; + int irq; + /* + * for preventing to add a new wait + */ + bool staled; + bool touched; +}; + +struct dept_class { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * unique information about the class + */ + const char *name; + unsigned long key; + int sub_id; + + /* + * for BFS + */ + unsigned int bfs_gen; + int bfs_dist; + struct dept_class *bfs_parent; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking all classes + */ + struct list_head all_node; + + /* + * for associating its dependencies + */ + struct list_head dep_head; + struct list_head dep_rev_head; + + /* + * for tracking IRQ dependencies + */ + struct dept_iecxt iecxt[DEPT_IRQS_NR]; + struct dept_iwait iwait[DEPT_IRQS_NR]; + + /* + * classified by a map embedded in task_struct, + * not an explicit map + */ + bool sched_map; + }; + }; +}; + +struct dept_key { + union { + /* + * Each byte-wise address will be used as its key. + */ + char base[DEPT_MAX_SUBCLASSES]; + + /* + * for caching the main class pointer + */ + struct dept_class *classes[DEPT_MAX_SUBCLASSES_CACHE]; + }; +}; + +struct dept_map { + const char *name; + struct dept_key *keys; + + /* + * subclass that can be set from user + */ + int sub_u; + + /* + * It's local copy for fast access to the associated classes. + * Also used for dept_key for static maps. + */ + struct dept_key map_key; + + /* + * wait timestamp associated to this map + */ + unsigned int wgen; + + /* + * whether this map should be going to be checked or not + */ + bool nocheck; +}; + +#define DEPT_MAP_INITIALIZER(n, k) \ +{ \ + .name = #n, \ + .keys = (struct dept_key *)(k), \ + .sub_u = 0, \ + .map_key = { .classes = { NULL, } }, \ + .wgen = 0U, \ + .nocheck = false, \ +} + +struct dept_stack { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * backtrace entries + */ + unsigned long raw[DEPT_MAX_STACK_ENTRY]; + int nr; + }; + }; +}; + +struct dept_ecxt { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function that entered to this ecxt + */ + const char *ecxt_fn; + + /* + * event function + */ + const char *event_fn; + + /* + * associated class + */ + struct dept_class *class; + + /* + * flag indicating which IRQ has been + * enabled within the event context + */ + unsigned long enirqf; + + /* + * where the IRQ-enabled happened + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + + /* + * where the event context started + */ + unsigned long ecxt_ip; + struct dept_stack *ecxt_stack; + + /* + * where the event triggered + */ + unsigned long event_ip; + struct dept_stack *event_stack; + }; + }; +}; + +struct dept_wait { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * function causing this wait + */ + const char *wait_fn; + + /* + * the associated class + */ + struct dept_class *class; + + /* + * which IRQ the wait was placed in + */ + unsigned long irqf; + + /* + * where the IRQ wait happened + */ + unsigned long irq_ip[DEPT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_IRQS_NR]; + + /* + * where the wait happened + */ + unsigned long wait_ip; + struct dept_stack *wait_stack; + + /* + * whether this wait is for commit in scheduler + */ + bool sched_sleep; + }; + }; +}; + +struct dept_dep { + union { + struct llist_node pool_node; + struct { + /* + * reference counter for object management + */ + atomic_t ref; + + /* + * key data of dependency + */ + struct dept_ecxt *ecxt; + struct dept_wait *wait; + + /* + * This object can be referred without dept_lock + * held but with IRQ disabled, e.g. for hash + * lookup. So deferred deletion is needed. + */ + struct rcu_head rh; + + /* + * for BFS + */ + struct list_head bfs_node; + + /* + * for hashing this object + */ + struct hlist_node hash_node; + + /* + * for linking to a class object + */ + struct list_head dep_node; + struct list_head dep_rev_node; + }; + }; +}; + +struct dept_hash { + /* + * hash table + */ + struct hlist_head *table; + + /* + * size of the table e.i. 2^bits + */ + int bits; +}; + +struct dept_pool { + const char *name; + + /* + * object size + */ + size_t obj_sz; + + /* + * the number of the static array + */ + atomic_t obj_nr; + + /* + * offset of ->pool_node + */ + size_t node_off; + + /* + * pointer to the pool + */ + void *spool; + struct llist_head boot_pool; + struct llist_head __percpu *lpool; +}; + +struct dept_ecxt_held { + /* + * associated event context + */ + struct dept_ecxt *ecxt; + + /* + * unique key for this dept_ecxt_held + */ + struct dept_map *map; + + /* + * class of the ecxt of this dept_ecxt_held + */ + struct dept_class *class; + + /* + * the wgen when the event context started + */ + unsigned int wgen; + + /* + * subclass that only works in the local context + */ + int sub_l; +}; + +struct dept_wait_hist { + /* + * associated wait + */ + struct dept_wait *wait; + + /* + * unique id of all waits system-wise until wrapped + */ + unsigned int wgen; + + /* + * local context id to identify IRQ context + */ + unsigned int ctxt_id; +}; + +struct dept_task { + /* + * all event contexts that have entered and before exiting + */ + struct dept_ecxt_held ecxt_held[DEPT_MAX_ECXT_HELD]; + int ecxt_held_pos; + + /* + * ring buffer holding all waits that have happened + */ + struct dept_wait_hist wait_hist[DEPT_MAX_WAIT_HIST]; + int wait_hist_pos; + + /* + * sequential id to identify each IRQ context + */ + unsigned int irq_id[DEPT_IRQS_NR]; + + /* + * for tracking IRQ-enabled points with cross-event + */ + unsigned int wgen_enirq[DEPT_IRQS_NR]; + + /* + * for keeping up-to-date IRQ-enabled points + */ + unsigned long enirq_ip[DEPT_IRQS_NR]; + + /* + * for reserving a current stack instance at each operation + */ + struct dept_stack *stack; + + /* + * for preventing recursive call into DEPT engine + */ + int recursive; + + /* + * for staging data to commit a wait + */ + struct dept_map stage_m; + bool stage_sched_map; + const char *stage_w_fn; + unsigned long stage_ip; + + /* + * the number of missing ecxts + */ + int missing_ecxt; + + /* + * for tracking IRQ-enable state + */ + bool hardirqs_enabled; + bool softirqs_enabled; + + /* + * whether the current is on do_exit() + */ + bool task_exit; + + /* + * whether the current is running __schedule() + */ + bool in_sched; +}; + +#define DEPT_TASK_INITIALIZER(t) \ +{ \ + .wait_hist = { { .wait = NULL, } }, \ + .ecxt_held_pos = 0, \ + .wait_hist_pos = 0, \ + .irq_id = { 0U }, \ + .wgen_enirq = { 0U }, \ + .enirq_ip = { 0UL }, \ + .stack = NULL, \ + .recursive = 0, \ + .stage_m = DEPT_MAP_INITIALIZER((t)->stage_m, NULL), \ + .stage_sched_map = false, \ + .stage_w_fn = NULL, \ + .stage_ip = 0UL, \ + .missing_ecxt = 0, \ + .hardirqs_enabled = false, \ + .softirqs_enabled = false, \ + .task_exit = false, \ + .in_sched = false, \ +} + +extern void dept_on(void); +extern void dept_off(void); +extern void dept_init(void); +extern void dept_task_init(struct task_struct *t); +extern void dept_task_exit(struct task_struct *t); +extern void dept_free_range(void *start, unsigned int sz); +extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_map_copy(struct dept_map *to, struct dept_map *from); + +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_request_event_wait_commit(void); +extern void dept_clean_stage(void); +extern void dept_stage_event(struct task_struct *t, unsigned long ip); +extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); +extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); +extern void dept_request_event(struct dept_map *m); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); +extern void dept_sched_enter(void); +extern void dept_sched_exit(void); + +static inline void dept_ecxt_enter_nokeep(struct dept_map *m) +{ + dept_ecxt_enter(m, 0UL, 0UL, NULL, NULL, 0); +} + +/* + * for users who want to manage external keys + */ +extern void dept_key_init(struct dept_key *k); +extern void dept_key_destroy(struct dept_key *k); +extern void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, struct dept_key *new_k, unsigned long new_e_f, unsigned long new_ip, const char *new_c_fn, const char *new_e_fn, int new_sub_l); + +extern void dept_softirq_enter(void); +extern void dept_hardirq_enter(void); +extern void dept_softirqs_on_ip(unsigned long ip); +extern void dept_hardirqs_on(void); +extern void dept_softirqs_off(void); +extern void dept_hardirqs_off(void); +#else /* !CONFIG_DEPT */ +struct dept_key { }; +struct dept_map { }; +struct dept_task { }; + +#define DEPT_MAP_INITIALIZER(n, k) { } +#define DEPT_TASK_INITIALIZER(t) { } + +#define dept_on() do { } while (0) +#define dept_off() do { } while (0) +#define dept_init() do { } while (0) +#define dept_task_init(t) do { } while (0) +#define dept_task_exit(t) do { } while (0) +#define dept_free_range(s, sz) do { } while (0) +#define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_map_copy(t, f) do { } while (0) + +#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_request_event_wait_commit() do { } while (0) +#define dept_clean_stage() do { } while (0) +#define dept_stage_event(t, ip) do { } while (0) +#define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) +#define dept_ecxt_holding(m, e_f) false +#define dept_request_event(m) do { } while (0) +#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_ecxt_exit(m, e_f, ip) do { } while (0) +#define dept_sched_enter() do { } while (0) +#define dept_sched_exit() do { } while (0) +#define dept_ecxt_enter_nokeep(m) do { } while (0) +#define dept_key_init(k) do { (void)(k); } while (0) +#define dept_key_destroy(k) do { (void)(k); } while (0) +#define dept_map_ecxt_modify(m, e_f, n_k, n_e_f, n_ip, n_c_fn, n_e_fn, n_sl) do { (void)(n_k); (void)(n_c_fn); (void)(n_e_fn); } while (0) + +#define dept_softirq_enter() do { } while (0) +#define dept_hardirq_enter() do { } while (0) +#define dept_softirqs_on_ip(ip) do { } while (0) +#define dept_hardirqs_on() do { } while (0) +#define dept_softirqs_off() do { } while (0) +#define dept_hardirqs_off() do { } while (0) +#endif +#endif /* __LINUX_DEPT_H */ diff --git a/include/linux/hardirq.h b/include/linux/hardirq.h index d57cab4d4c06..bb279dbbe748 100644 --- a/include/linux/hardirq.h +++ b/include/linux/hardirq.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include #include @@ -106,6 +107,7 @@ void irq_exit_rcu(void); */ #define __nmi_enter() \ do { \ + dept_off(); \ lockdep_off(); \ arch_nmi_enter(); \ BUG_ON(in_nmi() == NMI_MASK); \ @@ -128,6 +130,7 @@ void irq_exit_rcu(void); __preempt_count_sub(NMI_OFFSET + HARDIRQ_OFFSET); \ arch_nmi_exit(); \ lockdep_on(); \ + dept_on(); \ } while (0) #define nmi_exit() \ diff --git a/include/linux/sched.h b/include/linux/sched.h index 292c31697248..6dafdc93b462 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -38,6 +38,7 @@ #include #include #include +#include /* task_struct member predeclarations (sorted alphabetically): */ struct audit_context; @@ -1178,6 +1179,8 @@ struct task_struct { struct held_lock held_locks[MAX_LOCK_DEPTH]; #endif + struct dept_task dept_task; + #if defined(CONFIG_UBSAN) && !defined(CONFIG_UBSAN_TRAP) unsigned int in_ubsan; #endif diff --git a/init/init_task.c b/init/init_task.c index 5727d42149c3..171572fbdb43 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -12,6 +12,7 @@ #include #include #include +#include #include @@ -194,6 +195,7 @@ struct task_struct init_task .curr_chain_key = INITIAL_CHAIN_KEY, .lockdep_recursion = 0, #endif + .dept_task = DEPT_TASK_INITIALIZER(init_task), #ifdef CONFIG_FUNCTION_GRAPH_TRACER .ret_stack = NULL, .tracing_graph_pause = ATOMIC_INIT(0), diff --git a/init/main.c b/init/main.c index e24b0780fdff..39dac78fafc1 100644 --- a/init/main.c +++ b/init/main.c @@ -65,6 +65,7 @@ #include #include #include +#include #include #include #include @@ -1011,6 +1012,7 @@ void start_kernel(void) panic_param); lockdep_init(); + dept_init(); /* * Need to run this when irqs are enabled, because it wants diff --git a/kernel/Makefile b/kernel/Makefile index 3947122d618b..e864b66d6470 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -51,6 +51,7 @@ obj-y += livepatch/ obj-y += dma/ obj-y += entry/ obj-$(CONFIG_MODULES) += module/ +obj-y += dependency/ obj-$(CONFIG_KCMP) += kcmp.o obj-$(CONFIG_FREEZER) += freezer.o diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile new file mode 100644 index 000000000000..b5cfb8a03c0c --- /dev/null +++ b/kernel/dependency/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_DEPT) += dept.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c new file mode 100644 index 000000000000..a3e774479f94 --- /dev/null +++ b/kernel/dependency/dept.c @@ -0,0 +1,2966 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DEPT(DEPendency Tracker) - Runtime dependency tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + * + * DEPT provides a general way to detect deadlock possibility in runtime + * and the interest is not limited to typical lock but to every + * syncronization primitives. + * + * The following ideas were borrowed from LOCKDEP: + * + * 1) Use a graph to track relationship between classes. + * 2) Prevent performance regression using hash. + * + * The following items were enhanced from LOCKDEP: + * + * 1) Cover more deadlock cases. + * 2) Allow muliple reports. + * + * TODO: Both LOCKDEP and DEPT should co-exist until DEPT is considered + * stable. Then the dependency check routine should be replaced with + * DEPT after. It should finally look like: + * + * + * + * As is: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | +-------------------------------------+ | + * | | Dependency check | | + * | | (by tracking lock acquisition order)| | + * | +-------------------------------------+ | + * | | + * +-----------------------------------------+ + * + * DEPT + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + * + * + * + * To be: + * + * LOCKDEP + * +-----------------------------------------+ + * | Lock usage correctness check | <-> locks + * | | + * | | + * | (Request dependency check) | + * | T | + * +--------------------|--------------------+ + * | + * DEPT V + * +-----------------------------------------+ + * | Dependency check | <-> waits/events + * | (by tracking wait and event context) | + * +-----------------------------------------+ + */ + +#include +#include +#include +#include +#include +#include +#include +#include + +static int dept_stop; +static int dept_per_cpu_ready; + +#define DEPT_READY_WARN (!oops_in_progress) + +/* + * Make all operations using DEPT_WARN_ON() fail on oops_in_progress and + * prevent warning message. + */ +#define DEPT_WARN_ON_ONCE(c) \ + ({ \ + int __ret = 0; \ + \ + if (likely(DEPT_READY_WARN)) \ + __ret = WARN_ONCE(c, "DEPT_WARN_ON_ONCE: " #c); \ + __ret; \ + }) + +#define DEPT_WARN_ONCE(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) \ + WARN_ONCE(1, "DEPT_WARN_ONCE: " s); \ + }) + +#define DEPT_WARN_ON(c) \ + ({ \ + int __ret = 0; \ + \ + if (likely(DEPT_READY_WARN)) \ + __ret = WARN(c, "DEPT_WARN_ON: " #c); \ + __ret; \ + }) + +#define DEPT_WARN(s...) \ + ({ \ + if (likely(DEPT_READY_WARN)) \ + WARN(1, "DEPT_WARN: " s); \ + }) + +#define DEPT_STOP(s...) \ + ({ \ + WRITE_ONCE(dept_stop, 1); \ + if (likely(DEPT_READY_WARN)) \ + WARN(1, "DEPT_STOP: " s); \ + }) + +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) + +static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; + +/* + * DEPT internal engine should be careful in using outside functions + * e.g. printk at reporting since that kind of usage might cause + * untrackable deadlock. + */ +static atomic_t dept_outworld = ATOMIC_INIT(0); + +static void dept_outworld_enter(void) +{ + atomic_inc(&dept_outworld); +} + +static void dept_outworld_exit(void) +{ + atomic_dec(&dept_outworld); +} + +static bool dept_outworld_entered(void) +{ + return atomic_read(&dept_outworld); +} + +static bool dept_lock(void) +{ + while (!arch_spin_trylock(&dept_spin)) + if (unlikely(dept_outworld_entered())) + return false; + return true; +} + +static void dept_unlock(void) +{ + arch_spin_unlock(&dept_spin); +} + +/* + * whether to stack-trace on every wait or every ecxt + */ +static bool rich_stack = true; + +enum bfs_ret { + BFS_CONTINUE, + BFS_CONTINUE_REV, + BFS_DONE, + BFS_SKIP, +}; + +static bool after(unsigned int a, unsigned int b) +{ + return (int)(b - a) < 0; +} + +static bool before(unsigned int a, unsigned int b) +{ + return (int)(a - b) < 0; +} + +static bool valid_stack(struct dept_stack *s) +{ + return s && s->nr > 0; +} + +static bool valid_class(struct dept_class *c) +{ + return c->key; +} + +static void invalidate_class(struct dept_class *c) +{ + c->key = 0UL; +} + +static struct dept_ecxt *dep_e(struct dept_dep *d) +{ + return d->ecxt; +} + +static struct dept_wait *dep_w(struct dept_dep *d) +{ + return d->wait; +} + +static struct dept_class *dep_fc(struct dept_dep *d) +{ + return dep_e(d)->class; +} + +static struct dept_class *dep_tc(struct dept_dep *d) +{ + return dep_w(d)->class; +} + +static const char *irq_str(int irq) +{ + if (irq == DEPT_SIRQ) + return "softirq"; + if (irq == DEPT_HIRQ) + return "hardirq"; + return "(unknown)"; +} + +static inline struct dept_task *dept_task(void) +{ + return ¤t->dept_task; +} + +/* + * Dept doesn't work either when it's stopped by DEPT_STOP() or in a nmi + * context. + */ +static bool dept_working(void) +{ + return !READ_ONCE(dept_stop) && !in_nmi(); +} + +/* + * Even k == NULL is considered as a valid key because it would use + * &->map_key as the key in that case. + */ +struct dept_key __dept_no_validate__; +static bool valid_key(struct dept_key *k) +{ + return &__dept_no_validate__ != k; +} + +/* + * Pool + * ===================================================================== + * DEPT maintains pools to provide objects in a safe way. + * + * 1) Static pool is used at the beginning of booting time. + * 2) Local pool is tried first before the static pool. Objects that + * have been freed will be placed. + */ + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +#define OBJECT(id, nr) \ +static struct dept_##id spool_##id[nr]; \ +static DEFINE_PER_CPU(struct llist_head, lpool_##id); + #include "dept_object.h" +#undef OBJECT + +static struct dept_pool pool[OBJECT_NR] = { +#define OBJECT(id, nr) { \ + .name = #id, \ + .obj_sz = sizeof(struct dept_##id), \ + .obj_nr = ATOMIC_INIT(nr), \ + .node_off = offsetof(struct dept_##id, pool_node), \ + .spool = spool_##id, \ + .lpool = &lpool_##id, }, + #include "dept_object.h" +#undef OBJECT +}; + +/* + * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is + * enabled or not because NMI and other contexts in the same CPU never + * run inside of DEPT concurrently by preventing reentrance. + */ +static void *from_pool(enum object_t t) +{ + struct dept_pool *p; + struct llist_head *h; + struct llist_node *n; + + /* + * llist_del_first() doesn't allow concurrent access e.g. + * between process and IRQ context. + */ + if (DEPT_WARN_ON(!irqs_disabled())) + return NULL; + + p = &pool[t]; + + /* + * Try local pool first. + */ + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + n = llist_del_first(h); + if (n) + return (void *)n - p->node_off; + + /* + * Try static pool. + */ + if (atomic_read(&p->obj_nr) > 0) { + int idx = atomic_dec_return(&p->obj_nr); + + if (idx >= 0) + return p->spool + (idx * p->obj_sz); + } + + DEPT_INFO_ONCE("---------------------------------------------\n" + " Some of Dept internal resources are run out.\n" + " Dept might still work if the resources get freed.\n" + " However, the chances are Dept will suffer from\n" + " the lack from now. Needs to extend the internal\n" + " resource pools. Ask max.byungchul.park@gmail.com\n"); + return NULL; +} + +static void to_pool(void *o, enum object_t t) +{ + struct dept_pool *p = &pool[t]; + struct llist_head *h; + + preempt_disable(); + if (likely(dept_per_cpu_ready)) + h = this_cpu_ptr(p->lpool); + else + h = &p->boot_pool; + + llist_add(o + p->node_off, h); + preempt_enable(); +} + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); \ +static void (*dtor_##id)(struct dept_##id *a); \ +static struct dept_##id *new_##id(void) \ +{ \ + struct dept_##id *a; \ + \ + a = (struct dept_##id *)from_pool(OBJECT_##id); \ + if (unlikely(!a)) \ + return NULL; \ + \ + atomic_set(&a->ref, 1); \ + \ + if (ctor_##id) \ + ctor_##id(a); \ + \ + return a; \ +} \ + \ +static struct dept_##id *get_##id(struct dept_##id *a) \ +{ \ + atomic_inc(&a->ref); \ + return a; \ +} \ + \ +static void put_##id(struct dept_##id *a) \ +{ \ + if (!atomic_dec_return(&a->ref)) { \ + if (dtor_##id) \ + dtor_##id(a); \ + to_pool(a, OBJECT_##id); \ + } \ +} \ + \ +static void del_##id(struct dept_##id *a) \ +{ \ + put_##id(a); \ +} \ + \ +static bool __maybe_unused id##_consumed(struct dept_##id *a) \ +{ \ + return a && atomic_read(&a->ref) > 1; \ +} +#include "dept_object.h" +#undef OBJECT + +#define SET_CONSTRUCTOR(id, f) \ +static void (*ctor_##id)(struct dept_##id *a) = f + +static void initialize_dep(struct dept_dep *d) +{ + INIT_LIST_HEAD(&d->bfs_node); + INIT_LIST_HEAD(&d->dep_node); + INIT_LIST_HEAD(&d->dep_rev_node); +} +SET_CONSTRUCTOR(dep, initialize_dep); + +static void initialize_class(struct dept_class *c) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iecxt *ie = &c->iecxt[i]; + struct dept_iwait *iw = &c->iwait[i]; + + ie->ecxt = NULL; + ie->enirq = i; + ie->staled = false; + + iw->wait = NULL; + iw->irq = i; + iw->staled = false; + iw->touched = false; + } + c->bfs_gen = 0U; + + INIT_LIST_HEAD(&c->all_node); + INIT_LIST_HEAD(&c->dep_head); + INIT_LIST_HEAD(&c->dep_rev_head); +} +SET_CONSTRUCTOR(class, initialize_class); + +static void initialize_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + e->enirq_stack[i] = NULL; + e->enirq_ip[i] = 0UL; + } + e->ecxt_ip = 0UL; + e->ecxt_stack = NULL; + e->enirqf = 0UL; + e->event_ip = 0UL; + e->event_stack = NULL; +} +SET_CONSTRUCTOR(ecxt, initialize_ecxt); + +static void initialize_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) { + w->irq_stack[i] = NULL; + w->irq_ip[i] = 0UL; + } + w->wait_ip = 0UL; + w->wait_stack = NULL; + w->irqf = 0UL; +} +SET_CONSTRUCTOR(wait, initialize_wait); + +static void initialize_stack(struct dept_stack *s) +{ + s->nr = 0; +} +SET_CONSTRUCTOR(stack, initialize_stack); + +#define OBJECT(id, nr) \ +static void (*ctor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_CONSTRUCTOR + +#define SET_DESTRUCTOR(id, f) \ +static void (*dtor_##id)(struct dept_##id *a) = f + +static void destroy_dep(struct dept_dep *d) +{ + if (dep_e(d)) + put_ecxt(dep_e(d)); + if (dep_w(d)) + put_wait(dep_w(d)); +} +SET_DESTRUCTOR(dep, destroy_dep); + +static void destroy_ecxt(struct dept_ecxt *e) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (e->enirq_stack[i]) + put_stack(e->enirq_stack[i]); + if (e->class) + put_class(e->class); + if (e->ecxt_stack) + put_stack(e->ecxt_stack); + if (e->event_stack) + put_stack(e->event_stack); +} +SET_DESTRUCTOR(ecxt, destroy_ecxt); + +static void destroy_wait(struct dept_wait *w) +{ + int i; + + for (i = 0; i < DEPT_IRQS_NR; i++) + if (w->irq_stack[i]) + put_stack(w->irq_stack[i]); + if (w->class) + put_class(w->class); + if (w->wait_stack) + put_stack(w->wait_stack); +} +SET_DESTRUCTOR(wait, destroy_wait); + +#define OBJECT(id, nr) \ +static void (*dtor_##id)(struct dept_##id *a); + #include "dept_object.h" +#undef OBJECT + +#undef SET_DESTRUCTOR + +/* + * Caching and hashing + * ===================================================================== + * DEPT makes use of caching and hashing to improve performance. Each + * object can be obtained in O(1) with its key. + * + * NOTE: Currently we assume all the objects in the hashs will never be + * removed. Implement it when needed. + */ + +/* + * Some information might be lost but it's only for hashing key. + */ +static unsigned long mix(unsigned long a, unsigned long b) +{ + int halfbits = sizeof(unsigned long) * 8 / 2; + unsigned long halfmask = (1UL << halfbits) - 1UL; + + return (a << halfbits) | (b & halfmask); +} + +static bool cmp_dep(struct dept_dep *d1, struct dept_dep *d2) +{ + return dep_fc(d1)->key == dep_fc(d2)->key && + dep_tc(d1)->key == dep_tc(d2)->key; +} + +static unsigned long key_dep(struct dept_dep *d) +{ + return mix(dep_fc(d)->key, dep_tc(d)->key); +} + +static bool cmp_class(struct dept_class *c1, struct dept_class *c2) +{ + return c1->key == c2->key; +} + +static unsigned long key_class(struct dept_class *c) +{ + return c->key; +} + +#define HASH(id, bits) \ +static struct hlist_head table_##id[1 << (bits)]; \ + \ +static struct hlist_head *head_##id(struct dept_##id *a) \ +{ \ + return table_##id + hash_long(key_##id(a), bits); \ +} \ + \ +static struct dept_##id *hash_lookup_##id(struct dept_##id *a) \ +{ \ + struct dept_##id *b; \ + \ + hlist_for_each_entry_rcu(b, head_##id(a), hash_node) \ + if (cmp_##id(a, b)) \ + return b; \ + return NULL; \ +} \ + \ +static void hash_add_##id(struct dept_##id *a) \ +{ \ + get_##id(a); \ + hlist_add_head_rcu(&a->hash_node, head_##id(a)); \ +} \ + \ +static void hash_del_##id(struct dept_##id *a) \ +{ \ + hlist_del_rcu(&a->hash_node); \ + put_##id(a); \ +} +#include "dept_hash.h" +#undef HASH + +static struct dept_dep *lookup_dep(struct dept_class *fc, + struct dept_class *tc) +{ + struct dept_ecxt onetime_e = { .class = fc }; + struct dept_wait onetime_w = { .class = tc }; + struct dept_dep onetime_d = { .ecxt = &onetime_e, + .wait = &onetime_w }; + return hash_lookup_dep(&onetime_d); +} + +static struct dept_class *lookup_class(unsigned long key) +{ + struct dept_class onetime_c = { .key = key }; + + return hash_lookup_class(&onetime_c); +} + +/* + * Report + * ===================================================================== + * DEPT prints useful information to help debuging on detection of + * problematic dependency. + */ + +static void print_ip_stack(unsigned long ip, struct dept_stack *s) +{ + if (ip) + print_ip_sym(KERN_WARNING, ip); + + if (valid_stack(s)) { + pr_warn("stacktrace:\n"); + stack_trace_print(s->raw, s->nr, 5); + } + + if (!ip && !valid_stack(s)) + pr_warn("(N/A)\n"); +} + +#define print_spc(spc, fmt, ...) \ + pr_warn("%*c" fmt, (spc) * 4, ' ', ##__VA_ARGS__) + +static void print_diagram(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + bool firstline = true; + int spc = 1; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + if (!firstline) + pr_warn("\nor\n\n"); + firstline = false; + + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, " <%s interrupt>\n", irq_str(irq)); + print_spc(spc + 1, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } + + if (!irqf) { + print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); + print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); + } +} + +static void print_dep(struct dept_dep *d) +{ + struct dept_ecxt *e = dep_e(d); + struct dept_wait *w = dep_w(d); + struct dept_class *fc = dep_fc(d); + struct dept_class *tc = dep_tc(d); + unsigned long irqf; + int irq; + const char *w_fn = w->wait_fn ?: "(unknown)"; + const char *e_fn = e->event_fn ?: "(unknown)"; + const char *c_fn = e->ecxt_fn ?: "(unknown)"; + const char *fc_n = fc->sched_map ? "" : (fc->name ?: "(unknown)"); + const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); + + irqf = e->enirqf & w->irqf; + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + pr_warn("%s has been enabled:\n", irq_str(irq)); + print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); + pr_warn("\n"); + + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d) in %s context:\n", + w_fn, tc_n, tc->sub_id, irq_str(irq)); + print_ip_stack(w->irq_ip[irq], w->irq_stack[irq]); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } + + if (!irqf) { + pr_warn("[S] %s(%s:%d):\n", c_fn, fc_n, fc->sub_id); + print_ip_stack(e->ecxt_ip, e->ecxt_stack); + pr_warn("\n"); + + pr_warn("[W] %s(%s:%d):\n", w_fn, tc_n, tc->sub_id); + print_ip_stack(w->wait_ip, w->wait_stack); + pr_warn("\n"); + + pr_warn("[E] %s(%s:%d):\n", e_fn, fc_n, fc->sub_id); + print_ip_stack(e->event_ip, e->event_stack); + } +} + +static void save_current_stack(int skip); + +/* + * Print all classes in a circle. + */ +static void print_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + int i; + + dept_outworld_enter(); + save_current_stack(6); + + pr_warn("===================================================\n"); + pr_warn("DEPT: Circular dependency has been detected.\n"); + pr_warn("%s %.*s %s\n", init_utsname()->release, + (int)strcspn(init_utsname()->version, " "), + init_utsname()->version, + print_tainted()); + pr_warn("---------------------------------------------------\n"); + pr_warn("summary\n"); + pr_warn("---------------------------------------------------\n"); + + if (fc == tc) + pr_warn("*** AA DEADLOCK ***\n\n"); + else + pr_warn("*** DEADLOCK ***\n\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + if (fc != c) + pr_warn("\n"); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("\n"); + pr_warn("[S]: start of the event context\n"); + pr_warn("[W]: the wait blocked\n"); + pr_warn("[E]: the event not reachable\n"); + + i = 0; + do { + struct dept_dep *d = lookup_dep(fc, tc); + + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c's detail\n", 'A' + i); + pr_warn("---------------------------------------------------\n"); + pr_warn("context %c\n", 'A' + (i++)); + print_diagram(d); + pr_warn("\n"); + print_dep(d); + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + pr_warn("---------------------------------------------------\n"); + pr_warn("information that might be helpful\n"); + pr_warn("---------------------------------------------------\n"); + dump_stack(); + + dept_outworld_exit(); +} + +/* + * BFS(Breadth First Search) + * ===================================================================== + * Whenever a new dependency is added into the graph, search the graph + * for a new circular dependency. + */ + +static void enqueue(struct list_head *h, struct dept_dep *d) +{ + list_add_tail(&d->bfs_node, h); +} + +static struct dept_dep *dequeue(struct list_head *h) +{ + struct dept_dep *d; + + d = list_first_entry(h, struct dept_dep, bfs_node); + list_del(&d->bfs_node); + return d; +} + +static bool empty(struct list_head *h) +{ + return list_empty(h); +} + +static void extend_queue(struct list_head *h, struct dept_class *cur) +{ + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_head, dep_node) { + struct dept_class *next = dep_tc(d); + + if (cur->bfs_gen == next->bfs_gen) + continue; + next->bfs_gen = cur->bfs_gen; + next->bfs_dist = cur->bfs_dist + 1; + next->bfs_parent = cur; + enqueue(h, d); + } +} + +static void extend_queue_rev(struct list_head *h, struct dept_class *cur) +{ + struct dept_dep *d; + + list_for_each_entry(d, &cur->dep_rev_head, dep_rev_node) { + struct dept_class *next = dep_fc(d); + + if (cur->bfs_gen == next->bfs_gen) + continue; + next->bfs_gen = cur->bfs_gen; + next->bfs_dist = cur->bfs_dist + 1; + next->bfs_parent = cur; + enqueue(h, d); + } +} + +typedef enum bfs_ret bfs_f(struct dept_dep *d, void *in, void **out); +static unsigned int bfs_gen; + +/* + * NOTE: Must be called with dept_lock held. + */ +static void bfs(struct dept_class *c, bfs_f *cb, void *in, void **out) +{ + LIST_HEAD(q); + enum bfs_ret ret; + + if (DEPT_WARN_ON(!cb)) + return; + + /* + * Avoid zero bfs_gen. + */ + bfs_gen = bfs_gen + 1 ?: 1; + + c->bfs_gen = bfs_gen; + c->bfs_dist = 0; + c->bfs_parent = c; + + ret = cb(NULL, in, out); + if (ret == BFS_DONE) + return; + if (ret == BFS_SKIP) + return; + if (ret == BFS_CONTINUE) + extend_queue(&q, c); + if (ret == BFS_CONTINUE_REV) + extend_queue_rev(&q, c); + + while (!empty(&q)) { + struct dept_dep *d = dequeue(&q); + + ret = cb(d, in, out); + if (ret == BFS_DONE) + break; + if (ret == BFS_SKIP) + continue; + if (ret == BFS_CONTINUE) + extend_queue(&q, dep_tc(d)); + if (ret == BFS_CONTINUE_REV) + extend_queue_rev(&q, dep_fc(d)); + } + + while (!empty(&q)) + dequeue(&q); +} + +/* + * Main operations + * ===================================================================== + * Add dependencies - Each new dependency is added into the graph and + * checked if it forms a circular dependency. + * + * Track waits - Waits are queued into the ring buffer for later use to + * generate appropriate dependencies with cross-event. + * + * Track event contexts(ecxt) - Event contexts are pushed into local + * stack for later use to generate appropriate dependencies with waits. + */ + +static unsigned long cur_enirqf(void); +static int cur_irq(void); +static unsigned int cur_ctxt_id(void); + +static struct dept_iecxt *iecxt(struct dept_class *c, int irq) +{ + return &c->iecxt[irq]; +} + +static struct dept_iwait *iwait(struct dept_class *c, int irq) +{ + return &c->iwait[irq]; +} + +static void stale_iecxt(struct dept_iecxt *ie) +{ + if (ie->ecxt) + put_ecxt(ie->ecxt); + + WRITE_ONCE(ie->ecxt, NULL); + WRITE_ONCE(ie->staled, true); +} + +static void set_iecxt(struct dept_iecxt *ie, struct dept_ecxt *e) +{ + /* + * ->ecxt will never be updated once getting set until the class + * gets removed. + */ + if (ie->ecxt) + DEPT_WARN_ON(1); + else + WRITE_ONCE(ie->ecxt, get_ecxt(e)); +} + +static void stale_iwait(struct dept_iwait *iw) +{ + if (iw->wait) + put_wait(iw->wait); + + WRITE_ONCE(iw->wait, NULL); + WRITE_ONCE(iw->staled, true); +} + +static void set_iwait(struct dept_iwait *iw, struct dept_wait *w) +{ + /* + * ->wait will never be updated once getting set until the class + * gets removed. + */ + if (iw->wait) + DEPT_WARN_ON(1); + else + WRITE_ONCE(iw->wait, get_wait(w)); + + iw->touched = true; +} + +static void touch_iwait(struct dept_iwait *iw) +{ + iw->touched = true; +} + +static void untouch_iwait(struct dept_iwait *iw) +{ + iw->touched = false; +} + +static struct dept_stack *get_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + return s ? get_stack(s) : NULL; +} + +static void prepare_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + /* + * The dept_stack is already ready. + */ + if (s && !stack_consumed(s)) { + s->nr = 0; + return; + } + + if (s) + put_stack(s); + + s = dept_task()->stack = new_stack(); + if (!s) + return; + + get_stack(s); + del_stack(s); +} + +static void save_current_stack(int skip) +{ + struct dept_stack *s = dept_task()->stack; + + if (!s) + return; + if (valid_stack(s)) + return; + + s->nr = stack_trace_save(s->raw, DEPT_MAX_STACK_ENTRY, skip); +} + +static void finish_current_stack(void) +{ + struct dept_stack *s = dept_task()->stack; + + if (stack_consumed(s)) + save_current_stack(2); +} + +/* + * FIXME: For now, disable LOCKDEP while DEPT is working. + * + * Both LOCKDEP and DEPT report it on a deadlock detection using + * printk taking the risk of another deadlock that might be caused by + * locks of console or printk between inside and outside of them. + * + * For DEPT, it's no problem since multiple reports are allowed. But it + * would be a bad idea for LOCKDEP since it will stop even on a singe + * report. So we need to prevent LOCKDEP from its reporting the risk + * DEPT would take when reporting something. + */ +#include + +void noinstr dept_off(void) +{ + dept_task()->recursive++; + lockdep_off(); +} + +void noinstr dept_on(void) +{ + dept_task()->recursive--; + lockdep_on(); +} + +static unsigned long dept_enter(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + dept_off(); + prepare_current_stack(); + return flags; +} + +static void dept_exit(unsigned long flags) +{ + finish_current_stack(); + dept_on(); + arch_local_irq_restore(flags); +} + +static unsigned long dept_enter_recursive(void) +{ + unsigned long flags; + + flags = arch_local_irq_save(); + return flags; +} + +static void dept_exit_recursive(unsigned long flags) +{ + arch_local_irq_restore(flags); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static struct dept_dep *__add_dep(struct dept_ecxt *e, + struct dept_wait *w) +{ + struct dept_dep *d; + + if (DEPT_WARN_ON(!valid_class(e->class))) + return NULL; + + if (DEPT_WARN_ON(!valid_class(w->class))) + return NULL; + + if (lookup_dep(e->class, w->class)) + return NULL; + + d = new_dep(); + if (unlikely(!d)) + return NULL; + + d->ecxt = get_ecxt(e); + d->wait = get_wait(w); + + /* + * Add the dependency into hash and graph. + */ + hash_add_dep(d); + list_add(&d->dep_node, &dep_fc(d)->dep_head); + list_add(&d->dep_rev_node, &dep_tc(d)->dep_rev_head); + return d; +} + +static enum bfs_ret cb_check_dl(struct dept_dep *d, + void *in, void **out) +{ + struct dept_dep *new = (struct dept_dep *)in; + + /* + * initial condition for this BFS search + */ + if (!d) { + dep_tc(new)->bfs_parent = dep_fc(new); + + if (dep_tc(new) != dep_fc(new)) + return BFS_CONTINUE; + + /* + * AA circle does not make additional deadlock. We don't + * have to continue this BFS search. + */ + print_circle(dep_tc(new)); + return BFS_DONE; + } + + /* + * Allow multiple reports. + */ + if (dep_tc(d) == dep_fc(new)) + print_circle(dep_tc(new)); + + return BFS_CONTINUE; +} + +/* + * This function is actually in charge of reporting. + */ +static void check_dl_bfs(struct dept_dep *d) +{ + bfs(dep_tc(d), cb_check_dl, (void *)d, NULL); +} + +static enum bfs_ret cb_find_iw(struct dept_dep *d, void *in, void **out) +{ + int irq = *(int *)in; + struct dept_class *fc; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + /* + * initial condition for this BFS search + */ + if (!d) + return BFS_CONTINUE_REV; + + fc = dep_fc(d); + iw = iwait(fc, irq); + + /* + * If any parent's ->wait was set, then the children would've + * been touched. + */ + if (!iw->touched) + return BFS_SKIP; + + if (!iw->wait) + return BFS_CONTINUE_REV; + + *out = iw; + return BFS_DONE; +} + +static struct dept_iwait *find_iw_bfs(struct dept_class *c, int irq) +{ + struct dept_iwait *iw = iwait(c, irq); + struct dept_iwait *found = NULL; + + if (iw->wait) + return iw; + + /* + * '->touched == false' guarantees there's no parent that has + * been set ->wait. + */ + if (!iw->touched) + return NULL; + + bfs(c, cb_find_iw, (void *)&irq, (void **)&found); + + if (found) + return found; + + untouch_iwait(iw); + return NULL; +} + +static enum bfs_ret cb_touch_iw_find_ie(struct dept_dep *d, void *in, + void **out) +{ + int irq = *(int *)in; + struct dept_class *tc; + struct dept_iecxt *ie; + struct dept_iwait *iw; + + if (DEPT_WARN_ON(!out)) + return BFS_DONE; + + /* + * initial condition for this BFS search + */ + if (!d) + return BFS_CONTINUE; + + tc = dep_tc(d); + ie = iecxt(tc, irq); + iw = iwait(tc, irq); + + touch_iwait(iw); + + if (!ie->ecxt) + return BFS_CONTINUE; + + if (!*out) + *out = ie; + + return BFS_CONTINUE; +} + +static struct dept_iecxt *touch_iw_find_ie_bfs(struct dept_class *c, + int irq) +{ + struct dept_iecxt *ie = iecxt(c, irq); + struct dept_iwait *iw = iwait(c, irq); + struct dept_iecxt *found = ie->ecxt ? ie : NULL; + + touch_iwait(iw); + bfs(c, cb_touch_iw_find_ie, (void *)&irq, (void **)&found); + return found; +} + +/* + * Should be called with dept_lock held. + */ +static void __add_idep(struct dept_iecxt *ie, struct dept_iwait *iw) +{ + struct dept_dep *new; + + /* + * There's nothing to do. + */ + if (!ie || !iw || !ie->ecxt || !iw->wait) + return; + + new = __add_dep(ie->ecxt, iw->wait); + + /* + * Deadlock detected. Let check_dl_bfs() report it. + */ + if (new) { + check_dl_bfs(new); + stale_iecxt(ie); + stale_iwait(iw); + } + + /* + * If !new, it would be the case of lack of object resource. + * Just let it go and get checked by other chances. Retrying is + * meaningless in that case. + */ +} + +static void set_check_iecxt(struct dept_class *c, int irq, + struct dept_ecxt *e) +{ + struct dept_iecxt *ie = iecxt(c, irq); + + set_iecxt(ie, e); + __add_idep(ie, find_iw_bfs(c, irq)); +} + +static void set_check_iwait(struct dept_class *c, int irq, + struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + set_iwait(iw, w); + __add_idep(touch_iw_find_ie_bfs(c, irq), iw); +} + +static void add_iecxt(struct dept_class *c, int irq, struct dept_ecxt *e, + bool stack) +{ + /* + * This access is safe since we ensure e->class has set locally. + */ + struct dept_task *dt = dept_task(); + struct dept_iecxt *ie = iecxt(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(ie->staled))) + return; + + /* + * Skip add_iecxt() if ie->ecxt has ever been set at least once. + * Which means it has a valid ->ecxt or been staled. + */ + if (READ_ONCE(ie->ecxt)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(ie->staled)) + goto unlock; + if (ie->ecxt) + goto unlock; + + e->enirqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * enirq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(e->enirq_ip[irq]); + DEPT_WARN_ON(e->enirq_stack[irq]); + + e->enirq_ip[irq] = dt->enirq_ip[irq]; + e->enirq_stack[irq] = stack ? get_current_stack() : NULL; + + set_check_iecxt(c, irq, e); +unlock: + dept_unlock(); +} + +static void add_iwait(struct dept_class *c, int irq, struct dept_wait *w) +{ + struct dept_iwait *iw = iwait(c, irq); + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (unlikely(READ_ONCE(iw->staled))) + return; + + /* + * Skip add_iwait() if iw->wait has ever been set at least once. + * Which means it has a valid ->wait or been staled. + */ + if (READ_ONCE(iw->wait)) + return; + + if (unlikely(!dept_lock())) + return; + + if (unlikely(iw->staled)) + goto unlock; + if (iw->wait) + goto unlock; + + w->irqf |= (1UL << irq); + + /* + * Should be NULL since it's the first time that these + * irq_{ip,stack}[irq] have ever set. + */ + DEPT_WARN_ON(w->irq_ip[irq]); + DEPT_WARN_ON(w->irq_stack[irq]); + + w->irq_ip[irq] = w->wait_ip; + w->irq_stack[irq] = get_current_stack(); + + set_check_iwait(c, irq, w); +unlock: + dept_unlock(); +} + +static struct dept_wait_hist *hist(int pos) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist + (pos % DEPT_MAX_WAIT_HIST); +} + +static int hist_pos_next(void) +{ + struct dept_task *dt = dept_task(); + + return dt->wait_hist_pos % DEPT_MAX_WAIT_HIST; +} + +static void hist_advance(void) +{ + struct dept_task *dt = dept_task(); + + dt->wait_hist_pos++; + dt->wait_hist_pos %= DEPT_MAX_WAIT_HIST; +} + +static struct dept_wait_hist *new_hist(void) +{ + struct dept_wait_hist *wh = hist(hist_pos_next()); + + hist_advance(); + return wh; +} + +static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) +{ + struct dept_wait_hist *wh = new_hist(); + + if (likely(wh->wait)) + put_wait(wh->wait); + + wh->wait = get_wait(w); + wh->wgen = wg; + wh->ctxt_id = ctxt_id; +} + +/* + * Should be called after setting up e's iecxt and w's iwait. + */ +static void add_dep(struct dept_ecxt *e, struct dept_wait *w) +{ + struct dept_class *fc = e->class; + struct dept_class *tc = w->class; + struct dept_dep *d; + int i; + + if (lookup_dep(fc, tc)) + return; + + if (unlikely(!dept_lock())) + return; + + /* + * __add_dep() will lookup_dep() again with lock held. + */ + d = __add_dep(e, w); + if (d) { + check_dl_bfs(d); + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_iwait *fiw = iwait(fc, i); + struct dept_iecxt *found_ie; + struct dept_iwait *found_iw; + + /* + * '->touched == false' guarantees there's no + * parent that has been set ->wait. + */ + if (!fiw->touched) + continue; + + /* + * find_iw_bfs() will untouch the iwait if + * not found. + */ + found_iw = find_iw_bfs(fc, i); + + if (!found_iw) + continue; + + found_ie = touch_iw_find_ie_bfs(tc, i); + __add_idep(found_ie, found_iw); + } + } + dept_unlock(); +} + +static atomic_t wgen = ATOMIC_INIT(1); + +static void add_wait(struct dept_class *c, unsigned long ip, + const char *w_fn, int sub_l, bool sched_sleep) +{ + struct dept_task *dt = dept_task(); + struct dept_wait *w; + unsigned int wg = 0U; + int irq; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + w = new_wait(); + if (unlikely(!w)) + return; + + WRITE_ONCE(w->class, get_class(c)); + w->wait_ip = ip; + w->wait_fn = w_fn; + w->wait_stack = get_current_stack(); + w->sched_sleep = sched_sleep; + + irq = cur_irq(); + if (irq < DEPT_IRQS_NR) + add_iwait(c, irq, w); + + /* + * Avoid adding dependency between user aware nested ecxt and + * wait. + */ + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + + /* + * the case of invalid key'ed one + */ + if (!eh->ecxt) + continue; + + if (eh->ecxt->class != c || eh->sub_l == sub_l) + add_dep(eh->ecxt, w); + } + + if (!wait_consumed(w) && !rich_stack) { + if (w->wait_stack) + put_stack(w->wait_stack); + w->wait_stack = NULL; + } + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + add_hist(w, wg, cur_ctxt_id()); + + del_wait(w); +} + +static bool add_ecxt(struct dept_map *m, struct dept_class *c, + unsigned long ip, const char *c_fn, + const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + unsigned long irqf; + int irq; + + if (DEPT_WARN_ON(!valid_class(c))) + return false; + + if (DEPT_WARN_ON_ONCE(dt->ecxt_held_pos >= DEPT_MAX_ECXT_HELD)) + return false; + + if (m->nocheck) { + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = NULL; + eh->map = m; + eh->class = get_class(c); + eh->wgen = atomic_read(&wgen); + eh->sub_l = sub_l; + + return true; + } + + e = new_ecxt(); + if (unlikely(!e)) + return false; + + e->class = get_class(c); + e->ecxt_ip = ip; + e->ecxt_stack = ip && rich_stack ? get_current_stack() : NULL; + e->event_fn = e_fn; + e->ecxt_fn = c_fn; + + eh = dt->ecxt_held + (dt->ecxt_held_pos++); + eh->ecxt = get_ecxt(e); + eh->map = m; + eh->class = get_class(c); + eh->wgen = atomic_read(&wgen); + eh->sub_l = sub_l; + + irqf = cur_enirqf(); + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + add_iecxt(c, irq, e, false); + + del_ecxt(e); + return true; +} + +static int find_ecxt_pos(struct dept_map *m, struct dept_class *c, + bool newfirst) +{ + struct dept_task *dt = dept_task(); + int i; + + if (newfirst) { + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } else { + for (i = 0; i < dt->ecxt_held_pos; i++) { + struct dept_ecxt_held *eh; + + eh = dt->ecxt_held + i; + if (eh->map == m && eh->class == c) + return i; + } + } + return -1; +} + +static bool pop_ecxt(struct dept_map *m, struct dept_class *c) +{ + struct dept_task *dt = dept_task(); + int pos; + int i; + + pos = find_ecxt_pos(m, c, true); + if (pos == -1) + return false; + + if (dt->ecxt_held[pos].class) + put_class(dt->ecxt_held[pos].class); + + if (dt->ecxt_held[pos].ecxt) + put_ecxt(dt->ecxt_held[pos].ecxt); + + dt->ecxt_held_pos--; + + for (i = pos; i < dt->ecxt_held_pos; i++) + dt->ecxt_held[i] = dt->ecxt_held[i + 1]; + return true; +} + +static bool good_hist(struct dept_wait_hist *wh, unsigned int wg) +{ + return wh->wait != NULL && before(wg, wh->wgen); +} + +/* + * Binary-search the ring buffer for the earliest valid wait. + */ +static int find_hist_pos(unsigned int wg) +{ + int oldest; + int l; + int r; + int pos; + + oldest = hist_pos_next(); + if (unlikely(good_hist(hist(oldest), wg))) { + DEPT_INFO_ONCE("Need to expand the ring buffer.\n"); + return oldest; + } + + l = oldest + 1; + r = oldest + DEPT_MAX_WAIT_HIST - 1; + for (pos = (l + r) / 2; l <= r; pos = (l + r) / 2) { + struct dept_wait_hist *p = hist(pos - 1); + struct dept_wait_hist *wh = hist(pos); + + if (!good_hist(p, wg) && good_hist(wh, wg)) + return pos % DEPT_MAX_WAIT_HIST; + if (good_hist(wh, wg)) + r = pos - 1; + else + l = pos + 1; + } + return -1; +} + +static void do_event(struct dept_map *m, struct dept_class *c, + unsigned int wg, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + struct dept_wait_hist *wh; + struct dept_ecxt_held *eh; + unsigned int ctxt_id; + int end; + int pos; + int i; + + if (DEPT_WARN_ON(!valid_class(c))) + return; + + if (m->nocheck) + return; + + /* + * The event was triggered before wait. + */ + if (!wg) + return; + + pos = find_ecxt_pos(m, c, false); + if (pos == -1) + return; + + eh = dt->ecxt_held + pos; + + if (DEPT_WARN_ON(!eh->ecxt)) + return; + + eh->ecxt->event_ip = ip; + eh->ecxt->event_stack = get_current_stack(); + + /* + * The ecxt already has done what it needs. + */ + if (!before(wg, eh->wgen)) + return; + + pos = find_hist_pos(wg); + if (pos == -1) + return; + + ctxt_id = cur_ctxt_id(); + end = hist_pos_next(); + end = end > pos ? end : end + DEPT_MAX_WAIT_HIST; + for (wh = hist(pos); pos < end; wh = hist(++pos)) { + if (after(wh->wgen, eh->wgen)) + break; + + if (dt->in_sched && wh->wait->sched_sleep) + continue; + + if (wh->ctxt_id == ctxt_id) + add_dep(eh->ecxt, wh->wait); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + struct dept_ecxt *e; + + if (before(dt->wgen_enirq[i], wg)) + continue; + + e = eh->ecxt; + add_iecxt(e->class, i, e, false); + } +} + +static void del_dep_rcu(struct rcu_head *rh) +{ + struct dept_dep *d = container_of(rh, struct dept_dep, rh); + + preempt_disable(); + del_dep(d); + preempt_enable(); +} + +/* + * NOTE: Must be called with dept_lock held. + */ +static void disconnect_class(struct dept_class *c) +{ + struct dept_dep *d, *n; + int i; + + list_for_each_entry_safe(d, n, &c->dep_head, dep_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + list_for_each_entry_safe(d, n, &c->dep_rev_head, dep_rev_node) { + list_del_rcu(&d->dep_node); + list_del_rcu(&d->dep_rev_node); + hash_del_dep(d); + call_rcu(&d->rh, del_dep_rcu); + } + + for (i = 0; i < DEPT_IRQS_NR; i++) { + stale_iecxt(iecxt(c, i)); + stale_iwait(iwait(c, i)); + } +} + +/* + * Context control + * ===================================================================== + * Whether a wait is in {hard,soft}-IRQ context or whether + * {hard,soft}-IRQ has been enabled on the way to an event is very + * important to check dependency. All those things should be tracked. + */ + +static unsigned long cur_enirqf(void) +{ + struct dept_task *dt = dept_task(); + int he = dt->hardirqs_enabled; + int se = dt->softirqs_enabled; + + if (he) + return DEPT_HIRQF | (se ? DEPT_SIRQF : 0UL); + return 0UL; +} + +static int cur_irq(void) +{ + if (lockdep_softirq_context(current)) + return DEPT_SIRQ; + if (lockdep_hardirq_context()) + return DEPT_HIRQ; + return DEPT_IRQS_NR; +} + +static unsigned int cur_ctxt_id(void) +{ + struct dept_task *dt = dept_task(); + int irq = cur_irq(); + + /* + * Normal process context + */ + if (irq == DEPT_IRQS_NR) + return 0U; + + return dt->irq_id[irq] | (1UL << irq); +} + +static void enirq_transition(int irq) +{ + struct dept_task *dt = dept_task(); + int i; + + /* + * IRQ can cut in on the way to the event. Used for cross-event + * detection. + * + * wait context event context(ecxt) + * ------------ ------------------- + * wait event + * UPDATE wgen + * observe IRQ enabled + * UPDATE wgen + * keep the wgen locally + * + * on the event + * check the wgen kept + */ + + /* + * Avoid zero wgen. + */ + dt->wgen_enirq[irq] = atomic_inc_return(&wgen) ?: + atomic_inc_return(&wgen); + + for (i = dt->ecxt_held_pos - 1; i >= 0; i--) { + struct dept_ecxt_held *eh; + struct dept_ecxt *e; + + eh = dt->ecxt_held + i; + e = eh->ecxt; + if (e) + add_iecxt(e->class, irq, e, true); + } +} + +static void dept_enirq(unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long irqf = cur_enirqf(); + int irq; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * IRQ ON/OFF transition might happen while Dept is working. + * We cannot handle recursive entrance. Just ingnore it. + * Only transitions outside of Dept will be considered. + */ + if (dt->recursive) + return; + + flags = dept_enter(); + + for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + dt->enirq_ip[irq] = ip; + enirq_transition(irq); + } + + dept_exit(flags); +} + +void dept_softirqs_on_ip(unsigned long ip) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = true; + dept_enirq(ip); +} + +void dept_hardirqs_on(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = true; + dept_enirq(_RET_IP_); +} + +void dept_softirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->softirqs_enabled = false; +} + +void dept_hardirqs_off(void) +{ + /* + * Assumes that it's called with IRQ disabled so that accessing + * current's fields is not racy. + */ + dept_task()->hardirqs_enabled = false; +} + +/* + * Ensure it's the outmost softirq context. + */ +void dept_softirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; +} + +/* + * Ensure it's the outmost hardirq context. + */ +void dept_hardirq_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; +} + +void dept_sched_enter(void) +{ + dept_task()->in_sched = true; +} + +void dept_sched_exit(void) +{ + dept_task()->in_sched = false; +} + +/* + * Exposed APIs + * ===================================================================== + */ + +static void clean_classes_cache(struct dept_key *k) +{ + int i; + + for (i = 0; i < DEPT_MAX_SUBCLASSES_CACHE; i++) { + if (!READ_ONCE(k->classes[i])) + continue; + + WRITE_ONCE(k->classes[i], NULL); + } +} + +void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u < 0)) { + m->nocheck = true; + return; + } + + if (DEPT_WARN_ON(sub_u >= DEPT_MAX_SUBCLASSES_USR)) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + clean_classes_cache(&m->map_key); + + m->keys = k; + m->sub_u = sub_u; + m->name = n; + m->wgen = 0U; + m->nocheck = !valid_key(k); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_init); + +void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, + const char *n) +{ + unsigned long flags; + + if (unlikely(!dept_working())) { + m->nocheck = true; + return; + } + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + if (k) { + clean_classes_cache(&m->map_key); + m->keys = k; + m->nocheck = !valid_key(k); + } + + if (sub_u >= 0 && sub_u < DEPT_MAX_SUBCLASSES_USR) + m->sub_u = sub_u; + + if (n) + m->name = n; + + m->wgen = 0U; + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_map_reinit); + +void dept_map_copy(struct dept_map *to, struct dept_map *from) +{ + if (unlikely(!dept_working())) { + to->nocheck = true; + return; + } + + *to = *from; + + /* + * XXX: 'to' might be in a stack or something. Using the address + * in a stack segment as a key is meaningless. Just ignore the + * case for now. + */ + if (!to->keys) { + to->nocheck = true; + return; + } + + /* + * Since the class cache can be modified concurrently we could + * observe half pointers (64bit arch using 32bit copy insns). + * Therefore clear the caches and take the performance hit. + * + * XXX: Doesn't work well with lockdep_set_class_and_subclass() + * since that relies on cache abuse. + */ + clean_classes_cache(&to->map_key); +} + +static LIST_HEAD(classes); + +static bool within(const void *addr, void *start, unsigned long size) +{ + return addr >= start && addr < start + size; +} + +void dept_free_range(void *start, unsigned int sz) +{ + struct dept_task *dt = dept_task(); + struct dept_class *c, *n; + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Failed to successfully free Dept objects.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_free_range() should not fail. + * + * FIXME: Should be fixed if dept_free_range() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + list_for_each_entry_safe(c, n, &classes, all_node) { + if (!within((void *)c->key, start, sz) && + !within(c->name, start, sz)) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} + +static int sub_id(struct dept_map *m, int e) +{ + return (m ? m->sub_u : 0) + e * DEPT_MAX_SUBCLASSES_USR; +} + +static struct dept_class *check_new_class(struct dept_key *local, + struct dept_key *k, int sub_id, + const char *n, bool sched_map) +{ + struct dept_class *c = NULL; + + if (DEPT_WARN_ON(sub_id >= DEPT_MAX_SUBCLASSES)) + return NULL; + + if (DEPT_WARN_ON(!k)) + return NULL; + + /* + * XXX: Assume that users prevent the map from using if any of + * the cached keys has been invalidated. If not, the cache, + * local->classes should not be used because it would be racy + * with class deletion. + */ + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + c = READ_ONCE(local->classes[sub_id]); + + if (c) + return c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (c) + goto caching; + + if (unlikely(!dept_lock())) + return NULL; + + c = lookup_class((unsigned long)k->base + sub_id); + if (unlikely(c)) + goto unlock; + + c = new_class(); + if (unlikely(!c)) + goto unlock; + + c->name = n; + c->sched_map = sched_map; + c->sub_id = sub_id; + c->key = (unsigned long)(k->base + sub_id); + hash_add_class(c); + list_add(&c->all_node, &classes); +unlock: + dept_unlock(); +caching: + if (local && sub_id < DEPT_MAX_SUBCLASSES_CACHE) + WRITE_ONCE(local->classes[sub_id], c); + + return c; +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l, + bool sched_sleep, bool sched_map) +{ + int e; + + /* + * Be as conservative as possible. In case of mulitple waits for + * a single dept_map, we are going to keep only the last wait's + * wgen for simplicity - keeping all wgens seems overengineering. + * + * Of course, it might cause missing some dependencies that + * would rarely, probabily never, happen but it helps avoid + * false positive report. + */ + for_each_set_bit(e, &w_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, sched_map); + if (!c) + continue; + + add_wait(c, ip, w_fn, sub_l, sched_sleep); + } +} + +/* + * Called between dept_enter() and dept_exit(). + */ +static void __dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn, + bool sched_map) +{ + struct dept_class *c; + struct dept_key *k; + int e; + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (DEPT_WARN_ON(e >= DEPT_MAX_SUBCLASSES_EVT)) + return; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); + + if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { + do_event(m, c, READ_ONCE(m->wgen), ip); + pop_ecxt(m, c); + } +} + +void dept_wait(struct dept_map *m, unsigned long w_f, + unsigned long ip, const char *w_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + if (m->nocheck) + return; + + flags = dept_enter(); + + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_wait); + +void dept_stage_wait(struct dept_map *m, struct dept_key *k, + unsigned long ip, const char *w_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m && m->nocheck) + return; + + /* + * Either m or k should be passed. Which means Dept relies on + * either its own map or the caller's position in the code when + * determining its class. + */ + if (DEPT_WARN_ON(!m && !k)) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + /* + * Ensure the outmost dept_stage_wait() works. + */ + if (dt->stage_m.keys) + goto exit; + + if (m) { + dt->stage_m = *m; + + /* + * Ensure dt->stage_m.keys != NULL and it works with the + * map's map_key, not stage_m's one when ->keys == NULL. + */ + if (!m->keys) + dt->stage_m.keys = &m->map_key; + } else { + dt->stage_m.name = w_fn; + dt->stage_sched_map = true; + } + + /* + * dept_map_reinit() includes WRITE_ONCE(->wgen, 0U) that + * effectively disables the map just in case real sleep won't + * happen. dept_request_event_wait_commit() will enable it. + */ + dept_map_reinit(&dt->stage_m, k, -1, NULL); + + dt->stage_w_fn = w_fn; + dt->stage_ip = ip; +exit: + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_stage_wait); + +static void __dept_clean_stage(struct dept_task *dt) +{ + memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); + dt->stage_sched_map = false; + dt->stage_w_fn = NULL; + dt->stage_ip = 0UL; +} + +void dept_clean_stage(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + __dept_clean_stage(dt); + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_clean_stage); + +/* + * Always called from __schedule(). + */ +void dept_request_event_wait_commit(void) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + unsigned int wg; + unsigned long ip; + const char *w_fn; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + /* + * It's impossible that __schedule() is called while Dept is + * working that already disabled IRQ at the entrance. + */ + if (DEPT_WARN_ON(dt->recursive)) + return; + + flags = dept_enter(); + + /* + * Checks if current has staged a wait. + */ + if (!dt->stage_m.keys) + goto exit; + + w_fn = dt->stage_w_fn; + ip = dt->stage_ip; + sched_map = dt->stage_sched_map; + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + WRITE_ONCE(dt->stage_m.wgen, wg); + + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); +exit: + dept_exit(flags); +} + +/* + * Always called from try_to_wake_up(). + */ +void dept_stage_event(struct task_struct *requestor, unsigned long ip) +{ + struct dept_task *dt = dept_task(); + struct dept_task *dt_req = &requestor->dept_task; + unsigned long flags; + struct dept_map m; + bool sched_map; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) + return; + + flags = dept_enter(); + + /* + * Serializing is unnecessary as long as it always comes from + * try_to_wake_up(). + */ + m = dt_req->stage_m; + sched_map = dt_req->stage_sched_map; + __dept_clean_stage(dt_req); + + /* + * ->stage_m.keys should not be NULL if it's in use. Should + * make sure that it's not NULL when staging a valid map. + */ + if (!m.keys) + goto exit; + + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); +exit: + dept_exit(flags); +} + +/* + * Modifies the latest ecxt corresponding to m and e_f. + */ +void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, + struct dept_key *new_k, unsigned long new_e_f, + unsigned long new_ip, const char *new_c_fn, + const char *new_e_fn, int new_sub_l) +{ + struct dept_task *dt = dept_task(); + struct dept_ecxt_held *eh; + struct dept_class *c; + struct dept_key *k; + unsigned long flags; + int pos = -1; + int new_e; + int e; + + if (unlikely(!dept_working())) + return; + + /* + * XXX: Couldn't handle re-enterance cases. Ingore it for now. + */ + if (dt->recursive) + return; + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + pos = find_ecxt_pos(m, c, true); + if (pos != -1) + break; + } + + if (unlikely(pos == -1)) + goto exit; + + eh = dt->ecxt_held + pos; + new_sub_l = new_sub_l >= 0 ? new_sub_l : eh->sub_l; + + new_e = find_first_bit(&new_e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (new_e < DEPT_MAX_SUBCLASSES_EVT) + /* + * Let it work with the first bit anyway. + */ + DEPT_WARN_ON(1UL << new_e != new_e_f); + else + new_e = e; + + pop_ecxt(m, c); + + /* + * Apply the key to the map. + */ + if (new_k) + dept_map_reinit(m, new_k, -1, NULL); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); + + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + goto exit; + + /* + * Successfully pop_ecxt()ed but failed to add_ecxt(). + */ + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_map_ecxt_modify); + +void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, + const char *c_fn, const char *e_fn, int sub_l) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + struct dept_class *c; + struct dept_key *k; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt++; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + e = find_first_bit(&e_f, DEPT_MAX_SUBCLASSES_EVT); + + if (e >= DEPT_MAX_SUBCLASSES_EVT) + goto missing_ecxt; + + /* + * An event is an event. If the caller passed more than single + * event, then warn it and handle the event corresponding to + * the first bit anyway. + */ + DEPT_WARN_ON(1UL << e != e_f); + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); + + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + goto exit; +missing_ecxt: + dt->missing_ecxt++; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_enter); + +bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + bool ret = false; + int e; + + if (unlikely(!dept_working())) + return false; + + if (dt->recursive) + return false; + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + if (find_ecxt_pos(m, c, true) != -1) { + ret = true; + break; + } + } + + dept_exit(flags); + + return ret; +} +EXPORT_SYMBOL_GPL(dept_ecxt_holding); + +void dept_request_event(struct dept_map *m) +{ + unsigned long flags; + unsigned int wg; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + /* + * Allow recursive entrance. + */ + flags = dept_enter_recursive(); + + /* + * Avoid zero wgen. + */ + wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); + WRITE_ONCE(m->wgen, wg); + + dept_exit_recursive(flags); +} +EXPORT_SYMBOL_GPL(dept_request_event); + +void dept_event(struct dept_map *m, unsigned long e_f, + unsigned long ip, const char *e_fn) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + + if (unlikely(!dept_working())) + return; + + if (m->nocheck) + return; + + if (dt->recursive) { + /* + * Dept won't work with this even though an event + * context has been asked. Don't make it confused at + * handling the event. Disable it until the next. + */ + WRITE_ONCE(m->wgen, 0U); + return; + } + + flags = dept_enter(); + + __dept_event(m, e_f, ip, e_fn, false); + + /* + * Keep the map diabled until the next sleep. + */ + WRITE_ONCE(m->wgen, 0U); + + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_event); + +void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, + unsigned long ip) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int e; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + dt->missing_ecxt--; + return; + } + + /* + * Should go ahead no matter whether ->nocheck == true or not + * because ->nocheck value can be changed within the ecxt area + * delimitated by dept_ecxt_enter() and dept_ecxt_exit(). + */ + + flags = dept_enter(); + + for_each_set_bit(e, &e_f, DEPT_MAX_SUBCLASSES_EVT) { + struct dept_class *c; + struct dept_key *k; + + k = m->keys ?: &m->map_key; + c = check_new_class(&m->map_key, k, + sub_id(m, e), m->name, false); + if (!c) + continue; + + /* + * When it found an ecxt for any event in e_f, done. + */ + if (pop_ecxt(m, c)) + goto exit; + } + + dt->missing_ecxt--; +exit: + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_ecxt_exit); + +void dept_task_exit(struct task_struct *t) +{ + struct dept_task *dt = &t->dept_task; + int i; + + if (unlikely(!dept_working())) + return; + + raw_local_irq_disable(); + + if (dt->stack) + put_stack(dt->stack); + + for (i = 0; i < dt->ecxt_held_pos; i++) { + if (dt->ecxt_held[i].class) + put_class(dt->ecxt_held[i].class); + if (dt->ecxt_held[i].ecxt) + put_ecxt(dt->ecxt_held[i].ecxt); + } + + for (i = 0; i < DEPT_MAX_WAIT_HIST; i++) + if (dt->wait_hist[i].wait) + put_wait(dt->wait_hist[i].wait); + + dt->task_exit = true; + dept_off(); + + raw_local_irq_enable(); +} + +void dept_task_init(struct task_struct *t) +{ + memset(&t->dept_task, 0x0, sizeof(struct dept_task)); +} + +void dept_key_init(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive) { + DEPT_STOP("Key initialization fails.\n"); + return; + } + + flags = dept_enter(); + + clean_classes_cache(k); + + /* + * dept_key_init() should not fail. + * + * FIXME: Should be fixed if dept_key_init() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + DEPT_STOP("The class(%s/%d) has not been removed.\n", + c->name, sub_id); + break; + } + + dept_unlock(); + dept_exit(flags); +} +EXPORT_SYMBOL_GPL(dept_key_init); + +void dept_key_destroy(struct dept_key *k) +{ + struct dept_task *dt = dept_task(); + unsigned long flags; + int sub_id; + + if (unlikely(!dept_working())) + return; + + if (dt->recursive == 1 && dt->task_exit) { + /* + * Need to allow to go ahead in this case where + * ->recursive has been set to 1 by dept_off() in + * dept_task_exit() and ->task_exit has been set to + * true in dept_task_exit(). + */ + } else if (dt->recursive) { + DEPT_STOP("Key destroying fails.\n"); + return; + } + + flags = dept_enter(); + + /* + * dept_key_destroy() should not fail. + * + * FIXME: Should be fixed if dept_key_destroy() causes deadlock + * with dept_lock(). + */ + while (unlikely(!dept_lock())) + cpu_relax(); + + for (sub_id = 0; sub_id < DEPT_MAX_SUBCLASSES; sub_id++) { + struct dept_class *c; + + c = lookup_class((unsigned long)k->base + sub_id); + if (!c) + continue; + + hash_del_class(c); + disconnect_class(c); + list_del(&c->all_node); + invalidate_class(c); + + /* + * Actual deletion will happen on the rcu callback + * that has been added in disconnect_class(). + */ + del_class(c); + } + + dept_unlock(); + dept_exit(flags); + + /* + * Wait until even lockless hash_lookup_class() for the class + * returns NULL. + */ + might_sleep(); + synchronize_rcu(); +} +EXPORT_SYMBOL_GPL(dept_key_destroy); + +static void move_llist(struct llist_head *to, struct llist_head *from) +{ + struct llist_node *first = llist_del_all(from); + struct llist_node *last; + + if (!first) + return; + + for (last = first; last->next; last = last->next); + llist_add_batch(first, last, to); +} + +static void migrate_per_cpu_pool(void) +{ + const int boot_cpu = 0; + int i; + + /* + * The boot CPU has been using the temperal local pool so far. + * From now on that per_cpu areas have been ready, use the + * per_cpu local pool instead. + */ + DEPT_WARN_ON(smp_processor_id() != boot_cpu); + for (i = 0; i < OBJECT_NR; i++) { + struct llist_head *from; + struct llist_head *to; + + from = &pool[i].boot_pool; + to = per_cpu_ptr(pool[i].lpool, boot_cpu); + move_llist(to, from); + } +} + +#define B2KB(B) ((B) / 1024) + +/* + * Should be called after setup_per_cpu_areas() and before no non-boot + * CPUs have been on. + */ +void __init dept_init(void) +{ + size_t mem_total = 0; + + local_irq_disable(); + dept_per_cpu_ready = 1; + migrate_per_cpu_pool(); + local_irq_enable(); + +#define HASH(id, bits) BUILD_BUG_ON(1 << (bits) <= 0); + #include "dept_hash.h" +#undef HASH +#define OBJECT(id, nr) mem_total += sizeof(struct dept_##id) * nr; + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) mem_total += sizeof(struct hlist_head) * (1 << (bits)); + #include "dept_hash.h" +#undef HASH + + pr_info("DEPendency Tracker: Copyright (c) 2020 LG Electronics, Inc., Byungchul Park\n"); + pr_info("... DEPT_MAX_STACK_ENTRY: %d\n", DEPT_MAX_STACK_ENTRY); + pr_info("... DEPT_MAX_WAIT_HIST : %d\n", DEPT_MAX_WAIT_HIST); + pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); + pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); +#define OBJECT(id, nr) \ + pr_info("... memory used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct dept_##id) * nr)); + #include "dept_object.h" +#undef OBJECT +#define HASH(id, bits) \ + pr_info("... hash list head used by %s: %zu KB\n", \ + #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); + #include "dept_hash.h" +#undef HASH + pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); +} diff --git a/kernel/dependency/dept_hash.h b/kernel/dependency/dept_hash.h new file mode 100644 index 000000000000..fd85aab1fdfb --- /dev/null +++ b/kernel/dependency/dept_hash.h @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * HASH(id, bits) + * + * id : Id for the object of struct dept_##id. + * bits: 1UL << bits is the hash table size. + */ + +HASH(dep, 12) +HASH(class, 12) diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h new file mode 100644 index 000000000000..0b7eb16fe9fb --- /dev/null +++ b/kernel/dependency/dept_object.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * OBJECT(id, nr) + * + * id: Id for the object of struct dept_##id. + * nr: # of the object that should be kept in the pool. + */ + +OBJECT(dep, 1024 * 8) +OBJECT(class, 1024 * 8) +OBJECT(stack, 1024 * 32) +OBJECT(ecxt, 1024 * 16) +OBJECT(wait, 1024 * 32) diff --git a/kernel/exit.c b/kernel/exit.c index aedc0832c9f4..32b7f551674b 100644 --- a/kernel/exit.c +++ b/kernel/exit.c @@ -917,6 +917,7 @@ void __noreturn do_exit(long code) exit_tasks_rcu_finish(); lockdep_free_task(tsk); + dept_task_exit(tsk); do_task_dead(); } diff --git a/kernel/fork.c b/kernel/fork.c index 10917c3e1f03..2f1f5dbec154 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -99,6 +99,7 @@ #include #include #include +#include #include #include @@ -2455,6 +2456,7 @@ __latent_entropy struct task_struct *copy_process( #ifdef CONFIG_LOCKDEP lockdep_init_task(p); #endif + dept_task_init(p); #ifdef CONFIG_DEBUG_MUTEXES p->blocked_on = NULL; /* not blocked yet */ diff --git a/kernel/module/main.c b/kernel/module/main.c index 98fedfdb8db5..507c73b10258 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1228,12 +1228,14 @@ static void free_mod_mem(struct module *mod) /* Free lock-classes; relies on the preceding sync_rcu(). */ lockdep_free_key_range(mod_mem->base, mod_mem->size); + dept_free_range(mod_mem->base, mod_mem->size); if (mod_mem->size) module_memory_free(mod_mem->base, type); } /* MOD_DATA hosts mod, so free it at last */ lockdep_free_key_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); + dept_free_range(mod->mem[MOD_DATA].base, mod->mem[MOD_DATA].size); module_memory_free(mod->mem[MOD_DATA].base, MOD_DATA); } @@ -3020,6 +3022,8 @@ static int load_module(struct load_info *info, const char __user *uargs, for_class_mod_mem_type(type, core_data) { lockdep_free_key_range(mod->mem[type].base, mod->mem[type].size); + dept_free_range(mod->mem[type].base, + mod->mem[type].size); } module_deallocate(mod, info); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a708d225c28e..e2a7ee076a58 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -64,6 +64,7 @@ #include #include #include +#include #ifdef CONFIG_PREEMPT_DYNAMIC # ifdef CONFIG_GENERIC_ENTRY @@ -4197,6 +4198,8 @@ int try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags) guard(preempt)(); int cpu, success = 0; + dept_stage_event(p, _RET_IP_); + if (p == current) { /* * We're waking current, this means 'p->on_rq' and 'task_cpu(p) @@ -6578,6 +6581,12 @@ static void __sched notrace __schedule(unsigned int sched_mode) rq = cpu_rq(cpu); prev = rq->curr; + prev_state = READ_ONCE(prev->__state); + if (sched_mode != SM_PREEMPT && prev_state & TASK_NORMAL) + dept_request_event_wait_commit(); + + dept_sched_enter(); + schedule_debug(prev, !!sched_mode); if (sched_feat(HRTICK) || sched_feat(HRTICK_DL)) @@ -6691,6 +6700,7 @@ static void __sched notrace __schedule(unsigned int sched_mode) __balance_callbacks(rq); raw_spin_rq_unlock_irq(rq); } + dept_sched_exit(); } void __noreturn do_task_dead(void) diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 4405f81248fb..9602f41ad8e8 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1285,6 +1285,33 @@ config DEBUG_PREEMPT menu "Lock Debugging (spinlocks, mutexes, etc...)" +config DEPT + bool "Dependency tracking (EXPERIMENTAL)" + depends on DEBUG_KERNEL && LOCK_DEBUGGING_SUPPORT + select DEBUG_SPINLOCK + select DEBUG_MUTEXES + select DEBUG_RT_MUTEXES if RT_MUTEXES + select DEBUG_RWSEMS + select DEBUG_WW_MUTEX_SLOWPATH + select DEBUG_LOCK_ALLOC + select TRACE_IRQFLAGS + select STACKTRACE + select FRAME_POINTER if !MIPS && !PPC && !ARM && !S390 && !MICROBLAZE && !ARC && !X86 + select KALLSYMS + select KALLSYMS_ALL + select PROVE_LOCKING + default n + help + Check dependencies between wait and event and report it if + deadlock possibility has been detected. Multiple reports are + allowed if there are more than a single problem. + + This feature is considered EXPERIMENTAL that might produce + false positive reports because new dependencies start to be + tracked, that have never been tracked before. It's worth + noting, to mitigate the impact by the false positives, multi + reporting has been supported. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT diff --git a/lib/locking-selftest.c b/lib/locking-selftest.c index 6f6a5fc85b42..2558ae57b117 100644 --- a/lib/locking-selftest.c +++ b/lib/locking-selftest.c @@ -1398,6 +1398,8 @@ static void reset_locks(void) local_irq_disable(); lockdep_free_key_range(&ww_lockdep.acquire_key, 1); lockdep_free_key_range(&ww_lockdep.mutex_key, 1); + dept_free_range(&ww_lockdep.acquire_key, 1); + dept_free_range(&ww_lockdep.mutex_key, 1); I1(A); I1(B); I1(C); I1(D); I1(X1); I1(X2); I1(Y1); I1(Y2); I1(Z1); I1(Z2); From patchwork Wed Feb 21 09:49:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565307 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 1C4D53D3B1; Wed, 21 Feb 2024 09:49:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; cv=none; b=uOgLGU7QR56LTySDzqywWaq819xGo93dAbs+ZePlDRZtpdBDm4bO57c3A4pyOVGCWLqjs6BCJ167DovJQvmdZN1qOvsWxLXH0ucxQuuUeo08wnUse9aEnJyW6JuTJggq0sDBsvizuddvB5GSGZ/hT7SQIo9kTGBywUicWrbakrA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; c=relaxed/simple; bh=yYyo9J9LbrfncpB2CWVjqJhM6aD/eyXV6B0l1bR9NkI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=LMIHwgfL+TWa4Ifbw4Zx7KNpoiLzG2vsGLkhWZWBaODF7Mhbdjj2ODsJWj/JWeQQeM6LX2tIriKxYvzH5yyyqVnMyQ8pDs+W75txviDTE+MGTUUfkVxZCBLzUNMVOv4zZHkJHV1dxEseX4r5cEhPOxZeTRMpisr+KLl+4awcJFg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-b6-65d5c7385d59 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 03/27] dept: Add single event dependency tracker APIs Date: Wed, 21 Feb 2024 18:49:09 +0900 Message-Id: <20240221094933.36348-4-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe2xLcRTH97vvda1cRVyTsDRZyGze5IQRkeCKiNk8EiSU3lizh2npzCub 1WuvzOjKNmwjVVspLZlXqWK1LRirKWYowbLN2LTUNrTDPyeffM85n5w/DoNLr5HhjDJ1i6BK lSfLKBEh6hRXxIDzqTDR/C4GDuVNBO+3AwSUmU0UNJ6vRmC6lIVB270F8MzXgaD3wSMc9LpG BBVvX+FwqbYVgc24h4Km94PA5e2ioE6XS0H2KTMFj9v7MGgpLsKg2rIYGgorMbD7PxKgb6Og VJ+NBconDPyGKhoMmZHgMZbQ0Pd2EtS1NpNgezEOjp1ooeCGrY6A2iseDJqulVHQavpNQkPt fQIaD+WTcO5zJQXtPgMOBm8XDU/s5Rhc0AZE+3p+keDMt2Ow7/RFDFzPryO4eeANBhZTMwV3 vB0YWC06HH6euYfAU9BJw948Pw2lWQUIcvcWE/Co30mCtmUa9P4oo+bM4O90dOG81prO23zl BF9fyfFXS17RvPbmC5ovt2zlrcYo/tSNNoyv6PaSvKXqIMVbuotoPqfThfGfHz6k+ftHewn+ vUuPxY1cJYpVCMlKjaCaMHudKNFRWEKmPRu+zacfnYlc0hwUynDsVO6Drp7+z1n911GQKXYM 53b78SAPZSM4a/4HMgeJGJzdH8YZvzyggo0h7ELO/fouGWSCjeRMxuyBZQk7jdtjKyH/Skdz 1RfsA6JQdjp3trRjIJcGZp4+vowHpRz7m+EcPQf/XTGCu210E4VIUo5CqpBUmapJkSuTp45P zEhVbhu/YVOKBQU+yrCrb/UV1N2Y4EAsg2RiSWKNS5CSco06I8WBOAaXDZUQ6YFIopBnbBdU m9aqtiYLagcayRCy4ZLJvnSFlN0o3yIkCUKaoPrfxZjQ8EwUqxkbvX9+VOZXdvca38zC5bdi ZG09sTumLHlSqj25WTEsPmHR2pCIlSvmmual1d8OLzaqj4clLa0enB99RO497J+nEU95uX5R Q9x37bD2sOW7i8iiy07xzoplIX57jdjqnBVvU8QkRfebKc+KGjrL3RzuKGjiSI/uqtnm+BEZ P0pGqBPlk6JwlVr+BxcEsThNAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRTHe573NleLl2X5doFiJMWyG2Se6EL0paeo6EIEfcmRL27lJba8 UqBtmlmKRrbSVaax1lxps4tWk6HlWpGXnGmiKyUqa2q3WfNSTaMvhx///zm/T0dCyU3MHIkm 4aioTVDFKVgpLd2xVr8UXO3iitqvs6DwzArw/8ihwVRpY6HlVgUC251MDP1PNkPHsA/B6Itm CoxFLQiu9vZQcKfRi8BhOcFC27vp4PEPseAuOs2CvryShdbPYxi6z5/FUGHfDs8LyjA4Ax9o MPazUGLU4+D4iCFgtnJgzgiHPksxB2O9K8HtfcVAwyU3A46uJXDxcjcLjxxuGhpr+jC0PTCx 4LX9YeB541MaWgrzGLg5WMbC52EzBWb/EAcvnaUYqgxBW/b33wy48pwYsq/dxuB5/RBBXc5b DHbbKxYa/D4M1fYiCkauP0HQlz/AQdaZAAclmfkITmedp6F53MWAoTsSRn+Z2I1rSYNviCKG 6hTiGC6lybMygdQW93DEUNfFkVJ7Eqm2KEn5o35Mrn7zM8RuPcUS+7ezHMkd8GAy2NTEkacX RmnyzmPEO+ftl66LEeM0yaJ2+YZoqbq+oJg50hGWOmycn4E88lwUIhH4VULm+EM0wSy/SOjs DFATHMovEKrz3jO5SCqh+JNTBcuXF+xEMYPfInS+ecxMMM2HCzaLfvJYxkcKJxzFzD/pfKGi yjkpCuFXCzdKfJO5PLjT3nqXKkDSUjTFikI1CcnxKk1c5DLdYXVagiZ12cHEeDsK/oz5+Fhh DfrRtrke8RKkmCZT3/eIckaVrEuLr0eChFKEyuiUYCSLUaWli9rEA9qkOFFXj+ZKaEWYbOs+ MVrOx6qOiodF8Yio/d9iScicDCQt+FCumLvNW+XxLhz/GRigp09LOae5tFc7EosH76k62q1O os/nFudEVH4K89Fd5fyulJiokb2mG5uuiM2Ju22z4tdHeNc4e9VK8ybrn76pB57lz3absnsM C5LGZORUqmO5cs8hZXpenXG2oqTsWGaUa2bnVgPlaore0OpuyVLQOrVqpZLS6lR/AYOUIwov AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Wrapped the base APIs for easier annotation on wait and event. Start with supporting waiters on each single event. More general support for multiple events is a future work. Do more when the need arises. How to annotate (the simplest way): 1. Initaialize a map for the interesting wait. /* * Recommand to place along with the wait instance. */ struct dept_map my_wait; /* * Recommand to place in the initialization code. */ sdt_map_init(&my_wait); 2. Place the following at the wait code. sdt_wait(&my_wait); 3. Place the following at the event code. sdt_event(&my_wait); That's it! Signed-off-by: Byungchul Park --- include/linux/dept_sdt.h | 62 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 include/linux/dept_sdt.h diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h new file mode 100644 index 000000000000..12a793b90c7e --- /dev/null +++ b/include/linux/dept_sdt.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Single-event Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_SDT_H +#define __LINUX_DEPT_SDT_H + +#include +#include + +#ifdef CONFIG_DEPT +#define sdt_map_init(m) \ + do { \ + static struct dept_key __key; \ + dept_map_init(m, &__key, 0, #m); \ + } while (0) + +#define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) + +#define sdt_wait(m) \ + do { \ + dept_request_event(m); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + } while (0) + +/* + * sdt_might_sleep() and its family will be committed in __schedule() + * when it actually gets to __schedule(). Both dept_request_event() and + * dept_wait() will be performed on the commit. + */ + +/* + * Use the code location as the class key if an explicit map is not used. + */ +#define sdt_might_sleep_start(m) \ + do { \ + struct dept_map *__m = m; \ + static struct dept_key __key; \ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + } while (0) + +#define sdt_might_sleep_end() dept_clean_stage() + +#define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) +#else /* !CONFIG_DEPT */ +#define sdt_map_init(m) do { } while (0) +#define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start(m) do { } while (0) +#define sdt_might_sleep_end() do { } while (0) +#define sdt_ecxt_enter(m) do { } while (0) +#define sdt_event(m) do { } while (0) +#define sdt_ecxt_exit(m) do { } while (0) +#endif +#endif /* __LINUX_DEPT_SDT_H */ From patchwork Wed Feb 21 09:49:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565309 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D8F83D3BB; Wed, 21 Feb 2024 09:49:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; cv=none; b=uBhiWMhKqX2jsawZw99JMTTryC7oEL6k/17CkTiZzUcPCqg+n0hXCxAxC3fvF0IfAGSLWSNR3sD+vv5s+KC5HYHOtAUQlyvfA/dP15T0o4twkpgsFjciXdRr1VVJlzb3EX7GpSMyFIfOvogNy5F169JnRjU5319s4pS5Dx5pKoo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; c=relaxed/simple; bh=ZnDomY68etRURkOftpc7pgxDDdwCW9QdH8k+FPh35/w=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=iAhUCi6+CtnX6814TL3Z6pRcwcLrktv/Yc2pxdAxnKTrTzqm5/dW9tay/FNzWohEB+/OTFwVZtRldX3SpC4RJT/x1KZivHND4uow7/9fMZr3p4OnEHRKUggeDoCFqwJtqYMXyjlROOiD+l3b2OcaUwhMAPN7rGfHOAaedm9M7KI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-c6-65d5c7380bae From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 04/27] dept: Add lock dependency tracker APIs Date: Wed, 21 Feb 2024 18:49:10 +0900 Message-Id: <20240221094933.36348-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0yTZxTGfd/v2krJlyLx9TaXZsaFTQTEeTTOSWKyz8RbsrAlQuIqfIGG a1oBu8SIowhWMEoslUtIgdlViuJKl6njUjDcJEIZVZgiAWaGhEsXthIruK2V+M/Jk+c555eT k8NTyvvMRl6TeUbSZqrTVaycls+H1O6AnidSlHNsN1wriQLfP8U0VDc1suC+Y0fQ6LyAYabr SxhZmkOw/HiQArPJjaB28gUFzu5xBK2271kYfhkKHp+XhT7TZRYK6ptYGJpdwTBWXobB7jgK /VfrMLj80zSYZ1ioMhfgQHmFwW9t4MCavw2mbJUcrExGQ9/4UwZan30CFTVjLLS09tHQfW8K w/CDahbGG/9joL+7lwb3tVIGbi/UsTC7ZKXA6vNy8JvLguGuIQC6+Pe/DPSUujBc/OEnDJ7f f0XQVjyBwdH4lIWHvjkMzQ4TBW9+7EIwdWWeg8ISPwdVF64guFxYTsPg2x4GDIGbLL+uZg/u Ex/OeSnR0Jwnti5ZaPFRHRHvV77gREPbM060OHLEZluEWN8yg8XaRR8jOhousaJjsYwTjfMe LC4MDHBi741lWnzpMeMTm07K9ydL6ZpcSbvzwLfyVENgu+xX68/edOWjfGRXGpGMJ0IsudlV gYyIf6fbJ3cFbVbYTkZH/VRQrxM+JM2lfzJGJOcpoWgtsf31mA0GYUIcMbqn2eAsLWwjzpKk oK0QdpOR2T/QKn4rsd91vePIhM/Irao5JqiVgZ4nQz9TQSYRimTkgfM2Xh3YQDpso/RVpLCg NQ1IqcnMzVBr0mMjU/WZmrORSVkZDhR4KOu5lYR7aNH9VScSeKQKUaT+4pGUjDpXp8/oRISn VOsUdF7AUiSr9d9J2qxT2px0SdeJNvG0ar0iZikvWSmkqM9IaZKULWnfp5iXbcxH8fbEOx/F fJ3TseeStXygtq5mIX5zlv6k7+DetOiQ+S2De1xFpsLQirSUeHlEh2ZvNXP988HiiUN9iYe+ +WB07cCW02HEy4Q9b+LjwhNj38Sd791pCf+4/fhCf5WzIC2hbMgb2/WFTH/i+Pg+m5nE9KQM m46cGvtUN7Gr/5j/dELLYRWtS1VHR1Banfp/xiq3vEwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x46z3074zcPYTWyImrLPZIbZ7jvD/OFhY8OpH916dJfI Y3SiU6itosIpTk/IrzyrbpceLlR0EdIUk9YRca1THu5s/vnstff783nv/ceHp1S5zCReFx0n 6aO1kWpWQSvWhCT5Q32bFGBLnQbpqQHg+nGChrwbpSy0XC9BUFpxBENvrQZeDjoRDD9tpiA7 swXBpa63FFTUdSKoLDzKQuuHseBw9bNgzzzJQlLBDRae9Y1g6MjKwFAir4bHZ/IxWN09NGT3 spCbnYQ94xMGt6WYA0uiH3QX5nAw0hUI9s4XDNSctzNQ+XoOnLvQwcLDSjsNdXe7MbTez2Oh s/QPA4/rGmhoSU9j4NqXfBb6Bi0UWFz9HDy3mjGUGT1pyd9/M1CfZsWQfPkmBserBwiqTrzD IJe+YKHG5cRQLmdS8PNqLYLuU585OJbq5iD3yCkEJ49l0dD8q54BY0cwDA/lsUtDSI2znyLG 8j2kctBMk8Z8kdzLecsRY9Vrjpjl3aS8cDYpeNiLyaUBF0Pk4hSWyAMZHDF9dmDypamJIw1n h2nywZGN107ZpFgcJkXq4iX9/CXbFOFGT7vYTxP3XrEmokRUojIhnheFILG6a4EJ+fCsMEts b3dTXvYVpovlaR8ZE1LwlHB8tFj49SnrNcYJy0RTSw/rvaUFP7EiNdQrK4Vg8WXfe+RlUZgm lpRZ/+X4CAvFolwn42WVZ6ft2S3qDFKY0ahi5KuLjo/S6iKD5xkiwhOidXvnhcZEycjzMpaD I+l30Y9WjQ0JPFKPUYbfcUgqRhtvSIiyIZGn1L5Keo9HUoZpE/ZJ+pit+t2RksGGJvO0eqJy 5UZpm0rYqY2TIiQpVtL/dzHvMykRmQ9naefGucrWrl/yKi+hquDBeGWMfyiqSktRFS3TbMnE AW3fyJ/pDaN2aVYNy44rFXEHpkzYuvyQ27Zf2jCzyylrMtQf9dv9TmcERdyeUa18nrJIPVS0 4smj5M27FrlqB7+mbDJOvRbT2L5jnf3wBNuM5l8X39S1XW8M3Hh6YCgk552aNoRrA2dTeoP2 L3nMtLYuAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Wrapped the base APIs for easier annotation on typical lock. Signed-off-by: Byungchul Park --- include/linux/dept_ldt.h | 77 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 include/linux/dept_ldt.h diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h new file mode 100644 index 000000000000..062613e89fc3 --- /dev/null +++ b/include/linux/dept_ldt.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Lock Dependency Tracker + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __LINUX_DEPT_LDT_H +#define __LINUX_DEPT_LDT_H + +#include + +#ifdef CONFIG_DEPT +#define LDT_EVT_L 1UL +#define LDT_EVT_R 2UL +#define LDT_EVT_W 1UL +#define LDT_EVT_RW (LDT_EVT_R | LDT_EVT_W) +#define LDT_EVT_ALL (LDT_EVT_L | LDT_EVT_RW) + +#define ldt_init(m, k, su, n) dept_map_init(m, k, su, n) +#define ldt_lock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ + } \ + } while (0) + +#define ldt_rlock(m, sl, t, n, i, q) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ + else { \ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ + } \ + } while (0) + +#define ldt_wlock(m, sl, t, n, i) \ + do { \ + if (n) \ + dept_ecxt_enter_nokeep(m); \ + else if (t) \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ + else { \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ + } \ + } while (0) + +#define ldt_unlock(m, i) dept_ecxt_exit(m, LDT_EVT_ALL, i) + +#define ldt_downgrade(m, i) \ + do { \ + if (dept_ecxt_holding(m, LDT_EVT_W)) \ + dept_map_ecxt_modify(m, LDT_EVT_W, NULL, LDT_EVT_R, i, "downgrade", "read_unlock", -1);\ + } while (0) + +#define ldt_set_class(m, n, k, sl, i) dept_map_ecxt_modify(m, LDT_EVT_ALL, k, 0UL, i, "lock_set_class", "(any)unlock", sl) +#else /* !CONFIG_DEPT */ +#define ldt_init(m, k, su, n) do { (void)(k); } while (0) +#define ldt_lock(m, sl, t, n, i) do { } while (0) +#define ldt_rlock(m, sl, t, n, i, q) do { } while (0) +#define ldt_wlock(m, sl, t, n, i) do { } while (0) +#define ldt_unlock(m, i) do { } while (0) +#define ldt_downgrade(m, i) do { } while (0) +#define ldt_set_class(m, n, k, sl, i) do { } while (0) +#endif +#endif /* __LINUX_DEPT_LDT_H */ From patchwork Wed Feb 21 09:49:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565310 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D94C3D541; Wed, 21 Feb 2024 09:49:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; cv=none; b=X+JjW3s787rkFZhC5kW7GP/iySSwSo0JW1hKbsDbQYEzyTXLoImK5l39wkh8EUqrs56Bd7ZZVmyBn6cAM832hRl8RGiCDgbQb+H9hgalafgnuoO7TV+PYmTZdPYV8KsDSk7SogYYIndDLOCaqsS603ViknFvjOPvDPSvc5kQ7bE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508996; c=relaxed/simple; bh=QWHRCvl6WHVJCxC3TSQ+F2P9dkdR+c40EtoDliptNOM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=arczJjH1o7/DiDQjc2JnwSMwUu0F5AN9MIryRh4VPJGntLe1+xJ2WrI5U22zr2DcvAy1l3A/4IlcTgJEnlanDdWQLyshyvLd1O5nN67B8Mv8fXPJbugN2KswkknQdTRunpRBl2+Ebim+AX6xMtQl/ObR+kqeycSOrdtdkW5O9M4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-d6-65d5c738f4e1 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 05/27] dept: Tie to Lockdep and IRQ tracing Date: Wed, 21 Feb 2024 18:49:11 +0900 Message-Id: <20240221094933.36348-6-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSaUxTaRSG/b67UqneFBOvYOLYBHGJa9ScMe7JZK7JTDLRH0aN0SpX2kir abGAxoTVQRSjZhCUzljA1FJQoBX3EigpiAZspSoiVEWigxSr1OJUQGwZ/XPy5LxvnpwfhyVk d6hYVqVJEbUaRbKclpCSwejShT+3PBaX2JwknDm5BIKf8kgwVFfR4LpaiaDqWiaGfuev8HTY h2Ck7SEBRYUuBKWvegi41uxFYDdn0dDRNwU8QT8NrYUnaMgur6bBPTCKofvcWQyV1t/hweky DA2htyQU9dNQUpSNw+NfDCGThQFTRjz0mi8wMPpqKbR6n1Bg71oA5//ppuGuvZWE5pu9GDpu G2jwVo1T8KD5HgmuMwUUXHlfRsPAsIkAU9DPwKMGI4aanLDoWOArBS0FDRiOXarF4Hl2B0F9 3ksM1qonNDQFfRhs1kICvlx2Iug9NchA7skQAyWZpxCcyD1HwsOxFgpyulfAyH8Gev0qocnn J4QcW6pgHzaSwv0yXrh1oYcRcuq7GMFoPSTYzPOF8rv9WCgdClKC1XKcFqxDZxkhf9CDhfft 7Yxwr3iEFPo8RfiPuO2S1Yliskovahev3S1RVoyVkgf/OpL2zBpgMtBHRT6KYnluOe/tGcM/ +PrVWibCNJfAd3aGiAhP437ibQVvqHwkYQnuz8m8+UMbHQliuHV80OKgIkxy8fztrvEJkZRb wRuMdeT/0ll8ZU3DhCiKW8lXlPgm+rJw57G7johIeS47in9e0ff9ihl8o7mTPI2kRjTJgmQq jV6tUCUvX6RM16jSFu09oLai8EuZjo7uuImGXFsciGORPFqqvOERZZRCr0tXOxDPEvJpUjI1 vJImKtIPi9oDu7SHkkWdA8WxpHy6dNlwaqKMS1KkiPtF8aCo/ZFiNio2A1Efire6swbn6vy7 Ay/ebsysIZxbZ6Y1TorV73R1xri34TkJ87JeEhfX+ELuo3qtZaU9yI6Lf/+2eApV1+TP3BAs 3uSI3paUmxAb+GVAPfvremPb1PK8Ofrjn2bH7XsnrTXw7emvP8fPYn2b1Xucfl91/PbCL7kp HepaZaDHofHKSZ1SsXQ+odUpvgGIL+v+TgMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa2xLYRjHve+5dYfKMcNBhTVkLnFLlMclLp+8IUSCiFtoONHaBS1jRLLa BWNiYzq70HZStZXNGXPbpNlYzdjKamaZYZnL2AVbF7MNLfHlyS///z+/T4+CCs5mRin0UXsl Q5Q2Qs3yNL9yftxUcL+QZvTJkyDl5AzwdR2jISvfyYLnWh4C5w0ThpaHS+FldyuC3qfVFJjT PAis715TcKO8EUGJ4wgLNc2DwevrYKEi7QQLcTn5LDz70oeh4Vwqhjx5BVSetmFw9XykwdzC QqY5DvvPJww99lwO7LEToMmRwUHfu5lQ0VjLQFl2BQMl9VPg/IUGFopLKmgov92EoeZuFguN zt8MVJY/osGTkszA1XYbC1+67RTYfR0cPHdZMBTE+22Jnb8YcCe7MCReuo7B++oegvvH3mKQ nbUslPlaMRTKaRT8vPwQQdOpNg4STvZwkGk6heBEwjkaqvvdDMQ3aKD3Rxa7eD4pa+2gSHzh flLSbaHJY5tI7mS85kj8/XqOWOR9pNAxmeQUt2Bi/e5jiJx7nCXy91SOJLV5MWmvquLIo/Re mjR7zXiVagO/YLsUoY+WDNMXbuV1V/qt9O6zhw68kju5WPRNm4SCFKIwSyy6dp0LMCuEiXV1 PVSAQ4RxYmHyByYJ8QpKODpQdHx9ygaKocIi0ZdbygSYFiaId+t/4wArBY2YZblJ/5OOFfMK XH9FQcJs8Upm6999sH/z4tlN6jTiLWhALgrRR0VHavURmmnGcF1MlP7AtG27ImXkfxr74b6U 26irZmkpEhRIPUipu+WVghlttDEmshSJCkodoqT3+yPldm3MQcmwa4thX4RkLEWjFbR6hHLZ OmlrsLBDu1cKl6TdkuF/ixVBo2KR52pze7Zbl+cqGv4+lf+VE7t8j+YbWiMusmU0bwpb/UDV Xj9sc3FVWLr7fcyQfJdGlb7OkKOaKJn7z98TC8KLavl5lRfveBK6HGT8k7FnjqzuDJlTZ12+ 0Rbaxn50JralVNtMzqQCk2rMzsQtob0jl5heelas9X1+88H6Zv1c3qqmjTrtzMmUwaj9A76H vmkwAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: How to place Dept this way looks so ugly. But it's inevitable for now. The way should be enhanced gradually. Signed-off-by: Byungchul Park --- include/linux/irqflags.h | 7 +- include/linux/local_lock_internal.h | 1 + include/linux/lockdep.h | 102 ++++++++++++++++++++++------ include/linux/lockdep_types.h | 3 + include/linux/mutex.h | 1 + include/linux/percpu-rwsem.h | 2 +- include/linux/rtmutex.h | 1 + include/linux/rwlock_types.h | 1 + include/linux/rwsem.h | 1 + include/linux/seqlock.h | 2 +- include/linux/spinlock_types_raw.h | 3 + include/linux/srcu.h | 2 +- kernel/dependency/dept.c | 8 +-- kernel/locking/lockdep.c | 22 ++++++ 14 files changed, 127 insertions(+), 29 deletions(-) diff --git a/include/linux/irqflags.h b/include/linux/irqflags.h index 2b665c32f5fe..672dac1c3059 100644 --- a/include/linux/irqflags.h +++ b/include/linux/irqflags.h @@ -14,6 +14,7 @@ #include #include +#include #include #include @@ -61,8 +62,10 @@ extern void trace_hardirqs_off(void); # define lockdep_softirqs_enabled(p) ((p)->softirqs_enabled) # define lockdep_hardirq_enter() \ do { \ - if (__this_cpu_inc_return(hardirq_context) == 1)\ + if (__this_cpu_inc_return(hardirq_context) == 1) { \ current->hardirq_threaded = 0; \ + dept_hardirq_enter(); \ + } \ } while (0) # define lockdep_hardirq_threaded() \ do { \ @@ -137,6 +140,8 @@ do { \ # define lockdep_softirq_enter() \ do { \ current->softirq_context++; \ + if (current->softirq_context == 1) \ + dept_softirq_enter(); \ } while (0) # define lockdep_softirq_exit() \ do { \ diff --git a/include/linux/local_lock_internal.h b/include/linux/local_lock_internal.h index 975e33b793a7..39f67788fd95 100644 --- a/include/linux/local_lock_internal.h +++ b/include/linux/local_lock_internal.h @@ -21,6 +21,7 @@ typedef struct { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, \ .owner = NULL, diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h index dc2844b071c2..8825f535d36d 100644 --- a/include/linux/lockdep.h +++ b/include/linux/lockdep.h @@ -12,6 +12,7 @@ #include #include +#include #include struct task_struct; @@ -39,6 +40,8 @@ static inline void lockdep_copy_map(struct lockdep_map *to, */ for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) to->class_cache[i] = NULL; + + dept_map_copy(&to->dmap, &from->dmap); } /* @@ -466,7 +469,8 @@ enum xhlock_context_t { * Note that _name must not be NULL. */ #define STATIC_LOCKDEP_MAP_INIT(_name, _key) \ - { .name = (_name), .key = (void *)(_key), } + { .name = (_name), .key = (void *)(_key), \ + .dmap = DEPT_MAP_INITIALIZER(_name, _key) } static inline void lockdep_invariant_state(bool force) {} static inline void lockdep_free_task(struct task_struct *task) {} @@ -548,33 +552,89 @@ extern bool read_lock_is_recursive(void); #define lock_acquire_shared(l, s, t, n, i) lock_acquire(l, s, t, 1, 1, n, i) #define lock_acquire_shared_recursive(l, s, t, n, i) lock_acquire(l, s, t, 2, 1, n, i) -#define spin_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define spin_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define spin_release(l, i) lock_release(l, i) - -#define rwlock_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) +#define spin_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define spin_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define spin_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwlock_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) #define rwlock_acquire_read(l, s, t, i) \ do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, !read_lock_is_recursive());\ if (read_lock_is_recursive()) \ lock_acquire_shared_recursive(l, s, t, NULL, i); \ else \ lock_acquire_shared(l, s, t, NULL, i); \ } while (0) - -#define rwlock_release(l, i) lock_release(l, i) - -#define seqcount_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define seqcount_acquire_read(l, s, t, i) lock_acquire_shared_recursive(l, s, t, NULL, i) -#define seqcount_release(l, i) lock_release(l, i) - -#define mutex_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define mutex_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define mutex_release(l, i) lock_release(l, i) - -#define rwsem_acquire(l, s, t, i) lock_acquire_exclusive(l, s, t, NULL, i) -#define rwsem_acquire_nest(l, s, t, n, i) lock_acquire_exclusive(l, s, t, n, i) -#define rwsem_acquire_read(l, s, t, i) lock_acquire_shared(l, s, t, NULL, i) -#define rwsem_release(l, i) lock_release(l, i) +#define rwlock_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define seqcount_acquire(l, s, t, i) \ +do { \ + ldt_wlock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_acquire_read(l, s, t, i) \ +do { \ + ldt_rlock(&(l)->dmap, s, t, NULL, i, false); \ + lock_acquire_shared_recursive(l, s, t, NULL, i); \ +} while (0) +#define seqcount_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define mutex_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define mutex_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define mutex_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) +#define rwsem_acquire(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_exclusive(l, s, t, NULL, i); \ +} while (0) +#define rwsem_acquire_nest(l, s, t, n, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, n, i); \ + lock_acquire_exclusive(l, s, t, n, i); \ +} while (0) +#define rwsem_acquire_read(l, s, t, i) \ +do { \ + ldt_lock(&(l)->dmap, s, t, NULL, i); \ + lock_acquire_shared(l, s, t, NULL, i); \ +} while (0) +#define rwsem_release(l, i) \ +do { \ + ldt_unlock(&(l)->dmap, i); \ + lock_release(l, i); \ +} while (0) #define lock_map_acquire(l) lock_acquire_exclusive(l, 0, 0, NULL, _THIS_IP_) #define lock_map_acquire_try(l) lock_acquire_exclusive(l, 0, 1, NULL, _THIS_IP_) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 2ebc323d345a..aecd65836b2c 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -11,6 +11,7 @@ #define __LINUX_LOCKDEP_TYPES_H #include +#include #define MAX_LOCKDEP_SUBCLASSES 8UL @@ -77,6 +78,7 @@ struct lock_class_key { struct hlist_node hash_entry; struct lockdep_subclass_key subkeys[MAX_LOCKDEP_SUBCLASSES]; }; + struct dept_key dkey; }; extern struct lock_class_key __lockdep_no_validate__; @@ -194,6 +196,7 @@ struct lockdep_map { int cpu; unsigned long ip; #endif + struct dept_map dmap; }; struct pin_cookie { unsigned int val; }; diff --git a/include/linux/mutex.h b/include/linux/mutex.h index a33aa9eb9fc3..04c41faace85 100644 --- a/include/linux/mutex.h +++ b/include/linux/mutex.h @@ -26,6 +26,7 @@ , .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define __DEP_MAP_MUTEX_INITIALIZER(lockname) diff --git a/include/linux/percpu-rwsem.h b/include/linux/percpu-rwsem.h index 36b942b67b7d..e871aca04645 100644 --- a/include/linux/percpu-rwsem.h +++ b/include/linux/percpu-rwsem.h @@ -21,7 +21,7 @@ struct percpu_rw_semaphore { }; #ifdef CONFIG_DEBUG_LOCK_ALLOC -#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname }, +#define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) }, #else #define __PERCPU_RWSEM_DEP_MAP_INIT(lockname) #endif diff --git a/include/linux/rtmutex.h b/include/linux/rtmutex.h index 7d049883a08a..35889ac5eeae 100644 --- a/include/linux/rtmutex.h +++ b/include/linux/rtmutex.h @@ -81,6 +81,7 @@ do { \ .dep_map = { \ .name = #mutexname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(mutexname, NULL),\ } #else #define __DEP_MAP_RT_MUTEX_INITIALIZER(mutexname) diff --git a/include/linux/rwlock_types.h b/include/linux/rwlock_types.h index 1948442e7750..6e58dfc84997 100644 --- a/include/linux/rwlock_types.h +++ b/include/linux/rwlock_types.h @@ -10,6 +10,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL), \ } #else # define RW_DEP_MAP_INIT(lockname) diff --git a/include/linux/rwsem.h b/include/linux/rwsem.h index 1dd530ce8b45..1fa391e7770a 100644 --- a/include/linux/rwsem.h +++ b/include/linux/rwsem.h @@ -22,6 +22,7 @@ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SLEEP, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ }, #else # define __RWSEM_DEP_MAP_INIT(lockname) diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h index e92f9d5577ba..dee83ab183e4 100644 --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -81,7 +81,7 @@ static inline void __seqcount_init(seqcount_t *s, const char *name, #ifdef CONFIG_DEBUG_LOCK_ALLOC # define SEQCOUNT_DEP_MAP_INIT(lockname) \ - .dep_map = { .name = #lockname } + .dep_map = { .name = #lockname, .dmap = DEPT_MAP_INITIALIZER(lockname, NULL) } /** * seqcount_init() - runtime initializer for seqcount_t diff --git a/include/linux/spinlock_types_raw.h b/include/linux/spinlock_types_raw.h index 91cb36b65a17..3dcc551ded25 100644 --- a/include/linux/spinlock_types_raw.h +++ b/include/linux/spinlock_types_raw.h @@ -31,11 +31,13 @@ typedef struct raw_spinlock { .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_SPIN, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define SPIN_DEP_MAP_INIT(lockname) \ .dep_map = { \ .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } # define LOCAL_SPIN_DEP_MAP_INIT(lockname) \ @@ -43,6 +45,7 @@ typedef struct raw_spinlock { .name = #lockname, \ .wait_type_inner = LD_WAIT_CONFIG, \ .lock_type = LD_LOCK_PERCPU, \ + .dmap = DEPT_MAP_INITIALIZER(lockname, NULL),\ } #else # define RAW_SPIN_DEP_MAP_INIT(lockname) diff --git a/include/linux/srcu.h b/include/linux/srcu.h index 127ef3b2e607..f6b8266a4bfd 100644 --- a/include/linux/srcu.h +++ b/include/linux/srcu.h @@ -35,7 +35,7 @@ int __init_srcu_struct(struct srcu_struct *ssp, const char *name, __init_srcu_struct((ssp), #ssp, &__srcu_key); \ }) -#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name }, +#define __SRCU_DEP_MAP_INIT(srcu_name) .dep_map = { .name = #srcu_name, .dmap = DEPT_MAP_INITIALIZER(srcu_name, NULL) }, #else /* #ifdef CONFIG_DEBUG_LOCK_ALLOC */ int init_srcu_struct(struct srcu_struct *ssp); diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a3e774479f94..7e12e46dc4b7 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -244,10 +244,10 @@ static bool dept_working(void) * Even k == NULL is considered as a valid key because it would use * &->map_key as the key in that case. */ -struct dept_key __dept_no_validate__; +extern struct lock_class_key __lockdep_no_validate__; static bool valid_key(struct dept_key *k) { - return &__dept_no_validate__ != k; + return &__lockdep_no_validate__.dkey != k; } /* @@ -1936,7 +1936,7 @@ void dept_softirqs_off(void) dept_task()->softirqs_enabled = false; } -void dept_hardirqs_off(void) +void noinstr dept_hardirqs_off(void) { /* * Assumes that it's called with IRQ disabled so that accessing @@ -1958,7 +1958,7 @@ void dept_softirq_enter(void) /* * Ensure it's the outmost hardirq context. */ -void dept_hardirq_enter(void) +void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index 151bd3de5936..e27cf9d17163 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -1215,6 +1215,8 @@ void lockdep_register_key(struct lock_class_key *key) struct lock_class_key *k; unsigned long flags; + dept_key_init(&key->dkey); + if (WARN_ON_ONCE(static_obj(key))) return; hash_head = keyhashentry(key); @@ -4310,6 +4312,8 @@ static void __trace_hardirqs_on_caller(void) */ void lockdep_hardirqs_on_prepare(void) { + dept_hardirqs_on(); + if (unlikely(!debug_locks)) return; @@ -4430,6 +4434,8 @@ EXPORT_SYMBOL_GPL(lockdep_hardirqs_on); */ void noinstr lockdep_hardirqs_off(unsigned long ip) { + dept_hardirqs_off(); + if (unlikely(!debug_locks)) return; @@ -4474,6 +4480,8 @@ void lockdep_softirqs_on(unsigned long ip) { struct irqtrace_events *trace = ¤t->irqtrace; + dept_softirqs_on_ip(ip); + if (unlikely(!lockdep_enabled())) return; @@ -4512,6 +4520,8 @@ void lockdep_softirqs_on(unsigned long ip) */ void lockdep_softirqs_off(unsigned long ip) { + dept_softirqs_off(); + if (unlikely(!lockdep_enabled())) return; @@ -4859,6 +4869,8 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, { int i; + ldt_init(&lock->dmap, &key->dkey, subclass, name); + for (i = 0; i < NR_LOCKDEP_CACHING_CLASSES; i++) lock->class_cache[i] = NULL; @@ -5630,6 +5642,12 @@ void lock_set_class(struct lockdep_map *lock, const char *name, { unsigned long flags; + /* + * dept_map_(re)init() might be called twice redundantly. But + * there's no choice as long as Dept relies on Lockdep. + */ + ldt_set_class(&lock->dmap, name, &key->dkey, subclass, ip); + if (unlikely(!lockdep_enabled())) return; @@ -5647,6 +5665,8 @@ void lock_downgrade(struct lockdep_map *lock, unsigned long ip) { unsigned long flags; + ldt_downgrade(&lock->dmap, ip); + if (unlikely(!lockdep_enabled())) return; @@ -6447,6 +6467,8 @@ void lockdep_unregister_key(struct lock_class_key *key) unsigned long flags; bool found = false; + dept_key_destroy(&key->dkey); + might_sleep(); if (WARN_ON_ONCE(static_obj(key))) From patchwork Wed Feb 21 09:49:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565314 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C37823D995; Wed, 21 Feb 2024 09:49:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; cv=none; b=Uh8XSmriq7soxYdo7G1OMWHuPijLQ7fHb64lDN7uxr+Lw+50BMUcZds/dHpd1NvSbgCytFaXulp5Njq61aNXejdDAtiJ6KECUjuq2mETcFcn/0e8Lx4l0zU+v6pCNzGYTaY73CWWv5dsYgGHZp09a+nyvSdp+CkbHVP1sIQqVCs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; c=relaxed/simple; bh=37cSYW8WA/TnfUxZD66nhjfMFS+Rdxdh6OVt7EPfSog=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=R8fihbSictf5QDUdWNnMAE92FoLQBWKs4JMHYvRh5F706FVO8Q19XgKtCNJl4sW8BPoecQYjdPetmPycm0OqPFgbN4HfvAigMBVcJjgRRT8qTRbEgvEbT8UjX3c1KuEwx86ui+fTP1c+bHMTTAcfyhmTVZajpgsDaz4tdW5FYyM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-e6-65d5c738595f From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 06/27] dept: Add proc knobs to show stats and dependency graph Date: Wed, 21 Feb 2024 18:49:12 +0900 Message-Id: <20240221094933.36348-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ8f3+fpeHh2Mo+ylRsz8iPR7cP8GjaPP2zG/IrhVs90U+Gi xGwdh4qSH4lqVmnn+kFcKHTJtX6JunKlqNRN0VyydHGK3DH/fPba5/N+v/76sKT8Me3FqiOO SJoIVZgCyyhZ/4Ss+UurmyX/qy85uHTBHxxDcRRkFBZgsNzNR1DwQEtAX+V6eDNsRzDyqoGE 1BQLgqzuDhIeVHUiMBlOYXj9YSJYHQMYalPOYzh9qxBD4+dRAtqvXSYg37gR6pKzCSh3fqQg tQ9DeuppwjU+EeDU5zGgj50FNkMaA6Pdi6C2s4UG01s/uHGzHUOpqZaCqhIbAa+fZGDoLBij oa6qhgLLpUQa7nzJxvB5WE+C3jHAQFN5JgH3dC7R2W+/aahOLCfgbM59AqxtTxGUxXURYCxo wVDhsBNQZEwh4eftSgS2pH4GzlxwMpCuTUJw/sw1Chp+VdOgaw+EkR8ZePUyscI+QIq6omjR NJxJiS+yBfFxWgcj6sreMmKm8ahYZJgr3irtI8SsQQctGvPisWgcvMyICf1WQvxSX8+INddH KPGDNZXY5B0kWx4ihamjJM3Clftkoc53Jw69UB5LfmjGsSh/XgLyYAV+iaDtacP/+Y0hnnQz 5mcLra3Ov+zJ+wpFib10ApKxJH9uvGD4+spVYNnJ/GZB+32fO0Pxs4Rih5ZxM8cHCuaOQuaf 00fIv1f+1+PBK4XcdDvtZrkr09z4kHQ7BX6MFSw/ktC/wjThuaGVSkZcJhqXh+TqiKhwlTps yYLQmAj1sQXBB8ONyPVQ+pOju0rQoGWLGfEsUkzgQoutkpxWRUXGhJuRwJIKT46Kdq24EFXM cUlzcK/maJgUaUbeLKWYygUMR4fI+f2qI9IBSTokaf5fCdbDKxZRW+/yXYttqkd7pGe+sis+ Y7u9t3XtXLdyXI6N/MocfpTYn9dr1/txAclDa7cX68J/KhdzQ/EtQXHWmT4Nc9L2lMQyvc07 eqZgafaz3LFlXoJyg1JbEjRpzf0t6Q71ivE9HjMu+jU2OaTpdWXbclLeW4NX+bUez/U0+W7m ajfWtymoyFDVormkJlL1BzqHsJpMAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ/fHadn1eYZbbit2UJYso+h8VfftTGzmQ1TNz10dRV3RKzt UlJXcTUVFTtl5+qiXLX86Op26epSiTsk1dQMESldnPLjyvzz2Wvv93uvvz4c6VdKL+UUiSdE VaJcKWMklGTXlvS10P5cXN87EQr5uevBPZVFQVlNNQO9d0wIquvTCBhti4CX02MIZrqfkFBc 2IvgxvAgCfX2IQQW4zkGnG8Xg8s9zoCjMIeB9IoaBp5+miVgoKiAAJN5JzzWlRNg9bynoHiU gdLidMJ7PhDgMVSxYNAEwYixhIXZ4Q3gGHpBQ+s1Bw2W/tVw9foAA00WBwX2eyMEOB+UMTBU /YeGx/YOCnrz82i4/aWcgU/TBhIM7nEWnln1BNRmeG2Z337T0J5nJSDz5l0CXK8eImjOekOA ufoFA63uMQLqzIUk/LzVhmDk4mcWzud6WChNu4gg53wRBU9+tdOQMRAGMz/KmO1bcOvYOIkz 6k5hy7Sewp3lAr5fMsjijOZ+FuvNJ3GdMRhXNI0S+Makm8bmqmwGmycLWKz97CLwl54eFndc maHwW1cxsTtwv2RrjKhUJIuqdeHRkljP67PHOjed1jXYGA0yrdEiH07gNwovjdnkHDP8KqGv zzPPAfwKoS7vHa1FEo7kLywUjF+7GS3iOH9+j5D2PXpuQ/FBQqM7jZ1jKR8m2AZr2H/O5YKp 1jrv8eE3CZWlY/Qc+3k3z582kDok0aMFVShAkZicIFcow0LU8bEpiYrTIYeTEszI+zKG1Nn8 e2jKGWFDPIdki6SxjS7Rj5Ynq1MSbEjgSFmAlDrljaQx8pQzoiopSnVSKaptaBlHyZZII/eJ 0X78UfkJMV4Uj4mq/y3B+SzVoHhnoPTg9Q++m/lgS9NBqigk8JHVdsUzFN6VrbNptlfGNeT2 dPWXw9aJ2cFIHx0XetkZlbouvM/31Yrwqcn9b3hj1nCLVmk6FJqJj/8u8M9Pjbt0Lqjl18rQ MFaun/CPhAmRcxyw636+s2s+hsQNV2QfUe29sO0b/NkRdfnHlIxSx8o3BJMqtfwvB7e1ny4D AAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: It'd be useful to show Dept internal stats and dependency graph on runtime via proc for better information. Introduced the knobs. Signed-off-by: Byungchul Park --- kernel/dependency/Makefile | 1 + kernel/dependency/dept.c | 24 +++----- kernel/dependency/dept_internal.h | 26 +++++++++ kernel/dependency/dept_proc.c | 95 +++++++++++++++++++++++++++++++ 4 files changed, 131 insertions(+), 15 deletions(-) create mode 100644 kernel/dependency/dept_internal.h create mode 100644 kernel/dependency/dept_proc.c diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile index b5cfb8a03c0c..92f165400187 100644 --- a/kernel/dependency/Makefile +++ b/kernel/dependency/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DEPT) += dept.o +obj-$(CONFIG_DEPT) += dept_proc.o diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 7e12e46dc4b7..19406093103e 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,7 @@ #include #include #include +#include "dept_internal.h" static int dept_stop; static int dept_per_cpu_ready; @@ -260,20 +261,13 @@ static bool valid_key(struct dept_key *k) * have been freed will be placed. */ -enum object_t { -#define OBJECT(id, nr) OBJECT_##id, - #include "dept_object.h" -#undef OBJECT - OBJECT_NR, -}; - #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT -static struct dept_pool pool[OBJECT_NR] = { +struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ @@ -303,7 +297,7 @@ static void *from_pool(enum object_t t) if (DEPT_WARN_ON(!irqs_disabled())) return NULL; - p = &pool[t]; + p = &dept_pool[t]; /* * Try local pool first. @@ -338,7 +332,7 @@ static void *from_pool(enum object_t t) static void to_pool(void *o, enum object_t t) { - struct dept_pool *p = &pool[t]; + struct dept_pool *p = &dept_pool[t]; struct llist_head *h; preempt_disable(); @@ -2092,7 +2086,7 @@ void dept_map_copy(struct dept_map *to, struct dept_map *from) clean_classes_cache(&to->map_key); } -static LIST_HEAD(classes); +LIST_HEAD(dept_classes); static bool within(const void *addr, void *start, unsigned long size) { @@ -2124,7 +2118,7 @@ void dept_free_range(void *start, unsigned int sz) while (unlikely(!dept_lock())) cpu_relax(); - list_for_each_entry_safe(c, n, &classes, all_node) { + list_for_each_entry_safe(c, n, &dept_classes, all_node) { if (!within((void *)c->key, start, sz) && !within(c->name, start, sz)) continue; @@ -2200,7 +2194,7 @@ static struct dept_class *check_new_class(struct dept_key *local, c->sub_id = sub_id; c->key = (unsigned long)(k->base + sub_id); hash_add_class(c); - list_add(&c->all_node, &classes); + list_add(&c->all_node, &dept_classes); unlock: dept_unlock(); caching: @@ -2915,8 +2909,8 @@ static void migrate_per_cpu_pool(void) struct llist_head *from; struct llist_head *to; - from = &pool[i].boot_pool; - to = per_cpu_ptr(pool[i].lpool, boot_cpu); + from = &dept_pool[i].boot_pool; + to = per_cpu_ptr(dept_pool[i].lpool, boot_cpu); move_llist(to, from); } } diff --git a/kernel/dependency/dept_internal.h b/kernel/dependency/dept_internal.h new file mode 100644 index 000000000000..007c1eec6bab --- /dev/null +++ b/kernel/dependency/dept_internal.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Dept(DEPendency Tracker) - runtime dependency tracker internal header + * + * Started by Byungchul Park : + * + * Copyright (c) 2020 LG Electronics, Inc., Byungchul Park + */ + +#ifndef __DEPT_INTERNAL_H +#define __DEPT_INTERNAL_H + +#ifdef CONFIG_DEPT + +enum object_t { +#define OBJECT(id, nr) OBJECT_##id, + #include "dept_object.h" +#undef OBJECT + OBJECT_NR, +}; + +extern struct list_head dept_classes; +extern struct dept_pool dept_pool[]; + +#endif +#endif /* __DEPT_INTERNAL_H */ diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c new file mode 100644 index 000000000000..7d61dfbc5865 --- /dev/null +++ b/kernel/dependency/dept_proc.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Procfs knobs for Dept(DEPendency Tracker) + * + * Started by Byungchul Park : + * + * Copyright (C) 2021 LG Electronics, Inc. , Byungchul Park + */ +#include +#include +#include +#include "dept_internal.h" + +static void *l_next(struct seq_file *m, void *v, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_next(v, &dept_classes, pos); +} + +static void *l_start(struct seq_file *m, loff_t *pos) +{ + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + return seq_list_start_head(&dept_classes, *pos); +} + +static void l_stop(struct seq_file *m, void *v) +{ +} + +static int l_show(struct seq_file *m, void *v) +{ + struct dept_class *fc = list_entry(v, struct dept_class, all_node); + struct dept_dep *d; + const char *prefix; + + if (v == &dept_classes) { + seq_puts(m, "All classes:\n\n"); + return 0; + } + + prefix = fc->sched_map ? " " : ""; + seq_printf(m, "[%p] %s%s\n", (void *)fc->key, prefix, fc->name); + + /* + * XXX: Serialize list traversal if needed. The following might + * give a wrong information on contention. + */ + list_for_each_entry(d, &fc->dep_head, dep_node) { + struct dept_class *tc = d->wait->class; + + prefix = tc->sched_map ? " " : ""; + seq_printf(m, " -> [%p] %s%s\n", (void *)tc->key, prefix, tc->name); + } + seq_puts(m, "\n"); + + return 0; +} + +static const struct seq_operations dept_deps_ops = { + .start = l_start, + .next = l_next, + .stop = l_stop, + .show = l_show, +}; + +static int dept_stats_show(struct seq_file *m, void *v) +{ + int r; + + seq_puts(m, "Availability in the static pools:\n\n"); +#define OBJECT(id, nr) \ + r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ + if (r < 0) \ + r = 0; \ + seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + #include "dept_object.h" +#undef OBJECT + + return 0; +} + +static int __init dept_proc_init(void) +{ + proc_create_seq("dept_deps", S_IRUSR, NULL, &dept_deps_ops); + proc_create_single("dept_stats", S_IRUSR, NULL, dept_stats_show); + return 0; +} + +__initcall(dept_proc_init); From patchwork Wed Feb 21 09:49:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565313 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C9AC13D998; Wed, 21 Feb 2024 09:49:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; cv=none; b=PQlz0BHluIorRCG68L6l4R50agwmloadsCenFvZptQNUjF3ChtZ2w8r6Cd6cf+h6UEtESrKrGzHnao/XtbU9Bn/iwTWzyFKi2wLE7piDys8uA9J//CiaU/uk9oDQcOmI5QUTghLDhIgGZKi4misb4dvhK9jw2CLD+wsVdPp8Qzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; c=relaxed/simple; bh=Z/lrsPfinthw0Keb2TOWgQylHJ0tuWxQmHrjSTWrwaY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Zzxp5vTk90drJazPHOubyHaEuy8gqBzpxDNkDUQbD9+VZKDl4EN4aCKSHREhNoyF/u6pSkSo40y1FmMvTtygXNCYN20ttMfBzuBIyzJvp3tcMP/gMzKOueAZZYg2TN+rd5Ad2lMqWfyyjSVKr64DWzD3mNAL0zW7qUAoo5z4UDc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-f7-65d5c7395078 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 07/27] dept: Apply sdt_might_sleep_{start,end}() to wait_for_completion()/complete() Date: Wed, 21 Feb 2024 18:49:13 +0900 Message-Id: <20240221094933.36348-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSWUyTaRSG/b5/pVLz2yHhVy/QJobEnQH0SNToqOG7MZp4YeLEaJXfabVF 0rLIZBzBIjIgLhhaBRSKk1qhLhSI6AhWUKBuFG0ddIAog0sVRNGiLKKtM96cPHnPe56rw1OK OmYqr0lMlvSJKq2SldGy/lDL3MUtXmlB08lwOHpwAfg/5NBQcsHOgvt8JQJ7TSYG3814+Huo D8Ho3TYKzIVuBJanXRTUNHcjqLftY+FB7yTw+AdYcBXmsWA8fYGF9tdjGDpNBRgqHWvg9pFy DM7hFzSYfSwUm404MF5iGLZWcGDNmAk9tiIOxp5Ggav7IQP1j2fDiVOdLFytd9HQXNeD4cGV Eha67V8YuN3cSoP7aD4D596Us/B6yEqB1T/AwX1nGYaLWQFR9vtxBlrynRiy/6zC4Hn0F4KG nCcYHPaHLDT5+zBUOwopGDlzE0HPoX4O9h8c5qA48xCCvP0mGto+tzCQ1RkLo59K2OVxpKlv gCJZ1WmkfqiMJrfKRXK5qIsjWQ2POVLmSCHVtlnk9FUfJpZBP0McFX+wxDFYwJHcfg8mb+7d 40jr8VGa9HrMeN20jbIlCZJWkyrp5y/bIlO7up8xSdcm7x4b35uBaiblohBeFGLE0vc9dC7i v/HHqg3BmBUixY6OYSrIYcJ0sTr/OZOLZDwlHJgo2t7eZYP9HwS1OJIdHezQwkzR+KiUDbJc iBVv7cvG/+kjxMqLzm+eEGGheLa4jwmyItDxttdSQacoGENEh9f5/8EU8bqtgz6C5GVoQgVS aBJTdSqNNmaeOj1Rs3vetl06Bwr8k3XP2M91aNC9vhEJPFKGytWXPJKCUaUa0nWNSOQpZZic TgtE8gRV+q+SftdmfYpWMjSiaTytDJf/OJSWoBB+USVLOyUpSdJ/32I+ZGoGiv/Np9uzN/yJ y3RJ5s5bv6hr7Zwzq39aOMdSmbNq64doT8PJ2pHLsdft0QO/e7dvnGBKau893LxjhiUsqkBx pU13zBWX/M+qog3mqvgbixZnrl3+qpSUFEWk73xXEXfDcG68ZcqJTdaIpRmRKStbzXcmrnju jfT9q1hRG0NM2lCFzvhZSRvUqqhZlN6g+gpNMSk1SwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzH+36f5/k+T8fxSHhGhtvMpoSRfYxhRr5jjM1mMxs3nnXHddld RTYTnSgVIUeFU3ZyRbrLoh+61fpxLKJDLDdufkWqpSun/Liz+ee91z6f195/vQUmLJ+bKmj1 ibJBr9apiIJVbFqWNm9py3N5gTkVQW7WAvANnmShsLyMQPvtUgRllUcxdDetg5dDPQhG2p4w YM5rR3Dt3RsGKps9COpKjhHoeD8O3L4+Aq68UwTSissJPP06iqHrwlkMpfaN8OhMEQan/xML 5m4CBeY0HIjPGPxWGw/W1NngLcnnYfTdQnB5XnDQeNnFQd3rSLh0pYtAbZ2LheZ7Xgwd1YUE PGV/OHjU3MpCe242B7d6iwh8HbIyYPX18fDMacFwxxRoS//+m4OWbCeG9OsVGNyvahA8OPkW g73sBYFGXw8Ghz2PgZ83mhB4c77xcDzLz0PB0RwEp45fYOHJrxYOTF0xMPKjkKxaRht7+hhq chygdUMWlj4skuj9/Dc8NT14zVOLPYk6SubS4tpuTK8N+Dhqt2UQah84y9PMb25Mex8/5mnr xRGWvneb8eaI7Yrle2SdNlk2zF+xS6FxeT5w++snHBz9fSQVVY7LRIIgiYul4YptmShUIOIc qbPTzwQ5XJwpObI/cplIITDiiTFSSX8bCfoTRY30M31R0GHF2VLaq6skyEoxRnp4LB0HWRJn SKV3nP96QsUl0s2CHi7IYQHn+dO7zBmksKAQGwrX6pPj1VpdTLRxnyZFrz0YvTsh3o4Ci7Ee Hs29hwY71jUgUUCqsUpNlVsO49TJxpT4BiQJjCpcyR4InJR71CmHZEPCTkOSTjY2oGkCq5qi XL9N3hUmxqkT5X2yvF82/P9iIXRqKjq/Wm+LSHmmi20frN1hkSO9aHjWrJWfFI2e/iavrheq byXQGgPZXbM0LiSj4Fz9irVtpcOSp5gbH7UkIsO7fQez17am/37i1ljzlmLHZCnkUNTHAVNL RaslujsrdiwzeTo3422Gk3wpzxEvn94QOqnKvzFqS63BWp/U+Wto63wVa9SoF85lDEb1X3Bj TTotAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index fb2915676574..bd2c207481d6 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -10,6 +10,7 @@ */ #include +#include /* * struct completion - structure used to maintain state for a "completion" @@ -26,14 +27,33 @@ struct completion { unsigned int done; struct swait_queue_head wait; + struct dept_map dmap; }; +#define init_completion(x) \ +do { \ + sdt_map_init(&(x)->dmap); \ + __init_completion(x); \ +} while (0) + +/* + * XXX: No use cases for now. Fill the body when needed. + */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) {} -static inline void complete_release(struct completion *x) {} + +static inline void complete_acquire(struct completion *x) +{ + sdt_might_sleep_start(&x->dmap); +} + +static inline void complete_release(struct completion *x) +{ + sdt_might_sleep_end(); +} #define COMPLETION_INITIALIZER(work) \ - { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait) } + { 0, __SWAIT_QUEUE_HEAD_INITIALIZER((work).wait), \ + .dmap = DEPT_MAP_INITIALIZER(work, NULL), } #define COMPLETION_INITIALIZER_ONSTACK_MAP(work, map) \ (*({ init_completion_map(&(work), &(map)); &(work); })) @@ -75,13 +95,13 @@ static inline void complete_release(struct completion *x) {} #endif /** - * init_completion - Initialize a dynamically allocated completion + * __init_completion - Initialize a dynamically allocated completion * @x: pointer to completion structure that is to be initialized * * This inline function will initialize a dynamically created completion * structure. */ -static inline void init_completion(struct completion *x) +static inline void __init_completion(struct completion *x) { x->done = 0; init_swait_queue_head(&x->wait); From patchwork Wed Feb 21 09:49:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565312 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CC2FD3DB8B; Wed, 21 Feb 2024 09:49:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; cv=none; b=K0x8PVKp1n1eahvJ0KB+uyJbWnmkH0hGYDZ21ifCckEO3stCM8Hs7rt0eEM/voOto4vDi8F2xp8uJYH7Ci5j3jDFYRWdU4H6WLice6Yra2WTfoZ9XUe/wSMZTVtPC+Cwnf6PuCTBu30Uja7wvxC0U48qMLDkZ1oRI5ZANhrYb9U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708508999; c=relaxed/simple; bh=tg0pA2ZlUNywKtzMfrdbwkG7GYbhIVlK2HrCAbcpiMY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ilXn6EIazSSsZn+F6JDyR0nnMirrCCJINn0G+muttEpJagmlTYMILiJe26RUUn5D9zxW8aKw4gwyu1MQ+FGBccmhltVEMgY4PH9at3wMcs75YOEjamg95iru1gWq0UC2zSOdEK+eHrWEh86AHygHNTjFm73C2XUJ0mGZ0P4HuLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-08-65d5c739c5b2 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 08/27] dept: Apply sdt_might_sleep_{start,end}() to swait Date: Wed, 21 Feb 2024 18:49:14 +0900 Message-Id: <20240221094933.36348-9-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0yTZxSHfd/v2mrZl87Ed97ThGxx8cKCenBu0czEz0SNyVjCNo1W+Vib FSTlrjFDKKxDQK0iWpqlgGkrMHXFOxQYhCIyoEBBNEAmGhAtYNASKjhtNf5z8uSc3+/56/CU 8g6zmNcmJEv6BLVOxcpp+fiC0tVRLb3SurHc5XA6fx34XxlpsFypYsFzuRJB1bXjGMaat8P9 aR+C2fZOCoqLPAhKHw1ScM09hMDlyGKh50kYeP2TLLQWnWAhu/wKC13P5zAMnDNhqHTugrZT ZRgaAqM0FI+xUFKcjYPjKYaArYIDW2Y4DDvMHMw9ioDWoT4GXA+/hAt/DrBQ62qlwX1rGEPP HQsLQ1VvGWhz36XBc7qAgb8mylh4Pm2jwOaf5KC7wYrhqiEoyn35PwMtBQ0Yci/+jcH7oAZB nfE/DM6qPhaa/D4M1c4iCl7bmxEMF45zkJMf4KDkeCGCEznnaOh808KAYWA9zM5Y2C2bxCbf JCUaqtNE17SVFu+VEfG2eZATDXUPOdHqTBGrHavE8toxLJZO+RnRWfEHKzqnTJyYN+7F4kRH ByfePT9Li0+8xXjPkp/km2MlnTZV0q/99oBcU2j0s4luPt2V6aMzUSebh2Q8ESLJDX89/siW phEUYlb4nPT3B6gQLxRWkuqCESYPyXlK+H0+cbxoD5Z5/lNhF2nr+CSUoYVw0mi/976rENaT i3YH98G5glRebXjvkQkbyKUSHxNiZTDT23WdCjmJkC0jA1km6kPhM/KPo58+hRRWNK8CKbUJ qfFqrS5yjSYjQZu+5tDheCcKfpTt2NzPt9CU5/tGJPBItUChuemVlIw6NSkjvhERnlItVNBp wZUiVp1xRNIf3q9P0UlJjWgJT6sWKb6aTotVCr+ok6VfJSlR0n+8Yl62OBMthZxv8nXRYTHz Jsq7V6SbLpEI0+r4nfpkS0yFKqtZGKktsmruHzWrR6M0YSc95h8NB523d5zd3Zdi3zf6dpn1 8vmZbbJS46aV8rMa84TU3xNXH7c30jgz1b3RFgPRyT/Y3STy3wPfNZ+ZG/ri663Lort+e9z+ rGfcEQCTuebFdRWdpFFHrKL0Sep38+YPL00DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfZ/fHcfjNJ5hw01j3VBbtY/fZm2+szH/YMymZzzTrR/sLiXG jpL0azIJHVJcpyKemNCRa10q/bquUCc04ubIzl2cChfzz2evfT7v9+uvD0eqjPRMTpuYJOkS xXg1o6AUG5enLVra2C2F1bg4yM8JA583kwJjVSUDHTcrEFTeOUqAq2EdPB92IxhpbSehsKAD wZW3r0i4Y+tHYDEfY6Dr3WRw+IYYaCrIZiCttIqBzk+jBDjPniagQt4ALadKCKjzf6Cg0MVA UWEaERgfCfCbylkwGUJgwHyBhdG34dDU30ND/cUmGiy9Gjh/yclAraWJAlvNAAFdD4wM9Ff+ pqHF9pSCjvxcGm58KWHg07CJBJNviAV7XTEBt9IDtoxvv2hozK0jIOPqbQIcLx8ieJT5hgC5 soeBep+bgGq5gISfZQ0IBvI+s3A8x89C0dE8BNnHz1LQPtZIQ7ozEkZ+GJk1y3G9e4jE6dUp 2DJcTOHmEgHfv/CKxemPellcLO/H1eZQXFrrIvAVj4/GcvlJBsue0yzO+uwg8Je2NhY/PTdC 4XeOQmLT7O2KFbuleG2ypFuyKkYRm5fpY/bZuAMWg5syoHYmCwVxAh8hGOsH0Tgz/ALhxQs/ Oc7B/FyhOneQzkIKjuRPTBTMX1sDBY6bxm8QWtqmjGcoPkSwljX/7Sr5SOFqmZn955wjVNyq ++sJ4qOE60VuepxVgUx3513yFFIUownlKFibmJwgauMjF+vjYlMTtQcW79qbIKPAz5gOj+bX IG/XOiviOaSepIy955BUtJisT02wIoEj1cFKKiWwUu4WUw9Kur07dfvjJb0VzeIo9Qzl+q1S jIrfIyZJcZK0T9L9vxJc0EwDerJJnCMHjS3TbXvwPe5Mz8Jrnqhs++WVoSvtmhOb98i3I9Z7 KO3q6LnerdGvG+xHpjo9O/rGYnCfxi9kWqcvzHAtWXFofrQh6jUdsZFf/TD4sn3W+57hPHv7 jy1uTcoH53bbPNUqVTd+PBgW4p3a2BwZahJvrv39bMZXb62mqypHTeljxfBQUqcX/wCj3/+Y LwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by swaits. Signed-off-by: Byungchul Park --- include/linux/swait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/swait.h b/include/linux/swait.h index d324419482a0..277ac74f61c3 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -6,6 +6,7 @@ #include #include #include +#include #include /* @@ -161,6 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); struct swait_queue __wait; \ long __ret = ret; \ \ + sdt_might_sleep_start(NULL); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ @@ -176,6 +178,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); cmd; \ } \ finish_swait(&wq, &__wait); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Wed Feb 21 09:49:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565315 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 127593EA71; Wed, 21 Feb 2024 09:49:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509000; cv=none; b=cobb+hG3vFK6DJMtk2zlGTXsApIClJIQ5//b6ELA4oIDyc0TM2i7EZHnrL544ztGEOJcF0L68T87PfYpG/8q3UCMP81N722IDim5xnYdrkzeR5cOGpoSDsokQBIoXfda5gy1koRoVVMQ7YLJ08b4i9cMHAeoFGa4a6BOx6bc8bA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509000; c=relaxed/simple; bh=RIc5udGVOneBYojEExyl4QYDrUVZJa4Vo+L4Ic7nXzw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=XgAwwcOYWFSCFrRb5aJIf4aBfIWpiOeQPtgDITyoQhmb2qhJ6ygBOfosL2pDMtoIgtG5jZXDjOUK6+SiWVazrcv2X4ZLXXRxM1pINOOV7EBtLHLXG05immMv0CFd2eUTqW6ttFlKurq3d4kO/VfiWo8KgX8eM6wrIXq3ocGqIv4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-19-65d5c7392c4f From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 09/27] dept: Apply sdt_might_sleep_{start,end}() to waitqueue wait Date: Wed, 21 Feb 2024 18:49:15 +0900 Message-Id: <20240221094933.36348-10-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSV0xTYRiG/f8zW62eVI3HEUfjxIUzn0bBO48aEw3qhcRII0fbWCppsYjR BGWKlgAKqBAthZQyXAUiojWACiKCVQuiAaKIg1BGkFaROlqMN1+evG/e5+pjCfk9agar1kaJ Oq1So6ClpLRvgnn5hvoWMTD220JIvxAI7uFkEnJvldLguFmCoLT8DIaeJ1vhjceFYLTpBQHZ mQ4EeR86CCiv60Rgt56l4XX3RHC6B2hoyDxPQ1z+LRpe9noxtGdlYCix7YTGNDOG6pEvJGT3 0JCTHYd95yuGEUsxA5bYBdBlvcqA98MqaOhspcD+bilcudZOwwN7Awl1lV0YXlfl0tBZ+oeC xrqnJDjSjRTc6DfT0OuxEGBxDzDwqtqE4Xa8T5T47TcF9cZqDIkFdzA4395H8DD5PQZbaSsN j9wuDGW2TAJ+Fj5B0JXax0DChREGcs6kIjifkEXCi1/1FMS3r4PRH7n0lo3CI9cAIcSXRQt2 j4kUnpl54d7VDkaIf/iOEUy240KZNUDIf9CDhbwhNyXYis/Rgm0ogxFS+pxY6G9uZoSnl0dJ oduZjXfN3C/dFC5q1AZRtzIoTKp69r2HiiySnBhMS6JjUSGTgliW59byn6pOpSDJGCYO1BB+ prlFfFvbyBhP4ebyZcbPVAqSsgSXNJ63DjbR/mIyt4/3lP/Afg/JLeAd9rFYxq3nLz5/z/xz zuFLblePeSS+vCjHRflZzq3jW15WEH4nz8VJ+Px+D/VvMJ2vsbaRaUhmQuOKkVytNUQo1Zq1 K1QxWvWJFYeORdiQ76Esp72hlWjIEVKLOBYpJshUd52inFIa9DERtYhnCcUUGRnti2ThypiT ou7YQd1xjaivRTNZUjFNttoTHS7njiijxKOiGCnq/reYlcyIRSZNSNj2Ym3IkSx2GvppKNBv uHRnanhFpCPrQP6sHWZ5kyrZsSs5eFbBksfza0KHA9G8L/cHd39fHKSa2Lm1du/04NGDAUUV WxTLFOP6Thtbzrpch66EXjcqKlqGvZMMbUmm2R+9a9QJsxubDy88sGZbYXdO+9Q9qzcHVfUG dhQmdL1SkHqVclUAodMr/wKrqYZqTAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRiH+//PdavFYQkeureKMNMMMt4woovkMejyqaIgHXVoy3lpMy9F YXltpqilq1xhGmvpum2GdpmY4mWapbkszUQlqtFMqaaZdplSX14enh88n16WkBupuaw6Jl7U xig1ClpKSneGpAasb3olBumNHOSfDwLP9ywSjHctNLTfqUBgqTyDwdUQBq9H3Qgm2l4QYChs R3B94B0BlY19COzmszR0vp8NTs8wDY7CbBpSy+7S0PF5EkNvUQGGCusOaM0rxVA7/pEEg4uG YkMq9p5PGMZN5QyYUpbDoPkKA5MDa8DR10VB/VUHBfYef7h8rZeGJ3YHCY3Vgxg6Hxlp6LP8 oaC1sZmE9vwcCm5/KaXh86iJAJNnmIGXtSUY7qV5axnfflPQlFOLIePGfQzO7scIarL6MVgt XTTUe9wYbNZCAn7ebEAwmDvEQPr5cQaKz+QiyE4vIuHFryYK0nqDYeKHkd4UItS7hwkhzZYo 2EdLSKGllBceXnnHCGk1PYxQYj0u2MwrhbInLixc/+qhBGv5OVqwfi1gBP2QEwtfnj9nhOZL E6Tw3mnAu+fvl244LGrUCaJ29cZIqaplzEXF3ZIkjeRl0inoJqNHEpbn1vIZw0+JKaa5Ffyb N+PT7MMt5m05Hyg9krIElzmTN4+00VPDHG4PP1r5A+sRy5Lccr7dPq1l3Dr+wrP+f81FfMW9 2umOxOtvFbupKZZzwfyrjgdEHpKWoBnlyEcdkxCtVGuCA3VRquQYdVLgodhoK/L+jOnUZH41 +t4ZVoc4FilmyVRVTlFOKRN0ydF1iGcJhY+MTPQq2WFl8glRGxuhPa4RdXVoHksqfGXb94qR cu6IMl6MEsU4Uft/xaxkbgoKsK/aVvfywr7Q7qOLRoZCTh70dRV05nYvXmqmLtrFnKi4rCUz ydJYviNk65ETk2Nt2NfVOmb7udDTk77gkmjQZGs2+PVJ/PWKMP+GZqsqPPRylTt8jzz+wLG9 p8/mdX1Q6dOXRe4o2+V4uyUg2xKxqTA0qsJ5au0qv2Obe7rWZTYpSJ1KuWYlodUp/wIpHJes LwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait.h b/include/linux/wait.h index 3473b663176f..ebeb4678859f 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -303,6 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ @@ -318,6 +320,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); cmd; \ } \ finish_wait(&wq_head, &__wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Wed Feb 21 09:49:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565316 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 49FD23F8C3; Wed, 21 Feb 2024 09:49:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509000; cv=none; b=N4/j/8dDWwpkGPbJuP+CI+ad4ayReBGxbBJ3vNgC/XO7tnT+Yg8w4YuTQ9xRn4gtbqWfIgN5l2jB8+KprWmv/tPJzrKSXeX3GFX1gWENuC2NMrHaJ/vaF7JiuTUJxesFxvI7LDle0FtICCQhThbZ6M+v70wpzGY+5+L/RwAO/NY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509000; c=relaxed/simple; bh=MruAYr/ze2aZU4Q6DDtEFdBspBpkL05uqzvhsrQG5Gc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=kIrSy8MqjJQ97vU39lyIBnJ3gqkAN6LbkunRPPHRFidzCTkkEFDgNvD8aoRzL9Ou4jK755JmOrssnmp7iMrp0TPSC7eNMQQGzY4kuHemBt2rnpR4Ad+ZcQinWE3kmIFYOdbNSuGmNiCJTIkooqut5/ylEjiVoglqk1iqfF9DTkA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-28-65d5c73939f9 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 10/27] dept: Apply sdt_might_sleep_{start,end}() to hashed-waitqueue wait Date: Wed, 21 Feb 2024 18:49:16 +0900 Message-Id: <20240221094933.36348-11-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAz2SfUzMcRzHfb+/xzuu/XaMX/mD3ZiNJdmxDzOLGb9/bB5m8xi3+k3HdeVS yTwUJxxRzYmrccXOVUe5qylkp+e0FB0d65paHtIlKxepcJfxz2evvd97v/76sIT8ERXCqrWH RZ1WpVHQUlI6MC0/dEXDK3FJeU0oZF1cAr5v50jIK7HR0HavGIGtLA1DX90G6BjxIhhraSUg x9iGIL/bQ0BZfReCKuspGtp7g8DlG6ShyXiBhtO3Smh40T+OofNqNoZi+0ZozizA4Bz9SEJO Hw25Oaex/3zCMGopYsCSOh96rCYGxrvDoanrNQVVbxfB9RudNDyuaiKhvqIHQ/vDPBq6bL8p aK5vJKEtK4OCu18KaOgfsRBg8Q0y8NJpxlCq94vSh39R0JDhxJB++z4G15tHCJ6ce4fBbntN Q43Pi8FhNxLw804dgp5LAwycuTjKQG7aJQQXzlwloXWigQJ95zIY+5FHR6wUaryDhKB3JAtV I2ZSeFbAC5UmDyPon7xlBLM9UXBYFwq3HvdhIX/IRwn2ovO0YB/KZgTDgAsLX54/Z4TGa2Ok 0OvKwZtm75SuihY16iRRF7Z6nzTmoyeLinezRyZqK5lU5KYNSMLynJI3mb7/5zpDKRlgmlvA u92jRIBncHN5R8YHyoCkLMGdncpbv7ZMDqZze3hzXvvkgOTm8+VXvDjAMm4539dqwH+lc/ji UuekSOLPC3O9VIDl3DL+1YtyIiDlubMSvlGfTf0dBPNPrW4yE8nMaEoRkqu1SbEqtUa5OCZF qz6yOCou1o78L2U5Pr6rAg21ba1GHIsU02QxD1yinFIlJaTEViOeJRQzZGSyP5JFq1KOirq4 vbpEjZhQjWazpGKWbOlIcrSc2686LB4UxXhR96/FrCQkFR03/nxf1vE09LJn5vDGtPD1hT9c O7qUaia9RRM0L7Iucezu9t2mZs9JnLxde0wpfXNAtmVdbUdIELnipTNkc2bkSmnG0UPLv9/M 8j1oXHRi5pqGcaYwIsKdNpEfGXWM+zxPubZkWzD+LAlbasu1aoYyHcb+fXPLeoeD6e7wd3GR WgWZEKMKX0joElR/AKHS4eZOAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfb+/x45rP2n85mHsxmyZiI6P1ZqHWT+MYW22MG7dbzoquVPE mutBOIqahC67HpxT4VxZoU4rlbQe6ErsapRRuh6WLnrwcGX++ey193vv118flvDIouazqoiT ojpCESajJaRkl1/Cyg21reLqqoF5kHplNThHLpKgf1RIQ/PDAgSFxXEYeqsD4d2oA8FEQxMB GenNCLI/dRBQXNOJoNwUT0PLZ3ewOQdpqEu/TENC7iMa3vRNYrDfSMNQYNkJ9ddyMFSMfSUh o5eGzIwE7Do9GMaM+QwYtcugy3SbgclPPlDX2UZBVVYdBeUfVsCtO3YaysrrSKgp7cLQ8kxP Q2fhHwrqa16R0JyaTMGDgRwa+kaNBBidgwy8rTBgMCe6bEnff1NQm1yBISnvMQbb++cIrBc/ YrAUttFQ5XRgKLKkEzB+rxpBV0o/A+evjDGQGZeC4PL5GyQ0/aqlINEuh4mfenqjn1DlGCSE xKJTQvmogRRe5/DC09sdjJBo/cAIBkuUUGTyEnLLerGQPeykBEv+JVqwDKcxgq7fhoWBxkZG eHVzghQ+2zLw7oXBEn+lGKaKFtWrAg5LQr92pFKR7ezpXy+fMlrUTuuQG8tzvny1zkxOMc0t 59vbx4gp9uSW8EXJXygdkrAEd2EmbxpqmB7M4Q7yBn3L9IDklvFPrjvwFEu5dXxvkw7/ky7m C8wV0yI3V34/00FNsQcn51vfPCGuIYkBzchHnqqI6HCFKkzurTkWGhOhOu0dcjzcglxPY4yd TC1FIy2BlYhjkWyWNLTEJnpQimhNTHgl4llC5iklT7kiqVIRc0ZUHz+kjgoTNZVoAUvK5km3 7xMPe3BHFCfFY6IYKar/t5h1m69Fce+z5sofl+wxamtLu1VNm4Lct/iKB63WoMCrlVH6pd1e e+8qDXJdnD5+Yoc4NJyXoNQcXWs863vAv23cfmJzG2FWbl/l3bojcmuD2R58K3lRgPvH9O5F 0br1S9lna1Zod/ulaSN/WEI6vr2N+bP/XOzsQNNOa0qP5sX66m1JxQHbZKQmVOHjRag1ir9t 5u28MAMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dependencies by hashed-waitqueue waits. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index 7725b7579b78..fe89282c3e96 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -6,6 +6,7 @@ * Linux wait-bit related types and methods: */ #include +#include struct wait_bit_key { void *flags; @@ -246,6 +247,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ + sdt_might_sleep_start(NULL); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ @@ -263,6 +265,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); cmd; \ } \ finish_wait(__wq_head, &__wbq_entry.wq_entry); \ + sdt_might_sleep_end(); \ __out: __ret; \ }) From patchwork Wed Feb 21 09:49:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565322 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9432D3FE2C; Wed, 21 Feb 2024 09:49:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509006; cv=none; b=Nxpvi7w6C37hOllc6f/e+43Os5nM4bT2nCbmQyaxFEs1ovfUvV38d67dLWTOVJgtzMbuXPuzkqzUYEXydfPbtHnzzzYby9A+KtEXktI74hHaxhxbegLd5Y4kZiThFOtR2f8qRzLxHe1A+EAy7LRQHy4FqYwj7gXcrAT/E83q9YQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509006; c=relaxed/simple; bh=nifTUInVysx7YljtcW2Qa4mywSC4rocnbBoHNOK15ls=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=B84YeYwuZh6Fe5Lqplm6emXmkhdX9nCJqkKa7kLaTqEGsOhqD5Q6nwb6jrC0nO9KBrQF76xyuoeTF/7rA4rSLxarNjrIkP0Tohl6b4z1CrRzUmCaFoJxbiM6aFEU5P00e+qmbktoj6+MU1FTrT4eiGhndtJ6NxWN7s54WijmVLk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-38-65d5c7393909 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 11/27] dept: Distinguish each syscall context from another Date: Wed, 21 Feb 2024 18:49:17 +0900 Message-Id: <20240221094933.36348-12-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0xTZxjH977nnPeUQpdjNXrwEk2RGFHxEtieGbPti/HN5twyLzH6QU7k ZK0WNK2AmCwDKQy5RUygE4iBaroKVbFgVC4GIYAdASlWbgIOUjeJFBagnUi9FIxfnvzy++f/ //QoGHUdt1KhSzwjGxIlvYYoWaU3rGLLV+1P5W0VozuhMG8b+GazWSi7ZSfQfbMKgb02HcN4 6x7o808gmO98zIC5qBtBxegwA7VtIwgabecJPPF8Dm7fFAFnUS6BjKu3CLheBTAMFV/CUOX4 ATouWjA0zf3LgnmcQKk5AwfPSwxz1koerGmRMGYr4SEwuh2cI70cNA5ugstXhgg0NDpZaLs3 huFJXRmBEft7DjraHrHQXZjPwY1JC4FXfisDVt8UDz1N5RiqTcGhrJl3HLTnN2HIunYbg3ug HsGD7L8xOOy9BFp8ExhqHEUMvPmzFcFYgZeHzLw5HkrTCxDkZhaz8PhtOwemoViYf11Gvt1J WyamGGqqSaGN/nKW/mUR6f2SYZ6aHgzytNyRRGtsUfRqwzimFdM+jjoqLxDqmL7E0xyvG9PJ ri6ePvpjnqUetxn/tOqIcle8rNcly4atX8cpta60BnLaGXfW5ZHSUP3eHBSiEIUYsbDjJf7E z7NfoAUmwgaxv3+OWeBlwjqxJv8fLgcpFYzwe6ho+6+TLARLhX3i5QLPIrNCpGgazV0sqIQv xN7/TdzH0bViVXXTog8J+uulE4teLcSKT113mIVRUcgIEWudI/zHQrj40NbPXkSqcvRZJVLr EpMTJJ0+Jlqbmqg7G338VIIDBT/K+mvg6D003b2/GQkKpAlTae+6ZTUnJRtTE5qRqGA0y1Rs SlCp4qXUc7Lh1DFDkl42NqNVClazQrXDnxKvFn6RzsgnZfm0bPiUYkXIyjS068JM9Aze4vmu c3USU20Js5woXVtPW8ts+HXelePVAz3Pjuze/PP68MHZwPWBqO/jArkxBzOdL+oKf4vw7w11 x+67P+tKMq5eZ2/d8+ak91xXw7UVAf3GLyPW9P0YF04ipBJtZOiS4hRnZv/bA95nA+aWHT2T wwl1WYcOL0/va9Z9o2GNWml7FGMwSh8ALK+Gwk0DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PHaefU/zQsKNFqJjsY5n5ix9TDJuNWf3Gb93NdXKXlI2V O0m5Viahq12x08MhVyZ6WKv1cJrq6nrQKtXy0EQeukhHrsw/n732eu/9/uvD4DIjuZxRqmNE jVpQySkJIQkL0W3a3tgpBlXkyiHjehA4J5IJMD62UND2qBiBpSwRg9H6PdA9OYZg+lUrDlmZ bQjyhvpxKGsYQFBVcJmCjpGF4HCOU2DLTKVAd+8xBfaPLgz6bt3AoNgaCs3p+RjUTL0nIGuU guwsHeY+HzCYMhfRYE7wheGCuzS4hjaDbaCLhLocGwlVvRvgTm4fBZVVNgIayocx6HhhpGDA MkNCc0MTAW0ZBhIefs6n4OOkGQezc5yG9hoTBiV691rS9z8kNBpqMEi6/wQDx+sKBNXJgxhY LV0U1DnHMCi1ZuLw60E9guG0TzRcuT5FQ3ZiGoLUK7cIaP3dSIK+LximfxqpXSF83dg4zutL z/NVkyaCf5nP8c/v9tO8vrqX5k3Wc3xpgT9/r3IU4/O+OUneWnSN4q3fbtB8yicHxn9uaaH5 ptvTBD/iyMIO+hyT7DglqpSxoiZwZ4REYU+opKJtEXH2ESEBVexPQR4Mx27l3iS/RbNMsX5c T88UPste7Gqu1PCOTEESBmevzucKvryiZoPFbBh3J21kjgnWl9MPpc4VpOw2ruuHnvw3uoor LqmZ8x5uX5g9NudlbDDXaX+KpyOJCc0rQl5KdWyUoFQFB2hPK+LVyriAk2eirMj9M+aLroxy NNGxpxaxDJIvkCqeOUQZKcRq46NqEcfgci8pcd6tpKeE+Aui5ky45pxK1NaiFQwhXyrdd1SM kLGRQox4WhSjRc3/FGM8lieg4YPqZU2dRq/ihYYcabQuOb1dpfgSygXGJa6v1dn2GzzDXCFn fa5tehKZ7/mw7oOKMRX6+Z2IG3yQ2D7TrXYNTQQt8RDk9bsvHW8sbGC+V2/ce3TL80KTz+9F 3jLL12b9seDDK4+E7FSGx7xb+2vNn/ab3paSEj/juqSmQ/X2GccBOaFVCJv9cY1W+As8mfyx LwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: It enters kernel mode on each syscall and each syscall handling should be considered independently from the point of view of Dept. Otherwise, Dept may wrongly track dependencies across different syscalls. That might be a real dependency from user mode. However, now that Dept just started to work, conservatively let Dept not track dependencies across different syscalls. Signed-off-by: Byungchul Park --- arch/arm64/kernel/syscall.c | 3 ++ arch/x86/entry/common.c | 4 +++ include/linux/dept.h | 39 ++++++++++++--------- kernel/dependency/dept.c | 67 +++++++++++++++++++------------------ 4 files changed, 64 insertions(+), 49 deletions(-) diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index 9a70d9746b66..96c18ad1dbf7 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -7,6 +7,7 @@ #include #include #include +#include #include #include @@ -100,6 +101,8 @@ static void el0_svc_common(struct pt_regs *regs, int scno, int sc_nr, * (Similarly for HVC and SMC elsewhere.) */ + dept_kernel_enter(); + if (flags & _TIF_MTE_ASYNC_FAULT) { /* * Process the asynchronous tag check fault before the actual diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index 6356060caaf3..445e70937b38 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -20,6 +20,7 @@ #include #include #include +#include #ifdef CONFIG_XEN_PV #include @@ -75,6 +76,7 @@ static __always_inline bool do_syscall_x32(struct pt_regs *regs, int nr) /* Returns true to return using SYSRET, or false to use IRET */ __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr) { + dept_kernel_enter(); add_random_kstack_offset(); nr = syscall_enter_from_user_mode(regs, nr); @@ -262,6 +264,7 @@ __visible noinstr void do_int80_syscall_32(struct pt_regs *regs) { int nr = syscall_32_enter(regs); + dept_kernel_enter(); add_random_kstack_offset(); /* * Subtlety here: if ptrace pokes something larger than 2^31-1 into @@ -283,6 +286,7 @@ static noinstr bool __do_fast_syscall_32(struct pt_regs *regs) int nr = syscall_32_enter(regs); int res; + dept_kernel_enter(); add_random_kstack_offset(); /* * This cannot use syscall_enter_from_user_mode() as it has to diff --git a/include/linux/dept.h b/include/linux/dept.h index c6e2291dd843..4e359f76ac3c 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -25,11 +25,16 @@ struct task_struct; #define DEPT_MAX_SUBCLASSES_USR (DEPT_MAX_SUBCLASSES / DEPT_MAX_SUBCLASSES_EVT) #define DEPT_MAX_SUBCLASSES_CACHE 2 -#define DEPT_SIRQ 0 -#define DEPT_HIRQ 1 -#define DEPT_IRQS_NR 2 -#define DEPT_SIRQF (1UL << DEPT_SIRQ) -#define DEPT_HIRQF (1UL << DEPT_HIRQ) +enum { + DEPT_CXT_SIRQ = 0, + DEPT_CXT_HIRQ, + DEPT_CXT_IRQS_NR, + DEPT_CXT_PROCESS = DEPT_CXT_IRQS_NR, + DEPT_CXTS_NR +}; + +#define DEPT_SIRQF (1UL << DEPT_CXT_SIRQ) +#define DEPT_HIRQF (1UL << DEPT_CXT_HIRQ) struct dept_ecxt; struct dept_iecxt { @@ -94,8 +99,8 @@ struct dept_class { /* * for tracking IRQ dependencies */ - struct dept_iecxt iecxt[DEPT_IRQS_NR]; - struct dept_iwait iwait[DEPT_IRQS_NR]; + struct dept_iecxt iecxt[DEPT_CXT_IRQS_NR]; + struct dept_iwait iwait[DEPT_CXT_IRQS_NR]; /* * classified by a map embedded in task_struct, @@ -207,8 +212,8 @@ struct dept_ecxt { /* * where the IRQ-enabled happened */ - unsigned long enirq_ip[DEPT_IRQS_NR]; - struct dept_stack *enirq_stack[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *enirq_stack[DEPT_CXT_IRQS_NR]; /* * where the event context started @@ -252,8 +257,8 @@ struct dept_wait { /* * where the IRQ wait happened */ - unsigned long irq_ip[DEPT_IRQS_NR]; - struct dept_stack *irq_stack[DEPT_IRQS_NR]; + unsigned long irq_ip[DEPT_CXT_IRQS_NR]; + struct dept_stack *irq_stack[DEPT_CXT_IRQS_NR]; /* * where the wait happened @@ -406,19 +411,19 @@ struct dept_task { int wait_hist_pos; /* - * sequential id to identify each IRQ context + * sequential id to identify each context */ - unsigned int irq_id[DEPT_IRQS_NR]; + unsigned int cxt_id[DEPT_CXTS_NR]; /* * for tracking IRQ-enabled points with cross-event */ - unsigned int wgen_enirq[DEPT_IRQS_NR]; + unsigned int wgen_enirq[DEPT_CXT_IRQS_NR]; /* * for keeping up-to-date IRQ-enabled points */ - unsigned long enirq_ip[DEPT_IRQS_NR]; + unsigned long enirq_ip[DEPT_CXT_IRQS_NR]; /* * for reserving a current stack instance at each operation @@ -465,7 +470,7 @@ struct dept_task { .wait_hist = { { .wait = NULL, } }, \ .ecxt_held_pos = 0, \ .wait_hist_pos = 0, \ - .irq_id = { 0U }, \ + .cxt_id = { 0U }, \ .wgen_enirq = { 0U }, \ .enirq_ip = { 0UL }, \ .stack = NULL, \ @@ -503,6 +508,7 @@ extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); +extern void dept_kernel_enter(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -552,6 +558,7 @@ struct dept_task { }; #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) +#define dept_kernel_enter() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 19406093103e..9aba9eb22760 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -220,9 +220,9 @@ static struct dept_class *dep_tc(struct dept_dep *d) static const char *irq_str(int irq) { - if (irq == DEPT_SIRQ) + if (irq == DEPT_CXT_SIRQ) return "softirq"; - if (irq == DEPT_HIRQ) + if (irq == DEPT_CXT_HIRQ) return "hardirq"; return "(unknown)"; } @@ -406,7 +406,7 @@ static void initialize_class(struct dept_class *c) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iecxt *ie = &c->iecxt[i]; struct dept_iwait *iw = &c->iwait[i]; @@ -431,7 +431,7 @@ static void initialize_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { e->enirq_stack[i] = NULL; e->enirq_ip[i] = 0UL; } @@ -447,7 +447,7 @@ static void initialize_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { w->irq_stack[i] = NULL; w->irq_ip[i] = 0UL; } @@ -486,7 +486,7 @@ static void destroy_ecxt(struct dept_ecxt *e) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (e->enirq_stack[i]) put_stack(e->enirq_stack[i]); if (e->class) @@ -502,7 +502,7 @@ static void destroy_wait(struct dept_wait *w) { int i; - for (i = 0; i < DEPT_IRQS_NR; i++) + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) if (w->irq_stack[i]) put_stack(w->irq_stack[i]); if (w->class) @@ -651,7 +651,7 @@ static void print_diagram(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { if (!firstline) pr_warn("\nor\n\n"); firstline = false; @@ -684,7 +684,7 @@ static void print_dep(struct dept_dep *d) const char *tc_n = tc->sched_map ? "" : (tc->name ?: "(unknown)"); irqf = e->enirqf & w->irqf; - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { pr_warn("%s has been enabled:\n", irq_str(irq)); print_ip_stack(e->enirq_ip[irq], e->enirq_stack[irq]); pr_warn("\n"); @@ -910,7 +910,7 @@ static void bfs(struct dept_class *c, bfs_f *cb, void *in, void **out) */ static unsigned long cur_enirqf(void); -static int cur_irq(void); +static int cur_cxt(void); static unsigned int cur_ctxt_id(void); static struct dept_iecxt *iecxt(struct dept_class *c, int irq) @@ -1458,7 +1458,7 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) if (d) { check_dl_bfs(d); - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_iwait *fiw = iwait(fc, i); struct dept_iecxt *found_ie; struct dept_iwait *found_iw; @@ -1494,7 +1494,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, struct dept_task *dt = dept_task(); struct dept_wait *w; unsigned int wg = 0U; - int irq; + int cxt; int i; if (DEPT_WARN_ON(!valid_class(c))) @@ -1510,9 +1510,9 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; - irq = cur_irq(); - if (irq < DEPT_IRQS_NR) - add_iwait(c, irq, w); + cxt = cur_cxt(); + if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) + add_iwait(c, cxt, w); /* * Avoid adding dependency between user aware nested ecxt and @@ -1593,7 +1593,7 @@ static bool add_ecxt(struct dept_map *m, struct dept_class *c, eh->sub_l = sub_l; irqf = cur_enirqf(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) add_iecxt(c, irq, e, false); del_ecxt(e); @@ -1745,7 +1745,7 @@ static void do_event(struct dept_map *m, struct dept_class *c, add_dep(eh->ecxt, wh->wait); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { struct dept_ecxt *e; if (before(dt->wgen_enirq[i], wg)) @@ -1787,7 +1787,7 @@ static void disconnect_class(struct dept_class *c) call_rcu(&d->rh, del_dep_rcu); } - for (i = 0; i < DEPT_IRQS_NR; i++) { + for (i = 0; i < DEPT_CXT_IRQS_NR; i++) { stale_iecxt(iecxt(c, i)); stale_iwait(iwait(c, i)); } @@ -1812,27 +1812,21 @@ static unsigned long cur_enirqf(void) return 0UL; } -static int cur_irq(void) +static int cur_cxt(void) { if (lockdep_softirq_context(current)) - return DEPT_SIRQ; + return DEPT_CXT_SIRQ; if (lockdep_hardirq_context()) - return DEPT_HIRQ; - return DEPT_IRQS_NR; + return DEPT_CXT_HIRQ; + return DEPT_CXT_PROCESS; } static unsigned int cur_ctxt_id(void) { struct dept_task *dt = dept_task(); - int irq = cur_irq(); + int cxt = cur_cxt(); - /* - * Normal process context - */ - if (irq == DEPT_IRQS_NR) - return 0U; - - return dt->irq_id[irq] | (1UL << irq); + return dt->cxt_id[cxt] | (1UL << cxt); } static void enirq_transition(int irq) @@ -1893,7 +1887,7 @@ static void dept_enirq(unsigned long ip) flags = dept_enter(); - for_each_set_bit(irq, &irqf, DEPT_IRQS_NR) { + for_each_set_bit(irq, &irqf, DEPT_CXT_IRQS_NR) { dt->enirq_ip[irq] = ip; enirq_transition(irq); } @@ -1939,6 +1933,13 @@ void noinstr dept_hardirqs_off(void) dept_task()->hardirqs_enabled = false; } +void noinstr dept_kernel_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + /* * Ensure it's the outmost softirq context. */ @@ -1946,7 +1947,7 @@ void dept_softirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_SIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_SIRQ] += 1UL << DEPT_CXTS_NR; } /* @@ -1956,7 +1957,7 @@ void noinstr dept_hardirq_enter(void) { struct dept_task *dt = dept_task(); - dt->irq_id[DEPT_HIRQ] += 1UL << DEPT_IRQS_NR; + dt->cxt_id[DEPT_CXT_HIRQ] += 1UL << DEPT_CXTS_NR; } void dept_sched_enter(void) From patchwork Wed Feb 21 09:49:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565317 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D869440BFA; Wed, 21 Feb 2024 09:49:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; cv=none; b=uqzA5H8KmuBNmyOBUrLa+GnOLoLlNsTb/OZGKMRUU5gusN0fUgyzXEtEKTRLJd2YFUW2mAMyJJQGWM+kTw7VsLRqRKjrk3QmHipTHCh0QQZ8Q/RYnoLAQokPKbdiV1/4bsYfFPddo7bjTal2iEfBFubtGzKkDTz/Jr92kDrZ2Tw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; c=relaxed/simple; bh=7Na745h6mqt6szPtcPf56L4nwkzcVctpfLScpiam1ys=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=td8pYmZqjS1UfmX9zIJJCZngNJfo0P8nYUGFxtRdndA5QGFcrT2rNMV/UTd41UwGql8rZLQ+MBisC7YXX4jcdL7Cf92cLO8sekicAOx/TWaQq4YbOiqeIcSogA27eilQBWzTca/Pqe1Xqu6yNpwKHfjzZGYcRvcOdNwMrFtirrc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-48-65d5c739e041 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 12/27] dept: Distinguish each work from another Date: Wed, 21 Feb 2024 18:49:18 +0900 Message-Id: <20240221094933.36348-13-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0yTZxjHfd/v2I6az4L6TWM0TZxRAuoA84xsi4eLvbtYYmK8GRejjk+p cpCiHEwWUaqyIgxYEAfEcFhqgTqwRQWkWDGCqIV2VDkjoE47wBLWdquwQ6vZzZNf/v/8f1cP TynbmXW8Ju2EpE1Tp6hYOS2fD6uNiu99Iu0YKsVQenEH+LwFNFQ3m1hw/NKEwNR6BoP7/hcw 5J9DsGQfoKCi3IGgdnqCgtaeSQRW41kWBl+sBJfPw0JfeSEL+fXNLDhnlzGMXyrD0GT+Ch6V 1GGwBV7RUOFmoaoiHwfPawwBQyMHhrzNMGOs5GB5eif0TT5lwDoaCT9dGWeh09pHQ0/bDIbB jmoWJk3/MvCo5wENjtIiBq69qWNh1m+gwODzcPCrrQZDiy4oOv/HPwz0FtkwnP/5OgbXyG0E XQVTGMympyzc881hsJjLKXh79T6CmeJ5Ds5dDHBQdaYYQeG5SzQM/N3LgG48Dpb+qmZ3x5N7 cx6K6CzZxOqvocnDOpG0V05wRNc1ypEa80liMW4j9Z1uTGoXfQwxN37PEvNiGUf08y5M3vT3 c+TB5SWavHBV4P3rv5Z/miSlaLIk7fbPE+XJlU4POv57eM5w2w9UHmoQ9EjGi0KseFlnxHrE v+OX3pWhmBW2iMPDASrEEcIm0VL0G6NHcp4SLnwgGhfsbKgIF/aJY3llXIhpYbP49mYxFfIo hF3ignP1e/1GsanF9s4jC8YNVXNMiJVCnPjEeYMKOUXhgkx0Xp2i3w8+FO8ah+kSpKhBKxqR UpOWlarWpMRGJ+emaXKiv01PNaPgQxm+W05oQ4uOA91I4JEqTJF8yyUpGXVWZm5qNxJ5ShWh oLODkSJJnXtK0qZ/oz2ZImV2o/U8rVqr+NifnaQUjqhPSMck6bik/b/FvGxdHtpz+BTZesdz 9JOhDc59p/+UolYZRsa7Erx7D9r6YHfMQiET7emNLzF5nr8s+Kjj7PP+wcLpLd4SOqsy0n13 bbo9cteXnx3a8GO7ce+1bH/MmpxEq94hC7NPhKdeX3Rr872HriTxgxmPH7ftsQR0G1fXZczm 229xsRHoWUPr6c4xuYrOTFbv3EZpM9X/ARgKPolMAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRjH+723s8mLWb0dkI7sMNMEraeMiA56CQwJoqg/cuhbLq/Y1JwQ zNSO2UwnZjqrqbVMLWuaWamJlmfe08wrG5VZ1qSc5NGxBf3z8OHzhc9fD4M768jljCwyWpBH SsMllIgQHfBP3Li1sVfYZE71gPTLm8A6dZGA3NISCjrvFyMoKU/AYPzlPng9PYFgrq0Dh6zM TgR574ZxKG8YQVBdeI6CnvdOYLJaKGjOTKEgsaCUgq4v8xgMXdViUGwMgNa0fAxqZ8YIyBqn QJeViNnOJwxmDEU0GFTuYC7MoWH+nQ80j/SRUH+9mYTqgQ2QfWOIgqrqZgIaKs0Y9DzNpWCk 5A8JrQ1NBHSma0i49y2fgi/TBhwMVgsN3bV6DB4k2Wrnf/wmoVFTi8H5Ww8xML15hqDm4igG xpI+CuqtExiUGTNxmL3zEoE59SsNyZdnaNAlpCJISb5KQMevRhKShvxg7mcutdOfr5+w4HxS 2Rm+elpP8C35HP8kZ5jmk2oGaF5vjOHLCj34gqpxjM/7biV5Y9Elijd+19K8+qsJ47+1t9N8 07U5gn9vysICVx4VbQ8RwmWxgtx7R5AoNKfLgk5/XhTXX3kFV6G7rBoxDMf6ch+mnNTIgaHY tVx//wxuZxfWlSvTfCTVSMTg7AVHrnCyjbIPi9jd3KBKS9uZYN252YpU3N4Rs5u5ya7Fds2x q7jiB7X/Og42fVc3QdrZmfXjerse4WlIpEcLipCLLDI2QioL9/NShIUqI2VxXsFREUZkexnD 2fn0SjTVs68OsQySLBSHPjYJzqQ0VqGMqEMcg0tcxMQZmxKHSJXxgjzquDwmXFDUoRUMIVkq 3n9YCHJmT0qjhTBBOC3I/68Y47BchTIeBWd0H5kclb1oLW3bEljhL1buGR9ep3+77Lb/2HFJ svcuh4LZBbLRIl1B/OD6JcdCtgSZW/I9HeVrNmrWZ7/K9VZZPvhoyoN/HZS47n1+o8Vt0K34 gtot8YCnV9wJz6hV25Sn7ufNa31zDqVpzwWsximXSknbzZR4devY75i3ZJWEUIRKfTxwuUL6 FwSUBtwuAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Workqueue already provides concurrency control. By that, any wait in a work doesn't prevents events in other works with the control enabled. Thus, each work would better be considered a different context. So let Dept assign a different context id to each work. Signed-off-by: Byungchul Park --- include/linux/dept.h | 2 ++ kernel/dependency/dept.c | 10 ++++++++++ kernel/workqueue.c | 3 +++ 3 files changed, 15 insertions(+) diff --git a/include/linux/dept.h b/include/linux/dept.h index 4e359f76ac3c..319a5b43df89 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -509,6 +509,7 @@ extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long extern void dept_sched_enter(void); extern void dept_sched_exit(void); extern void dept_kernel_enter(void); +extern void dept_work_enter(void); static inline void dept_ecxt_enter_nokeep(struct dept_map *m) { @@ -559,6 +560,7 @@ struct dept_task { }; #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) #define dept_kernel_enter() do { } while (0) +#define dept_work_enter() do { } while (0) #define dept_ecxt_enter_nokeep(m) do { } while (0) #define dept_key_init(k) do { (void)(k); } while (0) #define dept_key_destroy(k) do { (void)(k); } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 9aba9eb22760..a8e693fd590f 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1933,6 +1933,16 @@ void noinstr dept_hardirqs_off(void) dept_task()->hardirqs_enabled = false; } +/* + * Assign a different context id to each work. + */ +void dept_work_enter(void) +{ + struct dept_task *dt = dept_task(); + + dt->cxt_id[DEPT_CXT_PROCESS] += 1UL << DEPT_CXTS_NR; +} + void noinstr dept_kernel_enter(void) { struct dept_task *dt = dept_task(); diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 2989b57e154a..4452864b918b 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -53,6 +53,7 @@ #include #include #include +#include #include "workqueue_internal.h" @@ -2549,6 +2550,8 @@ __acquires(&pool->lock) lockdep_copy_map(&lockdep_map, &work->lockdep_map); #endif + dept_work_enter(); + /* ensure we're on the correct CPU */ WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); From patchwork Wed Feb 21 09:49:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565319 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 044BB41235; Wed, 21 Feb 2024 09:49:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; cv=none; b=Pn/jxIPfpM03D9ws4hu0uP2un9oZK3fmmZ+IbCF3XH2w1An7FvVWrSYeA6lxX5ffLA52B7sgoLvRaJf42Gh/pcIdvws4bR2J5q/du9h5Z5VlIxyksN9rwan5W2nFWUwXLS2X3/n7MAq6qiryQkubwktgftN7rNuRdMaenxHjAvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; c=relaxed/simple; bh=FoAZA+0jlnYhnPe3f+IjJ7FMJD3ZMq2wQkEX2mpeEjA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=Y6ayxPL3UwwG8lNwwqI+Mx8HL48oSNr5HE7Ljf3duC2M/OWTuZw2riVrhzAFs7i+Z1aFv9DD2fawZ4kWM0OmRI8lXEl21WE+fDztay/O1zCLU7XbMiM6KpwuSQ1DDVdS39MPhhaH8QjJbTPZuKse3Jpx5a5jPPFWARUJnlrcaNs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-58-65d5c73972c1 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 13/27] dept: Add a mechanism to refill the internal memory pools on running out Date: Wed, 21 Feb 2024 18:49:19 +0900 Message-Id: <20240221094933.36348-14-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzXSa0xTZxgHcN/3XKnWnHQuHjVB0kRJ0DkgXp4Qon7zLI4wY4gJZpfGnkil gLbIRWMCUm/lEiGBqgVTy1Iq1ImtOHAWGUSQEbDMg0NDieAFkUIN0MYKG2tx+/Lkl///yfPp YQlFG7We1WTnirpslVZJy0jZ9CrrV0k9Q2L8hVE5VJbFQ2D+Igm1tx00eH5pQuC4W4xh8tE+ +CvoQ7DQ/4QAU7UHwY0xLwF3u0cRuO1naXj6ejVIAT8NvdWlNJTU36ZhcGoRw0hNFYYmZwr0 XbZi6AhNkGCapMFsKsHh8Q5DyNbIgK1oE4zbrzGwOJYAvaPPKHC/2AJXr4/Q8MDdS0J36ziG p/draRh1LFHQ1/2YBE9lOQW3Zqw0TAVtBNgCfgb+7LBgaDaED52f+4eCnvIODOd/voNBev4b gvaLLzE4Hc9o6Ar4MLic1QR8aniEYLximoFzZSEGzMUVCErP1ZDw5O8eCgwjO2DhYy29N0no 8vkJweDKF9xBCyn8YeWFtmteRjC0v2AEi/Ok4LLHCfUPJrFwYzZACc7GS7TgnK1iBOO0hIWZ gQFGeHxlgRReSyb83YZ0WbJa1GryRN3Xu3+SZQz1mZnjnj0Fb5qDZBFqSzSiKJbntvMf/BJh ROyyP1YmRGKai+WHh0NExGu4GN5V/pYyIhlLcBdW8vYP/XSk+II7wocGp1DEJLeJdyx5yYjl 3E6+3FxCfL6/kW9q7lh2VDi/afZRESu4HfzQYMt/O6VRvN9Ff/Y6/nf7MHkZyS1oRSNSaLLz slQa7fZtGYXZmoJtR3KynCj8ULYzi4db0aznYCfiWKRcJc/4VRIVlCpPX5jViXiWUK6Rk/nh SK5WFZ4SdTk/6k5qRX0n2sCSyrXyxGC+WsEdVeWKmaJ4XNT932I2an0ROnDYUHNl5+z+51K0 1HJqf/Tc1pSl1Lq56jTjfMOxMWPu2fkTL732tFt1V5di29WOCsnnv/SpJn6GN325Z74yOUm/ b/D7uBO9qepvGjZbXumtmdEPvQ3vi6d2pXe1HazH32YWGmNT16omrIk/pMnStadjlIrkiZv3 CuoS+ze6DzFlSlKfoUqII3R61b9orYCFTAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzXSfUzMcRwH8L6/546z39LmJ4+7IaIS4mMsZqPfzOMaNg/jpt90qy7u0tNm SiGXrCLRXZzYSR2Xi1USqfVwtVK6EqskeWiVku50deTO5p/PXnu/9/n89WFwNw3pwcjkkYJC Lg2TUCJCtGtDovf62jZhpT1nFaRfXgmWsWQCNAY9Bc2PChDonyRg0F8dCG+tgwgmG1/jkJXZ jODOxy4cntR0IyjPO0dBa98MMFuGKTBlplCQeNdAQcuAHYPO6xkYFBh3QkNaLgYVtq8EZPVT oM5KxBzjGwY2XT4NuvjF0JuXTYP9ox+YuttJqMoxkVD+fjncvNVJwfNyEwE1Jb0YtD7TUNCt nyKhoaaOgOb0VBIefs+lYMCqw0FnGabhTYUWg8Ikx7ULP/+QUJtagcGFe48xML8rQ/AiuQcD o76dgirLIAZFxkwcJu5XI+i9MkTD+cs2GtQJVxCknL9OwOvftSQkdfrD5LiG2ryBrxocxvmk omi+3Kol+Ppcji/N7qL5pBfvaV5rPM0X5Xnxd5/3Y/ydUQvJG/MvUbxxNIPmVUNmjP/e1ETz dTcmCb7PnIXtmXtQtDFYCJNFCQrfgGOikLYGNX2yeVPM50IrEY9KV6kQw3DsGm483U+FXBmK 9eQ6Omy40+7sQq4o9QupQiIGZy9O4/JGGilnMZM9ztlaBpDTBLuY0091EU6L2bVcqjrx3zLH LuAKCiv+2dWRP1APkk67sf5cW8tTPA2JtMglH7nL5FHhUlmYv48yNCRWLovxOR4RbkSOn9Gd saeXoLHWwErEMkgyXRxSbBbcSGmUMja8EnEMLnEXE9GOSBwsjY0TFBFHFafDBGUlmsMQklni 7QeEY27sCWmkECoIJwXF/xZjXD3iUcCRdcmjTRGLEs7Ux7m8M6Tl90RqdowcySj2ORj46cfs jl19U0MBLxuCXVQJkwGr5e2fPcMn0N7dHfbeoMdRrd5ltUE9W/FXV21eZ0v3R89beMpjRc1S zbJ61Hh43/QllW99t22bn2JijIfmX9oyenvBToU223ot50PuLZl3mcHwS58tIZQhUj8vXKGU /gXrGl4dLwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Dept engine works in a constrained environment. For example, Dept cannot make use of dynamic allocation e.g. kmalloc(). So Dept has been using static pools to keep memory chunks Dept uses. However, Dept would barely work once any of the pools gets run out. So implemented a mechanism for the refill on the lack by any chance, using irq work and workqueue that fits on the contrained environment. Signed-off-by: Byungchul Park --- include/linux/dept.h | 19 ++++-- kernel/dependency/dept.c | 104 +++++++++++++++++++++++++++----- kernel/dependency/dept_object.h | 10 +-- kernel/dependency/dept_proc.c | 8 +-- 4 files changed, 112 insertions(+), 29 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 319a5b43df89..ca1a34be4127 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -336,9 +336,19 @@ struct dept_pool { size_t obj_sz; /* - * the number of the static array + * the remaining number of the object in spool */ - atomic_t obj_nr; + int obj_nr; + + /* + * the number of the object in spool + */ + int tot_nr; + + /* + * accumulated amount of memory used by the object in byte + */ + atomic_t acc_sz; /* * offset of ->pool_node @@ -348,9 +358,10 @@ struct dept_pool { /* * pointer to the pool */ - void *spool; + void *spool; /* static pool */ + void *rpool; /* reserved pool */ struct llist_head boot_pool; - struct llist_head __percpu *lpool; + struct llist_head __percpu *lpool; /* local pool */ }; struct dept_ecxt_held { diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index a8e693fd590f..8ca46ad98e10 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -74,6 +74,9 @@ #include #include #include +#include +#include +#include #include "dept_internal.h" static int dept_stop; @@ -122,9 +125,11 @@ static int dept_per_cpu_ready; WARN(1, "DEPT_STOP: " s); \ }) -#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO_ONCE(s...) pr_warn_once("DEPT_INFO_ONCE: " s) +#define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t dept_pool_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; /* * DEPT internal engine should be careful in using outside functions @@ -263,6 +268,7 @@ static bool valid_key(struct dept_key *k) #define OBJECT(id, nr) \ static struct dept_##id spool_##id[nr]; \ +static struct dept_##id rpool_##id[nr]; \ static DEFINE_PER_CPU(struct llist_head, lpool_##id); #include "dept_object.h" #undef OBJECT @@ -271,14 +277,70 @@ struct dept_pool dept_pool[OBJECT_NR] = { #define OBJECT(id, nr) { \ .name = #id, \ .obj_sz = sizeof(struct dept_##id), \ - .obj_nr = ATOMIC_INIT(nr), \ + .obj_nr = nr, \ + .tot_nr = nr, \ + .acc_sz = ATOMIC_INIT(sizeof(spool_##id) + sizeof(rpool_##id)), \ .node_off = offsetof(struct dept_##id, pool_node), \ .spool = spool_##id, \ + .rpool = rpool_##id, \ .lpool = &lpool_##id, }, #include "dept_object.h" #undef OBJECT }; +static void dept_wq_work_fn(struct work_struct *work) +{ + int i; + + for (i = 0; i < OBJECT_NR; i++) { + struct dept_pool *p = dept_pool + i; + int sz = p->tot_nr * p->obj_sz; + void *rpool; + bool need; + + arch_spin_lock(&dept_pool_spin); + need = !p->rpool; + arch_spin_unlock(&dept_pool_spin); + + if (!need) + continue; + + rpool = vmalloc(sz); + + if (!rpool) { + DEPT_STOP("Failed to extend internal resources.\n"); + break; + } + + arch_spin_lock(&dept_pool_spin); + if (!p->rpool) { + p->rpool = rpool; + rpool = NULL; + atomic_add(sz, &p->acc_sz); + } + arch_spin_unlock(&dept_pool_spin); + + if (rpool) + vfree(rpool); + else + DEPT_INFO("Dept object(%s) just got refilled successfully.\n", p->name); + } +} + +static DECLARE_WORK(dept_wq_work, dept_wq_work_fn); + +static void dept_irq_work_fn(struct irq_work *w) +{ + schedule_work(&dept_wq_work); +} + +static DEFINE_IRQ_WORK(dept_irq_work, dept_irq_work_fn); + +static void request_rpool_refill(void) +{ + irq_work_queue(&dept_irq_work); +} + /* * Can use llist no matter whether CONFIG_ARCH_HAVE_NMI_SAFE_CMPXCHG is * enabled or not because NMI and other contexts in the same CPU never @@ -314,19 +376,31 @@ static void *from_pool(enum object_t t) /* * Try static pool. */ - if (atomic_read(&p->obj_nr) > 0) { - int idx = atomic_dec_return(&p->obj_nr); + arch_spin_lock(&dept_pool_spin); + + if (!p->obj_nr) { + p->spool = p->rpool; + p->obj_nr = p->rpool ? p->tot_nr : 0; + p->rpool = NULL; + request_rpool_refill(); + } + + if (p->obj_nr) { + void *ret; + + p->obj_nr--; + ret = p->spool + (p->obj_nr * p->obj_sz); + arch_spin_unlock(&dept_pool_spin); - if (idx >= 0) - return p->spool + (idx * p->obj_sz); + return ret; } + arch_spin_unlock(&dept_pool_spin); - DEPT_INFO_ONCE("---------------------------------------------\n" - " Some of Dept internal resources are run out.\n" - " Dept might still work if the resources get freed.\n" - " However, the chances are Dept will suffer from\n" - " the lack from now. Needs to extend the internal\n" - " resource pools. Ask max.byungchul.park@gmail.com\n"); + DEPT_INFO("------------------------------------------\n" + " Dept object(%s) is run out.\n" + " Dept is trying to refill the object.\n" + " Nevertheless, if it fails, Dept will stop.\n", + p->name); return NULL; } @@ -2957,8 +3031,8 @@ void __init dept_init(void) pr_info("... DEPT_MAX_ECXT_HELD : %d\n", DEPT_MAX_ECXT_HELD); pr_info("... DEPT_MAX_SUBCLASSES : %d\n", DEPT_MAX_SUBCLASSES); #define OBJECT(id, nr) \ - pr_info("... memory used by %s: %zu KB\n", \ - #id, B2KB(sizeof(struct dept_##id) * nr)); + pr_info("... memory initially used by %s: %zu KB\n", \ + #id, B2KB(sizeof(spool_##id) + sizeof(rpool_##id))); #include "dept_object.h" #undef OBJECT #define HASH(id, bits) \ @@ -2966,6 +3040,6 @@ void __init dept_init(void) #id, B2KB(sizeof(struct hlist_head) * (1 << (bits)))); #include "dept_hash.h" #undef HASH - pr_info("... total memory used by objects and hashs: %zu KB\n", B2KB(mem_total)); + pr_info("... total memory initially used by objects and hashs: %zu KB\n", B2KB(mem_total)); pr_info("... per task memory footprint: %zu bytes\n", sizeof(struct dept_task)); } diff --git a/kernel/dependency/dept_object.h b/kernel/dependency/dept_object.h index 0b7eb16fe9fb..4f936adfa8ee 100644 --- a/kernel/dependency/dept_object.h +++ b/kernel/dependency/dept_object.h @@ -6,8 +6,8 @@ * nr: # of the object that should be kept in the pool. */ -OBJECT(dep, 1024 * 8) -OBJECT(class, 1024 * 8) -OBJECT(stack, 1024 * 32) -OBJECT(ecxt, 1024 * 16) -OBJECT(wait, 1024 * 32) +OBJECT(dep, 1024 * 4 * 2) +OBJECT(class, 1024 * 4) +OBJECT(stack, 1024 * 4 * 8) +OBJECT(ecxt, 1024 * 4 * 2) +OBJECT(wait, 1024 * 4 * 4) diff --git a/kernel/dependency/dept_proc.c b/kernel/dependency/dept_proc.c index 7d61dfbc5865..f07a512b203f 100644 --- a/kernel/dependency/dept_proc.c +++ b/kernel/dependency/dept_proc.c @@ -73,12 +73,10 @@ static int dept_stats_show(struct seq_file *m, void *v) { int r; - seq_puts(m, "Availability in the static pools:\n\n"); + seq_puts(m, "Accumulated amount of memory used by pools:\n\n"); #define OBJECT(id, nr) \ - r = atomic_read(&dept_pool[OBJECT_##id].obj_nr); \ - if (r < 0) \ - r = 0; \ - seq_printf(m, "%s\t%d/%d(%d%%)\n", #id, r, nr, (r * 100) / (nr)); + r = atomic_read(&dept_pool[OBJECT_##id].acc_sz); \ + seq_printf(m, "%s\t%d KB\n", #id, r / 1024); #include "dept_object.h" #undef OBJECT From patchwork Wed Feb 21 09:49:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565318 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 2D11242ABD; Wed, 21 Feb 2024 09:50:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; cv=none; b=lhNk79KVc+mlLvWJRCbKKws96xwFDOv/wzt0PhUVVexv+FDWnt8JucvvI/01DBAttPjz2snxzjkIL7ItAcQKwukhzrUg9ndLSg8mvpdVu0/gkF2p9ZR+gnhwPIEbBA5hiZKiTlHWF/XJjs/10YD0dqhiFU0Rt3/qBH2nAU/1G6g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; c=relaxed/simple; bh=32dhqEgXIybDFaJN1JuIlSUqelCcRZP5tULcFTO0inM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=gS3Av+tAytRiAVptky+gG0zHZziMajXog1pf5U+Uon4cjcwZCItswsI7WVOK55xLjdswf7HBbZI8iG6pqHxIbuV5bMXr/BZ7Ad1h0QD01gdQHGpIvlnurXSiB/LFMAxR9wy0Z8OhIVWPXnvfjOoz48D/YqD8l0Kc+I2lqsPxVQY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-68-65d5c73a67e7 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 14/27] cpu/hotplug: Use a weaker annotation in AP thread Date: Wed, 21 Feb 2024 18:49:20 +0900 Message-Id: <20240221094933.36348-15-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe2yLexjH/X7vtbXypiZ7Rw4nTRZM3IeHCP7ZOa9jE3GJGEHpG2usG93s IhGzSzmziZKt2DLdSHVbsXUuY7rUsAsZHS0lM9PIOWadyaal21xa4p8nn3y/+X7+elhCfpOa xKqT00RtsjJJQUtJaX9YxaylrS5xbk+5HPSFc8H36SgJZVcsNDgu1yCwXD2Moff+3/Dc70Uw 0vGYAEOxA0HFm1cEXG3pRmAz59Dw9O04cPoGaGgvPkZD7vkrNHT2jWLoKjmJocYaDw9PVGKw B/4nwdBLQ6khFwfPOwwBUzUDpuwo8JjPMjD6Zh60dz+jwPZyJpwp76Lhtq2dhJYGD4ant8po 6LZ8p+BhSxsJDn0RBZc+VNLQ5zcRYPINMPDEbsRQmxcU6Ya+UdBaZMegu1CHwfmiEUHT0R4M VsszGu76vBjqrcUEDF+8j8BzvJ+B/MIAA6WHjyM4ll9CwuOvrRTkdS2EkS9l9Mqlwl3vACHk 1WcINr+RFB5U8sLNs68YIa/pJSMYrfuFenO0cP52LxYqBn2UYK3+lxasgycZoaDfiYUPjx4x QtvpEVJ46zTgtZMTpMtUYpI6XdTOWb5DmlhZ6CD3vpZkekt2ZqPvTAGSsDwXw+eU55K/+aLr Mw4xzU3j3e4AEeJw7k++vug/qgBJWYI7MpY3f+ygQ8UELo7XWTw/RSQXxV96fy84YFkZt4i/ 3rTjl3MqX1Nr/+mRBOOqUi8VYjm3kHd1XiNCTp47IuEdbcPUr0Ekf8fsJk8gmRGNqUZydXK6 RqlOipmdmJWszpy9K0VjRcGHMh0c3dKABh3rmxHHIkWYLPGGU5RTyvTULE0z4llCES4jM4KR TKXMOiBqU7Zr9yeJqc1oMksqImTz/RkqObdbmSbuEcW9ovZ3i1nJpGwUualoRV/sai6gogpX 7VPOL27Vc1ULnLLxfVFxU+zbN9O2huGICP1fqshz7pySOs03z/RTpzYuTvlYu26CfHpCP3mv I3Yovk7nNN5wNMZO3DDjH1eMzdDWs2WnLDrNFR5WNUUTt6083L1mY2PCUGaOes1W/R/++LhO nWIJdShhfb6CTE1UzosmtKnKH7XfM6lMAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa1BMYRjHve855z2nZc2ZrRlnYlwWYyZKjMzDGsMwHAajL8z4oqVDS8Xs KmVcooRUKrZiq6lltlVL7EoubbNTdHGNNpIUMUjLkjZWuWyML8/85v+f/+/Tw1EKAxPIaWJ3 SdpYdbSSyGjZGlVy8LyGVil08NR8yE4PBU//URoKKiwEmi+WI7BcOYih5/ZyeDrgQjB4/yEF efpmBCWvXlBwpb4Tgd18iEDLm9Hg9LgJNOmPE0g+W0HgUe8Qho7cHAzl1tVwN8uIweF9R0Ne DwFDXjL2nfcYvKYyFkxJU6HbfIaFoVezoKnzCQN1hU0M2Nunw+miDgLV9iYa6q91Y2i5UUCg 0/Kbgbv1jTQ0Z2cwcOGTkUDvgIkCk8fNwmNHMYZLKT5b6tdfDDRkODCknruMwfnsJoKaoy8x WC1PCNR5XBhsVj0FP0pvI+jO/MjC4XQvC4aDmQiOH86l4eHPBgZSOsJg8HsBWaQS61xuSkyx 7RbtA8W0eMcoiNfPvGDFlJp2Viy2xok2c5B4troHiyV9Hka0lh0jorUvhxXTPjqx+OnBA1Zs zB+kxTfOPLx23AbZgkgpWhMvaWcujJBFGdOb6Z1dfgmu3E1J6Debhvw4gZ8jlLZ+w8NM+GlC W5uXGuYAfqJgy3jLpCEZR/FHRgrmz/fJcOHPrxJSLd1/xzQ/Vbjw4ZZvwHFyfq5wtSbin3OC UH7J8dfj54vPG1zMMCv4MKH1USWVhWTFaEQZCtDExseoNdFhIbrtUYmxmoSQzTtirMj3MqZ9 Q9nXUH/L8lrEc0g5Sh5V5ZQUjDpelxhTiwSOUgbI6d2+SB6pTtwjaXds1MZFS7paNJajlWPk K9dLEQp+q3qXtF2Sdkra/y3m/AKT0Ia6rOCE1+P3e2wk+FmoaiSZ4F5SFD5psvFmkN0649ji SjZdv3hvSeXS6hCqqrKvq29bxA2N4Uv+fHArisjzQ11blu3P7z99vXPvgXXO8GT9Hv+VVSe8 J9m34wP5SarCzNdz7h1po5bOXuFfEOPfWPojLrwo0hGa45ky2ay6PNSuUtK6KPWsIEqrU/8B w+OkGS4DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: cb92173d1f0 ("locking/lockdep, cpu/hotplug: Annotate AP thread") was introduced to make lockdep_assert_cpus_held() work in AP thread. However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire() as a potential wait and might report a deadlock by the wait. So replaced it with trylock version of annotation. Signed-off-by: Byungchul Park --- kernel/cpu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/cpu.c b/kernel/cpu.c index a86972a91991..b708989f789f 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -535,7 +535,7 @@ int lockdep_is_cpus_held(void) static void lockdep_acquire_cpus_lock(void) { - rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 0, _THIS_IP_); + rwsem_acquire(&cpu_hotplug_lock.dep_map, 0, 1, _THIS_IP_); } static void lockdep_release_cpus_lock(void) From patchwork Wed Feb 21 09:49:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565320 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6C3524596F; Wed, 21 Feb 2024 09:50:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; cv=none; b=fNStzsc41HSP6RWnq5Cb3PTFJkF1RR6yA+S3bYh5cxYV711f6XTRZHeh9Qjy5j4Veqpauj3pms8UK+jMKicD0372ktxkIkebXLdEaYlthF3Zzo+3Bv1ZtBnG4KEvSLcOkQHaIsXbS5YG4bHW/Qpg1KBbvSMkla5MxM33GoOkmK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509003; c=relaxed/simple; bh=t4Qh0s/9NGiebNos8RWbTI6PuT087Z/eTqnpXervcDg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=how7qUS24LbkjNtNTPy3YTDtVTuXnXLuwC4QeWPFk4X/X1C58ZBKOjWab0eKmYe2mFyfWL2HjvJIpWyQTQe55sLPlZXMhfXj24AiyiTjtaFAtW7sKTpfv+UhhnlTFYScVPAwjpF96xIsgNGivA8iK1C0VnoMQjsmIJxLJpMvjX0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-78-65d5c73a3e19 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 15/27] dept: Apply sdt_might_sleep_{start,end}() to dma fence wait Date: Wed, 21 Feb 2024 18:49:21 +0900 Message-Id: <20240221094933.36348-16-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSb0zMcRzHfX//O85+uzI/ekA3jbFSUftoGPPA74k/Y5nxgFM/7ri77Krr j7V15G9/RluFQnVcp6K6EuEsnUoopStJmg4l3ZXF3TpF7jRPPnvv/X5/Xp8nHwaX1JOLGYU6 XtCoZUopJSJEjnnFQZEt3UJI731/uJQZAs6f5wgorKygoONuOYKKWh0GI01b4a3LjmCq7TUO +bkdCIoHP+BQ2zyAwGw8SUHX5/lgdY5T0JqbQcEpfSUFnaPTGPTn5WBQbtoGLy+WYNDgHiYg f4SCgvxTmGd8xcBtKKPBkBYINuNVGqYHQ6F1oIcEc98quHK9n4LH5lYCmh/YMOh6WEjBQMUM CS+bnxPQcSmLhDtjJRSMugw4GJzjNLxpKMKgKt0DOvPjDwktWQ0YnLlZjYH13SMET859xMBU 0UOBxWnHoMaUi8Ov0iYEtmwHDacz3TQU6LIRZJzOI+D17xYS0vvDYWqykNoUyVvs4zifXpPI m11FBP+ihOPrr36g+fQnfTRfZErga4wref3jEYwvnnCSvKnsPMWbJnJo/oLDivFj7e00//zy FMF/tuZjO/33idbHCEqFVtCs3nhQJM9+346O68RJpht0GvoiuoB8GI5dy01+Gyb/6/fTv2iv ptjlXG+vG/dqP3YpV5M15OmIGJw9O5czfm+jvIEvu4f7VHfbU2IYgg3kbrVpvbaYjeDyJoeo WeYSrryq4R/Hx+PfLrD/uyVhw7nuznu4l8mxMwzXaamiZxcWcU+NvcRFJC5Cc8qQRKHWqmQK 5dpgebJakRQcHasyIc9DGVKn9z9AEx27GxHLIOk8sfy+VZCQMm1csqoRcQwu9RMTiR5LHCNL ThE0sQc0CUohrhH5M4R0oTjMlRgjYY/I4oVjgnBc0PxPMcZncRry2dGqjXY7YuWbizUK46t1 kkhze21beGqPjSgtKHRofv+4EbCkWRW8d9fqspwNvgFJ17hRvbluxnBApVtmEavHFl77GR1o jJrbVL1ie3yGUpqiz6g+FLVM98y46+EJuz42M2hLxN2wo/VRvpui1/iF2SpdWZfZJEvKAvn5 w+FduVIiTi4LXYlr4mR/ASk1HedMAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSeUiTcRjH+723q8XbknozKVmFZGgWGQ8ZURT00oVBGUmQo95y6Ky2tIwC l1q2qangkUd5tTzWNe2wnIrmVWias9Mkl0fiPDAnTle2Ff3z8OH7/fL562FwSTbpxsjDzwvK cFmYlBIRogP+Md5bmroEX7ONgJQEX7BOxhOQ81BPQfuDMgT6CjUGQw274cOUBcFs61scMtLa EeT3fsWhorEHgbH4KgWdfQvBZB2joCVNS0FM4UMKOobtGHSnp2JQZtgPb5ILMKi1DRKQMURB dkYM5jg/MLDpSmnQRa8Bc3EWDfbeDdDS856E+twWEoyf18Gt290UVBlbCGh8bsag80UOBT36 ORLeNDYT0J6SSML90QIKhqd0OOisYzS8q83D4FGsw3bt528SmhJrMbhW9BgD06eXCKrjv2Fg 0L+noN5qwaDckIbDzL0GBOakERriEmw0ZKuTEGjj0gl4+6uJhNhuP5idzqG2+/P1ljGcjy2/ wBun8gj+dQHHV2Z9pfnY6s80n2eI4MuLvfjCqiGMz5+wkryh9AbFGyZSaV4zYsL40bY2mm/O nCX4PlMGFuAeJNp6UgiTRwrK9duCRSFJX9rQWbX4ouEOHY36RRrkwnDsJu6LfYZ2MsV6ch8/ 2nAnu7IeXHniAKlBIgZnr8/nisdbKWexmA3kvj8tcYwYhmDXcHdbI52xmN3MpU8PUP+cK7my R7V/PS6OvCTbQjpZwvpxXR1P8GQkykPzSpGrPDxSIZOH+fmoQkOiwuUXfU6cURiQ42V0V+wp z9Fk5+46xDJIukAc8swkSEhZpCpKUYc4Bpe6iokLjkh8UhZ1SVCeOa6MCBNUdWg5Q0iXivcc EYIl7GnZeSFUEM4Kyv8txri4RSNv4pBR0HQUzezdqJ/epdDaIRg0lZsI26qcVwqFj7qsgAhc ttbd/KuofXXJIqOHIqhSEtFwMGVxaWC/C+3VtUIa0Csa1F4KrqnsTvO0bNx/eV/FimNZubTc NryEXXj8nDqh8PDRmy9X7xTXVGkb5nrYcWGHpr95LrNa/tR6SiolVCGyDV64UiX7A1jZPJ4u AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track dma fence waits. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 8aa8f8cb7071..76dba11f0dab 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -16,6 +16,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -783,6 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.task = current; list_add(&cb.base.node, &fence->cb_list); + sdt_might_sleep_start(NULL); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -796,6 +798,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); if (!list_empty(&cb.base.node)) list_del(&cb.base.node); @@ -885,6 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, } } + sdt_might_sleep_start(NULL); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); @@ -899,6 +903,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, if (ret > 0 && intr && signal_pending(current)) ret = -ERESTARTSYS; } + sdt_might_sleep_end(); __set_current_state(TASK_RUNNING); From patchwork Wed Feb 21 09:49:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565321 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AE85845C0C; Wed, 21 Feb 2024 09:50:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509004; cv=none; b=CemIBb5F1cuIk3l43vvl+KBGd+bBS2OnU/MgZSHPUOqxZ+YAoxjlqyc3W+yABWi7WBT5yOup5NLS39uUjmwaioRvGqyeB5aW4YTRg1uNT0Fk5ZVbYsqZuiEe6H/YkBToqbShpoq3ja3U52O+bf7ULgwEc0ogZb7KCVMrf7AXv4c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509004; c=relaxed/simple; bh=LIy0pGzgh1KfCx3V9I4vVrDFzUkgQh3CEYezt1V8iYA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=e2xIYjDdONc4uzKs5CGjNM3N7MM51TS3KPKY7MNWSa46BaupJ2eW7EyAfExIU03zACHgIeQe7rpAt0wgIWFMOAL84MnefMht9xqRC5d83xoP8gLLBgwzTfbvIgEPlIABeUB+sQ6ekU+dchAsCSSgjDMbVGzSiLx/i7COSOs78pY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-88-65d5c73aa802 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 16/27] dept: Track timeout waits separately with a new Kconfig Date: Wed, 21 Feb 2024 18:49:22 +0900 Message-Id: <20240221094933.36348-17-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PXZ39djI/GdptMZqklX0Wmc3wI20eNn+w4dRvOupqd+nB SClKyVSri2u5i86pKHceS62rVWLl0pWkB7U8pKuIa1Lkjvnns9fe78/n9deHwSVPSA9GrogR lApZhJQSEaIxN92awOZOwfdzlRdkX/IF+/d0Agoryimw3C1DUH4/GYORxu3wesqGYKb1JQ7q PAsC3WAfDveb+hHUGM5R0DE8H6z2CQpa8jIpSLlRQUH76CwGvfk5GJQZQ+DFlWIM6qY/EqAe oUCjTsEc4xMG0/pSGvRJXjBkuEbD7OA6aOnvIqGmxxuuFvVS8LSmhYCmx0MYdFQVUtBfPkfC i6ZnBFiys0i4M15MweiUHge9fYKGV3VaDCpTHaIL336T0JxVh8GFm/cwsL6pRlCb/g4DY3kX BQ12GwYmYx4OP281Ihi6PEbD+UvTNGiSLyPIPJ9PwMtfzSSk9gbAzI9CanMg32CbwPlUUxxf M6Ul+OfFHP/kWh/Np9b20LzWeJI3GVbzN56OYLxu0k7yxtKLFG+czKH5jDErxo+3tdH8s4IZ gh+2qrHdSw6INoYJEfJYQbl20xFReOOoiYr+dCRePcIkoQe7MpALw7H+XO7sZ/o/69W//zLF ruS6u6dxJ7uznpwp6wOZgUQMzqa5coYvrZSzWMDu5R6WfEVOJlgvTjuQ61hiGDG7nntbvfef czlXVln31+PiiG9rbKSTJWwA19n+AHc6OTbNhTNfLyH/HSzmzIZu4goSa9G8UiSRK2IjZfII f5/wBIU83ic0KtKIHA+lPzN78DGatOyrRyyDpG7i8EdWQULKYlUJkfWIY3Cpu5iIc0TiMFnC KUEZdVh5MkJQ1aMlDCFdJPabiguTsMdkMcIJQYgWlP9bjHHxSEJeGzwPZMTgX2o95mceCmXu Bfvd+TlawMJ+v2Dd9rjOoB89UfFL17BRO2+bG5uSh+ZslhyzYYVmLlEhL/LsC9qyQ1FveT94 tkLnvXZbSFlvWtuH+NMlJnuKeRWjMa36mDKwLPBotmstaXFvzW1bmPj1u6r0+Os9N7M8CePW Tem33KSEKly2bjWuVMn+AKGT6FJMAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/P83yfp+Ps2dXmWdeGm8ay8mPiM2GM8czyY/nD5sd09NCt 69idUsaUQkoIdbhwwjl1JXdFqbPUKvlV6cqPfqAZWpEfXZMSd8w/n732fr/3+uvDUQoT489p dLslvU6tVREZLVsdlhI8v75Vmll2Vg5Zx2aCeyCNhtybNgJNRQUIbCXJGHpqV8DzwT4Ew08a KTBmNyG4/LaTgpK6LgRO60ECLe/Gg8vdT6AhO4NAypWbBJp7RzB05JzCUGBfBY9O5mGoGvpA g7GHgMmYgj3nI4YhSz4LlqRA6LaeZ2Hk7Sxo6GpjoOZCAwPOV9Ph3MUOApXOBhrqyroxtNzN JdBl+83Ao7oHNDRlZTJQ+DmPQO+ghQKLu5+FZ1VmDMWpHtvh76MM1GdWYTh89RYG18sKBPfS 3mCw29oI1Lj7MDjs2RT8vF6LoPv4JxYOHRtiwZR8HEHGoRwaGn/VM5DaEQrDP3LJ4jCxpq+f ElMde0TnoJkWH+YJYvn5TlZMvfeKFc32ONFhDRKvVPZg8fI3NyPa848S0f7tFCumf3Jh8fPT p6z44OwwLb5zGfHagA2yBVGSVhMv6WcsipRF1/Y6yK6PkQnGHi4JlYanIx9O4OcIFuMo62XC TxVevBiivOzHTxIcme+ZdCTjKP7IWMH65QnxFr58hHD72lfkZZoPFMyvT3tGHCfn5wrtFRH/ nBOFguKqvx4fT3zD1Md4WcGHCq3NpdRJJDOjMfnIT6OLj1VrtKEhhpjoRJ0mIWTbzlg78ryM Zf9IVhkaaFlRjXgOqcbJo++4JAWjjjckxlYjgaNUfnJ6jyeSR6kT90r6nVv0cVrJUI2UHK2a IF+5XopU8DvUu6UYSdol6f+3mPPxT0JBZyTby7WGon3hG5ltAckV98eZfQ809k7J22xVNH+4 yoaHOEuLNJvLlwS0LaUy1kyvKRiJC99u9jtXaWLbC8/cvaR8fOdZzhzfeZtkMKojeNVjFVZm K2OUR1PSll0r1gaOnVZe/jD2xPbKiK07/PNdEycvms0tXx68buEYi2/rwGwVbYhWzwqi9Ab1 H6pm6TUuAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Waits with valid timeouts don't actually cause deadlocks. However, Dept has been reporting the cases as well because it's worth informing the circular dependency for some cases where, for example, timeout is used to avoid a deadlock but not meant to be expired. However, yes, there are also a lot of, even more, cases where timeout is used for its clear purpose and meant to be expired. Let Dept report these as an information rather than shouting DEADLOCK. Plus, introduced CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT Kconfig to make it optional so that any reports involving waits with timeouts can be turned on/off depending on the purpose. Signed-off-by: Byungchul Park --- include/linux/dept.h | 15 ++++++--- include/linux/dept_ldt.h | 6 ++-- include/linux/dept_sdt.h | 12 +++++--- kernel/dependency/dept.c | 66 ++++++++++++++++++++++++++++++++++------ lib/Kconfig.debug | 10 ++++++ 5 files changed, 89 insertions(+), 20 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index ca1a34be4127..0280e45cc2af 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -270,6 +270,11 @@ struct dept_wait { * whether this wait is for commit in scheduler */ bool sched_sleep; + + /* + * whether a timeout is set + */ + bool timeout; }; }; }; @@ -453,6 +458,7 @@ struct dept_task { bool stage_sched_map; const char *stage_w_fn; unsigned long stage_ip; + bool stage_timeout; /* * the number of missing ecxts @@ -490,6 +496,7 @@ struct dept_task { .stage_sched_map = false, \ .stage_w_fn = NULL, \ .stage_ip = 0UL, \ + .stage_timeout = false, \ .missing_ecxt = 0, \ .hardirqs_enabled = false, \ .softirqs_enabled = false, \ @@ -507,8 +514,8 @@ extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, con extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); -extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l); -extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn); +extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); +extern void dept_stage_wait(struct dept_map *m, struct dept_key *k, unsigned long ip, const char *w_fn, long timeout); extern void dept_request_event_wait_commit(void); extern void dept_clean_stage(void); extern void dept_stage_event(struct task_struct *t, unsigned long ip); @@ -558,8 +565,8 @@ struct dept_task { }; #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_copy(t, f) do { } while (0) -#define dept_wait(m, w_f, ip, w_fn, sl) do { (void)(w_fn); } while (0) -#define dept_stage_wait(m, k, ip, w_fn) do { (void)(k); (void)(w_fn); } while (0) +#define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) +#define dept_stage_wait(m, k, ip, w_fn, t) do { (void)(k); (void)(w_fn); } while (0) #define dept_request_event_wait_commit() do { } while (0) #define dept_clean_stage() do { } while (0) #define dept_stage_event(t, ip) do { } while (0) diff --git a/include/linux/dept_ldt.h b/include/linux/dept_ldt.h index 062613e89fc3..8adf298dfcb8 100644 --- a/include/linux/dept_ldt.h +++ b/include/linux/dept_ldt.h @@ -27,7 +27,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_L, i, "trylock", "unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_L, i, "lock", sl); \ + dept_wait(m, LDT_EVT_L, i, "lock", sl, false); \ dept_ecxt_enter(m, LDT_EVT_L, i, "lock", "unlock", sl);\ } \ } while (0) @@ -39,7 +39,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_R, i, "read_trylock", "read_unlock", sl);\ else { \ - dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl);\ + dept_wait(m, q ? LDT_EVT_RW : LDT_EVT_W, i, "read_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_R, i, "read_lock", "read_unlock", sl);\ } \ } while (0) @@ -51,7 +51,7 @@ else if (t) \ dept_ecxt_enter(m, LDT_EVT_W, i, "write_trylock", "write_unlock", sl);\ else { \ - dept_wait(m, LDT_EVT_RW, i, "write_lock", sl); \ + dept_wait(m, LDT_EVT_RW, i, "write_lock", sl, false);\ dept_ecxt_enter(m, LDT_EVT_W, i, "write_lock", "write_unlock", sl);\ } \ } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 12a793b90c7e..21fce525f031 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -22,11 +22,12 @@ #define sdt_map_init_key(m, k) dept_map_init(m, k, 0, #m) -#define sdt_wait(m) \ +#define sdt_wait_timeout(m, t) \ do { \ dept_request_event(m); \ - dept_wait(m, 1UL, _THIS_IP_, __func__, 0); \ + dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) +#define sdt_wait(m) sdt_wait_timeout(m, -1L) /* * sdt_might_sleep() and its family will be committed in __schedule() @@ -37,12 +38,13 @@ /* * Use the code location as the class key if an explicit map is not used. */ -#define sdt_might_sleep_start(m) \ +#define sdt_might_sleep_start_timeout(m, t) \ do { \ struct dept_map *__m = m; \ static struct dept_key __key; \ - dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__);\ + dept_stage_wait(__m, __m ? NULL : &__key, _THIS_IP_, __func__, t);\ } while (0) +#define sdt_might_sleep_start(m) sdt_might_sleep_start_timeout(m, -1L) #define sdt_might_sleep_end() dept_clean_stage() @@ -52,7 +54,9 @@ #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) #define sdt_map_init_key(m, k) do { (void)(k); } while (0) +#define sdt_wait_timeout(m, t) do { } while (0) #define sdt_wait(m) do { } while (0) +#define sdt_might_sleep_start_timeout(m, t) do { } while (0) #define sdt_might_sleep_start(m) do { } while (0) #define sdt_might_sleep_end() do { } while (0) #define sdt_ecxt_enter(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 8ca46ad98e10..1b8fa9f69d73 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -739,6 +739,8 @@ static void print_diagram(struct dept_dep *d) if (!irqf) { print_spc(spc, "[S] %s(%s:%d)\n", c_fn, fc_n, fc->sub_id); print_spc(spc, "[W] %s(%s:%d)\n", w_fn, tc_n, tc->sub_id); + if (w->timeout) + print_spc(spc, "--------------- >8 timeout ---------------\n"); print_spc(spc, "[E] %s(%s:%d)\n", e_fn, fc_n, fc->sub_id); } } @@ -792,6 +794,24 @@ static void print_dep(struct dept_dep *d) static void save_current_stack(int skip); +static bool is_timeout_wait_circle(struct dept_class *c) +{ + struct dept_class *fc = c->bfs_parent; + struct dept_class *tc = c; + + do { + struct dept_dep *d = lookup_dep(fc, tc); + + if (d->wait->timeout) + return true; + + tc = fc; + fc = fc->bfs_parent; + } while (tc != c); + + return false; +} + /* * Print all classes in a circle. */ @@ -814,10 +834,14 @@ static void print_circle(struct dept_class *c) pr_warn("summary\n"); pr_warn("---------------------------------------------------\n"); - if (fc == tc) + if (is_timeout_wait_circle(c)) { + pr_warn("NOT A DEADLOCK BUT A CIRCULAR DEPENDENCY\n"); + pr_warn("CHECK IF THE TIMEOUT IS INTENDED\n\n"); + } else if (fc == tc) { pr_warn("*** AA DEADLOCK ***\n\n"); - else + } else { pr_warn("*** DEADLOCK ***\n\n"); + } i = 0; do { @@ -1563,7 +1587,8 @@ static void add_dep(struct dept_ecxt *e, struct dept_wait *w) static atomic_t wgen = ATOMIC_INIT(1); static void add_wait(struct dept_class *c, unsigned long ip, - const char *w_fn, int sub_l, bool sched_sleep) + const char *w_fn, int sub_l, bool sched_sleep, + bool timeout) { struct dept_task *dt = dept_task(); struct dept_wait *w; @@ -1583,6 +1608,7 @@ static void add_wait(struct dept_class *c, unsigned long ip, w->wait_fn = w_fn; w->wait_stack = get_current_stack(); w->sched_sleep = sched_sleep; + w->timeout = timeout; cxt = cur_cxt(); if (cxt == DEPT_CXT_HIRQ || cxt == DEPT_CXT_SIRQ) @@ -2294,7 +2320,7 @@ static struct dept_class *check_new_class(struct dept_key *local, */ static void __dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, - bool sched_sleep, bool sched_map) + bool sched_sleep, bool sched_map, bool timeout) { int e; @@ -2317,7 +2343,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, if (!c) continue; - add_wait(c, ip, w_fn, sub_l, sched_sleep); + add_wait(c, ip, w_fn, sub_l, sched_sleep, timeout); } } @@ -2354,14 +2380,23 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, } void dept_wait(struct dept_map *m, unsigned long w_f, - unsigned long ip, const char *w_fn, int sub_l) + unsigned long ip, const char *w_fn, int sub_l, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (dt->recursive) return; @@ -2370,21 +2405,30 @@ void dept_wait(struct dept_map *m, unsigned long w_f, flags = dept_enter(); - __dept_wait(m, w_f, ip, w_fn, sub_l, false, false); + __dept_wait(m, w_f, ip, w_fn, sub_l, false, false, timeout); dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_wait); void dept_stage_wait(struct dept_map *m, struct dept_key *k, - unsigned long ip, const char *w_fn) + unsigned long ip, const char *w_fn, + long timeoutval) { struct dept_task *dt = dept_task(); unsigned long flags; + bool timeout; if (unlikely(!dept_working())) return; + timeout = timeoutval > 0 && timeoutval < MAX_SCHEDULE_TIMEOUT; + +#if !defined(CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT) + if (timeout) + return; +#endif + if (m && m->nocheck) return; @@ -2430,6 +2474,7 @@ void dept_stage_wait(struct dept_map *m, struct dept_key *k, dt->stage_w_fn = w_fn; dt->stage_ip = ip; + dt->stage_timeout = timeout; exit: dept_exit_recursive(flags); } @@ -2441,6 +2486,7 @@ static void __dept_clean_stage(struct dept_task *dt) dt->stage_sched_map = false; dt->stage_w_fn = NULL; dt->stage_ip = 0UL; + dt->stage_timeout = false; } void dept_clean_stage(void) @@ -2471,6 +2517,7 @@ void dept_request_event_wait_commit(void) unsigned long ip; const char *w_fn; bool sched_map; + bool timeout; if (unlikely(!dept_working())) return; @@ -2493,6 +2540,7 @@ void dept_request_event_wait_commit(void) w_fn = dt->stage_w_fn; ip = dt->stage_ip; sched_map = dt->stage_sched_map; + timeout = dt->stage_timeout; /* * Avoid zero wgen. @@ -2500,7 +2548,7 @@ void dept_request_event_wait_commit(void) wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(dt->stage_m.wgen, wg); - __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map); + __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: dept_exit(flags); } diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 9602f41ad8e8..0ec3addef504 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1312,6 +1312,16 @@ config DEPT noting, to mitigate the impact by the false positives, multi reporting has been supported. +config DEPT_AGGRESSIVE_TIMEOUT_WAIT + bool "Aggressively track even timeout waits" + depends on DEPT + default n + help + Timeout wait doesn't contribute to a deadlock. However, + informing a circular dependency might be helpful for cases + that timeout is used to avoid a deadlock. Say N if you'd like + to avoid verbose reports. + config LOCK_DEBUGGING_SUPPORT bool depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT From patchwork Wed Feb 21 09:49:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565323 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 8B4DD487B3; Wed, 21 Feb 2024 09:50:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509006; cv=none; b=sYCp4g9DH8wnz7qRX/AtncUdv0QGty3xACRH2G5crTxD+98zb+NXG+InUTs8Vj3vsf60/pcTUTpIui6ATIvvZ8MN3Ae/855uMbstrM9ruyYEeAAcGqWNQzQfHZk5vXSBk/UUhgg1xq758AexbNLXWEsuVHj14Rcthim5nbH07tU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509006; c=relaxed/simple; bh=TSPxdh3H9mjhA/wDLLlJUz6YZTY7hYg5ulgpaHBJJSM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=bMtV2yU5kNNrbBCGPcM4HhUU4QV0C5Wy/BFmGg0UvQADYxpN42DkH0dLS9ftWZZ1Z3fHhiimEK9yXjdwC3+ZkdDtcXrrJu9BlwFrRSh3Ylu0LaBJpIOjLWv/H6ZHU0pBuZQBvPO/vPdANl9LhccY2e/urhra1R2AECa4LfsC+E8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-98-65d5c73ae00c From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 17/27] dept: Apply timeout consideration to wait_for_completion()/complete() Date: Wed, 21 Feb 2024 18:49:23 +0900 Message-Id: <20240221094933.36348-18-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUyTZxSGfZ73k0qXN9WER/zB1oWYAcOPoDkxxo9fe2LiZrIs0WGiFd6M Iq1aFCiJCdUqghbRBDoomlJn6aAqtuoYWFMhUqoBQYuiAVRiUMKX4kpWwbmWzT8nV+77nOvX ERlVG5coavWHZINek6fmFaxiMr7+2/WBfnlVlzUNzp5eBeG/TrJQd9XNQ++VJgTu6yYMY3e/ gyezEwjmuh8wYK3qRVD/coiB653DCHyuozw8evUFhMLTPASrTvFw7OJVHvrG5zEMVp/D0OTZ BvcrHRj8kdcsWMd4sFmP4eh4gyHibBTAWZIMI65aAeZfrobg8GMOfM9SoebCIA+3fEEWOltG MDxqreNh2P2Jg/udXSz0nrVwcHnKwcP4rJMBZ3hagId+O4Zmc1R04v0/HAQsfgwnfruGIfS0 DcHtky8weNyPeegIT2DweqoY+NBwF8FIxaQAx09HBLCZKhCcOl7NwoOPAQ7Mg2th7u86fvN6 2jExzVCzt5D6Zu0svecg9M/aIYGabz8TqN1zmHpdKfTirTFM62fCHPU0lvHUM3NOoOWTIUyn enoE2vXrHEtfhax4+/KfFRuy5TxtgWxYuXGPIsf9sEY4cEFR5C3rwyXohViO4kQiZZDadyP4 M5vCoQXmpRVkYCDCxHip9CXxWka5cqQQGal0MXG97eZjxRJpDzEHHCjGrJRMWiJmLsZKaR2p mmv+X5pEmpr9C6K4aP67bWJhRyWtJf19N5iYlEifRBIMBpj/DpaRO64BthIp7WhRI1Jp9QU6 jTYvIz3HqNcWpWft13lQ9KWcR+YzW9BM74/tSBKROl6Z80dIVnGagnyjrh0RkVEvVbKF0UiZ rTEWy4b9uw2H8+T8drRcZNUJyjWzhdkq6RfNIXmfLB+QDZ9bLMYllqCUr7e2WbIspfJXzJkE R4dty/OQvSRx1yaTr3970vzknU3VuQ7baENG5U3DJX3ayhXbik2LCopufr83s0Z/OTm9tGzn dOvR4ncVUveOn0ZvVO9AuoNKZ8+HhCXxuzKtvNE/VfdD81B5Wn2uKVW90/yR0M3njSJRj5/P be35RpdUrGbzczSrUxhDvuZf/KP9M04DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTYRjHe99z3anlwQYdKioWERiZQcaDRURQvQRF0YegqBx10OW0OkvN otBcNy+RwbLSymas6VbZZmgXTTRvieZ0mYWaWVTDlWFttGaXrejLw4///8/v08NT0VeYGbw+ 7aCspOkMWlaghY3LcxcltD6X4+xeDRQVxIH/22kaSu84WOi+bUfgqM7B4G1eBy8CPgShzmcU FJu7EVx/M0hBdcsQgjrbcRZ6300Fj3+MhXZzPgu55XdYcI9OYBi4cB6D3bkBOs5ZMDQEP9BQ 7GWhpDgXh89HDEFrJQfW7PkwYrvMwcSbJdA+1MdA05V2BupeLYRLVwdYeFTXTkNL7QiG3gel LAw5fjPQ0dJGQ3dRIQO3PltYGA1YKbD6xzjoaSjDUGUK205+/cVAa2EDhpM37mLwvHyIoP70 MAano4+FJr8Pg8tppuDHzWYEI2c/cXCiIMhBSc5ZBPknLtDw7GcrA6aBeAh9L2VXLSdNvjGK mFyZpC5QRpOnFoncvzzIEVP9K46UOdOJyxZDyh95Mbk+7meIs/IMS5zj5zmS98mDyeeuLo60 XQzR5J2nGG+atU1YsUc26DNkZfHKRCHZ0XOJ239VOOQ648bZaJjPQypeEpdKOX4PjjArLpD6 +4NUhDXiXMlV+J7JQwJPiacmS7YvnWykmCYmSqZWC4owLc6XaoMmJsJqcZlkDlXhf9I5kr2q 4a9IFc4rSnx/N9FivPTcfY86h4QyNKkSafRpGak6vSE+1piSnJWmPxS7e1+qE4Wfxnp0oqgW fetd14hEHmmnqJNrPHI0o8swZqU2IomntBo1nRmO1Ht0WYdlZd8uJd0gGxvRTJ7WTlev3yon RotJuoNyiizvl5X/LeZVM7JRm+F2ihLVkfmk37Ll9Vv15oSK/MwcJfbx6t64a2+P7U2ybx9U BXoS4oeG3T6+Jl3hcaC2ft7hggOl87aotkctbG5a6b7n2r0oaXa1fN+ydkfPaF7fEfK+eVds TH3XUo+vplyIYio7yMejFyvW7EzXPEjwhgToLByvKCqwmZ19WtqYrFsSQylG3R8w6iL+MAMA AA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to wait_for_completion()/complete(). Signed-off-by: Byungchul Park --- include/linux/completion.h | 4 ++-- kernel/sched/completion.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/completion.h b/include/linux/completion.h index bd2c207481d6..3200b741de28 100644 --- a/include/linux/completion.h +++ b/include/linux/completion.h @@ -41,9 +41,9 @@ do { \ */ #define init_completion_map(x, m) init_completion(x) -static inline void complete_acquire(struct completion *x) +static inline void complete_acquire(struct completion *x, long timeout) { - sdt_might_sleep_start(&x->dmap); + sdt_might_sleep_start_timeout(&x->dmap, timeout); } static inline void complete_release(struct completion *x) diff --git a/kernel/sched/completion.c b/kernel/sched/completion.c index 3561ab533dd4..499b1fee9dc1 100644 --- a/kernel/sched/completion.c +++ b/kernel/sched/completion.c @@ -110,7 +110,7 @@ __wait_for_common(struct completion *x, { might_sleep(); - complete_acquire(x); + complete_acquire(x, timeout); raw_spin_lock_irq(&x->wait.lock); timeout = do_wait_for_common(x, action, timeout, state); From patchwork Wed Feb 21 09:49:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565324 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D2AE94C626; Wed, 21 Feb 2024 09:50:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509006; cv=none; b=py7sVyOj93Aa7g2IFgZK6L/KLSEMe1B8AjTptMNwGOFley5DlyOYKkUynK6E5lgQpPO4UXYg1Qx3KzoykcDofEWuemML2HiORelUd99DWaGQSvOKk2DmV4eHtKtyd0UQPjyMYSoVRtYPIGbCbonkd8HMsU2FsR7UYSEZXIA2myk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509006; c=relaxed/simple; bh=9o9SI0ONdb1WbiIWXrCSb5nX53qir4UGX5Gt0IQgpYM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=DCcN2Vv8/Xz83QHUgamQHnXah+K9Ma6pdcksIGV/vTEVadNj0ZLlTzey8iugKDyyYCbPSL2VYZHLDEquLyhXYHzBxZ2kDiWTgsNAE+0+Qdxe+HGvgxfseBy4LbmzcRAlx6uyVCfTHiFK4mBLlLK9ajUDpIKnsqzBk4bZ+ZjWeI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-a8-65d5c73ab21f From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 18/27] dept: Apply timeout consideration to swait Date: Wed, 21 Feb 2024 18:49:24 +0900 Message-Id: <20240221094933.36348-19-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0xTWRSG3fucs8+hWnOsTtxqjKbe4oXbjJglMeqLcUdj1Jh5wWS0ygl0 hlvK3WgsgjcQIiSlCqjc7FRgBAsyOlzSgXDTqDAUBgwwSIgOsQUFS0QQLRhfVv7861/f/7Ik TvNYWCnpI2IUQ4QuTEtUvMq1qNA7sKVL8at68BNkXvMD94crPOSVlxFov1+KoKwqCcNI0374 d9KJYPrZCw7MpnYEBa/6OahqHkBQZ71AoHN4MTjcYwTaTGkEkovKCXS8ncHQl52FodR2CJ5e L8Rgn3rDg3mEQK45GXvG/ximLCUiWIwbYMiaI8LMK39oG+gWoO7lVrh5u49AbV0bD82PhjB0 /pVHYKDsiwBPm1t5aM9MF+CP0UICbyctHFjcYyL8Y8/HUJHiAV2amBWgJd2O4VLxAwyO3hoE 9VcGMdjKugk0up0YKm0mDj793oRgKMMlwsVrUyLkJmUgSLuYzcOLzy0CpPQFwPTHPLI3kDU6 xziWUhnP6ibzefakkLLHOf0iS6l/KbJ8WyyrtG5hRbUjmBWMuwVmK7lKmG08S2SpLgdmo8+f i6z1xjTPhh1mfGRVkGpXsBKmj1MMvrtPqkLLh5NQ1ARJMDcVC0b0REhFXhKVt9NOZwX3XQ8O GtGcJvIm2tMzNe8vk9fSyvTXnrxK4uTLC6n13TOSiiRpqbyPzphWz0le3kAz02Pn4mp5B3V1 VPPfkGtoaYV9HuPl8e/lOudrNXIA7ep4yM0hqZzsRXvtNeTbwQr6t7WHv47U+WhBCdLoI+LC dfqw7T6hiRH6BJ/TkeE25Hkoy7mZ44/QePuxBiRLSLtIHfqnQ9EIurjoxPAGRCVOu0zNx3ss dbAu8YxiiDxhiA1TohvQKonXLlf/OBkfrJFDdDHKb4oSpRi+b7HktdKIIKCx+P2tg+fv7dj2 g6urev3hWZzr2jlxl1yt7t9YMLo70H9t7Js8Rwx1n+8N0f6aYxnM2LN4qa/cHHAyEpPWA0rQ l4xftg7oJxxZxUfP3PGVgl3Lxxbu/Zmamv47VR1ybMVpY3Yrsy65L58tjdLtSThbu1lQMr3X Bc2mNWy79fqmlo8O1flv4QzRuq8dqHy0TAMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRzHfX/PXc5+Tls/ysPOjJGnTXxGHuapL8P4izF047c6dWV3rgcP WykkwmU5unAqJxWdK4Rqt86VPFW6ItLSmvQkK9fUXXEx/3z22vvz3uvzz4cjZQZ6CqeMPCKq IxURckZCSbatSJy/vKpBXNTY5Q+684vA+TOZgszCAgZq7+cjKChOIKDTHgzvB3sQuN7UkKBP r0Vw68tnEoorWxCU5Z5koL59AjicfQxUp59jIDG7kIG6bjcBzVfSCMi3bIVXl7IIsA51UKDv ZMCgTyQ84xsBQ6Y8Fkzxs6AtN4MF95fFUN3SSIPtejUNZR/nwbUbzQyUllVTUFnSRkD900wG Wgp+0/Cq8gUFtbpUGu59z2Kge9BEgsnZx8I7q5EAc5LHdnpglIaqVCsBp3MeEOBoeoagPLmV AEtBIwM2Zw8BRZZ0Eobv2BG0Xehl4dT5IRYMCRcQnDt1hYKakSoakpoDwfUrk1mzAtt6+kic VBSDywaNFH6ZJeAnGZ9ZnFT+kcVGixYX5c7F2aWdBL7V76SxJe8sgy39aSxO6XUQ+Pvbtyx+ cdVF4XaHntjuv1sSdFCMUEaL6oWrQiRhhe0J6PAAE6u359Dx6CWdgrw4gV8itLbGozFm+NnC hw9D5Bj78DOEotSvno6EI/kz3kLujzdMCuK4SfwGwZ0+dQwpfpagS9WO1aX8UqG37hH1Tzld yDdb/2q8PPldQ8/fUzI+UGioe0heQhIjGpeHfJSR0SqFMiJwgSY8LC5SGbvgQJTKgjwvYzrh 1pWgn/XBFYjnkHy8NOyxQ5TRimhNnKoCCRwp95FSMZ5IelARd1RUR+1XayNETQXy4yi5r3Tz TjFExocqjojhonhYVP/fEpzXlHiUuMxuG3zNn1WZr0/0PjSnyxj3PCMmfBc/fGxvoSEkSFd8 Jz9oh7ZqpgOjhgDzqPVu077W4zW3NyVvK/XdQ9Mjq7/G5gX6l1+MjXLpbCrZWtc1bnil92Xr tPUnmi7aa9fd/J0yogkIWn/ULzh8wmUxZGNOWpT2k3my6OfuCB3dskVOacIUi+eSao3iD6uF AVEuAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to swait, assuming an input 'ret' in ___swait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/swait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/swait.h b/include/linux/swait.h index 277ac74f61c3..233acdf55e9b 100644 --- a/include/linux/swait.h +++ b/include/linux/swait.h @@ -162,7 +162,7 @@ extern void finish_swait(struct swait_queue_head *q, struct swait_queue *wait); struct swait_queue __wait; \ long __ret = ret; \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ INIT_LIST_HEAD(&__wait.task_list); \ for (;;) { \ long __int = prepare_to_swait_event(&wq, &__wait, state);\ From patchwork Wed Feb 21 09:49:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565325 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0F6C64CB35; Wed, 21 Feb 2024 09:50:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509007; cv=none; b=FO5ofJiCCEMLMgUPV5k85QBiSJIx/z7pJG8cylceD4biEWz80exXPXttt3kzAUV/uOMB6+boVnMKz7Zlx9vUTMzhnOJbYzP5GV1HLNDbBuMO03TRxUG8fQp1LTW0DzufEMEyYm6wwosAUvHb+YFp2LqVSZsvJB++aBt4icZ9nLo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509007; c=relaxed/simple; bh=RHlJ2GxVpEp+l17KtAmYPgKdeN+t8Sb6ly9lphI8Ui8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=ZhxWJ0v+XKYvXI4MI6UTz20A2fwGY476ktEpzTvONj9ToTXsbC7B51wMcZ/MYSs0Wczw80bTD1pqczlpUo+XcQctQPFJf9sN/bZk/NDVi05xKbe8VwtpheqhpSSfQNLhAHX33M7UMjer1cQxUjvF5OPzqMCAJRkso4NF019sf3o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-b8-65d5c73a0bd5 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 19/27] dept: Apply timeout consideration to waitqueue wait Date: Wed, 21 Feb 2024 18:49:25 +0900 Message-Id: <20240221094933.36348-20-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRjAfb+/105nv87bz9mwM2OMYtjjdf5gvmaM+S+Mo9909MJdSjZz dEd6ISwnJV3aOXV6uWreql01decm4aJaHZKXVCIuUl6uzD/PPvs8ez5/PTyluMcoeU1UjKSN UkeoWBkt6w00z19R3ySFfMzAcD4lBHzfEmnIKrKx0FhYgMBWdgJD14P18GKgB8HQo8cUmNIb EZhft1NQVudFUGk9ycKzznHg8fWx4EpPZiHhehELT7qHMbRduoChwL4J3Gm5GByD72kwdbGQ aUrA/vEBw6AlnwOLfhZ0WK9wMPx6Ibi8zxmobJ0HGdltLFRUumiou9OB4dm9LBa8tj8MuOuc NDSeT2Xg1qdcFroHLBRYfH0cPHXkYCg2+EOnvv5moD7VgeFUXgkGT8t9BFWJrzDYbc9ZqPX1 YCi1p1Pw88YDBB1nezkwpgxykHniLIJk4yUaHv+qZ8DQtgSGfmSxa5aT2p4+ihhK40jlQA5N HuaK5O6Vdo4Yqlo5kmM/TEqtc8n1ii5MzP0+htjzz7DE3n+BI0m9Hkw+NTRwxHl5iCadHhPe MjVUtjJMitDEStrg1btl4eYqJ3WwnDuSd7oV6ZGBTUIBvCgsFr9/cDP/+c3Lr6PMCrPF5uZB aoQnCDPE0tR3fi/jKeH0WNH6+dHo8Xhhs1j0tsnPPE8Ls0TTr3UjWi4sFZ0lD/G/5nSxoNgx 2gnw+5uZPaN9hbBEbHpSTo00ReEPL+oTTNy/gylitbWZTkPyHDQmHyk0UbGRak3E4gXh8VGa Iwv2Rkfakf+jLMeGt99B/Y3bapDAI1WgPPy2R1Iw6lhdfGQNEnlKNUFOx/mVPEwdf1TSRu/S Ho6QdDVoKk+rJssXDcSFKYR96hjpgCQdlLT/t5gPUOrRxbagc/vOjduwzKm0G6+JrGlZstvg ciw9s7Glu3pGWcs0a0Ogd8fmjkPzV5EDQ6GWq8PHja4afmf52pj3l7vbbzdcrVAmBs/J+hIy Z1XoosKUPVLVRFvmzLxJ3s/XXhiLW9JmrmjO2K/fGr2tZH1ieXZtsDKy06zrp6bHSZ3upKBq Fa0LVy+cS2l16r85r5L0TQMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUzMcRjAfb+/147j54Sfd45mmGQ6e4ZZZtZ3GbMxr39w9KObrrhLydhK l7yUFXKptCvtnIrLHfNWLUUvTEV3oeUor01vSxfpvFyZf5599nn2fP56eEqRzUzmNRFRki5C Ha5kZbRs/fKEhcuqnVJAZ48fpCUHgLvvJA3Z1iIWGm4UIii6FY+h/XEwvOzvQDD4rJ4CY3oD gtzWNxTcqnIhKLUcZ6Hxw2hwuLtZqE0/w0LCFSsLz796MLRcPIeh0LYOnqbmYSgf+EyDsZ2F LGMC9o4vGAbMBRyY4/ygzZLJgad1MdS6mhiovFzLQGnzAriU08JCSWktDVV32zA03s9mwVX0 h4GnVTU0NKSlMHC9K4+Fr/1mCszubg5elJswFBu8tRPffjNQnVKO4UT+TQyO1w8QlJ18h8FW 1MRCpbsDg92WTsHPq48RtJ3t5CAxeYCDrPizCM4kXqSh/lc1A4YWFQz+yGaDlpPKjm6KGOwx pLTfRJMneSK5l/mGI4ayZo6YbIeI3TKfXClpxyS3180QW8Eplth6z3HkdKcDk666Oo7UZAzS 5IPDiDdM3S5bESqFa6Il3aKVu2RhuWU11IHb3OH8pGYUhwzsaeTDi0Kg+P7tN2aIWWGu+OrV ADXEvsJM0Z7yyetlPCUkjRQtPc+GD8YJ60XrR6eXeZ4W/ETjrzVDWi4sFWtuPsH/mjPEwuLy 4Y6P11/L6hjuKwSV6Hx+m0pFMhMaUYB8NRHRWrUmXOWv3x8WG6E57L8nUmtD3p8xH/Ok3UV9 jcEVSOCRcpQ87I5DUjDqaH2stgKJPKX0ldMxXiUPVccekXSRO3WHwiV9BZrC08qJ8pAt0i6F sE8dJe2XpAOS7v8W8z6T49DakaqkS9bpx8gnY2Br0/cxVyf5BPeiqNWeRinulLVvdlCqfS6x sDNRck6MSwxKfbi1alNSzjut6cK8+xaVc5zHUbIDF1ouhPY8cu4OqR8/dolys7jpfODB4kTP tFWzJuyt25gZsmZb61StoivfZdhwOaDpht+UlqM//U1zrkXGZyhpfZh68XxKp1f/BSZcKxYv AwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to waitqueue wait, assuming an input 'ret' in ___wait_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait.h b/include/linux/wait.h index ebeb4678859f..e5e3fb2981f4 100644 --- a/include/linux/wait.h +++ b/include/linux/wait.h @@ -304,7 +304,7 @@ extern void init_wait_entry(struct wait_queue_entry *wq_entry, int flags); struct wait_queue_entry __wq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_entry(&__wq_entry, exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ long __int = prepare_to_wait_event(&wq_head, &__wq_entry, state);\ From patchwork Wed Feb 21 09:49:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565327 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4C4AF4CE09; Wed, 21 Feb 2024 09:50:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509008; cv=none; b=P9hvdRxKaDXOb6jqxE8QYNuQqDb6OooL0heXoJ+eCdfnk8jospJMatp0cFTE/UqD3J83+9W+sJ9s7efU4hzVi0gu2LcxOAeYe07vM9GVMDmTVBJJcsGLby+3v3i4HHItpAS2/vQycI8c5D8inhBHhbkpOggESCEGkcleyfHzApU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509008; c=relaxed/simple; bh=oVu1M/z+Be6zJoonqEicwRg8jn1j0IbzUk3u5AE/er0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=NqDX74gbpAmTGBHmBfU3tKrYJ9h8DmGNjzsvyw1F/gEM1MKUsAk07YlOgazSiGnrzVY2QbNBwA7oyo14eKdO5gRJrbVAMpIkqAMyS0edYML1uDKxjkvJC7FLNjtOUKhI5slk2qgZ47P/IR2SEwpPcJovo22eQkj78zasNHJq02A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-c8-65d5c73a9c46 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 20/27] dept: Apply timeout consideration to hashed-waitqueue wait Date: Wed, 21 Feb 2024 18:49:26 +0900 Message-Id: <20240221094933.36348-21-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUxTZxTHfZ5773NvqzV3ncue4cxMg4644WRRd6JuMS7Lni9GEqMfxpLZ wVXqSjFFQFymKFUYClEUitKQFl3t2k6wSNANkJfx6kTEKmiQCDNsjUUULVkH6lqmX07++f3P +X06Eqe9LMRIBtNuxWzSG3VEzavH5zni13beUlZ090tw/OgKCD0r4MFW7SXQd96DwHvxAIZA +5cwMBVEMH3tOgfW0j4EjpF7HFzsGEbQ6DpI4OaD+eAPTRDoLj1CIO9MNYEbD2cwDJWVYPD4 NsLVY1UYmsN/8WANEKiw5uHI+BtD2OkWwZm7BEZdp0WYGUmA7uHbAjTe/QBOVQ4RaGjs5qHj 0iiGm7/aCAx7XwpwtaOLh77jRQL88qiKwMMpJwfO0IQI/c12DDWWiOjw0xcCdBY1Yzh89gIG /53fEDQV3Mfg894m0BYKYqj1lXLw77l2BKPF4yIcOhoWoeJAMYIjh8p4uP68UwDL0CqY/sdG 1q9hbcEJjllqs1njlJ1nPVWUXT59T2SWprsis/syWa1rGTvTEMDMMRkSmM/9I2G+yRKRFY77 MXvU2yuyrvJpnj3wW3Hiwq/U61IUoyFLMX/02TZ16h951WhXsbjn8VARzkUjQiFSSVReSW1/ tvCFSJrNlv7YKCby+3RwMMxF8wJ5Ma0tGousqyVOzp9LXY+vkWjxpryF2p94uOgtLy+hAT+N Yo28mt5xnnulf496appnPaoI/7kiOMu18ip660YdF3VS+YSK5hU85f8/eIe2uAb5Y0hjR3Pc SGswZaXpDcaVy1NzTIY9y5PT03wo8lDOH2aSLqHJvs2tSJaQbp4mtd6vaAV9VkZOWiuiEqdb oOGzI0iTos/Zq5jTvzFnGpWMVrRQ4nVvaz6eyk7Ryjv0u5XvFGWXYn7dYkkVk4u+/9RT5o3z frh0zLjP1v5FZn5v08jOxOfJ+zYqxoG4E8FPdOOZ8a6lWkdxUqwqvMG1iVS6P4/9OtAz92TM hSsnBxLsJVsX1ySbNG11yeUF+w0vW+53teoS3lqbT2Lc8Yv66xvgjbp3v/09nY119DieaSup daytsry+yjSx/SeyvkHHZ6TqE5Zx5gz9f66TceRMAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0wTaRiF/b6Z+Waodp1gXUc00VSNiQaFCOR1JcToj36aqJvNGrOaKHWZ SGO5bIsVNGbRFkUUBBNEpbspoJUFFHdws3jBVIhgQZFLkYtIBA1KLOKFoly8tCb+OTl5znvO r1dgggu5EMGQmCKbEvVGLVGxqs1rrKE/NXTIYac+hELeyTDwjWayYK+sINBypRxBxbXDGIbu 6qBzzItg8sFDBgryWxAU9T9h4Fp9H4Ka0iME2p//AB7fCAF3/gkC1pJKAq2vpjD0njmNoVzZ BE25xRhc4y9YKBgiUFhgxX55iWHcWcaDM30JDJSe52GqPxzcfY84qPvLzUFNz3I493cvgVs1 bhbqqwcwtN+wE+ir+MJBU/09Flrysjm4/LqYwKsxJwNO3wgPbS4Hhqs2/9rR9585aMh2YTh6 4V8Mnu6bCG5nPsWgVDwiUOfzYqhS8hmYuHQXwUDOMA8ZJ8d5KDycg+BExhkWHn5q4MDWGwmT H+1k7Rpa5x1hqK1qP60Zc7C0sVii188/4antdg9PHco+WlW6jJbcGsK06J2Po0rZcUKVd6d5 mjXswfR1czNP752dZOlzTwH+ef52VXScbDRYZNPKmFhV/H1rJUrO4VPf9GbjdNTPZSFBkMQI yda2OAsFCURcKnV1jTMBrxEXSlXZg/4TlcCIx6ZLpW8ekEAwS9wqOd6WM4EuKy6RhjxSAKvF KKnbeYkLeElcIJVfdX3bCfLzfwq933iwGCl1tP7H5CKVA00rQxpDoiVBbzBGrjDvjU9LNKSu +D0pQUH+l3EemsqrRqPtulokCkg7Qx3/v0cO5vQWc1pCLZIERqtRs/v9SB2nTzsgm5J2mfYZ ZXMtmiew2jnqjdvk2GBxjz5F3ivLybLpe4qFoJB0FKMjus2NMy2zm35Li149Gffj7MGJInLQ qGjDOmy6PyLnouHdMZpjW1q9M6Ontf6ZFCpHbGjbsm5byGInRiW+Z3ZjR9uz5L5fXyZVj/ZE 5NzZOWdR86rBI9aJYkvu9Lcl13d1I1j/y8WpszvSXc1RnS/yNbmpmSmPYzPoF3e/Yo/SsuZ4 ffgyxmTWfwW0t9IeLgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to hashed-waitqueue wait, assuming an input 'ret' in ___wait_var_event() macro is used as a timeout value. Signed-off-by: Byungchul Park --- include/linux/wait_bit.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index fe89282c3e96..3ef450d9a7c5 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -247,7 +247,7 @@ extern wait_queue_head_t *__var_waitqueue(void *p); struct wait_bit_queue_entry __wbq_entry; \ long __ret = ret; /* explicit shadow */ \ \ - sdt_might_sleep_start(NULL); \ + sdt_might_sleep_start_timeout(NULL, __ret); \ init_wait_var_entry(&__wbq_entry, var, \ exclusive ? WQ_FLAG_EXCLUSIVE : 0); \ for (;;) { \ From patchwork Wed Feb 21 09:49:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565326 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 579FF3D566; Wed, 21 Feb 2024 09:50:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509008; cv=none; b=qITw3b96TM4/l2swGUDyzOSZn59J+Tb+KfFaedhdsgH4n1koaGAJ+JLrnFac9H62Jr90fRnznVlIBumllf5Hi4FiCbPO48hyT+BgvByNQXYVHoZsvj3K69T5s3LqUCTNWXeOIvWDcjczNp1n7+yq3iHGkWapt7ae3EmV9WqDtFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509008; c=relaxed/simple; bh=A0HNCh9MAJFiyY64P3s+ZnF2ZrT06e1PDF08t8YO27Y=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sGGqCmNpK2/2lHN0ziXL4SfQj3CBv4tngUY3qgcsozdQ/8BstdsHPX2nkVRyMqBeBT9cbuTrf7l46DK5RYsJH+pF37dVf+iUgwgCUKHxGUsaLMMijw/fBzMjoNIWz33x2RMxLkmtYGcIiKCukgu8Z15GjGlvLFL9tT3a2IRXuvA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-d8-65d5c73a08db From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 21/27] dept: Apply timeout consideration to dma fence wait Date: Wed, 21 Feb 2024 18:49:27 +0900 Message-Id: <20240221094933.36348-22-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe2xLcRTH/X73uVJuSuIaQRqCMa8Mx1uQuP94ReK1eNzYtTXalc42E2Sz hxmTTexhXaQtqVrLppt4bmroNvZi3RTbwgiWbSajY1uZduGfk08+J+d7zh+HJRT3qUBWFXlY 0kWKaiUtI2VdI0zBSysapbn9V0dB5tm54PmRSkJ+oY2G+htWBLaSBAztT9fBq95OBAM1dQTk ZNUjML5vIaDE2Yqg1HKShoaPI8Hl6aahKusMDYmXC2l40eHF0Jx9HoPVvh6eZ5gwOPo+k5DT ToM+JxH7yhcMfeYCBszxU6HNkseA9/08qGptoqD0zUy4eKmZhgelVSQ477RhaLiXT0OrbZCC 585KEuoz0ym4/tVEQ0evmQCzp5uBlw4DhqIkX1DK9z8UVKQ7MKRcuYnB9fo+grLUdxjstiYa Hns6MRTbswjov/oUQdu5LgaSz/YxoE84h+BMcjYJdb8rKEhqXgADv/LpVUuEx53dhJBUHCuU 9hpI4ZmJF+7mtTBCUtkbRjDYo4ViS5Bw+UE7Fow9HkqwF5ymBXvPeUZI63Jh4WttLSNU5g6Q wkdXDt40fqdsWZikVsVIujkr9soifn5IpA82sEcqi9xMPLIyaSiA5bkQ3mY04//s7mgZ8jQ3 jXe7+wg/j+Em88Xpn6g0JGMJ7tRw3vKthvY3RnMbeH21l/QzyU3lk9+1DQ3IuYW8oSnz34JJ vLXIMeQDfP6avpPys4JbwDe+uEX4Q3lukOXL2t3/rhjHP7K4yQwkN6BhBUihiozRiCp1yOyI uEjVkdn7tBo78r2U+bg39A7qqd9SjjgWKUfII267JAUlxkTFacoRzxLKMXIy1qfkYWLcUUmn 3aOLVktR5Wg8SyrHyuf3xoYpuHDxsHRAkg5Kuv9dzAYExqM52x4GJjQXhU8+6rhkD34d9L32 SoKod+5XV4+7+dZ6yzkl2hw866Up+gQzaXvAZjHXtuaRdm1wZf7ivJDcC0bvxmN1aEaKrCdj uXa1surX7q0Lj2mnVe/KfhLf6O03rtwcXjORDZ2gI5taZSX3cvcUDKbCKM2hRdp9eTsecrc1 5ulKMipCnBdE6KLEvzbeaqxOAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSW0iTcRjG+39nV6uPNezLomwgkVlqaLxUhFf1EVndRFAXOfKrDeeUTS0j wcNWHkMtW6XGMlmm5mErsoMhSuryMJfTNFQ8YZqnMmctzZpSNy8/fg/Pc/UyuKSQ9GKU6hhB o5arZJSIEJ04mLLnQFOXEDBoD4aczABwzqcSUFBZTkFHRRmC8mdJGEy8OwofF6YQLLbZcDDk dSB4ONSPw7PGAQS1JckUdI6uB4dzlgJrXgYFKY8qKbBPLmHQdycXgzJzKLRkF2FQ5/pMgGGC gnxDCuY+4xi4TKU0mBJ9YLjkPg1LQ4FgHegmoaHQSkLtp91w70EfBW9qrQQ01gxj0PmqgIKB 8j8ktDQ2E9CRk0XC05kiCiYXTDiYnLM0fKgzYlClc69d/75MQlNWHQbXi6sxcPS+RvA2dRAD c3k3BQ3OKQws5jwcfj1+h2D45jQN+kwXDflJNxFk6O8QYPvdRIKuLxgWfxZQIQf5hqlZnNdZ LvO1C0aCf1/E8S/v99O87u0nmjeaY3lLiS//6M0Exj+cc5K8uTSN4s1zuTSfPu3A+Jn2dppv vrtI8KMOA3Zq61nRoXBBpYwTNP6Hw0SKHyMpVHQnc6W5qodORGV0OvJgODaI65nsX2WK3cn1 9LjwFZay3pwla4xMRyIGZ2+s5Uq+tlErwUb2BJffukSsMMH6cPrB4dWCmN3PGbtz/o1u58qq 6la9h9s/yZ8iV1jCBnNd9ud4NhIZ0ZpSJFWq4yLlSlXwXm2EIl6tvLL3QlSkGbmfxpSwlFOD 5juP1iOWQbJ1YsULhyAh5XHa+Mh6xDG4TComLruVOFwef1XQRJ3XxKoEbT3awhCyTeJjZ4Qw CXtJHiNECEK0oPmfYoyHVyJ6vi/m3GDx8bYZBbuN+6JmPJPTOnTjZqN31lzrvcrkMUPsAO6Z cPHp8kjQi8wvp3/odjj7v4aPVze379oxLren7VfO6Cqkr/y56pOhla50i2g27JrfEIyExvZu +HZp2sfWEnJr3u9qxZE/NpvU09pgTzq/fmSXU68q2JxL3i5tlBFahTzQF9do5X8Bi6QiIjAD AAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Now that CONFIG_DEPT_AGGRESSIVE_TIMEOUT_WAIT was introduced, apply the consideration to dma fence wait. Signed-off-by: Byungchul Park --- drivers/dma-buf/dma-fence.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c index 76dba11f0dab..95121cbcc6b5 100644 --- a/drivers/dma-buf/dma-fence.c +++ b/drivers/dma-buf/dma-fence.c @@ -784,7 +784,7 @@ dma_fence_default_wait(struct dma_fence *fence, bool intr, signed long timeout) cb.task = current; list_add(&cb.base.node, &fence->cb_list); - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (!test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &fence->flags) && ret > 0) { if (intr) __set_current_state(TASK_INTERRUPTIBLE); @@ -888,7 +888,7 @@ dma_fence_wait_any_timeout(struct dma_fence **fences, uint32_t count, } } - sdt_might_sleep_start(NULL); + sdt_might_sleep_start_timeout(NULL, timeout); while (ret > 0) { if (intr) set_current_state(TASK_INTERRUPTIBLE); From patchwork Wed Feb 21 09:49:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565328 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5AC1B4E1C8; Wed, 21 Feb 2024 09:50:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509009; cv=none; b=kSrcsScbozJdeAOkh9F2Je2R8qNxJENbfwOGpEtx1HDvNHFBz+n/k4KwpA5gomBRDzACqHwYLllNVw8S6IqU8yD8NL73ESauQvL1ct4f5Q3rqUppftmEbo3J+LSdzSDfblwGkGk3q775UI+qC3A+ClpFbbBa+OuADxwgSUshjSM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509009; c=relaxed/simple; bh=rOOuSh+1eZIFPiRqy8Z4jjiN4PmFYfqaD61RmkKO4sk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=o9ZHXWOAeNjxdO3+4wbKsxhVhV4Pwmga0Od9JiM/0c4pY1mS5s9Fm0LXcHQu14gu4rOtp2UlFMe9GmJTjPv3dx7KLvk/Ku60A6j66BuWlELymDG1D+FSyIs8xfgBtn2Simul2+NLoawcirzp79XRSAM+z3/b083YZbxpFeqDi34= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-e8-65d5c73b25db From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 22/27] dept: Record the latest one out of consecutive waits of the same class Date: Wed, 21 Feb 2024 18:49:28 +0900 Message-Id: <20240221094933.36348-23-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ732mqXNx3J3kGM0oTMYABxOE8sWfyg8zGL25TEJTiDVd6N KhfTKghqgliVi3RChDraKJfZVVoRWx1MqUOIIDilzMqYKUQbNiUWWcA2IqhrvXw5+eX/P+f3 6fCU8goTzWtz90i6XE22ipXT8okFDQnq3nvSMpc9GqqOL4Pgs1IaLBccLHha7Agclw5hGL+x Dv4KBRDM3h6gwFTjQdDwcISCSz2jCNy2Ehbujn0A3uAkC301FSwcbrrAwuCTOQy+2moMducG uHWiEUPnzCMaTOMsmE2HcXg8xjBjbebAWhwHflsdB3MPk6FvdIgB9/2l8NNpHwsd7j4aetr9 GO5esbAw6njNwK2emzR4qioZOP+0kYUnISsF1uAkB3921mNoNYRFR6dfMdBb2Ynh6M8XMXj/ vorgWukDDE7HEAvdwQAGl7OGghe/3EDgN05wcOT4DAfmQ0YEFUdqaRh42cuAwbcCZp9b2NWr SHdgkiIGVwFxh+pp0t8okt/qRjhiuHafI/XOvcRliydNHeOYNEwFGeJsLmOJc6qaI+UTXkye 3rnDkZunZmky5jXhb2LS5amZUrY2X9Ilfb5NnjVweZrZPT5/n9E/hopRr6wcyXhRSBGbjSH0 ns84f3/DrPCJODw8Q0U4Slgsuir/ZcqRnKeEY/NF23+32XLE8x8KGjFwtiiyQwtxomNkCEdY IXwmNpjb3jkXifbWzjceWTg/Zw4wEVYKK8R7g5epiFMUKmTi4NBr6u3Bx+J12zB9Ainq0bxm pNTm5udotNkpiVmFudp9iTvycpwo/FHWg3Nb2tGUJ60LCTxSLVBktXklJaPJ1xfmdCGRp1RR CrogHCkyNYVFki4vQ7c3W9J3oRieVn2kWB4qyFQKP2j2SLskabeke99iXhZdjJKM/i3FS4s8 nl0llrblhk1qxj791RLS2uSLMp/s//67nau3xaZ07f+n0FHn7mh/1r9upTv560BszGbLvG4u dYf3QNyac/qEL36NLT3V/e3CdF9VqvrLs56MT0c2LiQma17Z/nRL2VzS1rUrYzZO/RhIy9y+ CF61qNe7bOjqH66SoFpF67M0yfGUTq/5H5Mq8HFNAwAA X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUiTaxjGe57309nixSP4UkE1jgRGHYM83BwrooieoqKCqA6BveR73Giu 2MxlVGgus5mmgdlRKz9qZ+n6moZW00TRZZ3mzGUZKrqkWk3ta9LSPraify5+XPfN76+Lp6LK mZm8Rpcm63WSVsUqaMXGxOyFic7HcnzgG4aik/EQ+JhLQ/k1Gwvuq7UIbPVZGHzta+DJhB/B 5MMuCkqK3QgqhwcoqO8YRNBkPcpCz8gM8ATGWegszmMhu/oaC91vpjD0nzmNoda+AR4UVmFo Cb6kocTHQllJNg7FKwxBSw0HlsxY8FpLOZgaXgydg70MtJ3rZKDp2QL493w/C46mTho6Gr0Y em6XszBo+8bAg457NLiL8hm4MlbFwpsJCwWWwDgHj1oqMFw3hWw5H74y4MxvwZBz8QYGT98d BM25Qxjstl4W2gJ+DHX2Ygo+/9eOwFswysGxk0EOyrIKEOQdO0ND1xcnA6b+BJj8VM6uSCRt /nGKmOqMpGmigib3q0Ryq3SAI6bmZxypsO8nddY4Uu3wYVL5PsAQe80Jltjfn+aIedSDyZjL xZF7ZydpMuIpwZtm/61YmixrNemy/o/luxTqrpsfmH2+yAMF3hGUiZwRZhTBi8IS8YL9Lgoz K8wXnz4NUmGOFuaKdfkvGDNS8JRwPFK0vn3ImhHP/yZIov/SwfAPLcSKtoFeHGal8KdYWdaA fjrniLXXW354IkL95TI/E+YoIUF83H2TKkSKCjStBkVrdOmpkkabsMiwR52h0xxYtHtvqh2F NmM5PFXUiD72rGlFAo9U05XqBo8cxUjphozUViTylCpaSRtDlTJZyjgo6/cm6fdrZUMrmsXT qhjlum3yrighRUqT98jyPln/64r5iJmZyLG6oLA3dzxytI9O/itGfacnbv3LtdtnbZE7SoOH ko5rQXruu59jzLtr7F4V+zq+ynTKqTtrnFc/1Cgve/S61JlypH1z1j8ujU0zt775xjv3WNvv J9KWNVzY+r+LJLmPrI9ZUv1kk2soh10p7bD2OS66dn7a4RhOnN0sm7suextTVLRBLS2Oo/QG 6TtCaSsnLwMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The current code records all the waits for later use to track relation between waits and events in each context. However, since the same class is handled the same way, it'd be okay to record only one on behalf of the others if they all have the same class. Even though it's the ideal to search the whole history buffer for that, since it'd cost too high, alternatively, let's keep the latest one at least when the same class'ed waits consecutively appear. Signed-off-by: Byungchul Park --- kernel/dependency/dept.c | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 1b8fa9f69d73..5c996f11abd5 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -1521,9 +1521,28 @@ static struct dept_wait_hist *new_hist(void) return wh; } +static struct dept_wait_hist *last_hist(void) +{ + int pos_n = hist_pos_next(); + struct dept_wait_hist *wh_n = hist(pos_n); + + /* + * This is the first try. + */ + if (!pos_n && !wh_n->wait) + return NULL; + + return hist(pos_n + DEPT_MAX_WAIT_HIST - 1); +} + static void add_hist(struct dept_wait *w, unsigned int wg, unsigned int ctxt_id) { - struct dept_wait_hist *wh = new_hist(); + struct dept_wait_hist *wh; + + wh = last_hist(); + + if (!wh || wh->wait->class != w->class || wh->ctxt_id != ctxt_id) + wh = new_hist(); if (likely(wh->wait)) put_wait(wh->wait); From patchwork Wed Feb 21 09:49:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565329 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DC4E44EB3F; Wed, 21 Feb 2024 09:50:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509010; cv=none; b=C6fqyG8BJiSkm/dWRyRLW9roK25sJCWj3/y6mYYWvKtxC0Z/5vBOK+rZQBqjw75HuToKEk22m6yxtjGuwgG6HbgGZHRT27aDADzQNbIk0jWUJJhv/ENX6FFPaSyx1XwURrBygBJqCdiQBOdVJzeZ7Lvo05HXDJ9Muu/2XYDZLSc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509010; c=relaxed/simple; bh=Zg3dPmxAeZfdPCpe0eFW6kAfwZet+/NUlIS50hmLxGs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=DdcL0raQ+I3VNbTllySCcG4KzjAHXjNUNVZRu0WAFqHIDvlKsBa6Z2qy46Mysh7EYpj1BvDXfSn5jv9Wl3mLfGY/cOVu3Ze/mjPAV36jqOZuBqDBlk/Lt+RCNu1VvOiDZuXNXIqlpnBDJtaa7HqQmLqpTmkMqK865vz55ZaRftU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-f8-65d5c73b58e0 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 23/27] dept: Make Dept able to work with an external wgen Date: Wed, 21 Feb 2024 18:49:29 +0900 Message-Id: <20240221094933.36348-24-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUiTaxjHvZ93l6uHFXSfCspFHSgtjYyL3qHC+4sRRcU5B6qVT22k5pnv QaC5ymbKKtRKyWm2lq5cm5GWK1PSlplm0zxhlp7ezJlhbWSul83oy8WP6/r/f58ugVbUsTME TUKypE1QxSk5GSMbDikLX9HSJUW0O0Ph5IkI8HzOYaCk2sJBx9UqBJaaLAoG70XDU68bwXhb Ow1FBR0Iyvqf01DT3IfAYT7MwZNXk8HlGeHAWZDLQfaFag4eD/ko6C08RUGVLQZaDeUUNIy9 ZaBokIPiomzKP95RMGaq5MGUOQ8GzOd48PVHgrOvmwXHs4Vw9nwvB/UOJwPNtQMUPLlZwkGf 5QcLrc33Geg4mcfClQ/lHAx5TTSYPCM8dDYYKbDq/KKjn76z0JLXQMHRimsUuP67heB2zksK bJZuDpo8bgrstgIavl66h2Agf5iHIyfGeCjOykeQe6SQgfZvLSzoeqNg/EsJt3Y5aXKP0ERn TyMOr5EhD8oxqTv3nCe62894YrSlELt5AblQP0iRslEPS2yVxzliGz3FE/2wiyIfHj3iyf0z 4wx55SqiNs38W7YyVorTpEraxat3ydQ177roRP2a9BrDZToTuZfokSBgcSnObdquR8ETaO1/ QQeYE//EPT1jEzxNnIPteW9YPZIJtHhsEjZ/bOMC3aliDL7j2xrIMOI8XNuqYwMsF5fhToeT ++WcjausDROeYP/+crF7IqMQo3DX4+t0wInF7GA8aO1lfhX+wHfNPYwByY0oqBIpNAmp8SpN 3NJF6owETfqiPQfibcj/UKZDvn9q0WjHlkYkCkgZIlffcEkKVpWalBHfiLBAK6fJmTT/Sh6r yjgoaQ/s1KbESUmNaKbAKKfLl3jTYhXiPlWytF+SEiXt7yslBM/IRDnC3L3WVf/X/wthrF2L S3emXAqfv3kyHxKtrugDb/JGRZtx2a0pkV9UJr3za4ki76FRsWn2ML5rmBSbs3zL7qD1WU99 qa/r5xqMn2J2hG4I73mR/iO6qv/0SxIRtC2/Oz10wHIxNKqzujus9P3DqxVD1xL/qsPyWTLj uoqwxYX5SiZJrYpcQGuTVD8BAivVvEwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRiGe9/v6Gr1saQ+LDAGHehokfWA0eFXb0UHiggq0JUfbXnIttKM AstZNrW0UEstbNY0XWnfDK2cLcXDksx0WtkylU7SzE6TTDtMoz8PF/d9c/16eEqVxwTwuuhD kj5aE6lmFbRiU0jigpCGdimoOmM8ZKQGgfd7Mg15pVYWWm6VILCWn8DQV7cWng16EAw/fkJB dmYLgqs9rygor+9CYC86yULbm4ng8g6w4MxMYSGxoJSFpx9HMLizzmMokTdCU7oZg2PoPQ3Z fSzkZidi3/mAYchSzIElYSb0FuVwMNKzGJxdHQzUXnYyYO+cB5euuFmosjtpqK/sxdB2L4+F LusfBprqG2loyUhj4OYnMwsfBy0UWLwDHLQ68jGUGX22U99+M9CQ5sBw6tptDK4X9xFUJ3dj kK0dLNR6PRhsciYFPwvrEPSe7ecgKXWIg9wTZxGkJGXR8ORXAwNGdzAM/8hjV4eQWs8ARYy2 OGIfzKfJI7NI7ua84oixupMj+fJhYiuaSwqq+jC5+tXLELn4DEvkr+c5Yup3YfKpuZkjjReH afLGlY23TN+pWBEuRepiJf2ilWEKbfmHdirGtOpIefoNKgF5lpiQHy8KS8WyntfUKLPCbPH5 86Ex9hdmiLa0d4wJKXhKOD1eLPr8mDUhnp8sbBQfjGwf3dDCTLGyyciMslJYJrbanew/Z6BY UuYY8/j58hu5nrGNSggW25/eodKRIh+NK0b+uujYKI0uMnihIUIbH607snDvgSgZ+X7Gcnwk oxJ9b1tbgwQeqScotRUuScVoYg3xUTVI5Cm1v5KO80XKcE38UUl/IFR/OFIy1KBpPK2eqly/ QwpTCfs0h6QISYqR9P9bzPsFJKCC+bjWYEld1TrjS9LWKmtWDB8Qvr9h/rYLok1rthWuuF5j fVbROC+BCw1NnDPLPWhL7lC57X86u5sm9QtTutdvWPcyyxwfWBoTVyg99AY6oHlNjpx07Jx6 e1jZns0VdZ/lgrSI5fdbi5cEBO1etFLY8Dal0x6xg7ywHb98cJdlk5o2aDWL51J6g+Yv+4ut jC8DAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: There is a case where total maps for its wait/event is so large in size. For instance, struct page for PG_locked and PG_writeback is the case. The additional memory size for the maps would be 'the # of pages * sizeof(struct dept_map)' if each struct page keeps its map all the way, which might be too big to accept. It'd be better to keep the minimum data in the case, which is timestamp called 'wgen' that Dept makes use of. So made Dept able to work with an external wgen when needed. Signed-off-by: Byungchul Park --- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 4 ++-- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 37 insertions(+), 15 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 0280e45cc2af..dea53ad5b356 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -482,6 +482,13 @@ struct dept_task { bool in_sched; }; +/* + * for subsystems that requires compact use of memory e.g. struct page + */ +struct dept_ext_wgen{ + unsigned int wgen; +}; + #define DEPT_TASK_INITIALIZER(t) \ { \ .wait_hist = { { .wait = NULL, } }, \ @@ -512,6 +519,7 @@ extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); @@ -521,8 +529,8 @@ extern void dept_clean_stage(void); extern void dept_stage_event(struct task_struct *t, unsigned long ip); extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); -extern void dept_request_event(struct dept_map *m); -extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, struct dept_ext_wgen *ewg); extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); @@ -551,6 +559,7 @@ extern void dept_hardirqs_off(void); struct dept_key { }; struct dept_map { }; struct dept_task { }; +struct dept_ext_wgen { }; #define DEPT_MAP_INITIALIZER(n, k) { } #define DEPT_TASK_INITIALIZER(t) { } @@ -563,6 +572,7 @@ struct dept_task { }; #define dept_free_range(s, sz) do { } while (0) #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_ext_wgen_init(wg) do { } while (0) #define dept_map_copy(t, f) do { } while (0) #define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) @@ -572,8 +582,8 @@ struct dept_task { }; #define dept_stage_event(t, ip) do { } while (0) #define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) #define dept_ecxt_holding(m, e_f) false -#define dept_request_event(m) do { } while (0) -#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event(m, wg) do { } while (0) +#define dept_event(m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 21fce525f031..8cdac7982036 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -24,7 +24,7 @@ #define sdt_wait_timeout(m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) #define sdt_wait(m) sdt_wait_timeout(m, -1L) @@ -49,7 +49,7 @@ #define sdt_might_sleep_end() dept_clean_stage() #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) -#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL) #define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 5c996f11abd5..fb33c3758c25 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2186,6 +2186,11 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, } EXPORT_SYMBOL_GPL(dept_map_reinit); +void dept_ext_wgen_init(struct dept_ext_wgen *ewg) +{ + ewg->wgen = 0U; +} + void dept_map_copy(struct dept_map *to, struct dept_map *from) { if (unlikely(!dept_working())) { @@ -2371,7 +2376,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map) + bool sched_map, unsigned int wg) { struct dept_class *c; struct dept_key *k; @@ -2393,7 +2398,7 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { - do_event(m, c, READ_ONCE(m->wgen), ip); + do_event(m, c, wg, ip); pop_ecxt(m, c); } } @@ -2606,7 +2611,7 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); exit: dept_exit(flags); } @@ -2785,10 +2790,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) } EXPORT_SYMBOL_GPL(dept_ecxt_holding); -void dept_request_event(struct dept_map *m) +void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { unsigned long flags; unsigned int wg; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2801,21 +2807,25 @@ void dept_request_event(struct dept_map *m) */ flags = dept_enter_recursive(); + wg_p = ewg ? &ewg->wgen : &m->wgen; + /* * Avoid zero wgen. */ wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); - WRITE_ONCE(m->wgen, wg); + WRITE_ONCE(*wg_p, wg); dept_exit_recursive(flags); } EXPORT_SYMBOL_GPL(dept_request_event); void dept_event(struct dept_map *m, unsigned long e_f, - unsigned long ip, const char *e_fn) + unsigned long ip, const char *e_fn, + struct dept_ext_wgen *ewg) { struct dept_task *dt = dept_task(); unsigned long flags; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2823,24 +2833,26 @@ void dept_event(struct dept_map *m, unsigned long e_f, if (m->nocheck) return; + wg_p = ewg ? &ewg->wgen : &m->wgen; + if (dt->recursive) { /* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); return; } flags = dept_enter(); - __dept_event(m, e_f, ip, e_fn, false); + __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); /* * Keep the map diabled until the next sleep. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); dept_exit(flags); } From patchwork Wed Feb 21 09:49:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565330 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 21ED04F608; Wed, 21 Feb 2024 09:50:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509011; cv=none; b=Ch9RfDKsAe11VjsToPA2TcjHwU4Niq2d4Y6NtsECdtfeuCYYADrLDHwwy7vQUPaX2aAMDTMGJVzmxnfvEkiLfndFwCgUgN6Olddyl5k9tqWBcKn8EZVpR+SFB0mvByF8Zv+LSCREDrLfNYC6Hl4VujrYoMb3o/Wk9CUhDPdCTko= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509011; c=relaxed/simple; bh=tC26hq3Ijka0/rt3bw+rfbu4EsnZw7fNBY7MOAe1vjA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=LkbWMSvDNjWIelz/JSZFuGBTsViP7KT+jBzgIG4EJ5Ss30o/znQcdsdC5RRWn9nWJvQWvO5Ernu+0i/gztfgePrMitawB14ULZBXMrTW1f1j3e01XpJ8gDRN11n9MP8X/uYQxEF4QNZ8tjLq+RFcHMz3cUMh80sojHJr0VdzeWI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-08-65d5c73b32cc From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 24/27] dept: Track PG_locked with dept Date: Wed, 21 Feb 2024 18:49:30 +0900 Message-Id: <20240221094933.36348-25-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSfUxTVxjGPefee+6lWHfXGXem0ZkmxsVFRaPba9HFZEZPnMYtmm2Zy1gj d6MZoGmliMli1aKOD4PGggI6QFM72vlRxKlQ0kH4KASlUhAZEm1kk1isAiXyMbXV+M+bX54n z++vV+I0NcJsyZC+WzGm61O1RMWrhqaXL05s7lISDnR9CsfyEiAyeoSH0osuAh0XnAhcV/Zj GGzcAHfGQggm229xUGTrQFD+4B4HV5r6EXgcBwh0PpwBgUiYgM+WS+Dg2YsE/I+nMPQVHsfg dG+GtoIKDN7x/3goGiRQUnQQR88jDOP2ShHslgUQdBSLMPVgGfj6uwXw9H4Mp870Eaj1+Hho uhbE0HmjlEC/66UAbU0tPHQcyxfgzycVBB6P2TmwR8Ii3PaWYbhkjYoOjbwQoDnfi+HQucsY AndrENQduY/B7eom0BAJYahy2ziYON+IIHh0SITsvHERSvYfRZCbXcjDrf+bBbD2rYTJ56Vk rY41hMIcs1ZlMs9YGc9aKyi7XnxPZNa6XpGVuTNYlWMRO1s7iFn5cERg7srfCHMPHxdZzlAA syc3b4qs5eQkzx4GivCXc75TrU5WUg1mxbj0sx9VKcF8O9lVu22Pze8RLWhkXQ6Kk6i8go5M TPBv+UTwqhhjIi+kPT3jXIxnyvNpVf6/Qg5SSZx8OJ46nraTWPGerKN3LOdxjHl5AR29/+w1 q+VPaPWNbvxG+iF1XvK+FsVF8z9KQkKMNfJK2uWv5mJSKh+Ooy0hP3kz+ID+7ejhC5C6DE2r RBpDujlNb0hdsSQlK92wZ8mOnWluFH0p+69T26+h4Y6t9UiWkHa6OuWvgKIR9GZTVlo9ohKn nanmM6OROlmftVcx7kwyZqQqpno0R+K176uXj2Uma+Sf9buVXxRll2J822IpbrYFfbP2+y0X drR3mr8tdFw+PRBef1dnS3y3rnL11aXbuzZxxrlUN1qS+HXry1z/ugSuYbHFJGm/oJM/ZLu2 tWbEb1pVU23+yTerX/e7yZnX4vVpv0IbI42fs4/2bTzBkgam2gqodc28p2rn1iRf76z4zf/U 2ZPMZ1bBs4GTTeF39l13aXlTin7ZIs5o0r8C33x9sE4DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/P83yf63TtUY1nMezMGvmRcfaZMDatR0b+sJkfjZOHbtXV 7irFWCmhZGquUHHKrtRRLj+ry6npB+uXrvxKyzGkEtU1p/y4Y/5577X3e3v99ZZQngWMj0Sl jhU1amWknEhp6ZaAlMUBjV2if321F2Sd8Qf72Cka8suNBNpvliEw3k7G0P84CJ6PDyKYaGmj IFfXjuDq2zcU3G7oRWAuOU6g870HWO3DBJp1GQRSisoJdAxMYujJycZQZtoMT88VYrA4PtKQ 208gLzcFO+MTBoehlAVD0nywlVxiYfLtMmju7WagvqCZAfMrP7h4uYdAjbmZhob7NgydVfkE eo2/GXja0ERDe1YmAze+FBIYGDdQYLAPs/DMosdQkeq0pY3+YqAx04Ih7dotDNaX1QhqT/Vh MBm7CdTbBzFUmnQU/Ch+jMB2doiFE2ccLOQln0WQcSKHhrafjQyk9ihg4ns+WRcg1A8OU0Jq 5SHBPK6nhSeFvPDg0htWSK19xQp6U5xQWbJQKKrpx8LVETsjmEpPE8E0ks0K6UNWLHxpbWWF pgsTtPDemou3ztopXb1fjFTFi5qla/dKw22ZBhJTsy1B12Fmk9DohnTkJuG5Ffx5213WxYTz 5V+8cFAu9ubm8pWZH5h0JJVQ3MmpfMnXFuIavLhV/POkYuximpvPj/V9+8sybiV/p6ob/5PO 4csqLH9Fbs7+et4g42JPTsF3ddyhziGpHk0pRd4qdXyUUhWpWKKNCE9UqxKWhEVHmZDzNIaj k1n30VhnUB3iJEjuLgu/ZxU9GWW8NjGqDvESSu4tow85K9l+ZeJhURO9RxMXKWrr0EwJLZ8h C94u7vXkDipjxQhRjBE1/1cscfNJQjNDPVbps/23LliTXTetI8riE92v2BcSVDUaksGGLP+a 8fDDlgHdcOCtmCPqd8mhisBWFBe43n22cZFPU1q8se3K9Hmnj8UkO1436XYwscS3JWzTO13w RjgaEXqx+HO6+7OhXdW/RyrWhRVZE3bbfHMKtX41+8bWzhlZ0de19NGBqXo5rQ1XLltIabTK P+dXv+4wAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Makes Dept able to track PG_locked waits and events. It's going to be useful in practice. See the following link that shows dept worked with PG_locked and can detect real issues: https://lore.kernel.org/lkml/1674268856-31807-1-git-send-email-byungchul.park@lge.com/ Signed-off-by: Byungchul Park --- include/linux/mm_types.h | 2 + include/linux/page-flags.h | 105 ++++++++++++++++++++++++++++++++----- include/linux/pagemap.h | 7 ++- mm/filemap.c | 26 +++++++++ mm/mm_init.c | 2 + 5 files changed, 129 insertions(+), 13 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 957ce38768b2..5c1112bc7a46 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -19,6 +19,7 @@ #include #include #include +#include #include @@ -203,6 +204,7 @@ struct page { struct page *kmsan_shadow; struct page *kmsan_origin; #endif + struct dept_ext_wgen PG_locked_wgen; } _struct_page_alignment; /* diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index a88e64acebfe..0a498f2c4543 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -198,6 +198,43 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H +#ifdef CONFIG_DEPT +#include +#include + +extern struct dept_map PG_locked_map; + +/* + * Place the following annotations in its suitable point in code: + * + * Annotate dept_page_set_bit() around firstly set_bit*() + * Annotate dept_page_clear_bit() around clear_bit*() + * Annotate dept_page_wait_on_bit() around wait_on_bit*() + */ + +static inline void dept_page_set_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_request_event(&PG_locked_map, &p->PG_locked_wgen); +} + +static inline void dept_page_clear_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_event(&PG_locked_map, 1UL, _RET_IP_, __func__, &p->PG_locked_wgen); +} + +static inline void dept_page_wait_on_bit(struct page *p, int bit_nr) +{ + if (bit_nr == PG_locked) + dept_wait(&PG_locked_map, 1UL, _RET_IP_, __func__, 0, -1L); +} +#else +#define dept_page_set_bit(p, bit_nr) do { } while (0) +#define dept_page_clear_bit(p, bit_nr) do { } while (0) +#define dept_page_wait_on_bit(p, bit_nr) do { } while (0) +#endif + #ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP DECLARE_STATIC_KEY_FALSE(hugetlb_optimize_vmemmap_key); @@ -379,44 +416,88 @@ static __always_inline int Page##uname(struct page *page) \ #define SETPAGEFLAG(uname, lname, policy) \ static __always_inline \ void folio_set_##lname(struct folio *folio) \ -{ set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_set_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void SetPage##uname(struct page *page) \ -{ set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline \ void folio_clear_##lname(struct folio *folio) \ -{ clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_clear_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void ClearPage##uname(struct page *page) \ -{ clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define __SETPAGEFLAG(uname, lname, policy) \ static __always_inline \ void __folio_set_##lname(struct folio *folio) \ -{ __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + __set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_set_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void __SetPage##uname(struct page *page) \ -{ __set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __set_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_set_bit(page, PG_##lname); \ +} #define __CLEARPAGEFLAG(uname, lname, policy) \ static __always_inline \ void __folio_clear_##lname(struct folio *folio) \ -{ __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + __clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); \ + dept_page_clear_bit(&folio->page, PG_##lname); \ +} \ static __always_inline void __ClearPage##uname(struct page *page) \ -{ __clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + __clear_bit(PG_##lname, &policy(page, 1)->flags); \ + dept_page_clear_bit(page, PG_##lname); \ +} #define TESTSETFLAG(uname, lname, policy) \ static __always_inline \ bool folio_test_set_##lname(struct folio *folio) \ -{ return test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + bool ret = test_and_set_bit(PG_##lname, folio_flags(folio, FOLIO_##policy));\ + if (!ret) \ + dept_page_set_bit(&folio->page, PG_##lname); \ + return ret; \ +} \ static __always_inline int TestSetPage##uname(struct page *page) \ -{ return test_and_set_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_set_bit(PG_##lname, &policy(page, 1)->flags);\ + if (!ret) \ + dept_page_set_bit(page, PG_##lname); \ + return ret; \ +} #define TESTCLEARFLAG(uname, lname, policy) \ static __always_inline \ bool folio_test_clear_##lname(struct folio *folio) \ -{ return test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy)); } \ +{ \ + bool ret = test_and_clear_bit(PG_##lname, folio_flags(folio, FOLIO_##policy));\ + if (ret) \ + dept_page_clear_bit(&folio->page, PG_##lname); \ + return ret; \ +} \ static __always_inline int TestClearPage##uname(struct page *page) \ -{ return test_and_clear_bit(PG_##lname, &policy(page, 1)->flags); } +{ \ + bool ret = test_and_clear_bit(PG_##lname, &policy(page, 1)->flags);\ + if (ret) \ + dept_page_clear_bit(page, PG_##lname); \ + return ret; \ +} #define PAGEFLAG(uname, lname, policy) \ TESTPAGEFLAG(uname, lname, policy) \ diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h index 06142ff7f9ce..c6683b228b20 100644 --- a/include/linux/pagemap.h +++ b/include/linux/pagemap.h @@ -991,7 +991,12 @@ void folio_unlock(struct folio *folio); */ static inline bool folio_trylock(struct folio *folio) { - return likely(!test_and_set_bit_lock(PG_locked, folio_flags(folio, 0))); + bool ret = !test_and_set_bit_lock(PG_locked, folio_flags(folio, 0)); + + if (ret) + dept_page_set_bit(&folio->page, PG_locked); + + return likely(ret); } /* diff --git a/mm/filemap.c b/mm/filemap.c index ad5b4aa049a3..241a67a363b0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -45,6 +45,7 @@ #include #include #include +#include #include #include #include "internal.h" @@ -1098,6 +1099,7 @@ static int wake_page_function(wait_queue_entry_t *wait, unsigned mode, int sync, if (flags & WQ_FLAG_CUSTOM) { if (test_and_set_bit(key->bit_nr, &key->folio->flags)) return -1; + dept_page_set_bit(&key->folio->page, key->bit_nr); flags |= WQ_FLAG_DONE; } } @@ -1181,6 +1183,7 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, if (wait->flags & WQ_FLAG_EXCLUSIVE) { if (test_and_set_bit(bit_nr, &folio->flags)) return false; + dept_page_set_bit(&folio->page, bit_nr); } else if (test_bit(bit_nr, &folio->flags)) return false; @@ -1191,6 +1194,9 @@ static inline bool folio_trylock_flag(struct folio *folio, int bit_nr, /* How many times do we accept lock stealing from under a waiter? */ int sysctl_page_lock_unfairness = 5; +struct dept_map __maybe_unused PG_locked_map = DEPT_MAP_INITIALIZER(PG_locked_map, NULL); +EXPORT_SYMBOL(PG_locked_map); + static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, int state, enum behavior behavior) { @@ -1202,6 +1208,8 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, unsigned long pflags; bool in_thrashing; + dept_page_wait_on_bit(&folio->page, bit_nr); + if (bit_nr == PG_locked && !folio_test_uptodate(folio) && folio_test_workingset(folio)) { delayacct_thrashing_start(&in_thrashing); @@ -1295,6 +1303,23 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, break; } + /* + * dept_page_set_bit() might have been called already in + * folio_trylock_flag(), wake_page_function() or somewhere. + * However, call it again to reset the wgen of dept to ensure + * dept_page_wait_on_bit() is called prior to + * dept_page_set_bit(). + * + * Remind dept considers all the waits between + * dept_page_set_bit() and dept_page_clear_bit() as potential + * event disturbers. Ensure the correct sequence so that dept + * can make correct decisions: + * + * wait -> acquire(set bit) -> release(clear bit) + */ + if (wait->flags & WQ_FLAG_DONE) + dept_page_set_bit(&folio->page, bit_nr); + /* * If a signal happened, this 'finish_wait()' may remove the last * waiter from the wait-queues, but the folio waiters bit will remain @@ -1471,6 +1496,7 @@ void folio_unlock(struct folio *folio) BUILD_BUG_ON(PG_waiters != 7); BUILD_BUG_ON(PG_locked > 7); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + dept_page_clear_bit(&folio->page, PG_locked); if (folio_xor_flags_has_waiters(folio, 1 << PG_locked)) folio_wake_bit(folio, PG_locked); } diff --git a/mm/mm_init.c b/mm/mm_init.c index 077bfe393b5e..fc150d7a3686 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -26,6 +26,7 @@ #include #include #include +#include #include "internal.h" #include "slab.h" #include "shuffle.h" @@ -564,6 +565,7 @@ void __meminit __init_single_page(struct page *page, unsigned long pfn, page_mapcount_reset(page); page_cpupid_reset_last(page); page_kasan_tag_reset(page); + dept_ext_wgen_init(&page->PG_locked_wgen); INIT_LIST_HEAD(&page->lru); #ifdef WANT_PAGE_VIRTUAL From patchwork Wed Feb 21 09:49:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565331 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7706251004; Wed, 21 Feb 2024 09:50:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509011; cv=none; b=RnX2cdKyQOQCWZuTLnIyqRxdX/T2qAqMWpsSHgZD57DFdQDD+lhq6G4Rj7F7NfXFhwVhuIjOiJi9tp9AXH5nsEL7LAXxps76/t6v4c49o0WJbDMVp9iT68fnWebvzhkG2z3d3X30TJVwGjrKywsGpWTaBxHkUBHxUwyIC8b6ppY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509011; c=relaxed/simple; bh=ul/wiSzDZit7bOUs1eKkFyN45Ghg8m4Ni94/4VNQ6UQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=sI0pLUtrNq4VVSvSyf1yBQuA3ekTfS3zWnAr/IuxVRqx72LqTdCn/zLBIIS5gN5J6P6pMD63+LHjVLdycjvEsIef010qeCePIbfZkZ/4dYQfqAsJFx4DkD+NheZuHo8/UX9OneEVM8kdQa7rxO7kJd0JixaegPHXE5iL+llQO08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-18-65d5c73b6e0d From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 25/27] dept: Print event context requestor's stacktrace on report Date: Wed, 21 Feb 2024 18:49:31 +0900 Message-Id: <20240221094933.36348-26-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSf0yMcRzHfb/P83yfu+PscRqPH8MOMyeJhc9ofvxhvv4wNr/Z6KaHburO LqpjpugHkSlLUaxi51yXcvld3RJK6Ie6I6QpkSiRrkmRu8Y/n732ee/9+uOzj4xRFXHjZTr9 Psmo14aqiYJVdI7Inh1Y4ZL8Xe8IJJ/0B3fPMRYy820Eaq/lIrDdiMHQ/mglvOztQNBfVcNA WmotguzmtwzcKG9CUGI5QqC+dSQ43V0EKlNPEDh6KZ/A8y8DGBrPpmDIta+Gp6dzMJT2tbGQ 1k4gI+0o9oxPGPrMVh7M0dOhxXKeh4HmuVDZ9IKDktez4NzFRgLFJZUslN9pwVB/L5NAk22Q g6flj1moTU7iIO9rDoEvvWYGzO4uHupKszAUxHpE8T/+cFCRVIoh/vJ1DM5XRQgcx95hsNte EHjg7sBQaE9l4NeVRwhaTnXyEHeyj4eMmFMITsSdZaHmdwUHsY3zof9nJlm2iD7o6GJobGEk LenNYumTHJHePf+Wp7GO1zzNsu+nhRYNvVTcjml2t5ujdutxQu3dKTxN7HRi+rW6mqeP0/tZ 2upMw2snbFUEBkuhugjJOGdJkCKk8kIns/f4iqgP7cVsNEpfmIjkMlEIEJ/ZGvF//n69ZoiJ MENsaOhjvOwjTBELkz5yiUghY4SE4aLlWxXxBqOFDeLd9Bjey6wwXWx19QyVlcIC8X1b0z/p ZDG3oHRIJPfsr2Z0cF5WCfNF1/ObjFcqCgly8aX14b/COPG+pYE9jZRZaJgVqXT6iDCtLjTA L8Sk10X57TSE2ZHnpcyHBrbdQd2168qQIEPqEcqQ205JxWkjwk1hZUiUMWofJRvpWSmDtaYD ktGww7g/VAovQxNkrHqscl5vZLBK2K3dJ+2RpL2S8X+KZfLxnuMcPNJzJlXvy00t2ul8H80p TPWrqre3ZDpSPvvUjUlwxAUuDUDJ3Z82Osqiyl1+k/I31TBBtw7DxJmjNTlhUYboaVbD2J+z /ON25X9efnHLM03C5nEVyUqFYY3c9Cp7cFBTtb25zvdevP6hykJGveEi81rn+XJWuj5xcdCw hkMz1Gx4iHauhjGGa/8CyBTNdk4DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+//PbS4Xp2V00sBaSGFlGZkvWBJCdEiKiCKKIIcedDhXbKlZ FNpWXsrIwkupMVeupat0XitXonlZVmrObpjksoulrouTlnbZir68/Hgent+nV0RISyl/kUJ1 UFCr5EoZLSbFWyO0KyI6+oVVDcYAyDu9ClwTWSSU3DTT0HOjEoG5NgPDSNsmeDY5imDqUTcB hfk9CMqGXhFQ2z6IwGo6TkPf8Cywu5w02PJP0aC9fJOG3k/TGAYKzmGotGyBrrMGDM3u9yQU jtBQXKjFnvMBg9tYwYAxPQgcposMTA+Fgm3wKQWtpTYKrC+XwYVLAzQ0WW0ktDc6MPTdLqFh 0Pybgq72ThJ68nIpuD5uoOHTpJEAo8vJwJNmPYYqncd28tsvCjpymzGcvFKNwf7iDoK7Wa8x WMxPaWh1jWKoseQT8ONqGwLHmTEGTpx2M1CccQbBqRMFJHT/7KBANxAGU99L6A0RfOuok+B1 Nam8dVJP8g8MHH/r4iuG1919yfB6SzJfYwrmLzeNYL7sq4viLRXZNG/5eo7hc8bsmB9//Jjh O4umSH7YXoi3LdgjXhcnKBUpgnplZIw4wVY6RhzI3njo7UgTmY6KwnOQj4hj13Bfqruxl2l2 Cff8uZvwsh+7kKvJfUflILGIYDNncqbPj2hvMYfdyd0qymC8TLJB3HD/xN+xhF3LvXk/iP9J A7nKqua/Ih9Pfq14lPKylA3j+nvriLNIrEczKpCfQpWSJFcow0I0iQlpKsWhkNj9SRbkeRrj 0em8RjTRt6kFsSIk85UkNNgFKSVP0aQltSBORMj8JGSqJ5LEydMOC+r9+9TJSkHTggJEpGye ZPMuIUbKxssPComCcEBQ/2+xyMc/HdWnLJkVgLdMxObEfnxyzzk3ULpVG07nrS8hy9fcf/0l ZDzLFBS1u261bmldVn1m777Qcl/u4Xz/uOC6qvtSc1vXendU/A2diomPdr3RG8zRw8mZuy+0 xEZeWR5XbljkDHdcUhy5tre+M+ZYQef5xZsTtanbA61DSmrHdC47O/qdQ0ZqEuShwYRaI/8D VUKORjADAAA= X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Currently, print nothing in place of [S] in report, which means stacktrace of event context's start if the event is not an unlock thing by typical lock but general event because it's not easy to specify the point in a general way, where the event context has started from. However, unfortunately it makes hard to interpret dept's report in that case. So made it print the event requestor's stacktrace instead of the event context's start, in place of [S] in report. Signed-off-by: Byungchul Park --- include/linux/dept.h | 13 +++++++ kernel/dependency/dept.c | 83 ++++++++++++++++++++++++++++++++-------- 2 files changed, 80 insertions(+), 16 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index dea53ad5b356..6db23d77905e 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -145,6 +145,11 @@ struct dept_map { */ unsigned int wgen; + /* + * requestor for the event context to run + */ + struct dept_stack *req_stack; + /* * whether this map should be going to be checked or not */ @@ -486,7 +491,15 @@ struct dept_task { * for subsystems that requires compact use of memory e.g. struct page */ struct dept_ext_wgen{ + /* + * wait timestamp associated to this map + */ unsigned int wgen; + + /* + * requestor for the event context to run + */ + struct dept_stack *req_stack; }; #define DEPT_TASK_INITIALIZER(t) \ diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index fb33c3758c25..abf1cdab0615 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -129,6 +129,7 @@ static int dept_per_cpu_ready; #define DEPT_INFO(s...) pr_warn("DEPT_INFO: " s) static arch_spinlock_t dept_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; +static arch_spinlock_t dept_req_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; static arch_spinlock_t dept_pool_spin = (arch_spinlock_t)__ARCH_SPIN_LOCK_UNLOCKED; /* @@ -1669,7 +1670,8 @@ static void add_wait(struct dept_class *c, unsigned long ip, static bool add_ecxt(struct dept_map *m, struct dept_class *c, unsigned long ip, const char *c_fn, - const char *e_fn, int sub_l) + const char *e_fn, int sub_l, + struct dept_stack *req_stack) { struct dept_task *dt = dept_task(); struct dept_ecxt_held *eh; @@ -1700,10 +1702,16 @@ static bool add_ecxt(struct dept_map *m, struct dept_class *c, e->class = get_class(c); e->ecxt_ip = ip; - e->ecxt_stack = ip && rich_stack ? get_current_stack() : NULL; e->event_fn = e_fn; e->ecxt_fn = c_fn; + if (req_stack) + e->ecxt_stack = get_stack(req_stack); + else if (ip && rich_stack) + e->ecxt_stack = get_current_stack(); + else + e->ecxt_stack = NULL; + eh = dt->ecxt_held + (dt->ecxt_held_pos++); eh->ecxt = get_ecxt(e); eh->map = m; @@ -2147,6 +2155,7 @@ void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, m->sub_u = sub_u; m->name = n; m->wgen = 0U; + m->req_stack = NULL; m->nocheck = !valid_key(k); dept_exit_recursive(flags); @@ -2181,6 +2190,7 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, m->name = n; m->wgen = 0U; + m->req_stack = NULL; dept_exit_recursive(flags); } @@ -2189,6 +2199,7 @@ EXPORT_SYMBOL_GPL(dept_map_reinit); void dept_ext_wgen_init(struct dept_ext_wgen *ewg) { ewg->wgen = 0U; + ewg->req_stack = NULL; } void dept_map_copy(struct dept_map *to, struct dept_map *from) @@ -2376,7 +2387,8 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map, unsigned int wg) + bool sched_map, unsigned int wg, + struct dept_stack *req_stack) { struct dept_class *c; struct dept_key *k; @@ -2397,7 +2409,7 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); - if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { + if (c && add_ecxt(m, c, 0UL, "(event requestor)", e_fn, 0, req_stack)) { do_event(m, c, wg, ip); pop_ecxt(m, c); } @@ -2506,6 +2518,8 @@ EXPORT_SYMBOL_GPL(dept_stage_wait); static void __dept_clean_stage(struct dept_task *dt) { + if (dt->stage_m.req_stack) + put_stack(dt->stage_m.req_stack); memset(&dt->stage_m, 0x0, sizeof(struct dept_map)); dt->stage_sched_map = false; dt->stage_w_fn = NULL; @@ -2571,6 +2585,7 @@ void dept_request_event_wait_commit(void) */ wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(dt->stage_m.wgen, wg); + dt->stage_m.req_stack = get_current_stack(); __dept_wait(&dt->stage_m, 1UL, ip, w_fn, 0, true, sched_map, timeout); exit: @@ -2602,6 +2617,8 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) */ m = dt_req->stage_m; sched_map = dt_req->stage_sched_map; + if (m.req_stack) + get_stack(m.req_stack); __dept_clean_stage(dt_req); /* @@ -2611,8 +2628,12 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen, + m.req_stack); exit: + if (m.req_stack) + put_stack(m.req_stack); + dept_exit(flags); } @@ -2692,7 +2713,7 @@ void dept_map_ecxt_modify(struct dept_map *m, unsigned long e_f, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, new_e), m->name, false); - if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l)) + if (c && add_ecxt(m, c, new_ip, new_c_fn, new_e_fn, new_sub_l, NULL)) goto exit; /* @@ -2744,7 +2765,7 @@ void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, k = m->keys ?: &m->map_key; c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, false); - if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l)) + if (c && add_ecxt(m, c, ip, c_fn, e_fn, sub_l, NULL)) goto exit; missing_ecxt: dt->missing_ecxt++; @@ -2792,9 +2813,11 @@ EXPORT_SYMBOL_GPL(dept_ecxt_holding); void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { + struct dept_task *dt = dept_task(); unsigned long flags; unsigned int wg; unsigned int *wg_p; + struct dept_stack **req_stack_p; if (unlikely(!dept_working())) return; @@ -2802,12 +2825,18 @@ void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) if (m->nocheck) return; - /* - * Allow recursive entrance. - */ - flags = dept_enter_recursive(); + if (dt->recursive) + return; - wg_p = ewg ? &ewg->wgen : &m->wgen; + flags = dept_enter(); + + if (ewg) { + wg_p = &ewg->wgen; + req_stack_p = &ewg->req_stack; + } else { + wg_p = &m->wgen; + req_stack_p = &m->req_stack; + } /* * Avoid zero wgen. @@ -2815,7 +2844,13 @@ void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); WRITE_ONCE(*wg_p, wg); - dept_exit_recursive(flags); + arch_spin_lock(&dept_req_spin); + if (*req_stack_p) + put_stack(*req_stack_p); + *req_stack_p = get_current_stack(); + arch_spin_unlock(&dept_req_spin); + + dept_exit(flags); } EXPORT_SYMBOL_GPL(dept_request_event); @@ -2826,6 +2861,8 @@ void dept_event(struct dept_map *m, unsigned long e_f, struct dept_task *dt = dept_task(); unsigned long flags; unsigned int *wg_p; + struct dept_stack **req_stack_p; + struct dept_stack *req_stack; if (unlikely(!dept_working())) return; @@ -2833,7 +2870,18 @@ void dept_event(struct dept_map *m, unsigned long e_f, if (m->nocheck) return; - wg_p = ewg ? &ewg->wgen : &m->wgen; + if (ewg) { + wg_p = &ewg->wgen; + req_stack_p = &ewg->req_stack; + } else { + wg_p = &m->wgen; + req_stack_p = &m->req_stack; + } + + arch_spin_lock(&dept_req_spin); + req_stack = *req_stack_p; + *req_stack_p = NULL; + arch_spin_unlock(&dept_req_spin); if (dt->recursive) { /* @@ -2842,17 +2890,20 @@ void dept_event(struct dept_map *m, unsigned long e_f, * handling the event. Disable it until the next. */ WRITE_ONCE(*wg_p, 0U); + if (req_stack) + put_stack(req_stack); return; } flags = dept_enter(); - - __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); + __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p), req_stack); /* * Keep the map diabled until the next sleep. */ WRITE_ONCE(*wg_p, 0U); + if (req_stack) + put_stack(req_stack); dept_exit(flags); } From patchwork Wed Feb 21 09:49:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565332 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id DB11958ACA; Wed, 21 Feb 2024 09:50:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509012; cv=none; b=r5HYY2+3duY9oSKibGEW5rG74GMumuHCaq+kwr/jpL2vcZe7ILm/U0ovLBE1cpyn9CYxauxDUaHFAoasBgAZ3oYn8UGRb8oRzALhc+AX+bYIhY+IxlH8AY/Q7+HLcMYLXJ2hPwxjhMyNVmppiRHkoBI7FtjQ5lMcPiHh05+UUK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509012; c=relaxed/simple; bh=5yRKL/D4WKMUyz5KF4LYQmab3vQapC/hAllcpi+yp24=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=jlICJBlAq/vClzL5JKYSRePEwWReElQkTCJLXaXr42WLPRlrWlMKePOOgfXnhg3whwPhx4b5dI0ABZVNIcYWKGeCthRU+fGM3ENf2sMKbSz47Z6+FGJsvwc24Ivc3LQuVMQ/gJl1JAizFyGAS5zrk9Sm8S2D36BnDYJsym/Be7E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-28-65d5c73b30c9 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 26/27] fs/jbd2: Use a weaker annotation in journal handling Date: Wed, 21 Feb 2024 18:49:32 +0900 Message-Id: <20240221094933.36348-27-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0xTZxjHfd9zzntOC5WTzoSDxmhqdEYD6qLkiSGbX+aOS+ZM/MaySzdO pFmprJWbCYRLtYhAuASqQlxBUzvoQAsGBMq4hEtVoGhlWAEFUSTcNlg7uexSJH558sv/+T3/ Tw9HKZuYrZxGd07S69RaFZHT8rngivConifSwUu9QVCYexB8f2XTUF5rJ+CuqUZgr8/AMN31 Gfzun0Ww2jdAgbnEjaBifJSC+u4xBE5bJoHHk5vB41sg4Cq5TCDrRi2BwZk1DCOlRRiqHV/A g4JKDG3LUzSYpwmUmbNwYLzBsGytYsGavhsmbNdYWBs/BK6xIQac3v1w9foIgRani4buxgkM j5vKCYzZ/2PgQXcvDe7CPAZ+na8kMOO3UmD1LbDwqM2C4bYxUHRx6V8GevLaMFy8eQeD52kz gtbsFxgc9iECnb5ZDHWOEgpWbnUhmMifY+FC7jILZRn5CC5fKKVh4J8eBowjR2D1bTk5dlTs nF2gRGNdkuj0W2jxfqUg3rs2yorGVi8rWhwJYp1tn3ijZRqLFYs+RnRUXSKiY7GIFXPmPFic 7+9nxd4rq7Q46THjU9ui5VExklaTKOkPfPydPLbm1SMq3h+UfL/rN5SOVmQ5SMYJ/GGhtKSd vOf0wSJ2nQn/oTA8vEyt8xZ+p1CX95rJQXKO4k1Bgu2PvncHH/BfCvWuDYnmdwvN3mm8zgo+ Uhhv+JndKN0hVN9ue+fIAvkvZbPMOiv5I8KTwbvUhpMlEx5mh29wmNBuG6YLkMKCNlUhpUaX GKfWaA9HxKboNMkRP5yNc6DAR1lT175qRIvu0x2I55AqWBHb4JGUjDrRkBLXgQSOUm1R0EmB SBGjTjkv6c9+q0/QSoYOtI2jVaGKj/xJMUr+jPqc9KMkxUv691vMybamI+4n4+cnJ7Yzx72k 2GIq3OUMyQxVFWqrTJ+0762fmcv//u/iowlNhrcRac3PoyPnw/pS/5wcGtcMtLzMcoVGrCQd O3Cr+ps7e55ZQvbklge1nF7aaTaNfpqme505ZE/VTeECU/TUQ3138QjjjTlRbLkSHK6oOfX1 plr33pClHhJ+XUUbYtWH9lF6g/p/OE3gK00DAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSe0hTcRzF+/3uvb87Z4ubSV0qykYWGb4g4wtGVBRdgp5/VESUo245fLbp yqLQXFaWQw1bpZVarKXLbBr20obiY9pj6iwTs7Tn0LSsjebM2or+OXw453D+OhIq4DIzU6JM TBFViYp4OZHS0g3RmaHRzV1ixCuHHPLORoDzxykaim6bCNgqyhGYqjMwOBrXwkvXEALP0+cU 6AtsCEr6X1NQ3dSHoNZ4nEDn+ylgd44QsBacIZB57TaB9sFxDL3n8zGUm9dDW24pBov7Ew16 B4FCfSb2ymcMbkMZC4b0YBgwXmJhvD8SrH0vGGi4bGWgtmcxXLzSS+BRrZWGpnsDGDofFBHo M/1moK2phQZbXg4Dt4ZLCQy6DBQYnCMsdFiKMVRqvWtZ3ycYaM6xYMi6fgeD/dVDBHWn3mIw m14QaHAOYagyF1AwdqMRwYDuCwsnzrpZKMzQIThz4jwNz381M6DtjQLPzyKyIlpoGBqhBG3V QaHWVUwLraW8cP/Sa1bQ1vWwQrE5VagyhgjXHjmwUDLqZARz2WkimEfzWSH7ix0Lw8+esULL BQ8tvLfr8abZO6TL9orxSo2oCl8eI42t+NBBJbv8D7U2PkbpaMwvG/lJeG4Jn96ez/qYcAv5 7m435eNALoivyvnIZCOphOJO+vPGr0+JL5jGbeSrrf9KNBfMP+xxYB/LuKV8f81V9t/oXL68 0vK34+f1bxYOMT4O4KL4rva7VC6SFqNJZShQmahJUCjjo8LUcbFpicpDYXuSEszIexrD0fG8 e+hH59p6xEmQfLIstsYuBjAKjTotoR7xEkoeKKMPei3ZXkXaYVGVtFuVGi+q69EsCS2fIVu3 TYwJ4PYrUsQ4UUwWVf9TLPGbmY6SybEDMR9HXStX9OvwytSk+efcs/T3k1Z7NAnaeUFvDovB Gvg0HD6Y8i2iaF/JV83W3DVbbLYHllWVdaYP05a3VXcuzjqS98aQrJk3RTeYkb3AX7Vol95T gza3TYRO/ZxvDMLKA98ypxrHpm9/NzF7Z0cJ6/9kTpruWEi3hlSkNMlpdawiMoRSqRV/AAj7 mKAwAwAA X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: jbd2 journal handling code doesn't want jbd2_might_wait_for_commit() to be placed between start_this_handle() and stop_this_handle(). So it marks the region with rwsem_acquire_read() and rwsem_release(). However, the annotation is too strong for that purpose. We don't have to use more than try lock annotation for that. rwsem_acquire_read() implies: 1. might be a waiter on contention of the lock. 2. enter to the critical section of the lock. All we need in here is to act 2, not 1. So trylock version of annotation is sufficient for that purpose. Now that dept partially relies on lockdep annotaions, dept interpets rwsem_acquire_read() as a potential wait and might report a deadlock by the wait. So replaced it with trylock version of annotation. Signed-off-by: Byungchul Park --- fs/jbd2/transaction.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/jbd2/transaction.c b/fs/jbd2/transaction.c index 5f08b5fd105a..2c159a547e15 100644 --- a/fs/jbd2/transaction.c +++ b/fs/jbd2/transaction.c @@ -460,7 +460,7 @@ static int start_this_handle(journal_t *journal, handle_t *handle, read_unlock(&journal->j_state_lock); current->journal_info = handle; - rwsem_acquire_read(&journal->j_trans_commit_map, 0, 0, _THIS_IP_); + rwsem_acquire_read(&journal->j_trans_commit_map, 0, 1, _THIS_IP_); jbd2_journal_free_transaction(new_transaction); /* * Ensure that no allocations done while the transaction is open are From patchwork Wed Feb 21 09:49:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565333 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id BABD35823C; Wed, 21 Feb 2024 09:50:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509012; cv=none; b=fKhI1kznNddmLqpfHop+N4q99eDpL6ti3EWnWWVqlvZxwEBuSSBgKMg+Y+v/d90eKr5sxnnFbE0sUtUXBqn28cRTif5DHhIp4pvgtBJWvAiZMTUkD8H2Ofxr5AKwzaRpNvBPW7ip680XuraIR2d2s4TMVolJ2g/9eg2gjG55f+s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708509012; c=relaxed/simple; bh=1CF3fT+0y7rY0XzqAKz8iCCaEnPZ1j3P7N8n5p5eJns=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References; b=UjKn5dBPkqjWzWosd3qQ3Z35QmJazWgKpQUkD0AKLlxAtI1VYpdMVPnw/9EeXqsWvgH0mhPirrOxCn93jTTCrydl7N8E8gN8KF73iqnYJLEVZv10PeLJhOQ/DTG7R5T9fV10e6QtnxjoA7r084Jk/W91Do/mGAcNVpYikuIdREQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d85ff70000001748-39-65d5c73b28e7 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 27/27] dept: Add 'Dept' documentation Date: Wed, 21 Feb 2024 18:49:33 +0900 Message-Id: <20240221094933.36348-28-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0yTZxTHfZ73SrX4ppr4Tkw0TRSnUUFBj4aZGW+PRqOJyT64LVrlVRoL aLlZEw1qqaUV4yUFFdQCpnaFTXyLERQMQgCRqKDlogGizKjEIhfXuo46bDF+Ofnl/z/n9+nw lKqamclrU9IlfYpGp2YVtGJwSvGihOYOKSZQo4Bzp2PA94+ZhqKb5Sy0/VWGoLzyOIaBxo3Q 5fciGHv8lIICWxuC4te9FFQ29SGodZ5g4fmbSPD4hlhosVlZOFl6k4X2D0EMPfnnMZTJW6H1 bAmGusA7GgoGWCgsOIlD4z2GgMPFgSN7LvQ7L3MQfB0LLX2dDNS+XAiXrvawUFPbQkNTVT+G 53eLWOgrH2egtekhDW3n8hj482MJCx/8DgocviEOntXZMVQYQyLTp/8ZaM6rw2C6fguD58U9 BPfNrzDI5Z0sNPi8GNyyjYL/bjQi6D8zyEHO6QAHhcfPILDm5NPw9EszA8aeeBj7t4j9eRVp 8A5RxOjOIrV+O00elYik+nIvR4z3X3LELmcQt3MBKa0ZwKR41McQ2ZXLEnn0PEcsgx5MPj55 wpGHF8do8sZTgLdH7VQkJEo6baakX7J6tyLJbsvhDl5JOFz2aYzKRv75FhTBi0Kc+O5EF2tB /AT33tgVjlkhWuzuDlBhni7MEd15bxkLUvCUcGqy6Bx+zIaLacJKsaG7kQkzLcwV3WYbFfYo heXiSNDwTT9bLKuom/BEhOI/Cr0T6yohXuxov02FnaJgjRCDX4LMt4MfxAfObvosUtrRJBdS aVMykzVaXdziJEOK9vDivanJMgo9lONo8NcqNNq2ox4JPFJPUSbd8UgqRpOZZkiuRyJPqacr 6axQpEzUGI5I+tRd+gydlFaPonhaPUO51J+VqBL2a9KlA5J0UNJ/bzEfMTMbWa3pOuO63MgX yzrhWK7DZP5pXqTBFF3zi4tq31waG1c1Lm8iK9S/Y8ucuGXT/OmpFZptJRuK46fuyV+zNnl9 QsaI9UhFx7PhWzuv7Zv/+ULPYGrr+G8xZmZ0+Me/7V3qjspFs2YsbdZbho9dz66WmY2ZnUf3 ZXk/LxyZbDrkitqSqKbTkjSxCyh9muYrda9QDUwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0hTcRjG+5/L/xxXi8M0PHShGEplZUZab2jXD3WIjAgqsg866uCW17ay DALzUmZZai0tLTarZWqlm91TlkvnutjMeUnU0qQSV+u2kWmXLejLy8Pvefh9ellSVkpPZVXJ e0V1siJRjiWUZGNk1oJIa4cYVtAdAoUnwsD9PZeCspvVGOw3qhBU1x0mYLhpHXR5nAjGnr8g oVhrR6Af6COhrrkfQX1FJob2ocngcLsw2LTHMWRduomhbWScgN6zRQRUGaPhaUE5AebR9xQU D2MoLc4ivOcDAaOGSgYMGcEwWHGegfGBRWDr76TBcsFGQ33PPDh3sRfDw3obBc13Bwlov1+G ob/6Dw1Pm1sosBfm03D9UzmGEY+BBIPbxcBLs46Ammyv7ci33zRY880EHLlcS4Dj1QMEDblv CDBWd2KwuJ0EmIxaEn5ebUIwePIjAzknRhkoPXwSwfGcsxS8+GWlIbs3AsZ+lOFVkYLF6SKF bNN+od6jo4Qn5bxw73wfI2Q39DCCzrhPMFWECJceDhOC/qubFoyVx7Bg/FrECHkfHYTwqbWV EVpKxihhyFFMbJoeI4naJSaq0kT1whVxEqVOm8OkXog6UPVtjMxAnjl5iGV5Lpzvuxqbh/xY zM3mu7tHSV8O4Gbxpvx3dB6SsCR3dCJf8fk59hX+3DLe0t1E+zLFBfOmXC3p80i5JfyX8XQf 5rmZfFWN+Z/Hz4uvlTr/zWVcBN/RdossQBIdmlCJAlTJaUkKVWJEqCZBmZ6sOhC6MyXJiLwv Yzg0XngXfW9f14g4FsknSZV3HKKMVqRp0pMaEc+S8gAptd+LpLsU6QdFdUqsel+iqGlE01hK Hihdv02Mk3Hxir1igiimiur/LcH6Tc1At/ccTIr3hEWx5dH2GF2tNefU0pYdrctbn1Hro+ML uWDnmgHuypTw0BlKw7z78wNjUiKCFk/yo3dbA7Uul/6aJXBD0Gv/npIZNtPcrCsjAz+3rDwz f1rDlu2bM1eTRTsrgx6/HXqvV5q/RIfiTP3prV11a2NluCShJffBoyZ7nP8POaVRKhaFkGqN 4i+4H0qqLgMAAA== X-CFilter-Loop: Reflected Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: This document describes the concept of Dept. Signed-off-by: Byungchul Park --- Documentation/dependency/dept.txt | 283 ++++++++++++++++++++++++++++++ 1 file changed, 283 insertions(+) create mode 100644 Documentation/dependency/dept.txt diff --git a/Documentation/dependency/dept.txt b/Documentation/dependency/dept.txt new file mode 100644 index 000000000000..7efe3bc59b2d --- /dev/null +++ b/Documentation/dependency/dept.txt @@ -0,0 +1,283 @@ +DEPT(DEPendency Tracker) +======================== + +Started by Byungchul Park + +How lockdep works +----------------- + +Lockdep tries to detect a deadlock by checking lock acquisition order. +For example, consider a graph built by lockdep like: + + A -> B - + \ + -> E + / + C -> D - + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep keeps adding each new acquisition order into the graph in +runtime. For example, 'E -> C' will be added when it's recognized that +the two locks have been acquired in that order like: + + A -> B - + \ + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +This graph contains a subgraph that demonstrates a loop like: + + -> E - + / \ + -> C -> D - \ + / / + \ / + ------------------ + + where 'A -> B' means that acquisition A is prior to acquisition B + with A still held. + +Lockdep reports it as a deadlock on detection of a loop. + +CONCLUSION + +Lockdep detects a deadlock by checking if a loop has been created after +expanding the graph. + + +Limitation of lockdep +--------------------- + +Lockdep works on typical lock e.g. spinlock and mutex, that are supposed +to be released within the acquisition context. However, a deadlock by +folio lock or other synchronization mechanisms cannot be detected by +lockdep that basically tracks lock acquisition order. + +Can we detect the following deadlock? + + CONTEXT X CONTEXT Y CONTEXT Z + + mutex_lock A + folio_lock B + folio_lock B + mutex_lock A /* DEADLOCK */ + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +No, we can't. What about the following? + + CONTEXT X CONTEXT Y + + mutex_lock A + mutex_lock A + wait_for_complete B /* DEADLOCK */ + complete B + mutex_unlock A + mutex_unlock A + +No, we can't. + +CONCLUSION + +Given the limitation, lockdep cannot detect a deadlock by folio lock or +other synchronization mechanisms. + + +What leads a deadlock +--------------------- + +A deadlock occurs when one or multi contexts are waiting for events that +will never happen. For example: + + CONTEXT X CONTEXT Y CONTEXT Z + + | | | + v | | + (1) wait for A v | + . (2) wait for C v + event C . (3) wait for B + event B . + event A + +Event C cannot be triggered because context X is stuck at (1), event B +cannot be triggered because context Y is stuck at (2), and event A +cannot be triggered because context Z is stuck at (3). All the contexts +are stuck. We call the situation a *deadlock*. + +If an event occurrence is a prerequisite to reaching another event, we +call it *dependency*. In the example above: + + Event A occurrence is a prerequisite to reaching event C. + Event C occurrence is a prerequisite to reaching event B. + Event B occurrence is a prerequisite to reaching event A. + +In terms of dependency: + + Event C depends on event A. + Event B depends on event C. + Event A depends on event B. + +Dependencies in a graph look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +A circular dependency exists. Such a circular dependency leads a +deadlock since no waiters can have desired events triggered. + +CONCLUSION + +A circular dependency leads a deadlock. + + +Introduce DEPT +-------------- + +DEPT(DEPendency Tracker) tracks wait and event instead of lock +acquisition order so as to recognize the following situation: + + CONTEXT X CONTEXT Y CONTEXT Z + + | | | + v | | + wait for A v | + . wait for C v + event C . wait for B + event B . + event A + +and builds up a dependency graph in runtime, similar to lockdep. The +graph would look like: + + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT keeps adding each new dependency into the graph in runtime. For +example, 'B -> D' will be added when it's recognized that event D +occurrence is a prerequisite to reaching event B, in other words, event +B depends on event D like: + + | + v + wait for D + . + event B + +After adding 'B -> D' dependency into the graph, the graph would look +like: + + -> D + / + -> C -> A -> B - + / \ + \ / + ---------------- + + where 'A -> B' means that event A depends on event B. + +DEPT is going to report a deadlock on detection of a new loop. + +CONCLUSION + +DEPT works on wait and event so as to theoretically detect all the +potential deadlocks. + + +How DEPT works +-------------- + +Let's take a look how DEPT works with an example that was mentioned in +the section 'Limitation of lockdep'. + + CONTEXT X CONTEXT Y CONTEXT Z + + mutex_lock A + folio_lock B + folio_lock B + mutex_lock A /* DEADLOCK */ + folio_unlock B + folio_unlock B + mutex_unlock A + mutex_unlock A + +Add comments to describe DEPT's view using terms of wait and event. + + CONTEXT X CONTEXT Y CONTEXT Z + + mutex_lock A + /* start to take into account event A context */ + folio_lock B + /* start to take into account event B context */ + + folio_lock B + /* wait for B */ + (1) + mutex_lock A /* DEADLOCK */ + /* wait for A */ + (2) + + folio_unlock B + /* event B */ + folio_unlock B + /* not interest until reaching (1) */ + + mutex_unlock A + /* event A */ + mutex_unlock A + /* not interest until reaching (2) */ + +Focusing on wait and event, the example can be simplified like: + + CONTEXT X CONTEXT Y CONTEXT Z + + | | + | | + v | + wait for B v + . wait for A + . . + . event B + event A + +Event A occurrence is a prerequisite to reaching event B, and event B +occurrence is a prerequisite to reaching event A. + +In terms of dependency: + + Event B depends on event A. + Event A depends on event B. + +Dependencies in the dependency graph look like: + + -> A -> B - + / \ + \ / + ----------- + + where 'A -> B' means that event A depends on event B. + +A loop has been created. So DEPT can report it as a deadlock. + +CONCLUSION + +DEPT works well with any synchronization mechanisms by focusing on wait +and event.