From patchwork Wed Feb 21 09:49:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13565301 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F222C5478A for ; Wed, 21 Feb 2024 09:50:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C11196B009D; Wed, 21 Feb 2024 04:50:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BC1086B009F; Wed, 21 Feb 2024 04:50:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81B0E6B00A1; Wed, 21 Feb 2024 04:50:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5F5276B009D for ; Wed, 21 Feb 2024 04:50:07 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 3C4E116068F for ; Wed, 21 Feb 2024 09:50:07 +0000 (UTC) X-FDA: 81815340054.22.2FD00F1 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf01.hostedemail.com (Postfix) with ESMTP id 5F18940003 for ; Wed, 21 Feb 2024 09:50:05 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708509005; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=9ZiTPQysUQHlJ1BXJmNuI2s3qOwAjRRUQkbxY4APPts=; b=qAPjo9Ks0hkVmWmpt70DCYKFg0TAL7lbQ7gHv9kGoosvAPZW2DkMzE0sOZJUQbSSdEam4T k5IT+EPJGVxHIZu2hlSJ1KWXfqZunFDyED6Rax3P0w65ueOYr388Kd1VgTHBU9k69RqwzV R2EqDBojV4LbT1MxWWtY4hvkpA9hhuk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708509005; a=rsa-sha256; cv=none; b=GML6d5b5UFIvvYhpQA4+ZDtedNjEFEI1abqrXbvbDIUmQBN539wVMm38eHPEBnIo0uVqLS /eK5v775E6sPqCIXTFtbIFft+gWHKfQv8vf3QOwBMyDIoDHKnXRslAksUCmQkrZ21BE3Tx 2T5KQqUC97EKzEPiNyIN4Mbpt9mraVQ= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none X-AuditID: a67dfc5b-d85ff70000001748-f8-65d5c73b58e0 From: Byungchul Park To: linux-kernel@vger.kernel.org Cc: kernel_team@skhynix.com, torvalds@linux-foundation.org, damien.lemoal@opensource.wdc.com, linux-ide@vger.kernel.org, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, will@kernel.org, tglx@linutronix.de, rostedt@goodmis.org, joel@joelfernandes.org, sashal@kernel.org, daniel.vetter@ffwll.ch, duyuyang@gmail.com, johannes.berg@intel.com, tj@kernel.org, tytso@mit.edu, willy@infradead.org, david@fromorbit.com, amir73il@gmail.com, gregkh@linuxfoundation.org, kernel-team@lge.com, linux-mm@kvack.org, akpm@linux-foundation.org, mhocko@kernel.org, minchan@kernel.org, hannes@cmpxchg.org, vdavydov.dev@gmail.com, sj@kernel.org, jglisse@redhat.com, dennis@kernel.org, cl@linux.com, penberg@kernel.org, rientjes@google.com, vbabka@suse.cz, ngupta@vflare.org, linux-block@vger.kernel.org, josef@toxicpanda.com, linux-fsdevel@vger.kernel.org, viro@zeniv.linux.org.uk, jack@suse.cz, jlayton@kernel.org, dan.j.williams@intel.com, hch@infradead.org, djwong@kernel.org, dri-devel@lists.freedesktop.org, rodrigosiqueiramelo@gmail.com, melissa.srw@gmail.com, hamohammed.sa@gmail.com, 42.hyeyoo@gmail.com, chris.p.wilson@intel.com, gwan-gyeong.mun@intel.com, max.byungchul.park@gmail.com, boqun.feng@gmail.com, longman@redhat.com, hdanton@sina.com, her0gyugyu@gmail.com Subject: [PATCH v12 23/27] dept: Make Dept able to work with an external wgen Date: Wed, 21 Feb 2024 18:49:29 +0900 Message-Id: <20240221094933.36348-24-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240221094933.36348-1-byungchul@sk.com> References: <20240221094933.36348-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAAzWSbUiTaxjHvZ93l6uHFXSfCspFHSgtjYyL3qHC+4sRRcU5B6qVT22k5pnv QaC5ymbKKtRKyWm2lq5cm5GWK1PSlplm0zxhlp7ezJlhbWSul83oy8WP6/r/f58ugVbUsTME TUKypE1QxSk5GSMbDikLX9HSJUW0O0Ph5IkI8HzOYaCk2sJBx9UqBJaaLAoG70XDU68bwXhb Ow1FBR0Iyvqf01DT3IfAYT7MwZNXk8HlGeHAWZDLQfaFag4eD/ko6C08RUGVLQZaDeUUNIy9 ZaBokIPiomzKP95RMGaq5MGUOQ8GzOd48PVHgrOvmwXHs4Vw9nwvB/UOJwPNtQMUPLlZwkGf 5QcLrc33Geg4mcfClQ/lHAx5TTSYPCM8dDYYKbDq/KKjn76z0JLXQMHRimsUuP67heB2zksK bJZuDpo8bgrstgIavl66h2Agf5iHIyfGeCjOykeQe6SQgfZvLSzoeqNg/EsJt3Y5aXKP0ERn TyMOr5EhD8oxqTv3nCe62894YrSlELt5AblQP0iRslEPS2yVxzliGz3FE/2wiyIfHj3iyf0z 4wx55SqiNs38W7YyVorTpEraxat3ydQ177roRP2a9BrDZToTuZfokSBgcSnObdquR8ETaO1/ QQeYE//EPT1jEzxNnIPteW9YPZIJtHhsEjZ/bOMC3aliDL7j2xrIMOI8XNuqYwMsF5fhToeT ++WcjausDROeYP/+crF7IqMQo3DX4+t0wInF7GA8aO1lfhX+wHfNPYwByY0oqBIpNAmp8SpN 3NJF6owETfqiPQfibcj/UKZDvn9q0WjHlkYkCkgZIlffcEkKVpWalBHfiLBAK6fJmTT/Sh6r yjgoaQ/s1KbESUmNaKbAKKfLl3jTYhXiPlWytF+SEiXt7yslBM/IRDnC3L3WVf/X/wthrF2L S3emXAqfv3kyHxKtrugDb/JGRZtx2a0pkV9UJr3za4ki76FRsWn2ML5rmBSbs3zL7qD1WU99 qa/r5xqMn2J2hG4I73mR/iO6qv/0SxIRtC2/Oz10wHIxNKqzujus9P3DqxVD1xL/qsPyWTLj uoqwxYX5SiZJrYpcQGuTVD8BAivVvEwDAAA= X-Brightmail-Tracker: H4sIAAAAAAAAAzWSa0iTYRiGe9/v6Gr1saQ+LDAGHehokfWA0eFXb0UHiggq0JUfbXnIttKM AstZNrW0UEstbNY0XWnfDK2cLcXDksx0WtkylU7SzE6TTDtMoz8PF/d9c/16eEqVxwTwuuhD kj5aE6lmFbRiU0jigpCGdimoOmM8ZKQGgfd7Mg15pVYWWm6VILCWn8DQV7cWng16EAw/fkJB dmYLgqs9rygor+9CYC86yULbm4ng8g6w4MxMYSGxoJSFpx9HMLizzmMokTdCU7oZg2PoPQ3Z fSzkZidi3/mAYchSzIElYSb0FuVwMNKzGJxdHQzUXnYyYO+cB5euuFmosjtpqK/sxdB2L4+F LusfBprqG2loyUhj4OYnMwsfBy0UWLwDHLQ68jGUGX22U99+M9CQ5sBw6tptDK4X9xFUJ3dj kK0dLNR6PRhsciYFPwvrEPSe7ecgKXWIg9wTZxGkJGXR8ORXAwNGdzAM/8hjV4eQWs8ARYy2 OGIfzKfJI7NI7ua84oixupMj+fJhYiuaSwqq+jC5+tXLELn4DEvkr+c5Yup3YfKpuZkjjReH afLGlY23TN+pWBEuRepiJf2ilWEKbfmHdirGtOpIefoNKgF5lpiQHy8KS8WyntfUKLPCbPH5 86Ex9hdmiLa0d4wJKXhKOD1eLPr8mDUhnp8sbBQfjGwf3dDCTLGyyciMslJYJrbanew/Z6BY UuYY8/j58hu5nrGNSggW25/eodKRIh+NK0b+uujYKI0uMnihIUIbH607snDvgSgZ+X7Gcnwk oxJ9b1tbgwQeqScotRUuScVoYg3xUTVI5Cm1v5KO80XKcE38UUl/IFR/OFIy1KBpPK2eqly/ QwpTCfs0h6QISYqR9P9bzPsFJKCC+bjWYEld1TrjS9LWKmtWDB8Qvr9h/rYLok1rthWuuF5j fVbROC+BCw1NnDPLPWhL7lC57X86u5sm9QtTutdvWPcyyxwfWBoTVyg99AY6oHlNjpx07Jx6 e1jZns0VdZ/lgrSI5fdbi5cEBO1etFLY8Dal0x6xg7ywHb98cJdlk5o2aDWL51J6g+Yv+4ut jC8DAAA= X-CFilter-Loop: Reflected X-Stat-Signature: 7iy68bs4crr8ribyq5btfdjxbamm96ni X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 5F18940003 X-Rspam-User: X-HE-Tag: 1708509005-541203 X-HE-Meta: U2FsdGVkX1+cSsgm19Uo5kLtAFyvr9Kk2eRB9gu+5HMqgPL9u9B+fzoZ0s0neJG3B+lJFiJM+YERMAtRh1uWpF354pIQoqO51HrhLl/whshOdvbsOu/cKXRHCqRw+6GXuB8yPQYOQcqYhv4b+Q56iiojXrHjcWwt57cO9nza18jmr6XcKWZiHYaWyPCocueOyFzLf9plvf3eimo8e1XL8fmf/zwkommibwjko75oDjCiex4ErWf8v3ng4coXpYkDeQpfEXsAu87eqr+NclcSW5fQtustLf6EOdtrtoboyqgHATb8uQbho2xFyuiDIwI5fyZSXmcx0X2fb4c2IVjFrI4O1f5yto3HxzgPfI4tzMD2XSUGs7sZjNsl9i+YVvoZI+ANwFfuvT51xKtI2aDrEYS0as/k6pj5lmcN3i9hJMddaHTPNR8s08mrWvwhISKsXHVDu5z8Cr/6tKQOffuCsSwIgM6j8b6pyRS5naQJ7iyWqmeT0SaJmePvLjmVOJkSaeb3FYO8kzoy9cZDvoiQv7eqT+iqBPNL/Cvo1LxQxZmJkhWGg6AGosvfsGJTzZUcWteHdvZl8+Zl0oxgQ0sHcqIhJJQuyvNKCjsYNn3ZTjDIwftpJANNX5FZ+PUYA+m74jxvb5rqBFqI1v0RD2fuyxdoki2assWZE4x5R2dnYOdyJloRDjwy4n3eicC1w947/nkWy9mqwidspjS+gD9s7hca/xvP9ShRMBbUDuIJ/0ksXM2OtC5V3ZbosCbNDiI83m2AOnI0MfE4sT3ReD+R1nZvR1WFKnhytH7RYFG89E+xAhtADrF9q1LLhzcYubRb/fHQzxSYEHRjPjXpaoounhk63naA9V4QFujcbbsOKMDGldGfO0rpZicq1HZGEK0t9mYpCC0PNUlgZvHm0Mi1XbKB1HNxnq5AASm5OG2smWq2AvyDJOTj5KTNRqW6LxykpWsB2jRCSyzMUh9Sk6T cELTz/7B 1URt6P8fuxY5eSSxUKkNvfmj1UmB8r/Plspoufl6U9I4bHjKEM1CVKOQJtIOXapbX+/IR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is a case where total maps for its wait/event is so large in size. For instance, struct page for PG_locked and PG_writeback is the case. The additional memory size for the maps would be 'the # of pages * sizeof(struct dept_map)' if each struct page keeps its map all the way, which might be too big to accept. It'd be better to keep the minimum data in the case, which is timestamp called 'wgen' that Dept makes use of. So made Dept able to work with an external wgen when needed. Signed-off-by: Byungchul Park --- include/linux/dept.h | 18 ++++++++++++++---- include/linux/dept_sdt.h | 4 ++-- kernel/dependency/dept.c | 30 +++++++++++++++++++++--------- 3 files changed, 37 insertions(+), 15 deletions(-) diff --git a/include/linux/dept.h b/include/linux/dept.h index 0280e45cc2af..dea53ad5b356 100644 --- a/include/linux/dept.h +++ b/include/linux/dept.h @@ -482,6 +482,13 @@ struct dept_task { bool in_sched; }; +/* + * for subsystems that requires compact use of memory e.g. struct page + */ +struct dept_ext_wgen{ + unsigned int wgen; +}; + #define DEPT_TASK_INITIALIZER(t) \ { \ .wait_hist = { { .wait = NULL, } }, \ @@ -512,6 +519,7 @@ extern void dept_task_exit(struct task_struct *t); extern void dept_free_range(void *start, unsigned int sz); extern void dept_map_init(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); extern void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, const char *n); +extern void dept_ext_wgen_init(struct dept_ext_wgen *ewg); extern void dept_map_copy(struct dept_map *to, struct dept_map *from); extern void dept_wait(struct dept_map *m, unsigned long w_f, unsigned long ip, const char *w_fn, int sub_l, long timeout); @@ -521,8 +529,8 @@ extern void dept_clean_stage(void); extern void dept_stage_event(struct task_struct *t, unsigned long ip); extern void dept_ecxt_enter(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *c_fn, const char *e_fn, int sub_l); extern bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f); -extern void dept_request_event(struct dept_map *m); -extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn); +extern void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg); +extern void dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, struct dept_ext_wgen *ewg); extern void dept_ecxt_exit(struct dept_map *m, unsigned long e_f, unsigned long ip); extern void dept_sched_enter(void); extern void dept_sched_exit(void); @@ -551,6 +559,7 @@ extern void dept_hardirqs_off(void); struct dept_key { }; struct dept_map { }; struct dept_task { }; +struct dept_ext_wgen { }; #define DEPT_MAP_INITIALIZER(n, k) { } #define DEPT_TASK_INITIALIZER(t) { } @@ -563,6 +572,7 @@ struct dept_task { }; #define dept_free_range(s, sz) do { } while (0) #define dept_map_init(m, k, su, n) do { (void)(n); (void)(k); } while (0) #define dept_map_reinit(m, k, su, n) do { (void)(n); (void)(k); } while (0) +#define dept_ext_wgen_init(wg) do { } while (0) #define dept_map_copy(t, f) do { } while (0) #define dept_wait(m, w_f, ip, w_fn, sl, t) do { (void)(w_fn); } while (0) @@ -572,8 +582,8 @@ struct dept_task { }; #define dept_stage_event(t, ip) do { } while (0) #define dept_ecxt_enter(m, e_f, ip, c_fn, e_fn, sl) do { (void)(c_fn); (void)(e_fn); } while (0) #define dept_ecxt_holding(m, e_f) false -#define dept_request_event(m) do { } while (0) -#define dept_event(m, e_f, ip, e_fn) do { (void)(e_fn); } while (0) +#define dept_request_event(m, wg) do { } while (0) +#define dept_event(m, e_f, ip, e_fn, wg) do { (void)(e_fn); } while (0) #define dept_ecxt_exit(m, e_f, ip) do { } while (0) #define dept_sched_enter() do { } while (0) #define dept_sched_exit() do { } while (0) diff --git a/include/linux/dept_sdt.h b/include/linux/dept_sdt.h index 21fce525f031..8cdac7982036 100644 --- a/include/linux/dept_sdt.h +++ b/include/linux/dept_sdt.h @@ -24,7 +24,7 @@ #define sdt_wait_timeout(m, t) \ do { \ - dept_request_event(m); \ + dept_request_event(m, NULL); \ dept_wait(m, 1UL, _THIS_IP_, __func__, 0, t); \ } while (0) #define sdt_wait(m) sdt_wait_timeout(m, -1L) @@ -49,7 +49,7 @@ #define sdt_might_sleep_end() dept_clean_stage() #define sdt_ecxt_enter(m) dept_ecxt_enter(m, 1UL, _THIS_IP_, "start", "event", 0) -#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__) +#define sdt_event(m) dept_event(m, 1UL, _THIS_IP_, __func__, NULL) #define sdt_ecxt_exit(m) dept_ecxt_exit(m, 1UL, _THIS_IP_) #else /* !CONFIG_DEPT */ #define sdt_map_init(m) do { } while (0) diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c index 5c996f11abd5..fb33c3758c25 100644 --- a/kernel/dependency/dept.c +++ b/kernel/dependency/dept.c @@ -2186,6 +2186,11 @@ void dept_map_reinit(struct dept_map *m, struct dept_key *k, int sub_u, } EXPORT_SYMBOL_GPL(dept_map_reinit); +void dept_ext_wgen_init(struct dept_ext_wgen *ewg) +{ + ewg->wgen = 0U; +} + void dept_map_copy(struct dept_map *to, struct dept_map *from) { if (unlikely(!dept_working())) { @@ -2371,7 +2376,7 @@ static void __dept_wait(struct dept_map *m, unsigned long w_f, */ static void __dept_event(struct dept_map *m, unsigned long e_f, unsigned long ip, const char *e_fn, - bool sched_map) + bool sched_map, unsigned int wg) { struct dept_class *c; struct dept_key *k; @@ -2393,7 +2398,7 @@ static void __dept_event(struct dept_map *m, unsigned long e_f, c = check_new_class(&m->map_key, k, sub_id(m, e), m->name, sched_map); if (c && add_ecxt(m, c, 0UL, NULL, e_fn, 0)) { - do_event(m, c, READ_ONCE(m->wgen), ip); + do_event(m, c, wg, ip); pop_ecxt(m, c); } } @@ -2606,7 +2611,7 @@ void dept_stage_event(struct task_struct *requestor, unsigned long ip) if (!m.keys) goto exit; - __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map); + __dept_event(&m, 1UL, ip, "try_to_wake_up", sched_map, m.wgen); exit: dept_exit(flags); } @@ -2785,10 +2790,11 @@ bool dept_ecxt_holding(struct dept_map *m, unsigned long e_f) } EXPORT_SYMBOL_GPL(dept_ecxt_holding); -void dept_request_event(struct dept_map *m) +void dept_request_event(struct dept_map *m, struct dept_ext_wgen *ewg) { unsigned long flags; unsigned int wg; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2801,21 +2807,25 @@ void dept_request_event(struct dept_map *m) */ flags = dept_enter_recursive(); + wg_p = ewg ? &ewg->wgen : &m->wgen; + /* * Avoid zero wgen. */ wg = atomic_inc_return(&wgen) ?: atomic_inc_return(&wgen); - WRITE_ONCE(m->wgen, wg); + WRITE_ONCE(*wg_p, wg); dept_exit_recursive(flags); } EXPORT_SYMBOL_GPL(dept_request_event); void dept_event(struct dept_map *m, unsigned long e_f, - unsigned long ip, const char *e_fn) + unsigned long ip, const char *e_fn, + struct dept_ext_wgen *ewg) { struct dept_task *dt = dept_task(); unsigned long flags; + unsigned int *wg_p; if (unlikely(!dept_working())) return; @@ -2823,24 +2833,26 @@ void dept_event(struct dept_map *m, unsigned long e_f, if (m->nocheck) return; + wg_p = ewg ? &ewg->wgen : &m->wgen; + if (dt->recursive) { /* * Dept won't work with this even though an event * context has been asked. Don't make it confused at * handling the event. Disable it until the next. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); return; } flags = dept_enter(); - __dept_event(m, e_f, ip, e_fn, false); + __dept_event(m, e_f, ip, e_fn, false, READ_ONCE(*wg_p)); /* * Keep the map diabled until the next sleep. */ - WRITE_ONCE(m->wgen, 0U); + WRITE_ONCE(*wg_p, 0U); dept_exit(flags); }