From patchwork Thu Apr 10 01:49:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 14045750 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29F10C369A2 for ; Thu, 10 Apr 2025 01:50:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E4416B0250; Wed, 9 Apr 2025 21:49:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A94D280064; Wed, 9 Apr 2025 21:49:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA84F6B024E; Wed, 9 Apr 2025 21:49:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 668CE6B02A5 for ; Wed, 9 Apr 2025 21:49:57 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 99C0FB7E10 for ; Thu, 10 Apr 2025 01:49:58 +0000 (UTC) X-FDA: 83316453276.06.D2EE007 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf04.hostedemail.com (Postfix) with ESMTP id 227F04000D for ; Thu, 10 Apr 2025 01:49:56 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=ewoFzbvy; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine); spf=none (imf04.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744249797; a=rsa-sha256; cv=none; b=NOFsSnsSXEc9Ncr4IjS9PsNAAUr/ekE0K8ff9+Dj5RvkWI7+8JTwJngAumsGQLvbUvzpWY TZP5woghUDiYsDfe5aWuEAn59BsX18hhPswwKsmgUe+xe7NAISCz5T7/muo0qKiLbih+9r ihwQzKuYgLDqMLm31GKHBj/ifDxnrwE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=ewoFzbvy; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=quarantine); spf=none (imf04.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744249797; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lo+mNNCNF0Kcn2O97aAamvTUH5kTxQV3YmJyIlBZZYM=; b=zYhkgSkyKeT6EdlgLAmraaot+6JtbFpyklFr1sY449rCG8CVSi0TlSZfsCpByx6oLNnlVI +T9+m0N0XTNcf6FxFT2kaqM1o100yrRqdIMQD+BsjqGTDTB11FZeHZH10zU8BZryc4Zzy6 4OJZNQvtCSqO9T2TKtUkZjhnFdqa2Aw= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=Lo+mNNCNF0Kcn2O97aAamvTUH5kTxQV3YmJyIlBZZYM=; b=ewoFzbvyobG0uDxAjVoFDwjHWk WuZ/3S9yAjfKR2FmAUSEeXkkrkoEv+JUrk7lOQjdFVWExzB7lNCFT/WwVAeEYTqA/sy2xN3HQia6i /ghOVN2nNtzpBl+oOAfTpEl6qx2iF9VVGH+99o+w8iKGr0xEC5ojaCJ2SSQGhESiqnCkB87He/3r8 G+O96d8oiH4BoJHjpRwltFGCAvF/VC5EJEDVr5tvE1SIwBCillDTlLlrPdDeGASJydAFVQnvBxvOZ YZQBbE6LPTYlsCm4iSU/5Xr+xRWoVGrXbzvRonVbCbxo2U5PhfpawzAmeQ726cHZZIDKTeO97pgca Rfa3HOOg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98.2 #2 (Red Hat Linux)) id 1u2h36-00000008yvU-0WmT; Thu, 10 Apr 2025 01:49:48 +0000 From: Luis Chamberlain To: brauner@kernel.org, jack@suse.cz, tytso@mit.edu, adilger.kernel@dilger.ca, linux-ext4@vger.kernel.org, riel@surriel.com Cc: dave@stgolabs.net, willy@infradead.org, hannes@cmpxchg.org, oliver.sang@intel.com, david@redhat.com, axboe@kernel.dk, hare@suse.de, david@fromorbit.com, djwong@kernel.org, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org, linux-mm@kvack.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, mcgrof@kernel.org Subject: [PATCH v2 8/8] mm: add migration buffer-head debugfs interface Date: Wed, 9 Apr 2025 18:49:45 -0700 Message-ID: <20250410014945.2140781-9-mcgrof@kernel.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250410014945.2140781-1-mcgrof@kernel.org> References: <20250410014945.2140781-1-mcgrof@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 227F04000D X-Stat-Signature: jq9yjg1j974gy9aoba6c7siszue5rfqu X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Pre-Result: action=add header; module=dmarc; Action set by DMARC X-Rspam: Yes X-HE-Tag: 1744249796-793728 X-HE-Meta: U2FsdGVkX1/p9FGoKJCk0JU8vm85cvCR9dYLA55rUgmxrIaaTM/6ZqhNAM9TpFqqNbBzI/osbLIJUyGYfsENzbNUP8b8aw/ALBAxsmjtea0ZaSdAVY2dPsE49hI8fnYPWigsbpMRXPJOLiBWMS7mokbW19rs7KmlwNJU2V7j3WFGJ03iZ/ZFiRsf34TUFpBLHRzwU6Su5h3vEn10ANQRR6GNxKydn9pMPUYO5Ww5NHXgpyTHn1KRuW3qhSPJdilgR67UhlbZiICP7LcxkGgCG0Jy9zEFHABSKkK6dgTgxyvyhN4xvatBPuKigFqqDeF1AXlEmE6qKg6bKd0Dfz4yAm1sF9ZjVVrucJt0a9PoLwtH5xXbZOJSfB5qnPqtqTf0M/S2R63kMvbbY42gpKuAvvbX3u7ci+TGhcs8pU6FBrTLzCOLqA7haZAVY7+tqvb9YtEpC9xS3vuiTbLWJeBQidOQY8M4Q+eVmgJuip9N7NPqSr2uEpiPvEPyURsR/lWimjgJo5/rbo+wk4YyREBRZWrpFgm7V1lTSYaTGBjTH49x594IN3/pTrd7g9t90AGBgf/1b61OTgb6ug4Tpts4mM0z/o4/l+EGNlOoc5TMKqWxa+Buf27uaojv0rG5BbuHFnESuh9NnD6D5118n8dfDz58ZNR8COgaXz1pDwXh8EwisFxPXRTwV4gizQB4qPggkuxrpQG3GSWNku/RXqfSB8wafUwUryRsvi1V8Y9Sdb/I8xHteiHNPi2VZFCdn+RcAqT9aGqogd/HsNaiAYrPjuq+JNFXvb0DJ5hDrAiagvESgi+hvU/3q4npgD9PdSKHMNERFAfeQGcZekvwZ9/j9N61O144MGDla+JgPGsj1pHnNJCzLnLWjELEF5mF4CyjcPyAWPLj97Xun8LgG6FtTnNMZiV2fjvEwSrs0wdGDZpv/ZMcj39TJ3MPzi2Se2MQYCCDXCCjrIIPASrJ36v 3yLgvgoh SFnYMk03O4vWW/6fG+07qkT7FUsm9d4SL9iNUSQQx46/q5XRfzwhjj5RDAeUddIxP1miJJ96RgfEX5S4xWAipPwnru6J7E1YSJkNrQn34guSc4E8gaK7QhZWYzSxkDMhmF5MzbcEi9tGGRx4tnNEVSoEIZhdC4QPL+sGLrcjoC9cop7xeO0kELDt4KZ7tfqxzLjhphumYaJwkpEz7jev16/e5R/FPGLOcJSGT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If you are working on enhancing folio migration it is easy to not be certain on improvements. This debugfs interface enables you to evaluate gains on improvements on buffer-head folio migration. This can easily tell you *why* folio migration might fail, for example, here is the output of a generic/750 run for 18 hours: root@e3-ext4-2k ~ # cat /sys/kernel/debug/mm/migrate/bh/stats [buffer_migrate_folio] calls 50160811 success 50047572 fails 113239 [buffer_migrate_folio_norefs] calls 23577082468 success 2939858 fails 23574142610 jbd-meta 23425956714 no-head-success 102 no-head-fails 0 invalid 147919982 valid 2939881 valid-success 2939756 valid-fails 125 Success ratios: buffer_migrate_folio: 99% success (50047572/50160811) buffer_migrate_folio_norefs: 0% success (2939858/23577082468) Signed-off-by: Luis Chamberlain --- mm/migrate.c | 184 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 178 insertions(+), 6 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 8fed2655f2e8..c478e8218cb0 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -44,6 +44,7 @@ #include #include #include +#include #include @@ -791,6 +792,126 @@ int migrate_folio(struct address_space *mapping, struct folio *dst, EXPORT_SYMBOL(migrate_folio); #ifdef CONFIG_BUFFER_HEAD + +static const char * const bh_routine_names[] = { + "buffer_migrate_folio", + "buffer_migrate_folio_norefs", +}; + +#define BH_STATS(X) \ + X(bh_migrate_folio, 0, "calls") \ + X(bh_migrate_folio_success, 0, "success") \ + X(bh_migrate_folio_fails, 0, "fails") \ + X(bh_migrate_folio_norefs, 1, "calls") \ + X(bh_migrate_folio_norefs_success, 1, "success") \ + X(bh_migrate_folio_norefs_fails, 1, "fails") \ + X(bh_migrate_folio_norefs_meta, 1, "jbd-meta") \ + X(bh_migrate_folio_norefs_nohead_success, 1, "no-head-success") \ + X(bh_migrate_folio_norefs_nohead_fails, 1, "no-head-fails") \ + X(bh_migrate_folio_norefs_invalid, 1, "invalid") \ + X(bh_migrate_folio_norefs_valid, 1, "valid") \ + X(bh_migrate_folio_norefs_valid_success, 1, "valid-success") \ + X(bh_migrate_folio_norefs_valid_fails, 1, "valid-fails") + + +#define DECLARE_STAT(name, routine_idx, meaning) static atomic_long_t name; +BH_STATS(DECLARE_STAT) + +#define BH_STAT_PTR(name, routine_idx, meaning) &name, +static atomic_long_t * const bh_stat_array[] = { + BH_STATS(BH_STAT_PTR) +}; + +#define BH_STAT_ROUTINE_IDX(name, routine_idx, meaning) routine_idx, +static const int bh_stat_routine_index[] = { + BH_STATS(BH_STAT_ROUTINE_IDX) +}; + +#define BH_STAT_MEANING(name, routine_idx, meaning) meaning, +static const char * const bh_stat_meanings[] = { + BH_STATS(BH_STAT_MEANING) +}; + +#define NUM_BH_STATS ARRAY_SIZE(bh_stat_array) + +static ssize_t read_file_bh_migrate_stats(struct file *file, + char __user *user_buf, + size_t count, loff_t *ppos) +{ + char *buf; + unsigned int i, len = 0, size = NUM_BH_STATS * 128; + int ret, last_routine = -1; + unsigned long total, success, rate; + + BUILD_BUG_ON(ARRAY_SIZE(bh_stat_array) != ARRAY_SIZE(bh_stat_meanings)); + + buf = kzalloc(size, GFP_KERNEL); + if (buf == NULL) + return -ENOMEM; + + for (i = 0; i < NUM_BH_STATS; i++) { + int routine_idx = bh_stat_routine_index[i]; + + if (routine_idx != last_routine) { + len += scnprintf(buf + len, size - len, "\n[%s]\n", + bh_routine_names[routine_idx]); + last_routine = routine_idx; + } + + len += scnprintf(buf + len, size - len, "%25s\t%lu\n", + bh_stat_meanings[i], + atomic_long_read(bh_stat_array[i])); + + } + + len += scnprintf(buf + len, size - len, "\nSuccess ratios:\n"); + + total = atomic_long_read(&bh_migrate_folio); + success = atomic_long_read(&bh_migrate_folio_success); + rate = total ? (success * 100) / total : 0; + len += scnprintf(buf + len, size - len, + "%s: %lu%% success (%lu/%lu)\n", + "buffer_migrate_folio", rate, success, total); + + total = atomic_long_read(&bh_migrate_folio_norefs); + success = atomic_long_read(&bh_migrate_folio_norefs_success); + rate = total ? (success * 100) / total : 0; + len += scnprintf(buf + len, size - len, + "%s: %lu%% success (%lu/%lu)\n", + "buffer_migrate_folio_norefs", rate, success, total); + + ret = simple_read_from_buffer(user_buf, count, ppos, buf, len); + kfree(buf); + return ret; +} + +static const struct file_operations fops_bh_migrate_stats = { + .read = read_file_bh_migrate_stats, + .open = simple_open, + .owner = THIS_MODULE, + .llseek = default_llseek, +}; + +static void mm_migrate_bh_init(struct dentry *migrate_debug_root) +{ + struct dentry *parent_dirs[ARRAY_SIZE(bh_routine_names)] = { NULL }; + struct dentry *root = debugfs_create_dir("bh", migrate_debug_root); + int i; + + for (i = 0; i < ARRAY_SIZE(bh_routine_names); i++) + parent_dirs[i] = debugfs_create_dir(bh_routine_names[i], root); + + for (i = 0; i < NUM_BH_STATS; i++) { + int routine = bh_stat_routine_index[i]; + debugfs_create_ulong(bh_stat_meanings[i], 0400, + parent_dirs[routine], + (unsigned long *) + &bh_stat_array[i]->counter); + } + + debugfs_create_file("stats", 0400, root, root, &fops_bh_migrate_stats); +} + /* Returns true if all buffers are successfully locked */ static bool buffer_migrate_lock_buffers(struct buffer_head *head, enum migrate_mode mode) @@ -833,16 +954,26 @@ static int __buffer_migrate_folio(struct address_space *mapping, int expected_count; head = folio_buffers(src); - if (!head) - return migrate_folio(mapping, dst, src, mode); + if (!head) { + rc = migrate_folio(mapping, dst, src, mode); + if (check_refs) { + if (rc == 0) + atomic_long_inc(&bh_migrate_folio_norefs_nohead_success); + else + atomic_long_inc(&bh_migrate_folio_norefs_nohead_fails); + } + return rc; + } /* Check whether page does not have extra refs before we do more work */ expected_count = folio_expected_refs(mapping, src); if (folio_ref_count(src) != expected_count) return -EAGAIN; - if (buffer_meta(head)) + if (buffer_meta(head)) { + atomic_long_inc(&bh_migrate_folio_norefs_meta); return -EAGAIN; + } if (!buffer_migrate_lock_buffers(head, mode)) return -EAGAIN; @@ -868,17 +999,23 @@ static int __buffer_migrate_folio(struct address_space *mapping, if (busy) { if (invalidated) { rc = -EAGAIN; + atomic_long_inc(&bh_migrate_folio_norefs_invalid); goto unlock_buffers; } invalidate_bh_lrus(); invalidated = true; goto recheck_buffers; } + atomic_long_inc(&bh_migrate_folio_norefs_valid); } rc = filemap_migrate_folio(mapping, dst, src, mode); - if (rc != MIGRATEPAGE_SUCCESS) + if (rc != MIGRATEPAGE_SUCCESS) { + if (check_refs) + atomic_long_inc(&bh_migrate_folio_norefs_valid_fails); goto unlock_buffers; + } else if (check_refs) + atomic_long_inc(&bh_migrate_folio_norefs_valid_success); bh = head; do { @@ -915,7 +1052,16 @@ static int __buffer_migrate_folio(struct address_space *mapping, int buffer_migrate_folio(struct address_space *mapping, struct folio *dst, struct folio *src, enum migrate_mode mode) { - return __buffer_migrate_folio(mapping, dst, src, mode, false); + int ret; + atomic_long_inc(&bh_migrate_folio); + + ret = __buffer_migrate_folio(mapping, dst, src, mode, false); + if (ret == 0) + atomic_long_inc(&bh_migrate_folio_success); + else + atomic_long_inc(&bh_migrate_folio_fails); + + return ret; } EXPORT_SYMBOL(buffer_migrate_folio); @@ -936,9 +1082,21 @@ EXPORT_SYMBOL(buffer_migrate_folio); int buffer_migrate_folio_norefs(struct address_space *mapping, struct folio *dst, struct folio *src, enum migrate_mode mode) { - return __buffer_migrate_folio(mapping, dst, src, mode, true); + int ret; + + atomic_long_inc(&bh_migrate_folio_norefs); + + ret = __buffer_migrate_folio(mapping, dst, src, mode, true); + if (ret == 0) + atomic_long_inc(&bh_migrate_folio_norefs_success); + else + atomic_long_inc(&bh_migrate_folio_norefs_fails); + + return ret; } EXPORT_SYMBOL_GPL(buffer_migrate_folio_norefs); +#else +static inline void mm_migrate_bh_init(struct dentry *migrate_debug_root) { } #endif /* CONFIG_BUFFER_HEAD */ int filemap_migrate_folio(struct address_space *mapping, @@ -2737,3 +2895,17 @@ int migrate_misplaced_folio(struct folio *folio, int node) } #endif /* CONFIG_NUMA_BALANCING */ #endif /* CONFIG_NUMA */ + +static __init int mm_migrate_debugfs_init(void) +{ + struct dentry *mm_debug_root; + struct dentry *migrate_debug_root; + + mm_debug_root = debugfs_create_dir("mm", NULL); + migrate_debug_root = debugfs_create_dir("migrate", mm_debug_root); + + mm_migrate_bh_init(migrate_debug_root); + + return 0; +} +fs_initcall(mm_migrate_debugfs_init);