From patchwork Fri Mar 14 17:19:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gao Xiang X-Patchwork-Id: 14017194 Received: from out30-110.freemail.mail.aliyun.com (out30-110.freemail.mail.aliyun.com [115.124.30.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A234C78F2E for ; Fri, 14 Mar 2025 17:24:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.110 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741973095; cv=none; b=byFDHezVQsfShnnMHCDTUZki6+z35fHdR/szg4WlQpKDpl5G0b1bicrQ83pIfK2rgpSrU9i5IiSc1UNGAjE6uRqzh6hqqfzozpM2CyuiHZY4CTe6nSlG8ZL5+WmU393WmkV2Yq7kLoLKuVysHHTWI5spysDrstZDNr87oj3Cq5M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1741973095; c=relaxed/simple; bh=V/k3PwxUGOBJgikYinGyYixoa63fPzcJdljT8OQG//Q=; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject:Cc; b=L16Cu+RDEtL6XlVNGw5JlzJgSjX7Edwj8D/5Ugdy3C/YBuLh0ndpKwiK9gh7HGs6V9B+ny6E7mAZUttRzKskWK+8BebYtywyzdumnGgjHs/dS+Nc7mhCfP35UyZdPS4xckofFEuIMSEZuKDJJ0aF+cP34l41sZnu8vpAUmGQOZQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b=XTS35GQM; arc=none smtp.client-ip=115.124.30.110 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.alibaba.com header.i=@linux.alibaba.com header.b="XTS35GQM" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.alibaba.com; s=default; t=1741973089; h=Content-Type:Message-ID:Date:MIME-Version:To:From:Subject; bh=9ejQNoWW8rrRz3dF6/Acg0fk6alGD1FZ1xHq49BOt6w=; b=XTS35GQMW4cbFC/i1KnqBaGHh0UhPW4Fzx+ijfKhGzy8/8j8E1aP/WN/CzaSCInIcbxl6dzTM34oauStu9qSsVTyMDr2/DcflnuEPHQ4nMtuwQEAAOCjLMqO6OUerm1ZDvKoLrIAvjWrIPYCpS88QZZ39UvYWwEee9tMZ3L/0uU= Received: from 30.134.66.95(mailfrom:hsiangkao@linux.alibaba.com fp:SMTPD_---0WRM1JEe_1741972772 cluster:ay36) by smtp.aliyun-inc.com; Sat, 15 Mar 2025 01:19:32 +0800 Message-ID: <0849fc77-1a6e-46f8-a18d-15699f99158e@linux.alibaba.com> Date: Sat, 15 Mar 2025 01:19:31 +0800 Precedence: bulk X-Mailing-List: linux-xfs@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: linux-xfs From: Gao Xiang Subject: [report] Unixbench shell1 performance regression Cc: Zorro Lang Hi folks, Days ago, I received a XFS Unixbench[1] shell1 (high-concurrency) performance regression during a benchmark comparison between XFS and EXT4: The XFS result was lower than EXT4 by 15% on Linux 6.6.y with 144-core aarch64 (64K page size). Since Unixbench is somewhat important to indicate overall system performance for many end users, it's not a good result. shell1 test[2] basically runs in a loop that it executes commands to generate files (sort.$$, od.$$, grep.$$, wc.$$) and then remove them. The testcase lasts for one minute and then show the total number of iterations. While no difference was observed in single-threaded results, it showed a noticeable difference above if `./Run shell1 -c 144 -i 1` is used. The original report was on aarch64, but I could still reproduce some difference on Linux 6.13 with a X86 physical machine: Intel(R) Xeon(R) Platinum 8331C CPU @ 2.50GHz * 96 cores 512 GiB memory XFS (35649.6) is still lower than EXT4 (37146.0) by 4% and the kconfig is attached. However, I don't observe much difference on 5.10.y kernels. After collecting some off-CPU trace, I found there are many new agi buf lock waits compared with the correspoinding 5.10.y trace, as below: rm;el0t_64_sync;el0t_64_sync_handler;el0_svc;do_el0_svc;el0_svc_common.constprop.0;__arm64_sys_unlinkat;do_unlinkat;vfs_unlink;xfs_vn_unlink;xfs_remove;xfs_droplink;xfs_iunlink;xfs_read_agi;xfs_trans_read_buf_map;xfs_buf_read_map;xfs_buf_get_map;xfs_buf_lookup;xfs_buf_find_lock;xfs_buf_lock;down;__down;__down_common;___down_common;schedule_timeout;schedule;finish_task_switch.isra.0 2 .. rm;el0t_64_sync;el0t_64_sync_handler;el0_svc;do_el0_svc;el0_svc_common.constprop.0;__arm64_sys_unlinkat;do_unlinkat;vfs_unlink;xfs_vn_unlink;xfs_remove;xfs_droplink;xfs_iunlink;xfs_read_agi;xfs_trans_read_buf_map;xfs_buf_read_map;xfs_buf_get_map;xfs_buf_lookup;xfs_buf_find_lock;xfs_buf_lock;down;__down;__down_common;___down_common;schedule_timeout;schedule;finish_task_switch.isra.0 2 .. kworker/62:1;ret_from_fork;kthread;worker_thread;process_one_work;xfs_inodegc_worker;xfs_inodegc_inactivate;xfs_inactive;xfs_inactive_ifree;xfs_ifree;xfs_difree;xfs_ialloc_read_agi;xfs_read_agi;xfs_trans_read_buf_map;xfs_buf_read_map;xfs_buf_get_map;xfs_buf_lookup;xfs_buf_find_lock;xfs_buf_lock;down;__down;__down_common;___down_common;schedule_timeout;schedule;finish_task_switch.isra.0 5283 .. I tried to do some hack to disable defer inode inactivation as below, the shell1 benchmark then recovered: XFS (35649.6 -> 37810.9): I don't have extra slot for now, but hopefully this report could be useful ;) thanks! Thanks, Gao Xiang [1] https://github.com/kdlucas/byte-unixbench [2] https://github.com/kdlucas/byte-unixbench/blob/master/UnixBench/pgms/tst.sh diff --git a/fs/xfs/xfs_icache.c b/fs/xfs/xfs_icache.c index 7b6c026d01a1..d9fb2ef3686a 100644 --- a/fs/xfs/xfs_icache.c +++ b/fs/xfs/xfs_icache.c @@ -2059,6 +2059,7 @@ void xfs_inodegc_start( struct xfs_mount *mp) { + return; if (xfs_set_inodegc_enabled(mp)) return; @@ -2180,6 +2181,12 @@ xfs_inodegc_queue( ip->i_flags |= XFS_NEED_INACTIVE; spin_unlock(&ip->i_flags_lock); + if (1) { + xfs_iflags_set(ip, XFS_INACTIVATING); + xfs_inodegc_inactivate(ip); + return; + } + cpu_nr = get_cpu(); gc = this_cpu_ptr(mp->m_inodegc); llist_add(&ip->i_gclist, &gc->list);