From patchwork Sun Aug 27 13:28:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hao Xu X-Patchwork-Id: 13367140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A64BC83F01 for ; Sun, 27 Aug 2023 13:30:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B0D4A940007; Sun, 27 Aug 2023 09:30:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ABDC88E0001; Sun, 27 Aug 2023 09:30:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9858A940007; Sun, 27 Aug 2023 09:30:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8A7918E0001 for ; Sun, 27 Aug 2023 09:30:12 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5BF2C8020B for ; Sun, 27 Aug 2023 13:30:12 +0000 (UTC) X-FDA: 81169968264.09.D9A8372 Received: from out-249.mta1.migadu.com (out-249.mta1.migadu.com [95.215.58.249]) by imf01.hostedemail.com (Postfix) with ESMTP id 8E8FF40017 for ; Sun, 27 Aug 2023 13:30:10 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=D4glas+P; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of hao.xu@linux.dev designates 95.215.58.249 as permitted sender) smtp.mailfrom=hao.xu@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693143010; a=rsa-sha256; cv=none; b=LgyBQn3LxWAOkFOwU54ERggomJPZsXyBuJ7D9mQ+MvkZ+kq+zfC2gWRzPUTk4YFN9OLx0G t33iGRsXpL8u8th/x0RtOv3D5ZQlQbg6MF75wmAcZwosD3ta9S9kEd09MqkAFSsJUP8VYx g9Vc87m8ZXvOcL0o7NW6+rnCPvzPGFw= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=D4glas+P; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of hao.xu@linux.dev designates 95.215.58.249 as permitted sender) smtp.mailfrom=hao.xu@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693143010; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=hH9Z1s3VyDbNI+o0MhVSnq4vtAmGphAu/NP9Lh4Iqho=; b=Lhjdzou1U0QRleq2aG/2720LWHUWDzehWIH4T6pR3t7cEfwKAy2j+6W2977a1niP5WuuPs S/uU0qgknZjq0UVB7RFPL2fs+P/66MCbSerWbch0J82E2MCpWaTXVjQLGOHRAY93begA13 PiOeJonbUvyBlzMP7L6fVbhrq/I9o30= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1693143008; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=hH9Z1s3VyDbNI+o0MhVSnq4vtAmGphAu/NP9Lh4Iqho=; b=D4glas+PskglEfull56Ie/EaUGDVyzdgOX1ZIuTOt4hIdaNgYSKi8HuTaE82KbnTe8VrVg RYQOGm2sDsmgjyTzfYWA88b8swvCAy1S+AvUnlRILYK1CLtNBRiYb+57DIqLl/tE9VNEa0 45zeea98vHJXfwWbVyE91LWfpk5toEg= From: Hao Xu To: io-uring@vger.kernel.org, Jens Axboe Cc: Dominique Martinet , Pavel Begunkov , Christian Brauner , Alexander Viro , Stefan Roesch , Clay Harris , Dave Chinner , "Darrick J . Wong" , linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-ext4@vger.kernel.org, linux-cachefs@redhat.com, ecryptfs@vger.kernel.org, linux-nfs@vger.kernel.org, linux-unionfs@vger.kernel.org, bpf@vger.kernel.org, netdev@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-btrfs@vger.kernel.org, codalist@coda.cs.cmu.edu, linux-f2fs-devel@lists.sourceforge.net, cluster-devel@redhat.com, linux-mm@kvack.org, linux-nilfs@vger.kernel.org, devel@lists.orangefs.org, linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-mtd@lists.infradead.org, Wanpeng Li Subject: [PATCH v6 00/11] io_uring getdents Date: Sun, 27 Aug 2023 21:28:24 +0800 Message-Id: <20230827132835.1373581-1-hao.xu@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 8E8FF40017 X-Stat-Signature: ntge9ymd4jr4t7s1k8h9munxmcmwnq4y X-HE-Tag: 1693143010-673280 X-HE-Meta: U2FsdGVkX18+k+HfG0j40X5xiclueQ4SoaE9A1H+yCmoeBxD4LnLUlRZVdna/gI6dIhB1l6Z/0MfWzlyqYovN6j6vQL6YV7CbcjgmqCzQqVCXd1ofEcbxN/em47MNZlrhr4vGKsWNUD5Yz4hM0W4Eokzss3SJI/ZTyx8y1fZ7aZzuA6Z9Jivcg437voGz5RHC7vlJBdyxhKgi3t9RmkA81YPKD//ks2iAZqHflp8Ca56J2UFzY+K7Z/l9ZbjuQivMN2aOukwjFqNJ41+PnMQKqg653IDkW3A9OnjdxrKdKExeaFD4yoGPvS8J100x8I8Mjo5sHSrC2RxYb4SeDq555Voy1wAoMaHZcjj/nHBfH+0zvbZ7qpzYRiINY7HDgohYeIPvdFr0QguivQM3C0ipDpvNIBnCSv+hdn6aGFSMEGUeYv1cyyw2wgJlJNjJWL4L26+x2NQLzHrRH/xtLXojwn7XTOatZeLpzVfN77X8i1o4K49IfVvKvfbWJMImVxEwpf8mBSyFxPojUw/ct64NQiaBLYNw5mH3qRqf1bm6ZkXmA0CzxqbswwhLIdspExm/epoV4Feuvuq/lIGl9ltcM2hJ0MuFdIjhWN5l4QL1jsB1kzKMOt63zVuEGJOcUJt5BmWkmzZePzZ8il2RNsiDDZ/CkTXzai3ohs62R38xMXIQpzgmzOet3iWdtS3Jl6/D4Sp8zwkeQZuwFdJEbJ2XP7WK0qjkO/Hr7JW4ZIx8FaE0Y66j3jRFWKbz3R1+ADKO7ONjwB2o/5JR3f59oUCUXUoE/cZvdt3s/7c/zReXAJe8BOmx02pnBSrHhoD7uxGLYNzcwhUFLd2ySVQANONSftMMIrq78boiz0UOZ69PY2RCas15TMOlOQH/6GWnqc1pRUaeQCtDOpY91uDh0zO7/FQ32Uv/GF1AXupJjdIWR6i/z+LvR//Xi+7z7Dj2sKfDvoyMushQVPz4QiLRER 3CudTwDu fV5/ezJRrNcemEPYphkpyAJNflJxV01aEuzOnE7uc9wPKTXnlv2lT6epJz0QySBb4kotAkrfLvMnfJDi0T+E2ViAzWP8yUs+V82SSd0TqEdcpTE+VCW24C6YBCW9zwZL2JwB+Bk6QxzT85N151h4+fq6y9iUMdYhXGVctiuW76TwAmRq9HUeHsaFok7U64nma/rume96bhD/a/DWm3COgkZwRh6aSq09PUaLsjFzwAEt0jttbtMPRhVkodJ6+gzJSbzxRrtOZzGBUEWm7THhyr2cahAOtpoiluC+L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Hao Xu This series introduce getdents64 to io_uring, the code logic is similar with the snychronized version's. It first try nowait issue, and offload it to io-wq threads if the first try fails. Patch1 and Patch2 are some preparation Patch3 supports nowait for xfs getdents code Patch4-11 are vfs change, include adding helpers and trylock for locks Tests I've done: A liburing test case for functional test: https://github.com/HowHsu/liburing/commit/39dc9a8e19c06a8cebf8c2301b85320eb45c061e?diff=unified Tested it with a liburing performance test: https://github.com/HowHsu/liburing/blob/getdents/test/getdents2.c The test is controlled by the below script[2] which runs getdents2.t 100 times and calulate the avg. The result show that io_uring version is about 2.6% faster: note: [1] the number of getdents call/request in io_uring and normal sync version are made sure to be same beforehand. [2] run_getdents.py ```python3 import subprocess N = 100 sum = 0.0 args = ["/data/home/howeyxu/tmpdir", "sync"] for i in range(N): output = subprocess.check_output(["./liburing/test/getdents2.t"] + args) sum += float(output) average = sum / N print("Average of sync:", average) sum = 0.0 args = ["/data/home/howeyxu/tmpdir", "iouring"] for i in range(N): output = subprocess.check_output(["./liburing/test/getdents2.t"] + args) sum += float(output) average = sum / N print("Average of iouring:", average) ``` v5->v6: - remove xfs journal stuff since there are fundamental issues in the design. v4->v5: - move atime update to the beginning of getdents operation - trylock for i_rwsem - nowait semantics for involved xfs journal stuff v3->v4: - add Dave's xfs nowait code and fix a deadlock problem, with some code style tweak. - disable fixed file to avoid a race problem for now - add a test program. v2->v3: - removed the kernfs patches - add f_pos_lock logic - remove the "reduce last EOF getdents try" optimization since Dominique reports that doesn't make difference - remove the rewind logic, I think the right way is to introduce lseek to io_uring not to patch this logic to getdents. - add Singed-off-by of Stefan Roesch for patch 1 since checkpatch complained that Co-developed-by someone should be accompanied with Signed-off-by same person, I can remove them if Stefan thinks that's not proper. Dominique Martinet (1): fs: split off vfs_getdents function of getdents64 syscall Hao Xu (10): xfs: add NOWAIT semantics for readdir vfs: add nowait flag for struct dir_context vfs: add a vfs helper for io_uring file pos lock vfs: add file_pos_unlock() for io_uring usage vfs: add a nowait parameter for touch_atime() vfs: add nowait parameter for file_accessed() vfs: move file_accessed() to the beginning of iterate_dir() vfs: error out -EAGAIN if atime needs to be updated vfs: trylock inode->i_rwsem in iterate_dir() to support nowait io_uring: add support for getdents arch/s390/hypfs/inode.c | 2 +- block/fops.c | 2 +- fs/btrfs/file.c | 2 +- fs/btrfs/inode.c | 2 +- fs/cachefiles/namei.c | 2 +- fs/coda/dir.c | 4 +-- fs/ecryptfs/file.c | 4 +-- fs/ext2/file.c | 4 +-- fs/ext4/file.c | 6 ++-- fs/f2fs/file.c | 4 +-- fs/file.c | 13 ++++++++ fs/fuse/dax.c | 2 +- fs/fuse/file.c | 4 +-- fs/gfs2/file.c | 2 +- fs/hugetlbfs/inode.c | 2 +- fs/inode.c | 10 ++++-- fs/internal.h | 8 +++++ fs/namei.c | 4 +-- fs/nfsd/vfs.c | 2 +- fs/nilfs2/file.c | 2 +- fs/orangefs/file.c | 2 +- fs/orangefs/inode.c | 2 +- fs/overlayfs/file.c | 2 +- fs/overlayfs/inode.c | 2 +- fs/pipe.c | 2 +- fs/ramfs/file-nommu.c | 2 +- fs/readdir.c | 61 ++++++++++++++++++++++++++-------- fs/smb/client/cifsfs.c | 2 +- fs/splice.c | 2 +- fs/stat.c | 2 +- fs/ubifs/file.c | 2 +- fs/udf/file.c | 2 +- fs/xfs/libxfs/xfs_da_btree.c | 16 +++++++++ fs/xfs/libxfs/xfs_da_btree.h | 1 + fs/xfs/libxfs/xfs_dir2_block.c | 7 ++-- fs/xfs/libxfs/xfs_dir2_priv.h | 2 +- fs/xfs/scrub/dir.c | 2 +- fs/xfs/scrub/readdir.c | 2 +- fs/xfs/xfs_dir2_readdir.c | 49 +++++++++++++++++++++------ fs/xfs/xfs_file.c | 6 ++-- fs/xfs/xfs_inode.c | 27 +++++++++++++++ fs/xfs/xfs_inode.h | 17 ++++++---- fs/zonefs/file.c | 4 +-- include/linux/file.h | 7 ++++ include/linux/fs.h | 15 +++++++-- include/uapi/linux/io_uring.h | 1 + io_uring/fs.c | 53 +++++++++++++++++++++++++++++ io_uring/fs.h | 3 ++ io_uring/opdef.c | 8 +++++ kernel/bpf/inode.c | 4 +-- mm/filemap.c | 8 ++--- mm/shmem.c | 6 ++-- net/unix/af_unix.c | 4 +-- 53 files changed, 310 insertions(+), 96 deletions(-)