From patchwork Thu Feb 24 12:28:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758478 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B4BFC433FE for ; Thu, 24 Feb 2022 12:29:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234387AbiBXM3c (ORCPT ); Thu, 24 Feb 2022 07:29:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41912 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233175AbiBXM3Y (ORCPT ); Thu, 24 Feb 2022 07:29:24 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EA08B26836C for ; Thu, 24 Feb 2022 04:28:47 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id A4F27212B9 for ; Thu, 24 Feb 2022 12:28:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705726; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HeOppck5l0IV5Ij+JFDRe5f+tReMihAq4XoFcZqbXv8=; b=pHx295sx2VV5xWty8QTkXmsW0oro896gdKpDTS+lv+QiKQeHT42/oDqrlP+Wtq2engO3TZ rXrkMPvH//A84ErFiolVOcBfNSp49p7SORuGe+PyentNEdTiVy5fKzHBhXXT0x6VdGr/+L azaUnYRqhSvaTU9UOqVh6tiguANglB8= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id EAABC13AD9 for ; Thu, 24 Feb 2022 12:28:45 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 2PtaK/15F2LhAgAAMHmgww (envelope-from ) for ; Thu, 24 Feb 2022 12:28:45 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 1/7] btrfs: remove unused parameter for btrfs_add_inode_defrag() Date: Thu, 24 Feb 2022 20:28:35 +0800 Message-Id: <61861f9dbd64f31796a22a1de617600a869fdf09.1645705266.git.wqu@suse.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Since commit 543eabd5e192 ("Btrfs: don't auto defrag a file when doing directIO"), there is no more caller passing a transaction handler to btrfs_add_inode_defrag(). So the @trans parameter of btrfs_add_inode_defrag() is unnecessary and we can just remove it. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 3 +-- fs/btrfs/file.c | 8 ++------ fs/btrfs/inode.c | 2 +- 3 files changed, 4 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 5e02bf13fd1b..c72790c3bd41 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3345,8 +3345,7 @@ void btrfs_exclop_balance(struct btrfs_fs_info *fs_info, /* file.c */ int __init btrfs_auto_defrag_init(void); void __cold btrfs_auto_defrag_exit(void); -int btrfs_add_inode_defrag(struct btrfs_trans_handle *trans, - struct btrfs_inode *inode, u32 extent_thresh); +int btrfs_add_inode_defrag(struct btrfs_inode *inode, u32 extent_thresh); int btrfs_run_defrag_inodes(struct btrfs_fs_info *fs_info); void btrfs_cleanup_defrag_inodes(struct btrfs_fs_info *fs_info); int btrfs_sync_file(struct file *file, loff_t start, loff_t end, int datasync); diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 9f455c96c974..7d096c3f9127 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -136,8 +136,7 @@ static inline int __need_auto_defrag(struct btrfs_fs_info *fs_info) * insert a defrag record for this inode if auto defrag is * enabled */ -int btrfs_add_inode_defrag(struct btrfs_trans_handle *trans, - struct btrfs_inode *inode, u32 extent_thresh) +int btrfs_add_inode_defrag(struct btrfs_inode *inode, u32 extent_thresh) { struct btrfs_root *root = inode->root; struct btrfs_fs_info *fs_info = root->fs_info; @@ -151,10 +150,7 @@ int btrfs_add_inode_defrag(struct btrfs_trans_handle *trans, if (test_bit(BTRFS_INODE_IN_DEFRAG, &inode->runtime_flags)) return 0; - if (trans) - transid = trans->transid; - else - transid = inode->root->last_trans; + transid = inode->root->last_trans; defrag = kmem_cache_zalloc(btrfs_inode_defrag_cachep, GFP_NOFS); if (!defrag) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b2b26d786fde..24125d5860be 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -560,7 +560,7 @@ static inline void inode_should_defrag(struct btrfs_inode *inode, /* If this is a small write inside eof, kick off a defrag */ if (num_bytes < small_write && (start > 0 || end + 1 < inode->disk_i_size)) - btrfs_add_inode_defrag(NULL, inode, small_write); + btrfs_add_inode_defrag(inode, small_write); } /* From patchwork Thu Feb 24 12:28:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D142AC4332F for ; Thu, 24 Feb 2022 12:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234369AbiBXM31 (ORCPT ); Thu, 24 Feb 2022 07:29:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41924 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234282AbiBXM3Z (ORCPT ); Thu, 24 Feb 2022 07:29:25 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1754B269A83 for ; Thu, 24 Feb 2022 04:28:49 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C8F402113D for ; Thu, 24 Feb 2022 12:28:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705727; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+kR2Nj1jB4nbc1U4UvK8kRw/XHxgrb5YBlnpZtkG/3o=; b=Lv6PPeVsdQrfMj4spiftszfwi3I5JHxNTcdEo3Ovq+6eSiBIGokcSTWvy7pUAbnTKYmB/b lSka7DcsPkEaSY5KGOhHf89vCGLIOTcnpTGuHJQvez8xYW5Ed6Ttav8kxkzNt4tF5jW6IW HinHKwKiN53PiKwOeNufqR8prJNUwTs= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1BC5E13AD9 for ; Thu, 24 Feb 2022 12:28:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id cM/HNP55F2LhAgAAMHmgww (envelope-from ) for ; Thu, 24 Feb 2022 12:28:46 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 2/7] btrfs: refactor __btrfs_run_defrag_inode() Date: Thu, 24 Feb 2022 20:28:36 +0800 Message-Id: <254935398057591304da107b5e5d9967976b1265.1645705266.git.wqu@suse.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This refactor involves: - Use while () loop to replace open-coded loop - Add extra comments on btrfs_get_fs_root()/btrfs_iget() calls - Add extra comment on the meaning of range.start - Move @root, @inode and @range into the loop - Initial @range to 0 at its declaration So we don't need to reset it to zero. There is one behavior change: - Now we check @cur against @isize from last iteration. And the initial dummy @isize is set to (u64)-1 so we can always initial the loop. This slightly enlarge the race window where isize can be changed. Signed-off-by: Qu Wenruo --- fs/btrfs/file.c | 85 ++++++++++++++++++++++++++----------------------- 1 file changed, 45 insertions(+), 40 deletions(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 7d096c3f9127..5b11a8c72114 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -247,57 +247,62 @@ void btrfs_cleanup_defrag_inodes(struct btrfs_fs_info *fs_info) static int __btrfs_run_defrag_inode(struct btrfs_fs_info *fs_info, struct inode_defrag *defrag) { - struct btrfs_root *inode_root; - struct inode *inode; - struct btrfs_ioctl_defrag_range_args range; int ret = 0; + /* Initially we don't have the inode, use -1 as dummy isize */ + u64 isize = (u64)-1; u64 cur = 0; -again: - if (test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)) - goto cleanup; - if (!__need_auto_defrag(fs_info)) - goto cleanup; + while (cur < isize) { + struct btrfs_ioctl_defrag_range_args range = { 0 }; + struct btrfs_root *inode_root; + struct inode *inode; - /* get the inode */ - inode_root = btrfs_get_fs_root(fs_info, defrag->root, true); - if (IS_ERR(inode_root)) { - ret = PTR_ERR(inode_root); - goto cleanup; - } + if (test_bit(BTRFS_FS_STATE_REMOUNTING, &fs_info->fs_state)) + break; + if (!__need_auto_defrag(fs_info)) + break; - inode = btrfs_iget(fs_info->sb, defrag->ino, inode_root); - btrfs_put_root(inode_root); - if (IS_ERR(inode)) { - ret = PTR_ERR(inode); - goto cleanup; - } + /* Check if the root is still there and grab it */ + inode_root = btrfs_get_fs_root(fs_info, defrag->root, true); + if (IS_ERR(inode_root)) { + ret = PTR_ERR(inode_root); + break; + } - if (cur >= i_size_read(inode)) { - iput(inode); - goto cleanup; - } + /* Check if the inode is still there and grab it */ + inode = btrfs_iget(fs_info->sb, defrag->ino, inode_root); + btrfs_put_root(inode_root); + if (IS_ERR(inode)) { + ret = PTR_ERR(inode); + break; + } - /* do a chunk of defrag */ - clear_bit(BTRFS_INODE_IN_DEFRAG, &BTRFS_I(inode)->runtime_flags); - memset(&range, 0, sizeof(range)); - range.len = (u64)-1; - range.start = cur; - range.extent_thresh = defrag->extent_thresh; + isize = i_size_read(inode); - sb_start_write(fs_info->sb); - ret = btrfs_defrag_file(inode, NULL, &range, defrag->transid, - BTRFS_DEFRAG_BATCH); - sb_end_write(fs_info->sb); - iput(inode); + /* do a chunk of defrag */ + clear_bit(BTRFS_INODE_IN_DEFRAG, &BTRFS_I(inode)->runtime_flags); + range.len = (u64)-1; + range.start = cur; + range.extent_thresh = defrag->extent_thresh; - if (ret < 0) - goto cleanup; + sb_start_write(fs_info->sb); + ret = btrfs_defrag_file(inode, NULL, &range, defrag->transid, + BTRFS_DEFRAG_BATCH); + sb_end_write(fs_info->sb); + iput(inode); - cur = max(cur + fs_info->sectorsize, range.start); - goto again; + if (ret < 0) + break; -cleanup: + /* + * Range.start is the last scanned bytenr. + * + * And just in case the last scanned bytenr doesn't get + * increased at all, at least start from the next sector + * of current bytenr. + */ + cur = max(cur + fs_info->sectorsize, range.start); + } kmem_cache_free(btrfs_inode_defrag_cachep, defrag); return ret; } From patchwork Thu Feb 24 12:28:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758472 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50370C433EF for ; Thu, 24 Feb 2022 12:28:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234376AbiBXM31 (ORCPT ); Thu, 24 Feb 2022 07:29:27 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234368AbiBXM3Z (ORCPT ); Thu, 24 Feb 2022 07:29:25 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74B53264983 for ; Thu, 24 Feb 2022 04:28:50 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 317631F44A; Thu, 24 Feb 2022 12:28:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705729; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3nKggag2hJWv7dfhOOlLqh9aOcxM8bmHXMy6+G0dgRk=; b=VDuTOKT+/wgGkvixj92FZ4VNf+rpZTvQn1acJeW1QVeNx20m1QFvDfB2H2ntueioGeMWoM tArLzYf6C4+sbMwoyTjQ9H6XqUBihBnK4ifCSLicVFacwwNjxeEL5MjIrET4oAvaHnA8Xo ZzeCsJaJ5n4gpnEMPIlyAxqsYmPw6Wg= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 4032313AD9; Thu, 24 Feb 2022 12:28:48 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id kGvzAQB6F2LhAgAAMHmgww (envelope-from ); Thu, 24 Feb 2022 12:28:48 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Cc: Filipe Manana Subject: [PATCH 3/7] btrfs: uapi: introduce BTRFS_DEFRAG_RANGE_MASK for later sanity check Date: Thu, 24 Feb 2022 20:28:37 +0800 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org And since we're here, replace the hardcoded bit flags (1, 2) with (1UL << 0) and (1UL << 1), respectively. Signed-off-by: Qu Wenruo Reviewed-by: Filipe Manana --- include/uapi/linux/btrfs.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index d956b2993970..3d4bfdacf935 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -576,8 +576,10 @@ struct btrfs_ioctl_clone_range_args { * Used by: * struct btrfs_ioctl_defrag_range_args.flags */ -#define BTRFS_DEFRAG_RANGE_COMPRESS 1 -#define BTRFS_DEFRAG_RANGE_START_IO 2 +#define BTRFS_DEFRAG_RANGE_COMPRESS (1UL << 0) +#define BTRFS_DEFRAG_RANGE_START_IO (1UL << 1) +#define BTRFS_DEFRAG_RANGE_FLAGS_MASK (BTRFS_DEFRAG_RANGE_COMPRESS |\ + BTRFS_DEFRAG_RANGE_START_IO) struct btrfs_ioctl_defrag_range_args { /* start of the defrag operation */ __u64 start; From patchwork Thu Feb 24 12:28:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758474 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA7A5C433F5 for ; Thu, 24 Feb 2022 12:29:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234365AbiBXM32 (ORCPT ); Thu, 24 Feb 2022 07:29:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41938 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234115AbiBXM3Z (ORCPT ); Thu, 24 Feb 2022 07:29:25 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D068F269A8A for ; Thu, 24 Feb 2022 04:28:51 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8E05721155; Thu, 24 Feb 2022 12:28:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705730; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=wWAHNnO9bY1qa0fgAiXQkxIgj1h9XZS1HqWPc1IuRaE=; b=Vsj3nu0eZQ11l2Xfq46OuEhm/MWZQlyz1yjWyUiBJcarzGC+JR66DejmcC9TVfk44vlGxn 5A18k5jsWZkAkbYQxYbu+Lg/c2lkJrE7nyMwUNEfAM6AqkZxtMp+P6O2p2BOoPux+rPQ2e QFHJPtHn+5oI3Z82VWVKrbQRtWbg9xE= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 9CF4313AD9; Thu, 24 Feb 2022 12:28:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YMqZGAF6F2LhAgAAMHmgww (envelope-from ); Thu, 24 Feb 2022 12:28:49 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Cc: Filipe Manana Subject: [PATCH 4/7] btrfs: defrag: introduce btrfs_defrag_ctrl structure for later usage Date: Thu, 24 Feb 2022 20:28:38 +0800 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Currently btrfs_defrag_file() accepts not only btrfs_ioctl_defrag_range_args but also other parameters like @newer_than and @max_sectors_to_defrag for extra policies. Those extra values are hidden from defrag ioctl and even caused bugs in the past due to different behaviors based on those extra values. Here we introduce a new structure, btrfs_defrag_ctrl, to include: - all members in btrfs_ioctl_defrag_range_args - @max_sectors_to_defrag and @newer_than - Extra values which callers of btrfs_defrag_file() may care Like @sectors_defragged and @last_scanned. With the new structure, also introduce a new helper, btrfs_defrag_ioctl_args_to_ctrl() to: - Do extra sanity check on @compress and @flags - Do range alignment when possible - Set default values. Signed-off-by: Qu Wenruo Reviewed-by: Filipe Manana --- fs/btrfs/ctree.h | 19 +++++++++++++++++++ fs/btrfs/ioctl.c | 41 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index c72790c3bd41..97a9f17fc7f1 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3325,6 +3325,25 @@ int btrfs_fileattr_set(struct user_namespace *mnt_userns, int btrfs_ioctl_get_supported_features(void __user *arg); void btrfs_sync_inode_flags_to_i_flags(struct inode *inode); int __pure btrfs_is_empty_uuid(u8 *uuid); + +struct btrfs_defrag_ctrl { + /* Input, read-only fields */ + u64 start; + u64 len; + u32 extent_thresh; + u64 newer_than; + u64 max_sectors_to_defrag; + u8 compress; + u8 flags; + + /* Output fields */ + u64 sectors_defragged; + u64 last_scanned; /* Exclusive bytenr */ +}; +int btrfs_defrag_ioctl_args_to_ctrl(struct btrfs_fs_info *fs_info, + struct btrfs_ioctl_defrag_range_args *args, + struct btrfs_defrag_ctrl *ctrl, + u64 max_sectors_to_defrag, u64 newer_than); int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, struct btrfs_ioctl_defrag_range_args *range, u64 newer_than, unsigned long max_to_defrag); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 7183af333c98..1ee2db269bc3 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1749,6 +1749,47 @@ static int defrag_one_cluster(struct btrfs_inode *inode, return ret; } +/* + * Convert the old ioctl format to the new btrfs_defrag_ctrl structure. + * + * Will also do basic tasks like setting default values and sanity checks. + */ +int btrfs_defrag_ioctl_args_to_ctrl(struct btrfs_fs_info *fs_info, + struct btrfs_ioctl_defrag_range_args *args, + struct btrfs_defrag_ctrl *ctrl, + u64 max_sectors_to_defrag, u64 newer_than) +{ + u64 range_end; + + if (args->flags & ~BTRFS_DEFRAG_RANGE_FLAGS_MASK) + return -EOPNOTSUPP; + if (args->compress_type >= BTRFS_NR_COMPRESS_TYPES) + return -EOPNOTSUPP; + + ctrl->start = round_down(args->start, fs_info->sectorsize); + /* + * If @len does not overflow with @start nor is -1, align the length. + * Otherwise set it to (u64)-1 so later btrfs_defrag_file() will + * determine the length using isize. + */ + if (!check_add_overflow(args->start, args->len, &range_end) && + args->len != (u64)-1) + ctrl->len = round_up(range_end, fs_info->sectorsize) - + ctrl->start; + else + ctrl->len = -1; + ctrl->flags = args->flags; + ctrl->compress = args->compress_type; + if (args->extent_thresh == 0) + ctrl->extent_thresh = SZ_256K; + else + ctrl->extent_thresh = args->extent_thresh; + ctrl->newer_than = newer_than; + ctrl->last_scanned = 0; + ctrl->sectors_defragged = 0; + return 0; +} + /* * Entry point to file defragmentation. * From patchwork Thu Feb 24 12:28:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03578C4332F for ; Thu, 24 Feb 2022 12:29:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234380AbiBXM33 (ORCPT ); Thu, 24 Feb 2022 07:29:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41954 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234372AbiBXM3Z (ORCPT ); Thu, 24 Feb 2022 07:29:25 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04753269AA4 for ; Thu, 24 Feb 2022 04:28:53 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id B27412113D for ; Thu, 24 Feb 2022 12:28:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705731; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vrP6o5+46P9IirjXlbjeWk0IM3ggqDn7ZpEv9I+41Nc=; b=QvVIqVNLPa51P7+DWmiBJ9QmJXR+7t49wyvgIU8eTmls6fKXl8ZrVkhhDuv4qbvXqIpWOP Z6GUOVRnflKe0oeZUP3C1cwNAQM467zvMCnCnaTxCL/KwKsrKwR+znPSSYgYmzbFw35LSr Trmk/aYVNL410gR6iCKe9PiFYYHhP6g= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 04E4A13AD9 for ; Thu, 24 Feb 2022 12:28:50 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 6KwsLwJ6F2LhAgAAMHmgww (envelope-from ) for ; Thu, 24 Feb 2022 12:28:50 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 5/7] btrfs: defrag: use btrfs_defrag_ctrl to replace btrfs_ioctl_defrag_range_args for btrfs_defrag_file() Date: Thu, 24 Feb 2022 20:28:39 +0800 Message-Id: <4d18a7bef69bfb10de4d49ca24c171097343327a.1645705266.git.wqu@suse.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This brings the following benefits: - No more strange range->start update to indicate last scanned bytenr We have btrfs_defrag_ctrl::last_scanned (exclusive) for it directly. - No more return value to indicate defragged sectors Now btrfs_defrag_file() will just return 0 if no error happened. And btrfs_defrag_ctrl::sectors_defragged will show that value. - Less parameters to carry around Now most defrag_* functions only need to fetch its policy parameters from btrfs_defrag_ctrl directly. Signed-off-by: Qu Wenruo --- fs/btrfs/ctree.h | 3 +- fs/btrfs/file.c | 17 +++-- fs/btrfs/ioctl.c | 162 +++++++++++++++++++---------------------------- 3 files changed, 75 insertions(+), 107 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 97a9f17fc7f1..a8a3de10cead 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -3345,8 +3345,7 @@ int btrfs_defrag_ioctl_args_to_ctrl(struct btrfs_fs_info *fs_info, struct btrfs_defrag_ctrl *ctrl, u64 max_sectors_to_defrag, u64 newer_than); int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, - struct btrfs_ioctl_defrag_range_args *range, - u64 newer_than, unsigned long max_to_defrag); + struct btrfs_defrag_ctrl *ctrl); void btrfs_get_block_group_info(struct list_head *groups_list, struct btrfs_ioctl_space_info *space); void btrfs_update_ioctl_balance_args(struct btrfs_fs_info *fs_info, diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 5b11a8c72114..d50d6b042e67 100644 --- a/fs/btrfs/file.c +++ b/fs/btrfs/file.c @@ -253,7 +253,7 @@ static int __btrfs_run_defrag_inode(struct btrfs_fs_info *fs_info, u64 cur = 0; while (cur < isize) { - struct btrfs_ioctl_defrag_range_args range = { 0 }; + struct btrfs_defrag_ctrl ctrl = {0}; struct btrfs_root *inode_root; struct inode *inode; @@ -281,13 +281,14 @@ static int __btrfs_run_defrag_inode(struct btrfs_fs_info *fs_info, /* do a chunk of defrag */ clear_bit(BTRFS_INODE_IN_DEFRAG, &BTRFS_I(inode)->runtime_flags); - range.len = (u64)-1; - range.start = cur; - range.extent_thresh = defrag->extent_thresh; + ctrl.len = (u64)-1; + ctrl.start = cur; + ctrl.newer_than = defrag->transid; + ctrl.extent_thresh = defrag->extent_thresh; + ctrl.max_sectors_to_defrag = BTRFS_DEFRAG_BATCH; sb_start_write(fs_info->sb); - ret = btrfs_defrag_file(inode, NULL, &range, defrag->transid, - BTRFS_DEFRAG_BATCH); + ret = btrfs_defrag_file(inode, NULL, &ctrl); sb_end_write(fs_info->sb); iput(inode); @@ -295,13 +296,11 @@ static int __btrfs_run_defrag_inode(struct btrfs_fs_info *fs_info, break; /* - * Range.start is the last scanned bytenr. - * * And just in case the last scanned bytenr doesn't get * increased at all, at least start from the next sector * of current bytenr. */ - cur = max(cur + fs_info->sectorsize, range.start); + cur = max(cur + fs_info->sectorsize, ctrl.last_scanned); } kmem_cache_free(btrfs_inode_defrag_cachep, defrag); return ret; diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 1ee2db269bc3..72d157b89a17 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1376,23 +1376,20 @@ struct defrag_target_range { /* * Collect all valid target extents. * + * @ctrl: extra defrag policy control * @start: file offset to lookup * @len: length to lookup - * @extent_thresh: file extent size threshold, any extent size >= this value - * will be ignored - * @newer_than: only defrag extents newer than this value - * @do_compress: whether the defrag is doing compression - * if true, @extent_thresh will be ignored and all regular - * file extents meeting @newer_than will be targets. * @locked: if the range has already held extent lock * @target_list: list of targets file extents + * + * Will update ctrl::last_scanned. */ static int defrag_collect_targets(struct btrfs_inode *inode, - u64 start, u64 len, u32 extent_thresh, - u64 newer_than, bool do_compress, - bool locked, struct list_head *target_list, - u64 *last_scanned_ret) + struct btrfs_defrag_ctrl *ctrl, + u64 start, u32 len, bool locked, + struct list_head *target_list) { + bool do_compress = ctrl->flags & BTRFS_DEFRAG_RANGE_COMPRESS; bool last_is_target = false; u64 cur = start; int ret = 0; @@ -1405,7 +1402,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode, last_is_target = false; em = defrag_lookup_extent(&inode->vfs_inode, cur, - newer_than, locked); + ctrl->newer_than, locked); if (!em) break; @@ -1415,7 +1412,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode, goto next; /* Skip older extent */ - if (em->generation < newer_than) + if (em->generation < ctrl->newer_than) goto next; /* This em is under writeback, no need to defrag */ @@ -1459,7 +1456,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode, goto add; /* Skip too large extent */ - if (range_len >= extent_thresh) + if (range_len >= ctrl->extent_thresh) goto next; /* @@ -1531,16 +1528,16 @@ static int defrag_collect_targets(struct btrfs_inode *inode, kfree(entry); } } - if (!ret && last_scanned_ret) { + if (!ret) { /* * If the last extent is not a target, the caller can skip to * the end of that extent. * Otherwise, we can only go the end of the specified range. */ if (!last_is_target) - *last_scanned_ret = max(cur, *last_scanned_ret); + ctrl->last_scanned = max(cur, ctrl->last_scanned); else - *last_scanned_ret = max(start + len, *last_scanned_ret); + ctrl->last_scanned = max(start + len, ctrl->last_scanned); } return ret; } @@ -1600,9 +1597,8 @@ static int defrag_one_locked_target(struct btrfs_inode *inode, return ret; } -static int defrag_one_range(struct btrfs_inode *inode, u64 start, u32 len, - u32 extent_thresh, u64 newer_than, bool do_compress, - u64 *last_scanned_ret) +static int defrag_one_range(struct btrfs_inode *inode, + struct btrfs_defrag_ctrl *ctrl, u64 start, u32 len) { struct extent_state *cached_state = NULL; struct defrag_target_range *entry; @@ -1646,9 +1642,8 @@ static int defrag_one_range(struct btrfs_inode *inode, u64 start, u32 len, * And this time we have extent locked already, pass @locked = true * so that we won't relock the extent range and cause deadlock. */ - ret = defrag_collect_targets(inode, start, len, extent_thresh, - newer_than, do_compress, true, - &target_list, last_scanned_ret); + ret = defrag_collect_targets(inode, ctrl, start, len, true, + &target_list); if (ret < 0) goto unlock_extent; @@ -1678,13 +1673,17 @@ static int defrag_one_range(struct btrfs_inode *inode, u64 start, u32 len, return ret; } +/* + * Return <0 for error. + * Return >0 if we hit the ctrl->max_sectors_to_defrag limit + * Return 0 if we finished the range without error. + * + * For >= 0 case, ctrl->last_scanned and ctrl->sectors_defragged will be updated. + */ static int defrag_one_cluster(struct btrfs_inode *inode, struct file_ra_state *ra, - u64 start, u32 len, u32 extent_thresh, - u64 newer_than, bool do_compress, - unsigned long *sectors_defragged, - unsigned long max_sectors, - u64 *last_scanned_ret) + struct btrfs_defrag_ctrl *ctrl, + u64 start, u32 len) { const u32 sectorsize = inode->root->fs_info->sectorsize; struct defrag_target_range *entry; @@ -1692,9 +1691,8 @@ static int defrag_one_cluster(struct btrfs_inode *inode, LIST_HEAD(target_list); int ret; - ret = defrag_collect_targets(inode, start, len, extent_thresh, - newer_than, do_compress, false, - &target_list, NULL); + ret = defrag_collect_targets(inode, ctrl, start, len, false, + &target_list); if (ret < 0) goto out; @@ -1702,23 +1700,16 @@ static int defrag_one_cluster(struct btrfs_inode *inode, u32 range_len = entry->len; /* Reached or beyond the limit */ - if (max_sectors && *sectors_defragged >= max_sectors) { + if (ctrl->max_sectors_to_defrag && + ctrl->sectors_defragged >= ctrl->max_sectors_to_defrag) { ret = 1; break; } - if (max_sectors) + if (ctrl->max_sectors_to_defrag) range_len = min_t(u32, range_len, - (max_sectors - *sectors_defragged) * sectorsize); - - /* - * If defrag_one_range() has updated last_scanned_ret, - * our range may already be invalid (e.g. hole punched). - * Skip if our range is before last_scanned_ret, as there is - * no need to defrag the range anymore. - */ - if (entry->start + range_len <= *last_scanned_ret) - continue; + (ctrl->max_sectors_to_defrag - + ctrl->sectors_defragged) * sectorsize); if (ra) page_cache_sync_readahead(inode->vfs_inode.i_mapping, @@ -1731,21 +1722,17 @@ static int defrag_one_cluster(struct btrfs_inode *inode, * But that's fine, it only affects the @sectors_defragged * accounting. */ - ret = defrag_one_range(inode, entry->start, range_len, - extent_thresh, newer_than, do_compress, - last_scanned_ret); + ret = defrag_one_range(inode, ctrl, entry->start, range_len); if (ret < 0) break; - *sectors_defragged += range_len >> - inode->root->fs_info->sectorsize_bits; + ctrl->sectors_defragged += range_len >> + inode->root->fs_info->sectorsize_bits; } out: list_for_each_entry_safe(entry, tmp, &target_list, list) { list_del_init(&entry->list); kfree(entry); } - if (ret >= 0) - *last_scanned_ret = max(*last_scanned_ret, start + len); return ret; } @@ -1795,59 +1782,47 @@ int btrfs_defrag_ioctl_args_to_ctrl(struct btrfs_fs_info *fs_info, * * @inode: inode to be defragged * @ra: readahead state (can be NUL) - * @range: defrag options including range and flags - * @newer_than: minimum transid to defrag - * @max_to_defrag: max number of sectors to be defragged, if 0, the whole inode - * will be defragged. + * @ctrl: defrag options including range and various policy parameters * * Return <0 for error. - * Return >=0 for the number of sectors defragged, and range->start will be updated - * to indicate the file offset where next defrag should be started at. - * (Mostly for autodefrag, which sets @max_to_defrag thus we may exit early without - * defragging all the range). + * Return 0 if the defrag is done without error, ctrl->last_scanned and + * ctrl->sectors_defragged will be updated. */ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, - struct btrfs_ioctl_defrag_range_args *range, - u64 newer_than, unsigned long max_to_defrag) + struct btrfs_defrag_ctrl *ctrl) { struct btrfs_fs_info *fs_info = btrfs_sb(inode->i_sb); - unsigned long sectors_defragged = 0; u64 isize = i_size_read(inode); u64 cur; u64 last_byte; - bool do_compress = range->flags & BTRFS_DEFRAG_RANGE_COMPRESS; + bool do_compress = ctrl->flags & BTRFS_DEFRAG_RANGE_COMPRESS; bool ra_allocated = false; - int compress_type = BTRFS_COMPRESS_ZLIB; int ret = 0; - u32 extent_thresh = range->extent_thresh; pgoff_t start_index; if (isize == 0) return 0; - if (range->start >= isize) + if (ctrl->start >= isize) return -EINVAL; - if (do_compress) { - if (range->compress_type >= BTRFS_NR_COMPRESS_TYPES) - return -EINVAL; - if (range->compress_type) - compress_type = range->compress_type; - } + if (do_compress) + ASSERT(ctrl->compress < BTRFS_NR_COMPRESS_TYPES); - if (extent_thresh == 0) - extent_thresh = SZ_256K; + if (ctrl->extent_thresh == 0) + ctrl->extent_thresh = SZ_256K; - if (range->start + range->len > range->start) { + if (ctrl->start + ctrl->len > ctrl->start) { /* Got a specific range */ - last_byte = min(isize, range->start + range->len); + last_byte = min(isize, ctrl->start + ctrl->len); } else { /* Defrag until file end */ last_byte = isize; } /* Align the range */ - cur = round_down(range->start, fs_info->sectorsize); + cur = round_down(ctrl->start, fs_info->sectorsize); + ctrl->last_scanned = cur; last_byte = round_up(last_byte, fs_info->sectorsize) - 1; /* @@ -1871,8 +1846,7 @@ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, inode->i_mapping->writeback_index = start_index; while (cur < last_byte) { - const unsigned long prev_sectors_defragged = sectors_defragged; - u64 last_scanned = cur; + const unsigned long prev_sectors_defragged = ctrl->sectors_defragged; u64 cluster_end; if (btrfs_defrag_cancelled(fs_info)) { @@ -1896,19 +1870,17 @@ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, break; } if (do_compress) - BTRFS_I(inode)->defrag_compress = compress_type; - ret = defrag_one_cluster(BTRFS_I(inode), ra, cur, - cluster_end + 1 - cur, extent_thresh, - newer_than, do_compress, §ors_defragged, - max_to_defrag, &last_scanned); + BTRFS_I(inode)->defrag_compress = ctrl->compress; + ret = defrag_one_cluster(BTRFS_I(inode), ra, ctrl, cur, + cluster_end + 1 - cur); - if (sectors_defragged > prev_sectors_defragged) + if (ctrl->sectors_defragged > prev_sectors_defragged) balance_dirty_pages_ratelimited(inode->i_mapping); btrfs_inode_unlock(inode, 0); if (ret < 0) break; - cur = max(cluster_end + 1, last_scanned); + cur = max(cluster_end + 1, ctrl->last_scanned); if (ret > 0) { ret = 0; break; @@ -1918,27 +1890,21 @@ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, if (ra_allocated) kfree(ra); - /* - * Update range.start for autodefrag, this will indicate where to start - * in next run. - */ - range->start = cur; - if (sectors_defragged) { + if (ctrl->sectors_defragged) { /* * We have defragged some sectors, for compression case they * need to be written back immediately. */ - if (range->flags & BTRFS_DEFRAG_RANGE_START_IO) { + if (ctrl->flags & BTRFS_DEFRAG_RANGE_START_IO) { filemap_flush(inode->i_mapping); if (test_bit(BTRFS_INODE_HAS_ASYNC_EXTENT, &BTRFS_I(inode)->runtime_flags)) filemap_flush(inode->i_mapping); } - if (range->compress_type == BTRFS_COMPRESS_LZO) + if (ctrl->compress == BTRFS_COMPRESS_LZO) btrfs_set_fs_incompat(fs_info, COMPRESS_LZO); - else if (range->compress_type == BTRFS_COMPRESS_ZSTD) + else if (ctrl->compress == BTRFS_COMPRESS_ZSTD) btrfs_set_fs_incompat(fs_info, COMPRESS_ZSTD); - ret = sectors_defragged; } if (do_compress) { btrfs_inode_lock(inode, 0); @@ -3453,6 +3419,7 @@ static int btrfs_ioctl_defrag(struct file *file, void __user *argp) struct inode *inode = file_inode(file); struct btrfs_root *root = BTRFS_I(inode)->root; struct btrfs_ioctl_defrag_range_args range = {0}; + struct btrfs_defrag_ctrl ctrl = {0}; int ret; ret = mnt_want_write_file(file); @@ -3498,8 +3465,11 @@ static int btrfs_ioctl_defrag(struct file *file, void __user *argp) /* the rest are all set to zero by kzalloc */ range.len = (u64)-1; } - ret = btrfs_defrag_file(file_inode(file), &file->f_ra, - &range, BTRFS_OLDEST_GENERATION, 0); + ret = btrfs_defrag_ioctl_args_to_ctrl(root->fs_info, &range, + &ctrl, 0, BTRFS_OLDEST_GENERATION); + if (ret < 0) + break; + ret = btrfs_defrag_file(file_inode(file), &file->f_ra, &ctrl); if (ret > 0) ret = 0; break; From patchwork Thu Feb 24 12:28:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0416AC433EF for ; Thu, 24 Feb 2022 12:29:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234379AbiBXM3a (ORCPT ); Thu, 24 Feb 2022 07:29:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41934 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234377AbiBXM3Z (ORCPT ); Thu, 24 Feb 2022 07:29:25 -0500 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5CA67269AAC for ; Thu, 24 Feb 2022 04:28:54 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 1B89021155; Thu, 24 Feb 2022 12:28:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705733; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T5ulqCZi825YATFHCgWBmh8z9ETSPkh7Wmvh7vrmUR8=; b=bRjG14vxFscrbhWQjeUn4xxJZ4k5YSzBz52QVFlSlIujedQiy+M7kJOkK72amClr9dwCxl g9rSx5JL5PgSMDbAsynHuzP5uJSMfgzqH0okucnZ/IiFq545cu50ipzmewQm6XIzvfJYqj hA66Q+TfFsb5uYybrl1gk4ASkO9esPw= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 2A5B613AD9; Thu, 24 Feb 2022 12:28:51 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id YDZFOAN6F2LhAgAAMHmgww (envelope-from ); Thu, 24 Feb 2022 12:28:51 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Cc: Filipe Manana Subject: [PATCH 6/7] btrfs: defrag: make btrfs_defrag_file() to report accurate number of defragged sectors Date: Thu, 24 Feb 2022 20:28:40 +0800 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org Previously rework btrfs_defrag_file() can only report the number of sectors from the first run of defrag_collect_targets(). This number is not accurate as if holes are punched after the first defrag_collect_targets() call, we will not choose to defrag the holes. Originally this is to avoid passing @sectors_defragged to every involved functions. But now since we have btrfs_defrag_ctrl, there is no need to do such inaccurate accounting, just update btrfs_defrag_ctrl::sectors_defragged after a successful defrag_one_locked_target() call. Signed-off-by: Qu Wenruo Reviewed-by: Filipe Manana --- fs/btrfs/ioctl.c | 10 ++-------- 1 file changed, 2 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 72d157b89a17..1f0bf6181013 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1652,6 +1652,8 @@ static int defrag_one_range(struct btrfs_inode *inode, &cached_state); if (ret < 0) break; + ctrl->sectors_defragged += entry->len >> + inode->root->fs_info->sectorsize_bits; } list_for_each_entry_safe(entry, tmp, &target_list, list) { @@ -1716,17 +1718,9 @@ static int defrag_one_cluster(struct btrfs_inode *inode, ra, NULL, entry->start >> PAGE_SHIFT, ((entry->start + range_len - 1) >> PAGE_SHIFT) - (entry->start >> PAGE_SHIFT) + 1); - /* - * Here we may not defrag any range if holes are punched before - * we locked the pages. - * But that's fine, it only affects the @sectors_defragged - * accounting. - */ ret = defrag_one_range(inode, ctrl, entry->start, range_len); if (ret < 0) break; - ctrl->sectors_defragged += range_len >> - inode->root->fs_info->sectorsize_bits; } out: list_for_each_entry_safe(entry, tmp, &target_list, list) { From patchwork Thu Feb 24 12:28:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qu Wenruo X-Patchwork-Id: 12758475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17DD0C43217 for ; Thu, 24 Feb 2022 12:29:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234373AbiBXM3a (ORCPT ); Thu, 24 Feb 2022 07:29:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234379AbiBXM3Z (ORCPT ); Thu, 24 Feb 2022 07:29:25 -0500 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8447E26A38A for ; Thu, 24 Feb 2022 04:28:55 -0800 (PST) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 415431F44A for ; Thu, 24 Feb 2022 12:28:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1645705734; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dZSSFtcbrGBVNIcpYSN6FIBNxhpTs22gzVGtEDNuaGI=; b=hziNHRb8geiPNfc2jIlwTkyVYGjufTwJQOWI+ihP+n001d9uDF1xme3ud70nukY0pUudF5 ChoqwSOqxPaCiUJucRkqSSS0zpcCL1wgUbRm/LkT43A+YbklliTrUGpemmX5V4GswoUmbQ i+Q33X1c73V2vVyPdLmNfPzhSFu+IxQ= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8798713AD9 for ; Thu, 24 Feb 2022 12:28:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id CDZjEwV6F2LhAgAAMHmgww (envelope-from ) for ; Thu, 24 Feb 2022 12:28:53 +0000 From: Qu Wenruo To: linux-btrfs@vger.kernel.org Subject: [PATCH 7/7] btrfs: add trace events for defrag Date: Thu, 24 Feb 2022 20:28:41 +0800 Message-Id: X-Mailer: git-send-email 2.35.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org This patch will introduce the following trace events: - trace_defrag_add_target() - trace_defrag_one_locked_range() - trace_defrag_file_start() - trace_defrag_file_end() Under most cases, all of them are needed to debug policy related defrag bugs. The example output would look like this: (with TASK, CPU, TIMESTAMP and UUID skipped) defrag_file_start: : root=5 ino=257 start=0 len=131072 extent_thresh=262144 newer_than=7 flags=0x0 compress=0 max_sectors_to_defrag=1024 defrag_add_target: : root=5 ino=257 target_start=0 target_len=4096 found em=0 len=4096 generation=7 defrag_add_target: : root=5 ino=257 target_start=4096 target_len=4096 found em=4096 len=4096 generation=7 ... defrag_add_target: : root=5 ino=257 target_start=57344 target_len=4096 found em=57344 len=4096 generation=7 defrag_add_target: : root=5 ino=257 target_start=61440 target_len=4096 found em=61440 len=4096 generation=7 defrag_add_target: : root=5 ino=257 target_start=0 target_len=4096 found em=0 len=4096 generation=7 defrag_add_target: : root=5 ino=257 target_start=4096 target_len=4096 found em=4096 len=4096 generation=7 ... defrag_add_target: : root=5 ino=257 target_start=57344 target_len=4096 found em=57344 len=4096 generation=7 defrag_add_target: : root=5 ino=257 target_start=61440 target_len=4096 found em=61440 len=4096 generation=7 defrag_one_locked_range: : root=5 ino=257 start=0 len=65536 defrag_file_end: : root=5 ino=257 sectors_defragged=16 last_scanned=131072 ret=0 Although the defrag_add_target() part is lengthy, it shows some details of the extent map we get. With the extra info from defrag_file_start(), we can check if the target em is correct for our defrag policy. Signed-off-by: Qu Wenruo --- fs/btrfs/ioctl.c | 4 ++ include/trace/events/btrfs.h | 127 +++++++++++++++++++++++++++++++++++ 2 files changed, 131 insertions(+) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 1f0bf6181013..0d79dc28a9d4 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1486,6 +1486,7 @@ static int defrag_collect_targets(struct btrfs_inode *inode, add: last_is_target = true; range_len = min(extent_map_end(em), start + len) - cur; + trace_defrag_add_target(inode, em, cur, range_len); /* * This one is a good target, check if it can be merged into * last range of the target list. @@ -1581,6 +1582,7 @@ static int defrag_one_locked_target(struct btrfs_inode *inode, ret = btrfs_delalloc_reserve_space(inode, &data_reserved, start, len); if (ret < 0) return ret; + trace_defrag_one_locked_range(inode, start, (u32)len); clear_extent_bit(&inode->io_tree, start, start + len - 1, EXTENT_DELALLOC | EXTENT_DO_ACCOUNTING | EXTENT_DEFRAG, 0, 0, cached_state); @@ -1818,6 +1820,7 @@ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, cur = round_down(ctrl->start, fs_info->sectorsize); ctrl->last_scanned = cur; last_byte = round_up(last_byte, fs_info->sectorsize) - 1; + trace_defrag_file_start(BTRFS_I(inode), ctrl, cur, last_byte + 1 - cur); /* * If we were not given a ra, allocate a readahead context. As @@ -1905,6 +1908,7 @@ int btrfs_defrag_file(struct inode *inode, struct file_ra_state *ra, BTRFS_I(inode)->defrag_compress = BTRFS_COMPRESS_NONE; btrfs_inode_unlock(inode, 0); } + trace_defrag_file_end(BTRFS_I(inode), ctrl, ret); return ret; } diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index f068ff30d654..f46b9858154d 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -30,6 +30,7 @@ struct btrfs_qgroup; struct extent_io_tree; struct prelim_ref; struct btrfs_space_info; +struct btrfs_defrag_ctrl; #define show_ref_type(type) \ __print_symbolic(type, \ @@ -2264,6 +2265,132 @@ DEFINE_EVENT(btrfs__space_info_update, update_bytes_pinned, TP_ARGS(fs_info, sinfo, old, diff) ); +TRACE_EVENT(defrag_one_locked_range, + + TP_PROTO(const struct btrfs_inode *inode, u64 start, u32 len), + + TP_ARGS(inode, start, len), + + TP_STRUCT__entry_btrfs( + __field( u64, root ) + __field( u64, ino ) + __field( u64, start ) + __field( u32, len ) + ), + + TP_fast_assign_btrfs(inode->root->fs_info, + __entry->root = inode->root->root_key.objectid; + __entry->ino = btrfs_ino(inode); + __entry->start = start; + __entry->len = len; + ), + + TP_printk_btrfs("root=%llu ino=%llu start=%llu len=%u", + __entry->root, __entry->ino, __entry->start, __entry->len) +); + +TRACE_EVENT(defrag_add_target, + + TP_PROTO(const struct btrfs_inode *inode, const struct extent_map *em, + u64 start, u32 len), + + TP_ARGS(inode, em, start, len), + + TP_STRUCT__entry_btrfs( + __field( u64, root ) + __field( u64, ino ) + __field( u64, target_start ) + __field( u32, target_len ) + __field( u64, em_generation ) + __field( u64, em_start ) + __field( u64, em_len ) + ), + + TP_fast_assign_btrfs(inode->root->fs_info, + __entry->root = inode->root->root_key.objectid; + __entry->ino = btrfs_ino(inode); + __entry->target_start = start; + __entry->target_len = len; + __entry->em_generation = em->generation; + __entry->em_start = em->start; + __entry->em_len = em->len; + ), + + TP_printk_btrfs("root=%llu ino=%llu target_start=%llu target_len=%u " + "found em=%llu len=%llu generation=%llu", + __entry->root, __entry->ino, __entry->target_start, + __entry->target_len, __entry->em_start, __entry->em_len, + __entry->em_generation) +); + +TRACE_EVENT(defrag_file_start, + + TP_PROTO(const struct btrfs_inode *inode, + const struct btrfs_defrag_ctrl *ctrl, u64 start, u64 len), + + TP_ARGS(inode, ctrl, start, len), + + TP_STRUCT__entry_btrfs( + __field( u64, root ) + __field( u64, ino ) + __field( u64, start ) + __field( u64, len ) + __field( u64, newer_than ) + __field( u64, max_sectors_to_defrag ) + __field( u32, extent_thresh ) + __field( u8, flags ) + __field( u8, compress ) + ), + + TP_fast_assign_btrfs(inode->root->fs_info, + __entry->root = inode->root->root_key.objectid; + __entry->ino = btrfs_ino(inode); + __entry->start = start; + __entry->len = len; + __entry->extent_thresh = ctrl->extent_thresh; + __entry->newer_than = ctrl->newer_than; + __entry->max_sectors_to_defrag = ctrl->max_sectors_to_defrag; + __entry->flags = ctrl->flags; + __entry->compress = ctrl->compress; + ), + + TP_printk_btrfs("root=%llu ino=%llu start=%llu len=%llu " + "extent_thresh=%u newer_than=%llu flags=0x%x compress=%u " + "max_sectors_to_defrag=%llu", + __entry->root, __entry->ino, __entry->start, __entry->len, + __entry->extent_thresh, __entry->newer_than, __entry->flags, + __entry->compress, __entry->max_sectors_to_defrag) +); + +TRACE_EVENT(defrag_file_end, + + TP_PROTO(const struct btrfs_inode *inode, + const struct btrfs_defrag_ctrl *ctrl, int ret), + + TP_ARGS(inode, ctrl, ret), + + TP_STRUCT__entry_btrfs( + __field( u64, root ) + __field( u64, ino ) + __field( u64, sectors_defragged ) + __field( u64, last_scanned ) + __field( int, ret ) + ), + + TP_fast_assign_btrfs(inode->root->fs_info, + __entry->root = inode->root->root_key.objectid; + __entry->ino = btrfs_ino(inode); + __entry->sectors_defragged = ctrl->sectors_defragged; + __entry->last_scanned = ctrl->last_scanned; + __entry->ret = ret; + ), + + TP_printk_btrfs("root=%llu ino=%llu sectors_defragged=%llu " + "last_scanned=%llu ret=%d", + __entry->root, __entry->ino, __entry->sectors_defragged, + __entry->last_scanned, __entry->ret) +); + #endif /* _TRACE_BTRFS_H */ /* This part must be outside protection */